The fastest reliable way to find orphan pages after migration is to crawl the new store, then compare that crawl against URL lists the crawl can never produce by itself: the old server's access logs, the old XML sitemap, and Search Console. Screaming Frog will do the merge for you — connect the GSC and GA4 APIs, tick Crawl Linked XML Sitemaps, run Crawl Analysis, then open Reports > Orphan Pages. The part most people skip is the logs, and the logs are where the interesting damage hides.
Because here's the trap. Your new Shopify sitemap is generated from what exists in Shopify. Your internal links are generated from your nav, collections and theme. Both of them describe the store you built. Neither of them knows about the 300-URL long tail that was pulling in Google traffic on the old platform and didn't make it into the CSV.
Two different things get called an orphan, and only one is urgent
An orphan page in the strict SEO sense is a URL that exists and returns 200 but has no internal links pointing at it. Google can still find it via the sitemap, so it usually gets indexed, it just gets very little crawl priority and no internal PageRank.
The second kind is worse and nobody calls it an orphan: a URL that used to exist, still has inbound links and rankings, and now returns a 404 or gets bounced to the homepage. It's not in your crawl because nothing links to it. It's not in your sitemap because it isn't in Shopify. As far as every tool that only looks at the new store is concerned, it does not exist.
On a Shopify migration the second category is where the traffic loss comes from. Sort that first.
Pull the old access logs before you switch DNS
You cannot get server logs out of Shopify. There is no access log, no way to proxy the storefront and capture requests at the edge, and no plan tier that changes this. So the log file half of this method only works with logs from the platform you're leaving, which means you have to grab them before someone cancels the hosting.
Ask for twelve months of raw Apache or Nginx access logs, gzipped. On a WooCommerce or Magento box that's usually /var/log/nginx/ or the cPanel raw access download. Twelve months matters because of seasonality — a Diwali or Ramadan landing page that got 4,000 sessions last October will show zero traffic in a 30-day window and look safely deletable.
If the server is already gone, you're not finished, just poorer. Substitute the Wayback Machine's URL index for the old domain, the old CMS URL export if anyone kept a database dump, and your backlink tool's list of linked pages. It's a worse inventory. It's still an inventory.
Building the URL inventory
Five sources, in rough order of how much they earn their keep:
- Old access logs. Everything a human or a bot ever requested. Strip query strings, drop asset extensions, drop known bot user agents, deduplicate. This is the only source that catches URLs with no rankings, no links and no sitemap entry that still get real clicks from email or an old PDF catalogue.
- Search Console on the old property. Pages report, last 16 months, exported. The UI caps at 1,000 rows, so use the API or a bulk export if the catalogue is bigger than that — which it is.
- The archived old sitemap. Save the XML on the day you go live. It takes ten seconds and it is the single most useful file in the whole project.
- Backlink data. Any URL with a live inbound link that now 404s is measurable lost equity, and it's the easiest fix to justify to a founder.
- Campaign landing pages. Export destination URLs from your Meta ad account, Google Ads, and Klaviyo or Omnisend flows. Old flows keep sending traffic to URLs nobody on the migration call remembered.
Concatenate, deduplicate, rewrite the hostname to the new domain, save as a text file. That's your list.
Configuring Screaming Frog so Crawl Analysis is worth running
Run two crawls, not one.
Crawl one, Spider mode on the live store. Before you start: API Access > Google Search Console and GA4, both connected to the new property. Configuration > Spider > Crawl, tick Crawl Linked XML Sitemaps and add https://yourdomain.com/sitemap.xml manually, because Shopify's robots.txt reference isn't always picked up cleanly. Then throttle it. Shopify starts returning 429s and 430s if you crawl hard, and a crawl full of 430s looks exactly like a broken site. One or two threads, 2 URLs per second. A 15,000-URL store takes a couple of hours. Let it.
When it finishes, Crawl Analysis > Configure, tick Sitemaps, GA and GSC, then Start. Reports > Orphan Pages gives you every URL that appeared in the sitemap, in GA4 or in GSC but was never reached by a single internal link.
Crawl two, List mode with your inventory file. Upload, set it to always follow redirects, crawl. Then Reports > Redirects > All Redirects and check the final destination of every 301, not just the fact that a 301 exists.
A worked example
Say the logs give you 9,431 unique paths over twelve months. Strip assets, bot hits and anything requested fewer than twice and you're down to 4,102 real HTML URLs. Run those through List mode:
- 3,760 return a 301 to a genuinely matching page. Fine.
- 214 return a 301 to the homepage.
- 128 return a 404.
3,760 + 214 + 128 = 4,102. The 128 are obvious and someone will fix them by Friday. The 214 are the ones that get signed off as "redirects are in place" and quietly bleed for a year, because a blanket homepage redirect is treated as a soft 404 and the ranking goes anyway. If those 214 averaged even 8 organic sessions a month between them — 1,712 sessions — at a 1.6% conversion rate and ₹2,400 average order value, that's roughly ₹65,000 a month walking out. Whether your numbers are bigger or smaller, do that multiplication before you decide the redirect map is done.
Reading the orphan report without chasing ghosts
The Orphan Pages report will be noisy on Shopify. Ignore these:
/collections/alland Shopify's tag URLs (/collections/x/tag-y) — not in the sitemap by design, frequently unlinked, usually harmless.- Cart, account and checkout paths that GA4 recorded.
- App proxy pages and anything under a review or wishlist app's path.
What actually needs work: products that sit in no collection, blog posts orphaned when the old category structure disappeared, /pages/ URLs that were only ever linked from a nav item somebody deleted, and collection pages that exist, are in the sitemap, and appear nowhere in the menu because the new nav has fewer top-level items than the old one.
Products in zero collections are the common one. Shopify still lists them in sitemap_products_1.xml, so they're indexable, but the only route to them is site search. If a meaningful chunk of the catalogue lives that way, better on-site search and filtering is the honest fix rather than inventing collections nobody browses — which is what our FilterPro app is for. Either way, get them linked from somewhere.
Fixing each bucket
404s with traffic or links: 301 them to the closest equivalent page. Not the homepage, not the parent collection unless the parent collection is genuinely the right answer. Bulk-load via Online Store > Navigation > URL Redirects with a CSV; for very large sets use the Admin API or Matrixify, and expect the import to take a while.
404s with nothing: leave them. A 404 on a URL nobody wants is not a problem, and cluttering the redirect table makes the real entries harder to audit.
Live but unlinked: give them an internal link that a person would plausibly click. A collection description with three product links beats a footer link farm. Then re-submit the sitemap.
Redirected to the homepage: re-point them. This is usually a find-and-replace on the redirect CSV once you've matched old handles to new ones.
The cases that break this method
Multiple locales via Shopify Markets. Each market path (/en-ae/, /en-au/) generates its own sitemap and its own set of orphans, and Screaming Frog will happily report the same product as orphaned four times. Crawl one locale at a time, or filter the report by path prefix.
Headless storefronts. The sitemap is whatever your front end generates, which means it can be wrong in ways Shopify's own sitemap cannot. Crawl the sitemap in List mode separately and check every URL in it returns 200.
Password-protected staging. If you're auditing before launch, add the storefront password under Configuration > Authentication, and remember that a store behind a password serves a different robots.txt.
We get the old-to-new handle mapping wrong on maybe a third of URLs on the first pass, usually because the old platform used category-nested product paths and the mapping script matched on the wrong segment. Budget a second pass. It is not a sign that anything went badly.
Run it three times, not once
Day 3 after launch: List mode against the inventory, catch the hard 404s. Day 14: full Spider crawl plus Crawl Analysis, once GSC has enough data on the new property for the API pull to be useful. Day 45: repeat, and compare the Pages report on the old and new properties side by side. Anything that had impressions before and has none now is either still broken or was never redirected properly.
If you'd rather have someone else do the log parsing, send us the URL and we'll run the crawl and the redirect audit as part of a free store audit. If the problem is broader than redirects — thin collection copy, no internal linking structure to speak of — that's a content and SEO conversation instead.