Shopify SimGym is an AI shopper simulator. You point it at your storefront, hand a synthetic buyer a goal — find a gift under $80, buy the blue hoodie in size 12, check if it ships to Broome — and it reports where that shopper hesitated, backtracked or gave up. On 28 October it gained whole-journey runs and explicit shopping goals, which puts it in merchants' hands roughly three and a half weeks before Black Friday traffic arrives. For an Australian store doing a few thousand sessions a month, that timing is the point. You cannot A/B test your way to a better BFCM on 6,000 sessions. Simulation is the cheapest stand-in that exists, and it is wrong often enough that you need to know where.
What Shopify SimGym actually does on a storefront
It is a model reading your pages the way a browser renders them, following links, and narrating its reasoning as it goes. Before the October update, the useful output was mostly page-level: does this product page answer the questions a buyer has. With journeys and goals, you can set a destination — "complete a purchase of a sub-$100 gift that arrives before Christmas" — and get a trace of every step between the ad landing page and the checkout button, with the friction marked.
What it is not is traffic. It has no wallet, no Afterpay account, no memory of your brand from Instagram, and no actual impatience. It will not abandon because your hero image took 4.1 seconds to paint on a 4G connection in regional Victoria. Keep that gap in mind; most of the ways this tool misleads trace back to it.
Features move. Check the app listing for what is in the current build rather than trusting a blog post from last month, including ours.
Why a low-traffic Australian store can't A/B test its way out of this
Here is the arithmetic, because it settles the argument faster than opinion does.
Take a store at 6,000 sessions a month and a 2% conversion rate. You want to know whether a redesigned product page lifts that to 2.4% — a 20% relative gain, which is a big win, bigger than most redesigns deliver. The standard rule of thumb for sample size at 95% confidence and 80% power is roughly n ≈ 16 × p(1−p) / δ² per variant. Plug it in: p = 0.02, δ = 0.004, so p(1−p) = 0.0196, and 16 × 0.0196 = 0.3136. Divide by 0.000016 and you get 19,600 sessions per variant. That's 39,200 sessions total, or about six and a half months at your current traffic.
BFCM is in three weeks. The test finishes in May.
And that's for a 20% lift. If you're hunting a realistic 5% improvement, δ shrinks to 0.001 and the requirement balloons past 300,000 sessions per arm. Australia's population makes this worse than it is in the US: you're fishing in a pond of 26 million people, a good chunk of whom will never buy your category. Which is why retention and AOV carry so much more of the growth load here, and why spending your pre-BFCM weeks on a statistically underpowered split test is close to the worst use of the time.
So you do the thing that's actually available: structured qualitative review, run fast, against specific goals. That's the hole SimGym fills.
Running a SimGym simulation on your own theme, step by step
The way we'd sequence it for a client with a fortnight to spare:
- Write the goals before you open the tool. Five to eight, each phrased the way a real buyer's intent is phrased. "Buy a Christmas gift for my sister, under $70, delivered to Perth before the 20th." "Find out if this is actually made in Australia." "Pay in four instalments." Vague goals produce vague findings.
- Start the journey where your traffic starts. If 60% of November sessions land on a collection page from Meta, don't simulate from the homepage. The homepage is often the least-visited page on a Shopify store during a sale.
- Run the same goal against both themes if you're mid-redesign. Same prompt, same entry URL, same day.
- Read the transcript, not the score. The number is a summary. The sentence where the simulated shopper says it couldn't tell whether the price included GST is the finding.
- Fix, re-run, keep the before-and-after. Two passes catches most of it. Three has diminishing returns.
Budget a working day for the first round. Most of it goes on writing goals and reading output, not on the tool.
AI shopper testing vs real A/B testing: they answer different questions
An A/B test tells you whether a change moved money. A simulated shopper tells you whether a page is comprehensible. These are not substitutes, and treating them as such is how teams end up confidently shipping something that tanks.
Simulation is good at: missing information, confusing labels, dead ends, contradictory shipping promises, size guides that don't open, a returns policy that says 30 days in one place and 14 in another. It finds the stuff that is objectively broken or objectively unclear.
It is bad at: desire, price sensitivity, trust, urgency, and anything involving how a page feels. An AI shopper will not tell you that your product photography looks cheap. It will not tell you that "ethically sourced" reads as hollow without a supplier named — and under ACCC scrutiny of green claims, unsubstantiated sustainability language is a legal exposure, not just a conversion one. A simulation will happily accept a vague provenance claim that a real Australian buyer, and possibly a regulator, would not.
Rule we use: if a finding is about information, act on it. If it's about persuasion, treat it as a hypothesis and go find a human.
Where simulated shoppers mislead most: Afterpay and Zip placement
This is the one we'd flag hardest for Australian merchants.
Ask a synthetic shopper where instalment messaging should sit and it will reason its way to something sensible-sounding: near the price, above the fold, consistent across the page. Fine. But it has no stake in the outcome. It doesn't carry the specific feeling an Australian shopper has when a $78 jumper reconfigures itself into four payments of $19.50 — which is the entire mechanism. The instalment line isn't information. It's a reframe of the price, and reframes work on nerves the model doesn't have.
What we've found holds up in practice: the Afterpay or Zip line belongs immediately under the price, on the same visual block, before the variant pickers. Not in the trust-badge strip at the bottom. Not only at checkout. If it appears for the first time at checkout, you've already lost the buyers who bounced on the sticker price — and a simulation will never report that bounce, because it had no sticker shock to begin with.
Two practical notes. Keep the instalment figure accurate against the GST-inclusive price, since Australian display prices include the 10% and a mismatch reads as a bug. And if you offer both Afterpay and Zip, show one by default rather than stacking logos; a wall of payment badges under the price dilutes the reframe.
So: use simulation to check the instalment copy is present, legible and consistent. Do not use it to decide placement. That decision we'd make from heatmaps, session recordings and, where volume allows, a real test in January.
Simulating the checkout path before Black Friday
Checkout itself is largely out of your hands on Shopify, which is a mercy. The failures live just before it.
Goals worth running, specific to how Australian fulfilment actually behaves:
- Delivery to a 6xxx postcode. Regional WA is where shipping promises break. If your cart says "2–4 business days Australia-wide" and Australia Post quotes seven to Karratha, a simulated shopper reading the page will flag the contradiction. Real ones just leave.
- A cart sitting $8 below your free-shipping threshold. Does anything on the page tell them what to add? This is the single highest-leverage AOV lever on a small Australian store, because Sendle and Australia Post parcel costs eat a flat margin per order regardless of basket size.
- Discount code entry. If your BFCM code is auto-applied via a link, run the journey from that link and from a cold organic session, and confirm both end at the same price.
- Guest checkout. Still the path most first-time buyers take.
- If you ship from offshore, a goal that asks whether GST is included. Under the low-value imported goods rules, a registered overseas seller collects the 10% at sale; a page that is ambiguous about it generates support tickets at the worst possible moment.
Click Frenzy's main November event gives Australian merchants something most markets don't have: a live rehearsal a fortnight before Black Friday. Run your simulations now, ship the fixes, then use Click Frenzy's real sessions to check whether the fixes held.
Reading a theme comparison result without fooling yourself
If you run the same goal against two themes, you get two traces and some comparative scoring. Three things to keep in mind.
First, variance. The same prompt against the same page twice won't produce an identical transcript. Run each goal at least three times per theme and look for findings that repeat. A problem that shows up once is noise.
Second, a close result is a null result. If theme A scores marginally above theme B, that's not a reason to migrate. The cost of a theme change — QA, metafields, app re-integration, the apps that quietly stop working — needs a large, repeated, explainable gap to justify it, and "explainable" is the operative word. If you can't point at the sentence in the transcript that says why, you don't have a finding.
Third, these runs ignore performance. A theme that scores well on clarity may still ship 900KB of JavaScript and lose you more revenue on mobile than any copy fix recovers. Check Core Web Vitals separately and treat them as a hard constraint before BFCM traffic hits. SwiftStore handles the scan-and-monitor side of that, and for anything structural our speed work is where we'd start.
Storefront testing tools, compared honestly
A simulator replaces none of these. It sits alongside them, and it's cheaper than all of them.
- Session recording and heatmaps. Real behaviour, no statistical power needed. Still the best value tool a low-traffic store can buy. Watch 30 sessions, you'll find something.
- Split testing apps. Genuinely useful above roughly 50,000 monthly sessions. Below that they mostly generate false confidence, as the maths above shows.
- Five-second tests and moderated user research. Slow, small samples, and still better than simulation for anything involving taste or trust. Five real Australians on a video call will tell you more about your provenance claims than fifty synthetic runs.
- Analytics funnel reports. Shows you where people drop. Never why.
- AI shopper simulation. Fast, repeatable, available at any traffic level, blind to emotion.
The sensible stack for a small brand in November: simulation to find the broken and unclear, recordings to confirm it on real sessions, and no split tests until the new year.
A pre-BFCM pass that fits in a week
Day one, write goals and run the first simulation round. Day two, fix what's unambiguous — broken links, contradictory delivery copy, missing size guides, instalment messaging that only appears at checkout. Day three, run PageSpeed on your three highest-traffic templates and strip whatever app scripts you're no longer using; the tag manager container from an agency you stopped working with in 2023 is usually in there. Day four, re-run the simulations and watch 20 real session recordings. Day five, check your free-shipping threshold against your actual blended parcel cost and set the BFCM codes.
That's a week, and it's worth more than a redesign you can't finish in time.
Questions we get asked
Can simulation replace A/B testing entirely? No. It replaces the A/B tests you were never going to be able to run. Once you're past 50,000 sessions a month, start testing properly and relegate simulation to pre-launch QA.
How many goals should I run? Five to eight covers most catalogues. Beyond that you're reading the same finding in different words.
Will it find Afterpay problems? It'll find a missing or inconsistent instalment line. It will not tell you whether moving that line changes revenue. Different question, different method.
Is it worth it after BFCM? More than during, honestly. Run it before EOFY campaigns and before any theme or checkout change goes live. It's good QA with a conversion lens, which is a fine thing to have year-round.
If you'd rather someone else wrote the goals and read the transcripts, our team in Bengaluru runs this kind of pass for Australian brands regularly, usually as part of a store audit. Three weeks is enough time. Two is tight. One is housekeeping.