New: The $100M AI Visibility Roadmap, built from 7,960 live pages and 197 A/B tests. Get it free →

Home / Blog / 197 Shopify A/B Tests: What Wins and What Loses

CRO · 10 min read

197 Shopify A/B Tests: What Wins and What Loses

We ran 197 A/B tests on 26 e-commerce stores and published every result. Here is what won, what lost, where the revenue came from and how to use it on your own store.

Robert Battle ·

Key takeaways
  • 171 of 197 A/B tests across 26 e-commerce stores beat the original: an 86.8% win rate, a 12.8% average lift and $1,214,283 in added revenue.
  • Product pages added the most revenue ($512,176 from 62 tests) and had the highest average lift at 15.9%.
  • The homepage had the highest win rate of any page template at 94.1% across 51 tests.
  • An FAQ accordion on the homepage or a collection page won 19 of 19 tests.
  • Layout and color changes had the least certain records, so test them instead of debating them.

Across 197 A/B tests on 26 e-commerce stores, 171 beat the original page. That is an 86.8% win rate, an average lift of 12.8% per test and $1,214,283 in added revenue. Product pages added the most money, FAQ accordions never lost, and the few changes with mixed records were mostly layout and color changes.

This study is for Shopify and e-commerce teams who want A/B testing examples backed by a full record instead of ten hand-picked success stories. You'll get the win rate, average lift and revenue for each page type, the changes that won repeatedly, the ones with mixed records, a separate cut by industry, and a way to turn all of it into your own test plan.

What did 197 e-commerce A/B tests find?

Here is the short version. Every number below comes from QCK's own test tracker, which is published in full on our A/B test results page.

197 A/B tests at a glance
197A/B testsacross 26 e-commerce stores
86.8%Win rate171 wins
+12.8%Average lift per test
$1,214,283Added revenue

Source: QCK A/B test results, 197 tests, qck.co/ab-test-results

  • Most tests won. 171 of 197 tests made more money than the original, so 26 did not.
  • Product pages earned the most. 62 product page tests added $512,176, more than any other page type.
  • The homepage won most often among page templates. 51 homepage tests had a 94.1% win rate.
  • One change never lost. An FAQ accordion on the homepage or a collection page won 19 of 19 times.
  • Collections were the hardest place to win. 21 tests, an 81.0% win rate and the lowest average lift at 8.3%.

How this A/B testing study was run

Most lists of ecommerce A/B testing examples show the winners and leave out the rest. That makes testing look easier than it is, and it tells you nothing about which ideas are reliable. This study reports the whole record.

What is in the data

  • Tests: 197 A/B tests run by QCK for client brands.
  • Stores: 26 e-commerce stores.
  • Result measured: revenue. A win means the new version made more money than the original.
  • Reporting rule: every test is reported, including losses.

How to read the page-type numbers

Each test is tagged with the part of the store it changed: product pages, homepage, collections, sitewide elements such as the header, menu and cart, and FAQ sections. Some tests cover more than one page type. An FAQ accordion on the homepage counts under both Homepage and FAQ sections, for example. That is why the page-type rows add up to more than 197 tests and more than the $1,214,283 total. Read each row on its own, and do not add the rows together.

The data comes only from stores QCK works with, and the tests were chosen by CRO strategists after research on each store. So the win rate tells you how often a researched idea beat the original. It does not tell you how often a random idea would win.

Which page types win most often?

Win rate tells you how reliable testing was on each part of the store. FAQ sections won every time. Among the main page templates, the homepage had the highest win rate and collections had the lowest.

A/B test win rate by page type197 tests across 26 e-commerce stores; some tests cover more than one page type
FAQ sections100%
Homepage94.1%
Product pages87.1%
Sitewide85.2%
Collections81%
View the data
Value
FAQ sections100%
Homepage94.1%
Product pages87.1%
Sitewide85.2%
Collections81%

Source: QCK A/B test results, 197 tests, qck.co/ab-test-results

Page typeTestsWin rateAverage liftRevenue lift
Product pages6287.1%+15.9%$512,176
Homepage5194.1%+13.1%$348,756
Sitewide5485.2%+14.0%$175,238
Collections2181.0%+8.3%$143,515
FAQ sections19100%+11.7%$118,589

Three things stand out.

  • The homepage is a safe place to start. A 94.1% win rate across 51 tests is the highest of any page template. If you have never run a test, the homepage gives you the best odds of a first win.
  • Product pages had the biggest lifts. The average product page test lifted results by 15.9%, the highest of any page type.
  • Collections are harder. Collections had the lowest win rate and the lowest average lift. They still added $143,515 from only 21 tests, so they are worth testing, but expect more misses.
See which of these tests fit your store

We review your highest-traffic pages and pick the tests most likely to win on your store, drawn from the 197 we have already run, with the revenue math.

Get my test plan →

Where did the added revenue come from?

Win rate is only half the picture. A test that wins on a page with little traffic adds little money. Revenue lift shows where the wins were worth the most.

Revenue added by A/B test winners, by page typeSome tests cover more than one page type, so rows overlap
Product pages$512k
Homepage$349k
Sitewide$175k
Collections$144k
FAQ sections$119k
View the data
Value
Product pages$512k
Homepage$349k
Sitewide$175k
Collections$144k
FAQ sections$119k

Source: QCK A/B test results, 197 tests, qck.co/ab-test-results

Product pages produced $512,176 of added revenue from 62 tests. The homepage followed at $348,756 from 51 tests. Sitewide tests ran almost as often as homepage tests (54 tests) but added about half as much, at $175,238.

The practical read: product pages sit closest to the buying decision, so a change there touches almost every order. If you can only build a handful of tests this quarter, put most of them on your product page template. Our guide to product page optimization covers the product page tests in detail.

Which changes won again and again?

A single winning test can be luck. A change that wins on store after store is a pattern you can borrow. These are the changes QCK tested more than once, with the full record for each.

ChangeRecordWhere it ran
FAQ accordion19 of 19Homepage and collections
Review stars placement8 of 9Product pages
Cart drawer redesign6 of 7Sitewide
Image carousel below the main product image5 of 5Product pages
Menu reorder and restructure3 of 3Sitewide
Trust badge placement2 of 3Product pages
2 products per row1 of 2Collections
CTA color changes in the homepage second section1 of 2Homepage

The most-repeated winner on each page type

Win rate of the most-repeated test on each page typeNumber of tests shown in each label
FAQ accordion, FAQ sections (19 tests)100%
FAQ accordion, homepage (8 tests)100%
Review stars placement, product pages (9 tests)89%
FAQ accordion, collections (7 tests)86%
Button and CTA color, sitewide (18 tests)83%
View the data
Value
FAQ accordion, FAQ sections (19 tests)100%
FAQ accordion, homepage (8 tests)100%
Review stars placement, product pages (9 tests)89%
FAQ accordion, collections (7 tests)86%
Button and CTA color, sitewide (18 tests)83%

Source: QCK A/B test results, 197 tests, qck.co/ab-test-results

The FAQ accordion is the standout. It ran 8 times on homepages and won all 8. It ran 7 times on collections and won 86% of the time. Counted as its own category, FAQ sections went 19 for 19 with an average lift of 11.7%.

Why these changes keep winning

The data shows what won, not why. Our read, from building the tests, is that the repeat winners share one trait: they answer a question the shopper already has, at the moment they have it.

  • FAQ accordions answer shipping, returns and sizing questions without sending the shopper to another page.
  • Review stars near the title answer "do other people like this?" before the shopper scrolls.
  • A second image carousel answers "what does it look like in use?" for shoppers who skip the thumbnails.
  • A cleaner cart drawer answers "what am I paying?" before checkout.
  • A reordered menu answers "where do I find it?" in fewer taps.

What lost, and what it teaches

26 of the 197 tests did not beat the original. The losses are as useful as the wins, because they show where a confident opinion is not enough.

Layout and color changes had mixed records

  • 2 products per row on collections: 1 win in 2 tests. A bigger product image helped one store and not the other.
  • CTA color in the homepage second section: 1 win in 2 tests.
  • Button and CTA color, sitewide: the most-tested change of all, with 18 tests and an 83% win rate. That is a good record, but it sits below the overall 86.8%, and it means roughly one color test in six lost.
  • Trust badge placement: 2 wins in 3 tests on product pages.

Color and layout are the changes people argue about most in design reviews, and they are the ones where the test record is least certain. Changes that add information, such as an FAQ, reviews or more images, had cleaner records than changes that only restyle what is already there.

Even strong ideas lose sometimes

Review stars placement lost once in 9 tests. The cart drawer redesign lost once in 7. Neither loss makes the idea bad. It means the same change can behave differently on a different store, with different products, prices and shoppers. This is the case for testing instead of copying: if you had shipped every "best practice" in this study without testing, about 13% of your changes would have cost you money, and you would not know which ones.

Do results differ by industry?

QCK also publishes a separate cut of its test tracker by industry. It covers every concluded single-variant A/B test run on live Shopify storefronts: 183 tests across 31 brands, with 161 wins and 22 losses (an 88.0% win rate) and $932,170 in revenue added, as of October 2026. The scope is different from the 197-test set above, so the two sets of numbers should not be mixed or compared row for row.

These are the six sectors with the most tests in that set.

SectorTestsWin rateRevenue added
Automotive, Off-Road and Towing3878.9%$221,726
Apparel and Fashion3387.9%$357,137
Office, Stationery and Gifting2882.1%$77,724
Pet Products1984.2%$10,171
Health and Wellness16100%$29,056
Beauty and Personal Care15100%$28,016

Win rates ranged from 78.9% in automotive to 100% in health and beauty. The smallest sectors show why sample size matters: Food and Beverage had only 2 tests, won 1 and finished at a net loss of $981. Two tests cannot tell you anything about a whole sector. The full breakdown, including all 12 sectors, is on the CRO results by industry page.

Send better traffic into your winning pages

The free $100M AI Visibility Roadmap is built from 7,960 live pages and 197 A/B tests. It covers getting cited by ChatGPT, Google AI Overviews and Perplexity.

Get the free roadmap →

How to use these results on your store

Do not copy the winners straight into your theme. Use them to decide what to test first, then let your own shoppers decide. Here is the order we would follow.

  1. Find your highest-traffic template. In Shopify Analytics or GA4, check which template gets the most sessions: usually product pages, then the homepage, then collections.
  2. Find the biggest drop in your funnel. Compare sessions, add to cart, reached checkout and orders. The steepest fall tells you which page to test.
  3. Pick one repeat winner that fits that page. An FAQ accordion for the homepage or collections. Review stars placement or an image carousel for product pages. A cart drawer redesign if shoppers add to cart but do not check out.
  4. Write the hypothesis in one sentence. For example: "Moving review stars under the product title will raise revenue per visitor because shoppers will see proof before they scroll."
  5. Test one change against the original. Split traffic evenly and change only one thing, so you know what caused the result.
  6. Judge it on revenue. Use revenue per visitor, not clicks or add to cart alone.
  7. Record the result, win or lose. A written log of losses stops your team from re-running the same failed idea next year.

For the full ordered list, see our Shopify CRO checklist, which ranks each item by the test results behind it. If you need software to run the tests, we compare the options in best CRO tools for Shopify.

What this data cannot tell you

A study is only useful if you know its limits. These are the limits of this one.

  • It is one agency's data. All 197 tests were planned and built by QCK. Another team picking different ideas would get a different win rate.
  • Ideas were researched first. Tests were chosen after reviewing each store's analytics and shopper behavior. The 86.8% win rate applies to researched ideas, not to guesses.
  • Some samples are small. A 3 of 3 record for menu restructures is encouraging. It is not proof. Treat anything with fewer than 5 tests as a lead to check, not a rule.
  • Page types overlap. Some tests count under two page types, so the rows cannot be added together.
  • Stores differ. Lift and revenue depend on traffic, price point and product. Your result will not match the average.

What should you test first?

If you want one answer from 197 tests, it is this: start by adding information where shoppers hesitate. An FAQ accordion on the homepage or your top collection, and review stars near the product title, have the strongest records in the study and are quick to build. Save the color and layout debates for later, and test them when you do.

Then keep a full record. The value of a testing program comes from the list of what won and lost on your own store, which no outside study can give you. If you want help building that list, our Shopify CRO team runs this process every month, or you can get a custom plan with the first tests we would run on your store.

Want a test record like this for your store?

Book a free 30-minute call. We benchmark you against your top three competitors and show the pages ready to move, with prioritized fixes.

Get my custom plan →
FAQ

Questions & answers

What is a good A/B test win rate?
It depends on how tests are chosen and how a win is defined. In QCK's data, 171 of 197 researched tests (86.8%) made more money than the original. Ideas picked without research should be expected to win less often.
What should an e-commerce store A/B test first?
Start on your highest-traffic template with a change that adds information. In our 197 tests, an FAQ accordion won 19 of 19 times and review stars placement on product pages won 8 of 9.
Which page type adds the most revenue from testing?
Product pages. In QCK's study, 62 product page tests added $512,176, the most of any page type, with an average lift of 15.9%.
Do A/B tests ever lose?
Yes. 26 of the 197 tests in this study did not beat the original, and even strong ideas such as review stars placement lost once in 9 tests. That is why each change should be tested before it ships.
How do you decide if an A/B test won?
Judge it on revenue. In this study a test only counts as a win when the new version made more money than the original, not when it got more clicks.
Free custom plan

Find the revenue you're leaving on the table

A free 30-minute call where we build the first version of your plan with you. You leave with a plan either way.

  • Your score across Google, AI citations, branded demand, and reviews
  • The pages on your site that are ready to move right now
  • Your first Flagship page and the first three tests we'd run

Step 1 of 3

Get your free custom plan

Free. Takes about 30 seconds.