Stop redesigning your site with AI
I’ve seen companies follow AI recommendations and make their sites harder to buy from. Here’s what our redesign tests caught, and how a few changes turned things around.
Before you replace your website with a design AI recommended, test it against the site you already have.
I’ve worked with companies that followed AI recommendations and ended up with a worse buying experience. The ideas sounded reasonable. The designs looked good. Then customers started using them, and the problems showed up.
When that happens on a live site, you’re paying for it with lost sales while you figure out what went wrong.
We’ve tested over 3,000 website variations at Smart CRO. One thing I keep seeing is how much a seemingly small detail can change the result. A redesign can lose, then a version with a few adjustments can outperform the current site. You have to test it to find out.
The problem with following AI design advice
Ask AI how to improve your website and it will give you plenty of things to change. Simplify the page. Make the images bigger. Move the call to action. Add more information. Remove some information.
Some of those changes might help. Others might make it harder for someone to buy. You can read the explanation, agree with it, and still end up with a worse result.
A cleaner product page might hide something shoppers need. A new gallery might make it harder to compare options. A change to the purchase area might leave people wondering what they need to select before they can add to cart.
That’s why I want to see how customers respond before putting a new design in front of everyone. Here are a couple of examples from our testing.
Company A’s redesign lost. A refined version won.
We tested a product page redesign for Company A. In the early results, the full redesign was down 9.23% in revenue per visitor compared with the current page. A refined version of that redesign was up 33.72%.
Same site, same test, very different results. If we’d launched the full redesign and called the project finished, we would have missed the better option.
We kept the test running. The version we eventually selected finished with a 14.3% relative increase in conversion rate and was chosen for implementation.
How the versions performed
| Version | Result | Stage |
|---|---|---|
| Full redesign | −9.23% revenue per visitor | Early results |
| Refined redesign | +33.72% revenue per visitor | Same early results |
| Selected version | +14.3% conversion rate | Final results |
In a later redesign test for Company A, one version was showing a 59% drop in revenue per visitor before we fixed a scrolling problem. That’s the kind of issue you want to catch while you’re testing. We corrected the experience so we could evaluate the updated version.
A design review wouldn’t have told us what those changes would do to revenue. Testing gave us something we could act on.
Company B did better with fewer changes
For Company B, we tested several refinements to a product page. One focused change was leading at about 30% higher revenue per visitor. The version that combined all the changes performed poorly enough that we stopped sending traffic to it.
We were still collecting purchases on the leading version at that point. But we already had a reason to spend more time on that change and stop pushing the combined version.
This happens more often than people expect. You put several reasonable ideas together and assume the combined result will be better. Then the test shows that some of those changes are getting in the way.
Breaking the design into a few useful alternatives helps us figure out which parts deserve more work.
How we test a redesign
We start by figuring out where people are getting stuck on the current site. What questions are going unanswered? What makes the purchase harder than it needs to be? Which parts already work well?
Then we create the new design around those findings and build the versions we want to compare. We go through the buying experience on mobile and desktop, check that the interactions work, and make sure we’re recording the purchases or leads correctly.
We compare the new experience with the current site and watch the outcome that matters to the business. For an ecommerce store, that might be revenue per visitor or purchase rate. For a lead generation site, we also need to know whether the leads are any good.
If the new design loses, we look at what might be causing the problem and change direction. Sometimes we need another design. Sometimes a few adjustments make a big difference, like they did for Company A.
We also learn which ideas aren’t worth spending more time on. That leaves more time to optimize the crap out of the things that actually affect whether someone buys.
Once we’ve chosen the version to roll out, we check the permanent implementation too. It needs to work the same way the tested version did.
Questions about AI website redesigns
Can AI design a high-converting website?
AI can help you create a design and build it faster. Test it with your customers to find out whether it converts better than your current site. That’s how you find the changes worth keeping.
Should I redesign my Shopify store using AI recommendations?
Use the recommendations as ideas to investigate. Look at the problem each change is supposed to solve, check how it works with your theme and apps, and compare the new experience with the current one before replacing it.
Can a few changes fix a redesign that’s performing poorly?
Yes, sometimes. Company A’s refined redesign outperformed its original redesign, and Company B had a focused change performing better than the combined version. Look for what’s making the decision or purchase harder, then test a version that addresses it.
What if my site doesn’t have enough traffic for an A/B test?
Start with customer feedback and usability testing. Watch people try to complete the important steps and fix problems you can reproduce. A staged rollout can help you monitor what happens as more people use the new design.
Test it before you put it in front of everyone
You can have a design you love and still make it harder for customers to buy. Give yourself a way to catch that before replacing the current site.
We help create the design, test it, and make changes when something isn’t working. You end up knowing a lot more about what helps your customers and where your next improvement should come from.
If you’re planning a redesign, take a look at our website design and Shopify development services, or start with a conversion funnel audit.
