CRO Is Changing: Why Tactics That Work on People Fail on AI Shopping Agents

Also, what you need to know about LLMs.txt. HBR research shows what agents like and don’t like.

This post covers two topics:

1. The LLMs.txt file has been around for a long time. Google guidelines say don’t worry about it, but now they’ve said it can help.

2. CRO tactics and their impact on agents. HBR research breaks down positive change, no change and negative impact on agents.

WTF Is an llms.txt & Why Won’t It Go Away?

This is a topic that has been around for over 12 months. Originally, it felt like a hack or hype tactic to boost performance in ChatGPT. There was little evidence to support LLMs actually engaging with the file or sourcing it in answers, which is why we’ve never recommended it as a priority for clients.

It even says in Google’s guidelines, directly referencing ignored hacks like llms.txt:

However, over the past few weeks, Google released having llms.txt as a test to be agentic-ready in their Lighthouse audit, and John Muller confirmed it is a thing, but not how the industry (or LinkedIn) was claiming.

Correct - It’s not a discovery tool to hack your way to success.

Instead, he says it is more of a call to action for agents to read and reference when they visit the site.

For Shopify brands, Shopify is creating this as a default for you.

Just add the following URLs at the end of your Shopify domain to see it > /llms.txt

e.g., https://www.wearetala.com/llms.txt and https://yumove.co.uk/llms.txt

To make this slightly more confusing, Shopify has recently been endorsing agents.md. All Shopify sites also have this file live here > /agents.md

e.g. https://www.wearetala.com/agents.md and https://yumove.co.uk/agents.md

What’s the Difference Between agents.md and llms.txt?

Both files are identical, minus this line of text:

Agent discovery: the canonical agent-facing description of the store is at \/agents.md. You're reading /llms.txt, which mirrors that content.

This means that Shopify is referencing the agents.md file as the main original source. Likely just keeping LLMs.txt live as a fallback or backup just in case.

What If I Had a Custom llms.txt File?

Shopify would have overwritten this now. If you still want to maintain a custom LLMs.txt file, there is a workaround - you can see my social post here.

What If I’m Not on Shopify?

Non-Shopify sites, we’d recommend creating a file yourself and replicating a similar logic to the Shopify files linked above. Any clients of ours, we’ll be briefing these in and liaising with the developers when it fits in the priority queue.

Is It Going to Make a Difference?

Dan, our Delivery Director, phrased it as “It’s probs something you should be doing “just in case” anyway. If it could feasibly be good for AI, you should probably do it, although it’s not a super high priority,” - ever the poet.

Want More on This Topic?

This topic and more were discussed on our AI Search webinar recently, the link is here if you are interested in watching back:

Dan led this webinar with our friends at Glara and our client Abhishek from YuMove.

How Does CRO Impact Agents?

HBR research suggests that many eCommerce CRO tactics designed for people do not work reliably on AI shopping agents.

As agents shape more purchase decisions, tactics like scarcity, countdown timers, vouchers, strike-through pricing, and bundles can be ignored or even reduce selection depending on the model and category.

Here’s a summary of their research findings:

POSITIVE IMPACT (Reliably worked)

• Star ratings - the only tactic that consistently increased selection across all 4 AI models and product categories

• Social proof - purchase counts showed relatively stable effects (though not as consistent as ratings)

NO IMPACT / UNSTABLE (Unreliable)

• Strike-through pricing - produced different effects depending on model and product category

• Countdown timers - no stable pattern

• Vouchers - variable effects across models

• Bundling - showed mixed/unstable results; in at least one case, it reduced selection

NEGATIVE IMPACT (Backfired)

• Advanced reasoning models (GPT-5 and Gemini 2.5 Pro) appeared to penalise overt persuasion tactics, interpreting them as low-quality or manipulative signals

• More aggressive promotional tactics are increasingly becoming counterproductive with advanced models

• Scarcity cues (”Only 2 left!”) - more advanced models like GPT-5 actually reacted negatively in some categories

• Price - higher prices reliably reduced selection (as expected for both humans and AI), caveat being relative to the category being shopped, e.g., luxury comes with higher prices relative to competitors

Takeaway: The fundamentals matter most (competitive pricing + authentic reviews). Everything else requires model-specific testing and ongoing monitoring.

Across thousands of simulations, only star ratings consistently increased choice, and price consistently reduced it; other persuasion cues were unstable and often backfired in advanced models. Treat each model as a distinct segment, focus on fundamentals (competitive pricing and authentic reviews), and keep testing as models evolve.

Article here for additional reading > https://hbr.org/2026/05/research-traditional-marketing-doesnt-work-on-ai-shopping-agents?autocomplete=true&giftToken=1938012211781628298053

Thanks for reading. As always, we focus on non-algorithm hacking content, so we only grow through word of mouth. Please share this with anyone you think it will add value to.

---

And to address the elephant in the room: yes, we’ve had a glow-up ✨

You’ll have noticed things looking different around here this month as we’ve rebranded. Same team, same mission, sharper focus: NOVOS has grown from an SEO agency into an eCommerce growth agency, helping brands get found across search, AI, and paid channels.

More on what’s coming across events, roundtables, merch and new THRIVE drops in the weeks ahead. Be the answer™.

Speaking of which… you’re invited 🎟️

Join our Paid Media Director, Benoit Legendre, and John Brolly, Head of Partnerships at Athos Commerce, on 15 July as they unpack what it takes to scale eCommerce performance in 2026:

✅ How Meta’s evolving algorithm is changing the way eCommerce brands scale
✅ Why creative is no longer a volume challenge, but a systems challenge
✅ A practical framework for a creative system that drives weekly learnings — and faster iteration than your competitors
✅ How to get more from AI Max by connecting SEO, product data, PDPs and onsite content as one growth system

Sign up here →