AI-referred traffic to brand websites increased 632% in roughly 10 months, according to data from Contentsquare, an experience analytics platform. The move away from organic traffic is happening faster than the mobile revolution a decade ago, and most brand websites weren’t built for it. But here’s the twist.
A drop in referrals doesn’t necessarily mean fewer sales. One Contentsquare client lost 40% of its organic search traffic to a single financial services page. Still, conversion on that page actually increased. The people arriving from AI chatbots are more informed and more intentional. The problem is that brands are flying blind on which traffic actually matters. “What we used to think of as SEO success is changing. It’s not just about visibility anymore.
It’s about the accuracy of how your brand is being portrayed in the AI summary,” said Andrew Frank, distinguished VP analyst at Gartner. Here’s what brand marketers need to know about the shift.
The empty shell problem
The most immediate issue isn’t content strategy. It’s how websites are built. “The browser gets an empty shell, and then you render,” said Jane Austin, SVP of design at Contentsquare.
More Read
AI bots “do a plain fetch of the raw HTML. They don’t wait for the page to build. There is nothing for them to read.”
Many brand websites use client-side rendering. The page loads an empty HTML shell, and JavaScript fills in the content after the fact. Human visitors never notice because their browser executes the JavaScript.
But most AI bots don’t run JavaScript. They fetch the raw HTML, see nothing, and move on. Google’s crawler is the exception — it uses a headless Chrome service that can render JavaScript — but the AI crawlers powering ChatGPT Search, Claude, and Perplexity all fetch only raw HTML. A site can rank well in Google while being invisible to every AI answer engine. Machine-readable is not enough
The fix is server-side rendering, where all the content is assembled on the server and sent as a complete package.
Brands that want their product pages, comparison tables, and category pages visible to AI agents must ensure they are rendered server-side by collaborating with the engineering team.
That’s only part of the fix, however. Server-side rendering faces a second problem. The content that works for humans — rich images, videos, interactive modules — is nearly invisible to AI agents. Frank calls this the “dual-mode media” challenge. Every asset needs two layers: one for humans, which is the visual experience, and one for machines made up of transcripts, chapter headings, alt text, and structured metadata.
AI agents don’t watch a product video. They read its transcript instead, thereby missing the visual elements. But there may be ways to create transcripts that make up for that, somewhat, Frank suggests.
“AI is likely to pick up on nuances of the semantic presentation that are perhaps invisible to people,” Frank said. “People don’t usually read the transcripts of a video. If the transcript has descriptions that are not in the video, there is an opportunity to replace some of that lost information.”
B2B brands are accidentally ahead on this. Their content is naturally structured with comparison tables, spec sheets, FAQ sections, and pricing pages. AI agents parse this kind of content easily.
B2C brands that rely on rich visuals and minimal text are effectively invisible to AI agents. Austin noted that, according to Forrester, 51% of software buyers now start their research in an AI chatbot rather than a search engine, up from 29% the year before. For B2B brands, the urgency is already here. Unlock your $1M+ organic growth engine. Everything enterprise teams need to grow visibility across search and AI.
Learn more
Recommended Articles
The bot identity crisis
A separate challenge cuts across all of this. Brands can’t tell who the bots on their site are working for. Cloudflare and other security tools can categorize traffic as bot or human.
They can distinguish between crawlers, live agents, and scraping bots. But they cannot connect a bot to the specific person who directed it.
“That human binding agent is ultimately invisible,” Austin said. “The website treats that agent like a person, but there isn’t a fix for that identity gap.”
Automated traffic now makes up 53% of all web traffic, according to the Imperva Bad Bot Report, and 40% of that is malicious. The “empty internet” theory — that most web traffic is bots talking to bots — may already be here. For marketers, this creates a personalization problem. If a bot arrives representing a high-value prospect who has been researching a product for weeks, the brand has no way to know. And if a human eventually arrives after the bot has done the research, the connection between the two visits is lost.
Frank pointed out that AI agents can collapse the entire customer journey into a single chatbot session. “If you can do the whole journey with a chatbot that ends with a transaction, the role of the website is highly diminished,” he said. That raises uncomfortable questions about how to allocate marketing budgets. Gartner projects that $15 trillion of B2B spend will flow through AI agent exchanges. The economic model for how brands access the context of those AI-mediated conversations is still being negotiated. Get MarTech Insights That Matter
Platform news, strategy analysis, and industry trends.
Trusted by 40,000+ marketing professionals. What brands should do now
Both Austin and Frank converged on a set of practical steps that brand marketers can take today. Audit your rendering. Check whether your most important pages — product detail pages, pricing pages, comparison pages — are server-side rendered. Use a tool like curl or a bot simulator. If the page returns empty HTML, AI agents cannot see your content. Add machine layers to every asset.
Every video needs a transcript. Every image needs alt text that describes what matters from a brand perspective, not just SEO keywords.
Chapter headings and structured metadata turn visual content into machine-interpretable content. Build structured content. FAQs, comparison tables, spec sheets, and pricing pages are gold for AI agents because they parse easily. B2B brands already do this.
B2C brands need to catch up. Measure before you move. Establish a baseline of bot traffic versus human traffic. Track conversion rates separately for each. The old KPI was SEO traffic volume.
The new KPI may be the conversion rate from AI-mediated visits. Run small experiments. Test two versions of a high-traffic page.
See what converts from AI-referred traffic versus direct traffic. Learn what works for the specific category before making big investments.
Partner with your tech team. This isn’t a problem that marketing can solve alone. Server-side rendering, API exposure, and MCP server architecture require engineering. Austin recommends treating this as a cross-functional conversation starting now.
Don’t lose sight of the human
For all the talk about agents, Frank and Austin both emphasized that human experience still matters. Austin described searching for a specific pair of gold hoop earrings with sapphires through an AI interface. She got results, but she could not tell which brands were high quality.
“The brand signals and the trust signals are still needed,” she said. “You have to ensure that your brand appears, that it feels on brand, and that the experience of shopping still feels good.”
The brands that will win in this new environment, she said, are the ones that start measuring now and keep the human experience central while adapting their technical foundation for the machines arriving first. “You don’t need to panic,” Austin said. “But you do need to start.”
MarTech is owned by Semrush. We remain committed to providing high-quality coverage of marketing topics. Unless otherwise noted, this page’s content was written by either an employee or a paid contractor of Semrush Inc.
