What Are AI Tools Reading on My Site?What Are AI Tools Reading on My Site?

I wanted to see what actually happens between asking an AI tool a question and getting a recommendation back. What does it search for, which pages does it open, and how much of that ends up in the answer?

So I put five questions into ChatGPT, Perplexity, Gemini, and Claude, then followed whatever research activity each tool made visible.

Some questions asked for product recommendations. One asked for help implementing Google sign-in. Another asked why a website might appear in Google but rarely get cited by AI assistants.

I ran this on November 3, 2026. That gave me 20 question and platform combinations, with 17 usable answers captured.

A few things stood out, especially around what it means when your product shows up.

The question I typed wasn’t always the question it searched

One prompt asked for a documentation platform for a small SaaS team, with a budget of $250 a month. I mentioned frequent releases and keeping docs accurate for developers and AI agents. I didn’t name any vendors.

Perplexity turned that into searches including these.

Mintlify vs Fern vs GitBook documentation platform comparison 2026

documentation platform llms.txt MCP server for AI agents docs

It had already introduced three brands before reading the pages I could see.

The captured activity then showed it reading pages from GitBook, Fern, ReadMe, and another site. Its final answer recommended Fern, with links to Fern’s pricing and MCP documentation.

I couldn’t see why Fern won internally. But I could see how a broad question about a team’s needs became a much narrower comparison between named products.

That gives me something useful to check on my own site. I might have a page that answers the customer’s question, but does it answer the comparison the assistant actually went looking for?

Claude had a messier search along the way. A search for Fern pricing returned relevant material alongside FERC fee pages and a car dealer in Fernley. Later, it read official vendor pricing pages and linked those in its answer.

Those unrelated results appeared on screen. That doesn’t mean Claude used them.

Seeing my own product in the answer needed a closer look

ChatGPT recommended trying Thally Cloud for the documentation question. As someone building Thally, that would have been a nice result to count.

Except memory appeared in its Sources interface.

Claude also showed an explicit memory read for Thally. It mentioned Thally in its answer, but its selected option was Mintlify if the startup discount applied, with other options depending on the team’s needs.

So I had a recommendation from one tool and a mention from another, both with prior account context in the picture.

I couldn’t use either as evidence that someone who had never heard of Thally would discover it through the same question.

I had started every question in a fresh conversation. That still left account memory enabled.

Next time, I want to separate those tests. Seeing how an assistant helps me with everything it already knows is useful. Measuring whether it discovers my product without that history needs a different setup.

A search result, a citation, and a recommendation tell me different things

Following the activity made me more careful about what I counted.

If a page appeared in search results, I could say it was returned for that search. I couldn’t say the assistant opened it.

If the interface showed a completed page read, I had better evidence that the content was accessed. I still couldn’t tell which parts influenced the answer.

If the final answer linked the page, I could record a citation. I’d still need to check whether the page supported the claim beside it.

And a product could be discussed without being the one the assistant recommended, as happened with Thally in Claude’s answer.

That distinction changes how I’d investigate a missing recommendation. A page that never appears in the research needs a different investigation from one that gets read and cited while another product gets chosen.

It also means I can’t look at an uncited page and confidently say the assistant rejected it. The visible activity only showed part of the process.

The answers differed, even with the same prompt

For the Google sign-in question, ChatGPT and Perplexity built their answers around Auth.js v5. Claude read Better Auth documentation and used Better Auth for its main implementation, with Auth.js as an alternative.

For the named comparison of Mintlify, GitBook, and ReadMe, ChatGPT selected ReadMe Pro for the API requirements, with a qualification about AI answers and version scope. Perplexity and Claude leaned toward Mintlify for an engineering-led workflow.

The cross-chain stablecoin answers had more overlap. ChatGPT, Perplexity, and Claude all included Circle CCTP or Bridge Kit for USDC, then suggested other options for broader coverage.

These were the recommendations I captured. I didn’t run the implementation examples or independently verify every product claim, so I wouldn’t turn this into buying advice or a ranking of the assistants.

The question about AI visibility was interesting for another reason. The advice sounded fairly similar, but the visible sources differed.

ChatGPT linked OpenAI and Google documentation. Perplexity read Google’s AI-features documentation and linked a mix of provider guidance and other articles. Gemini’s citation controls led to commercial SEO articles. Claude showed memory activity, with no web-search event or external citation visible in the completed run I captured.

That would make me check the source before acting on advice to add a file, change schema, or rewrite a page. Similar answers can have very different evidence behind them.

What I’d do with this on my own site

I’d start with a few real customer questions and identify the page I’d expect to help answer each one.

Then I’d try the ordinary question, a search targeting my domain, and a direct request to read the page URL, in separate conversations.

If the assistant can read a supplied URL but never finds it on its own, I have a discovery problem to investigate. If it can’t read the page, I’d check access and what content it actually receives. If it reads the page but the answer still misses the point, I’d inspect the page against the question.

I’d also record mentions, citations, and recommendations separately, then check what each citation supports.

That gives me a specific thing to work on and retest. It’s much more useful than making a collection of changes because someone called them AI SEO.

What I’d change in the next run

This first pass had uneven settings. Gemini used Flash, Claude used Opus 5.5, and ChatGPT and Perplexity had mode differences across the assessment. Browser interruptions also meant some answers were recovered later.

Gemini returned usable answers for two questions. I didn’t capture usable answers for the other three after retries, and I don’t have enough evidence to explain every failure.

So next time I’d use accounts without prior brand context, record the modes more consistently, and repeat the questions across multiple days. I’d keep the failures too.

Then I’d change one demonstrated problem on a page and run the same questions again.

For now, the useful part was being able to trace a little more of what happened. I could see where brands entered the search, where memory complicated the result, and where a citation still needed checking.

Before I change another page to make it more likely to get recommended, I want to know how far that page is getting already.

The prompts, captured outcomes, selected source URLs, and limitations are in the public experiment record.