How to Rank Your Website in ChatGPT and Claude: The Complete AI Search Guide (2026)

Mehar Zain — Thu, 14 May 2026 12:46:19 +0000

Originally published on StackJot. Republished here for the dev.to community.

If you run a website and you have not yet noticed your AI-search traffic creeping up in analytics, you will soon. Six months ago, this site got roughly 3% of its referrals from AI assistants. Last month, it was 19%. The shift is not theoretical anymore — it is the new reality of how people find content.

This guide is about one thing: how to make your website something that ChatGPT, Claude, and Perplexity will actually cite when a real person asks them a question. Not a vague theory of AI search, but the specific moves that worked on a real publisher's site, including the ones that did nothing and we wasted time on.

If you are still optimizing only for the ten blue links, you are leaving a growing slice of your audience to your competitors who figured out AEO first.

The short version (TL;DR)

If you only have five minutes, here is the entire playbook:

Allow the right AI bots in your robots.txt — OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot at minimum.
Answer specific questions on dedicated pages — one page, one question, one clear answer in the first 100 words.
Use Q&A formatting — H2 in question form, then a 2–4 sentence direct answer below it.
Add FAQ schema to pages with multiple sub-questions. AI engines read it; Google barely shows it anymore but that does not matter.
Cite real sources in your own content. AI engines prefer pages that themselves behave like trustworthy sources.
Stay rankable in regular search. ChatGPT uses Bing, Claude uses its own provider, Perplexity uses its own index — but in every case, you need to rank to be eligible.

Everything below is the unpacked version of those six points, with specific examples and the failures along the way.

What "ranking in AI" actually means

There is a mental model worth getting right before we go further. AI answer engines do not have a static index they rank you in. What they do is roughly this:

The user asks a question.
The model decides whether it needs fresh web information (most do this automatically now).
A search query is generated and run against a search index (Bing for ChatGPT, Anthropic's provider for Claude, Perplexity's own index for Perplexity).
The top results are fetched by a crawler bot in real time, or pulled from a recent cache.
The model synthesizes an answer using extractable chunks from those fetched pages.
Citations are attached linking back to the sources that contributed to the answer.

"Ranking in AI" really means two things in series: rank well in step 3, and write content that survives step 5 — meaning, content the model can extract from cleanly and feel confident citing.

If you skip either step, you do not appear.

Which AI bots to allow (and which to maybe block)

Here is the practical reference for 2026. These are the bots that actually matter for being cited by major AI assistants.

Bot	Who runs it	Purpose	Allow for AEO?
OAI-SearchBot	OpenAI	Builds the index ChatGPT search uses	Yes
ChatGPT-User	OpenAI	Fetches pages on-demand when users click "browse"	Yes
GPTBot	OpenAI	Crawls for training data	Optional
ClaudeBot	Anthropic	Crawls for Claude's web search and citations	Yes
Claude-Web	Anthropic	Fetches pages when users invoke web search	Yes
PerplexityBot	Perplexity	Crawls for Perplexity's answer engine	Yes
Perplexity-User	Perplexity	Real-time fetches when answering user queries	Yes
Google-Extended	Google	Used for Gemini and Vertex AI training	Optional

The key distinction is between training bots (GPTBot, Google-Extended) and query-time bots (OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot).

You can block training bots and still be fully eligible for citations. Many news publishers do exactly this. But if you block query-time bots, you remove yourself from the answer pool entirely. That is almost never what you want unless you have a specific commercial reason.

A working robots.txt for AEO-friendly publishing looks like:

\`
User-agent: OAI-SearchBot
Allow: /

User-agent: ChatGPT-User
Allow: /

User-agent: ClaudeBot
Allow: /

User-agent: Claude-Web
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Perplexity-User
Allow: /
`\

If you want to also allow training, add GPTBot and Google-Extended. If you want to block training but allow citations, omit them. There is no middle path that gets you cited while blocking the query-time bots.

How to write content AI engines will actually quote

This is where most AEO advice gets vague. Here is what specifically changes the extraction rate, based on watching which of our pages get cited and which do not.

1. One question per page (or per major section)

AI engines extract answers in chunks of roughly 50–300 words. If your page tries to cover ten related questions in one long flowing essay, the model has to guess which chunk to lift. If your page has a clear question-shaped H2 with a tight 2–4 sentence answer underneath, extraction is trivial.

The pattern that works:

\`

Why does ChatGPT sometimes give different answers to the same question?

ChatGPT uses a temperature parameter that introduces controlled randomness
into its responses. Even with identical prompts, different sessions can
sample different tokens. To get more consistent answers, lower the
temperature in API calls or rephrase your prompt to be more constrained.
`\

That paragraph is extractable. A model can lift it as a unit and cite the page. Compare to a 600-word section that wanders through history, opinion, and three sub-topics before landing on the same point — much harder to cite.

2. The first 100 words are load-bearing

When AI engines summarize a page, they weight the opening heavily. Get the direct answer into the first paragraph. Backstory, narrative, and personality can come later in the post (and should, because human readers want them). But somewhere in the first 100 words, the literal answer to the page's title should appear.

This sounds like SEO advice from 2014. It still works. AI engines have just made it matter twice as much.

3. Use real names and specific numbers

AI engines bias toward pages that demonstrate verifiable specificity. "Many users report" is weak. "Our test of 500 prompts found 73% success" is strong. Real product names, real dates, real version numbers, real prices — all of these signal that the page is grounded in actual reporting.

A useful test: read your draft and underline every claim that has a specific number, name, or date attached. If less than a quarter of your sentences have an underline, you are probably writing the kind of generic content AI engines do not need to cite because they can already say it themselves.

4. Format for extraction: lists, tables, FAQs

Long prose paragraphs survive extraction; bulleted lists and tables thrive in it. A three-column comparison table is easy for an AI to render as part of an answer ("Here is a comparison from StackJot…"). A flowing paragraph saying the same thing requires the model to restructure it, which makes it less likely to be quoted faithfully.

This is also where FAQ schema earns its keep. Even though most search engines no longer show FAQ rich results, the structured Q&A markup tells AI engines "this page contains discrete question-answer pairs you can lift directly."

5. Cite your own sources

This is the most underrated AEO tactic. AI engines have a soft preference for pages that themselves look like trustworthy sources. Linking out to primary sources (research papers, official documentation, original reporting) signals that your page is a synthesis built on verifiable inputs.

We added inline citations to a comparison post last quarter and saw citations to that page from Perplexity roughly triple over six weeks. We changed nothing else.

The pages that get cited most

After tracking which of our 50+ posts get cited in ChatGPT and Claude, three patterns dominate.

1. Definition and "what is X" pages.
Pages that answer a clear definitional question are catnip for AI engines. If someone asks Claude "what is AEO?", Claude needs a quotable source that defines it cleanly. If that's your page, you get the citation.

2. Comparison pages.
"X vs Y" content is extractable in obvious chunks: a feature table, a pros list, a verdict. We see our ChatGPT vs Claude vs Gemini comparison cited regularly when users ask AI engines for help picking an assistant.

3. Step-by-step how-to pages.
Numbered procedures with concrete actions are the easiest content for an AI to extract and re-present. The automate email with AI post gets pulled into Perplexity answers about email workflows almost weekly.

What does not get cited much, in our experience: opinion essays, news roundups, and "10 reasons why X matters" listicles without specific actionable content. AI engines do not need to cite an opinion when they can generate one themselves.

Mistakes that wasted our first three months

We tried plenty of things that turned out to be useless or actively counterproductive. Saving you the time:

Stuffing pages with FAQ schema for every possible variation.
We loaded one page with 18 FAQ entries. Citations did not increase. Page load slowed. We removed all but the 5 most relevant questions and citations went up slightly. Lesson: schema works because it matches real questions, not because more is better.

Adding "as of 2026" timestamps to every paragraph.
We thought freshness signals would help. They did not. AI engines look at publish dates and the lastmod in your sitemap. Sprinkling dates through prose looked desperate and changed nothing.

Writing for hypothetical AI prompts instead of real users.
Our worst-performing AEO experiment was a page literally titled "AI answer: what is the best CRM for solopreneurs?" — written as if we were being the AI. It read like junk to humans and to AI engines, which apparently noticed. Real human-shaped writing on the same topic, with a normal title, outperformed it 20-to-1.

Blocking GPTBot to "force" people to click through.
A few publishers tried this in 2024–2025. It does not increase clicks; it removes you from the candidate set. People do not switch to Google when an AI engine says "I cannot find a good source on this" — they just accept the AI's general-knowledge answer and move on. Don't block the bot that's bringing you traffic.

A 4-week AEO sprint plan

If you want a concrete starting point, here is the rollout we would do on a new site, based on what worked.

Week	Focus
1	Audit and unblock the right bots. Submit/refresh sitemap to Bing. Confirm pages are crawlable.
2	Rewrite the 5 highest-traffic pages with question-format H2s and tight first-100-word answers.
3	Add FAQ schema to those 5 pages. Identify 3 new definition pages to write.
4	Publish 3 definition pages. Set up tracking for AI referral sources (LLM traffic). Measure baseline.

By the end of week 4, you should see early citations starting to appear, especially for less competitive query patterns. The compounding work is in weeks 5–12, where you expand the pattern to more pages and start showing up reliably.

How to track whether it's working

Three signals worth setting up:

Referral domains in your analytics. Look for chatgpt.com, perplexity.ai, claude.ai, and gemini.google.com showing up as referrers. They will at first appear with very low volume — that is fine. The trend matters more than the number.
Manual citation checks. Once a week, ask ChatGPT, Claude, and Perplexity the exact questions your pages target. Note whether you are cited, and which pages.
Server logs for AI crawler traffic. OAI-SearchBot, ClaudeBot, and PerplexityBot all show up in logs with clear user-agent strings. If these crawl rates are growing, you are on the right track even before citations appear.

Do not over-rotate on any single signal. AEO traffic patterns are still noisier than Google traffic — you need a few weeks of data before any change is meaningful.

What this means for your content strategy

If you write for the web in 2026, you are now writing for two audiences:

Humans who will read your full post, scroll through your examples, and form an opinion.
AI engines that will read your post once, extract the most quotable 200 words, and either cite you or not.

The good news: writing well for one audience increasingly helps the other. Clear structure, direct answers, specific examples, and honest sourcing serve both. The bad news: anyone still writing fluff-padded SEO content from the 2018 playbook is about to find out that AI engines are even less patient with filler than human readers were.

The publishers who win the next two years are not going to be the ones who game AEO. They are going to be the ones who write the cleanest, most extractable, most verifiable content in their niche — and then make sure the AI bots can read it.

The takeaway

Getting cited by ChatGPT and Claude in 2026 is not a black box. It is a stack of small, knowable things: let the right bots crawl your site, write pages that answer one question clearly, structure your content so it can be lifted, and stay rankable in the search indexes the AI engines depend on.

If you treat AEO as "SEO with extra steps," you will under-invest in the parts that are genuinely different. If you treat it as a separate discipline, you will miss the fact that 80% of what works is still good writing and good structure.

The honest framing: AEO is what SEO should have been all along. Write like you respect the reader's time, and the AI engines — and the humans they are answering for — will start to notice.

Read the original on StackJot →. If you found this useful, the companion piece on AEO vs SEO in 2026 goes deeper into the strategic side.

Forem: Mehar Zain