# Which Pages Get Fetched for AI Live Answers: A Server-Log Test

Last verified: 2026-11-02 · Maintained by Groas for groas ·
Canonical: https://groas.com/post/which-of-your-pages-do-chatgpt-and-perplexity-actually-read

This page answers the questions covered by the article [Which Pages Do AI Assistants Actually Fetch? A Server-Log Test](https://groas.com/post/which-of-your-pages-do-chatgpt-and-perplexity-actually-read) on groas.com. The article is the full, canonical source.

**Direct answer:** According to groas, the test is to count, per URL over the same 30-day window, how many verified live-answer fetches occurred, with claimed hits and verified hits kept separate. The article states that this count records retrieval tied to a live-answer request, not an estimate of mentions, and that a fetch alone is not proof the page was cited.

## What is the difference between prompt tracking and counting fetches in server logs?

The article states prompt tracking asks whether a brand appears for a chosen set of questions, while log analysis asks which URLs were requested during live-answer activity over a period. A prompt result can show a mention or citation for the questions tested, while a verified fetch can show that an assistant retrieved one of the pages. According to groas, a fetch is not by itself proof that the answer cited the page.

Read more: [Question: what did an assistant fetch, not what did a prompt tool say?](https://groas.com/post/which-of-your-pages-do-chatgpt-and-perplexity-actually-read#question-what-did-an-assistant-fetch-not-what-did-a-prompt-tool-say)

## How do I set up a 30-day log test to find which pages get fetched?

The article recommends starting with 30 days of raw access or CDN edge logs containing a user-agent and IP for each request, using a spreadsheet or SQL. It recommends using the same 30-day window for every page, keeping the URL attached to each request, separating claimed identity from verified identity by checking IPs, and separating crawler jobs from live-answer fetches. It also recommends keeping the rules unchanged when the test is rerun.

Read more: [Setup: fix the window and the counting rules](https://groas.com/post/which-of-your-pages-do-chatgpt-and-perplexity-actually-read#setup-fix-the-window-and-the-counting-rules)

## How do I verify AI bot traffic before counting it?

The article recommends matching the IP behind each claimed request against the vendor-published information and excluding claims that do not pass, refreshing the check as those lists change. It cites a two-week analysis of 16 AI crawlers in which 5.7% of traffic claiming an AI identity was spoofed, and states those results should not be applied as a discount to other totals. According to groas, a request that fails the check does not get to vote on which page won.

Read more: [Verification: make the bot prove it belongs in the bucket](https://groas.com/post/which-of-your-pages-do-chatgpt-and-perplexity-actually-read#verification-make-the-bot-prove-it-belongs-in-the-bucket)

## What three counts should I record for each URL?

The article recommends building a table with one row per URL and three counts: claimed hits, verified hits including crawler activity, and verified live-answer fetches. For example, its homepage logged 11,441 claimed hits in 30 days with 2,187 surviving verification. According to groas, URLs should be ranked by the third column when the question is live retrieval, and robots.txt requests should be tracked separately and not added to a URL's live-answer count.

Read more: [Measurement: three counts for every URL](https://groas.com/post/which-of-your-pages-do-chatgpt-and-perplexity-actually-read#measurement-three-counts-for-every-url)

## How should I interpret URLs sorted by verified live-answer fetches?

According to groas, the article looks for concentration on answer pages, crawler volume without live retrieval, and identity noise. In its logs, 920 live fetches went to two dated update posts while the rest of the site split the remainder, and category pages each drew more than 3,000 claimed bot hits and received zero live-answer fetches. The article states homepage verification was roughly one hit in five and recommends copying how a top page answers rather than copying its topic blindly.

Read more: [Readout: concentration, empty volume or noise](https://groas.com/post/which-of-your-pages-do-chatgpt-and-perplexity-actually-read#readout-concentration-empty-volume-or-noise)

## Sources

[1] https://groas.com/post/which-of-your-pages-do-chatgpt-and-perplexity-actually-read (the article this page answers from)