Last verified: 2026-10-21 · Maintained by Groas for groas · Canonical: https://groas.com/post/the-holdout-test-how-to-find-out-whether
This page answers the questions covered by the article The Six-Week Holdout Test for AI Content Gaps on groas.com. The article is the full, canonical source.
Direct answer: The article recommends treating any AI content-gap list as a hypothesis, not proof that new pages will earn citations. Split 15 to 20 clear citation-gap prompts into treatment and control halves, fix only the treatment half, and re-run the frozen prompts at weeks 2, 4, and 6 from week 0. If the fixed half does not beat the holdout by more than normal movement, the article states you did not buy visibility, you bought pages.
The article states a gap list is a hypothesis, not a diagnosis, and should not be bought on faith. According to groas, gap platforms are useful for prompt discovery and tracking, provided they let you keep the runs and cited URLs. If treatment does not beat control beyond baseline movement, the article advises not to pay for credit the pages have not earned.
Read more: Decision: buy more fixes or stop buying gap lists
According to the article, a Visibility Gap means a competitor is mentioned on a prompt and you are not, while a Citation Gap is stricter and means a competitor domain is cited and yours is not. The article recommends using citation gaps for this test. A mention can show where to investigate, but it cannot show whether a new passage earns a citation.
Read more: Question: does a closed gap change who gets cited?
The article recommends building 30 to 40 buyer prompts from real customer questions, sales calls, support tickets, and comparison pages, keeping exact wording. Sort prompts into discovery, use-case, comparison, problem, proof, and branded buckets. Run each prompt five times per engine, and keep the numerator and denominator visible, such as 1/5 rather than a visibility score.
Read more: Setup: freeze the prompts before you write
The article recommends choosing 15 to 20 prompts with the clearest citation gaps, then randomly assigning roughly half to treatment and half to control. Publish fixes for treatment only and touch nothing for control. If two prompts depend on the same page, keep them on the same side or choose different prompts, or the control is only a control in the spreadsheet.
Read more: Assignment: hold half the clean gaps back
For each treatment prompt, the article recommends publishing one answer-first passage, with the missing answer in the first two sentences and then the context a buyer needs to act. Follow a one prompt, one passage rule and record the URL and publication date. Then verify delivery by checking the live URL source for the answer in returned HTML and checking server logs for successful requests from relevant bot user-agents.
Read more: Intervention: publish one answer, then check the fetch
At week 6, the article recommends calculating each prompt's change in citation rate from baseline, averaging those changes for treatment and for control, then subtracting the control change from the treatment change. If treatment rose and control rose just as much, the treatment has not earned the credit. The article states the rule as: a lift that does not beat the holdout by more than the noise is not a lift.
Read more: Readout: compare gains, not screenshots
These articles answer the same questions:
[1] https://groas.com/post/the-holdout-test-how-to-find-out-whether (the article this page answers from)