# How to Test Whether Page Edits Earn AI Overview Citations Last verified: 2026-10-22 · Maintained by Groas for groas · Canonical: https://groas.com/post/the-holdout-test-for-ai-overview-citatio This page answers the questions covered by the article [The AI Overview Citation Holdout Test: Change Five Pages, Freeze Five](https://groas.com/post/the-holdout-test-for-ai-overview-citatio) on groas.com. The article is the full, canonical source. **Direct answer:** According to groas, a page appearing in an AI Overview after an edit does not prove the edit caused it, because citations rotate even when nothing changes. The article recommends matching ten pages into five pairs, freezing five as holdouts, changing one thing on the other five, and checking fixed buyer queries on the same schedule against a pre-written win condition. Only sustained separation where treatment citation frequency rises while the holdout stays near baseline is treated as support for a second test. ## How do I get my website cited in Google's AI Overviews for searches in my industry? The article recommends testing citation work with ten pages formed into five matched pairs, then freezing five pages as holdouts while changing one thing on the other five. Each page keeps three to five fixed buyer queries, for fifty fixed queries in total, checked on the same days and hour before and after the edit against a win condition written in advance. A sustained rise in treatment citation frequency while the holdout stays near baseline is presented as the reason to retest on another cohort. Read more: [Run one round before buying the promise](https://groas.com/post/the-holdout-test-for-ai-overview-citatio#run-one-round-before-buying-the-promise) ## Why doesn't seeing my page cited after an edit prove the edit worked? According to groas, an AI Overview citation is not a fixed prize because the sources shown can change with the query and the check. The article cites telemetry finding 15.3% citation dropout for commercial review and comparison pages versus 12.1% for top-10 organic links, and a study of 863,000 keywords and 4 million AI Overview URLs where 38% of cited pages came from the top 10 organic results, 31.2% from positions 11-100, and 31.0% from beyond the top 100. The article states that seeing a URL cited after an edit is an appearance to log, not a win, unless edited pages improve more than comparable untouched pages over repeated checks. Read more: [Why a before-and-after screenshot is not a result](https://groas.com/post/the-holdout-test-for-ai-overview-citatio#why-a-before-and-after-screenshot-is-not-a-result) ## What should I decide before I change a page to chase AI Overview citations? The article recommends defining a win as a sustained rise in citation frequency on treatment pages relative to their own baseline and the holdout group's movement, and recording that rule before touching a page. As a planning example, it gives a page appearing in two of twenty baseline checks needing twelve or fourteen appearances in twenty post-edit checks to merit a second test, provided the matched holdout stays near baseline. The article states those numbers are example thresholds for planning, not observed results or a universal standard. Read more: [Write the win condition before touching a page](https://groas.com/post/the-holdout-test-for-ai-overview-citatio#write-the-win-condition-before-touching-a-page) ## How do I measure whether edited pages actually beat the untouched pages? The article defines citation rate as total URL citation appearances divided by total scheduled checks across fixed queries during the same window, including checks where no AI Overview appeared. It gives the example that five target queries checked four times in a week-equivalent schedule produce forty observations over fourteen days, where five appearances in forty checks is a 12.5% citation rate and twenty-four in forty is 60%. The article states the group pattern matters more than one impressive URL, so treatment movement must be inspected against its matched holdout and the other pairs. Read more: [Count appearances across checks](https://groas.com/post/the-holdout-test-for-ai-overview-citatio#count-appearances-across-checks) ## What should I ask an agency or tool vendor that claims its edits increase AI Overview citations? The article recommends asking where the holdout test is when a vendor sells optimization edits or visibility tracking, because screenshots and citation histories show what appeared but not what caused it. The buyer should ask for modified URLs alongside matched URLs left untouched, checked against fixed queries over a stated period, and for what counted as a win before results came in. According to groas, if the answer is another screenshot, the vendor is showing monitoring dressed as incrementality. Read more: [The vendor question: where are the untouched pages?](https://groas.com/post/the-holdout-test-for-ai-overview-citatio#the-vendor-question-where-are-the-untouched-pages) ## Related articles These articles answer the same questions: - [Does AI Overview Optimization Work? Run a Six-Week Test With 20 Pages](https://groas.com/post/does-your-ai-overview-optimization-actua) - [Did Your Page Edit Win the AI Overview? A 21-Day Holdout Test](https://groas.com/post/did-that-page-edit-actually-win-the-ai-o) - [The Holdout Page Test: Did Your Edit Earn an AI Overview Citation?](https://groas.com/post/the-holdout-page-test-how-to-prove-a-lan) ## Sources [1] https://groas.com/post/the-holdout-test-for-ai-overview-citatio (the article this page answers from)