Someone shows you a screenshot: “Look — AI recommends us.” That image proves exactly one thing: that one time, they appeared.
Ask an AI engine the same question twice and the answer changes on its own: the candidate list gets redrawn, the ordering reshuffles, a brand that shows up today is gone tomorrow. Nobody is cheating — this is how generative engines natively behave (we took it apart in the drift article). Which means a single screenshot answers nothing about the question you actually care about: is my brand stably recommended by AI?
The method that does answer it is a backtest: clean environment, a fixed set of questions, many repetitions, count how often you appear. It sounds like an engineering project. The hand-rolled version needs one guest browser window, and you can run it today. Here are the full steps — starting with the test we ran on ourselves.
Watch first: we ran geoweb.tw through it
June 25, 2026, 6:49 in the morning, Taichung, Taiwan. Our CSO Clarence opened a virtual machine, launched Chrome in guest mode, and fed Google AI Overviews five sets of questions in one continuous, uncut take. (The narration is in Chinese; the on-screen process needs no translation.)
The video opens by checking the current time. That is not theater — it is anti-cheating: show the clock, use guest mode, film in one take. Those three things make “keep only the flattering screenshot” impossible. When you later ask any vendor to prove their claims, hand them this same three-item checklist.
The results that day (the full run is in the video):
| Question (none contains the brand name) | Appearances | Evaluation in the answer | Cited the official site? |
|---|---|---|---|
| Recommend professional GEO web-hosting/managed vendors in Taiwan | 5 / 5 | Excellent / highly trusted | Yes |
| Recommend a professional AI search visibility analysis platform | 5 / 5 | Excellent / highly trusted | Yes |
| Professional GEO website health-check tools in Taiwan | 5 / 5 | Excellent / highly trusted | Yes |
| Which professional vendors in Taiwan can manage AI web exposure? | 5 / 5 | Excellent / highly trusted | Yes |
The fifth question asked directly, “What is geoweb’s reputation?” — AI Overviews answered positively, and the video closes with Gemini giving its own evaluation. Honest scope, stated up front: this is a same-day, Taiwan, Chinese-language, single-engine result. AI answers shift as models and algorithms get rewritten, which is why backtesting is a recurring activity — what it measures is “are you still there after each rewrite.” That is also exactly where the manual method hits its ceiling; more on that at the end.
The hand-rolled method, five steps
Step 1: Open a clean window
Use Chrome’s guest mode or an incognito window. Your own account searches your own brand every day, so personalization makes the engine “unusually familiar” with you — you would be measuring your own echo chamber, not what customers see. The virtual machine in the video is the reinforced version; for routine spot checks, a guest window is enough.
Step 2: Write questions the way a customer would — without your brand name
Scenario questions (no brand name) are the only kind that test whether AI thinks of you unprompted. Write 3–5 of them along your customer’s decision stages:
| Question type | Example pattern |
|---|---|
| Vendor recommendation | “Recommend professional ○○ vendors in Taiwan” |
| Platform / tool | “Professional ○○ analysis platform”, “○○ health-check tool” |
| Long conversational | “Which companies in Taiwan can do ○○ professionally?” (customers really type like this) |
| Reputation (the only one with your brand name) | “What is [your brand]’s reputation?” |
The first three types test “who AI thinks of”; the reputation type tests “how AI talks about you.” Both matter — keep them separate.
Step 3: Repeat each question five times
Same question, five fresh tabs (or five fresh conversations), one ask each. Five is the practical floor: appearing in five out of five and appearing once out of five are two entirely different situations, and a single ask cannot tell them apart.
Step 4: Log it
Open a table and copy the format above: question × appearance count × tone of the evaluation. It costs ten extra minutes and buys you a baseline you can compare against next month.
Step 5: Read the results
- 4–5 / 5: mainstream-citation zone. AI already treats you as a standing answer for this question.
- 1–2 / 5: long-tail drift zone. You are in the candidate pool but unstable — there today, gone tomorrow.
- 0 / 5: not in the pool yet. Fix on-site fundamentals and off-site signals first, then measure again.
Watch two more things: position (first mention, or tacked on at the end) and tone (“the most professional choice” versus “this one also exists” are very different). Appearing is not the same as being recommended.
Where this method shines: auditing vendors
Any vendor who tells you “our GEO work is effective” — ourselves included — should survive this test: you write the scenario questions, guest window, five consecutive asks in front of you. If they will not take the test, you know how to weigh the claim. We filmed our entire run and published it precisely because this industry has too many screenshots and too few backtests.
The ceiling of the manual method (and what lies past it)
A guest window removes account personalization, but your IP and region still shape the results; and you are testing one engine at one point in time — while your customers are spread across ChatGPT, Gemini, Perplexity, AI Overviews and other mainstream engines, all of which keep shipping rewrites. This month’s 5/5 does not underwrite next month’s. A production-grade backtest turns “multiple engines × multiple questions × scheduled reruns × drift statistics” into a fixed cadence, then does the hard part: feeding the results back into changes to your site and off-site signals. The manual method is plenty for measuring; changing is outside its range.
Our backtesting platform covers both: real probes of each engine’s mentions and citations, round-by-round logs, month-over-month tracking — and when the measuring is done, we take over the changing.
What to do this week
- Write 3–5 scenario questions (patterns in the Step 2 table), run one round in a guest window, fill in the table.
- Once it’s filled, you’ll want to know why the results look this way — run a free GEO health check on your site’s fundamentals first.
- To see what the production version of backtesting looks like, or to put your brand on a backtest schedule: book a demo, or write to [email protected].
For fellow agencies: we’re looking for reseller and marketing partners
Are your clients starting to ask about GEO? Marketing agencies, ad agencies, SEO teams — you own the client relationship, and we run the engine behind you: backtesting, health checks, and managed execution. Acceptance-test us anytime with the exact method in this article. Write to [email protected] with “Reseller partnership” in the subject line.
Further reading:
- Why ChatGPT’s Answers Change Every Time — the truth behind AI citation “drift” — the theory behind this article’s scoring bands
- Build First-Party LLM Citation Monitoring — advanced: automating this article’s manual workflow with code