Your AI Visibility Score Is a Probability

Most AI visibility tools report some version of the same number: how often ChatGPT, Gemini or Google’s AI answers mention a brand, and where. Dashboards show it as a rank, a share of voice or a score out of 100. Buyers compare tools on it, and agencies report it to clients every month.

Two studies published in the past year suggest that number is often read wrong, because the answers it is built on change from one run to the next. That matters to everyone who relies on it, from enterprise trackers to services that sell the work of getting a brand mentioned. rankcited.com, for example, says it starts every engagement with a measurement report before any placement work begins. If the measurement moves, everything built on it moves too.

Same question, different answer, almost every time

In January 2026, Rand Fishkin of SparkToro and Patrick O’Donnell of Gumshoe published the results of a simple experiment. About 600 volunteers ran 12 identical prompts through ChatGPT, Claude and Google’s AI, 2,961 times in total, in November and December 2025. The prompts were ordinary buying questions: chef’s knives, headphones, digital marketing consultants, cancer care hospitals, cloud computing providers.

The result was blunt. Asked the same question 100 times, ChatGPT and Google’s AI had less than a 1 in 100 chance of giving the same list of brands in any two answers. The same list in the same order came up closer to once in a thousand runs. Claude repeated itself only slightly more often.

Fishkin did not conclude that tracking is pointless. Position inside a single answer is close to random, he wrote, but the share of answers a brand appears in holds up much better. When 142 volunteers each wrote their own prompt about headphones, almost no two prompts were alike, yet Bose, Sony, Sennheiser and Apple still appeared in 55% to 77% of the 994 answers. How often a brand shows up is a usable signal. Whether it was listed second or fourth in one answer tells you very little.

Same company, different engines, different sources

The second study came from Ahrefs in December 2025. It compared Google’s AI Mode with its AI Overviews, two products from the same company, across 540,000 query pairs from September 2025.

Only 13.7% of the cited URLs overlapped. Looking only at the top three citations raised that to 16.3%. The answers themselves were close in meaning, with an average semantic similarity of 86%, so the two engines said roughly the same things while pointing to different evidence. Wikipedia appeared in 28.9% of AI Mode citations against 18.1% in AI Overviews, and AI Mode cited Quora about 3.5 times as often. In AI Overviews, YouTube held the top position, cited more often than encyclopedic sources.

So a brand can be well covered by the sources one Google product prefers and nearly absent from the other’s. Add ChatGPT, Perplexity, Gemini, Claude and Copilot, and a single “AI ranking” stops making much sense.

What to check when comparing tools

None of this makes AI visibility tools useless. It changes what to look for in them, before signing up or before trusting a monthly report. Our guide on how to choose an AI visibility tool covers the wider checklist.

Sample size. How many times does the tool run each prompt, and how many prompts does it track? One run per prompt per week gives a number that moves for reasons unrelated to your marketing. Fishkin suggests running a prompt 60 to 100 times and averaging the results.

Frequency over position. Does the headline metric show how often you appear across runs, or where you placed in one answer? The first holds up across runs. The second mostly does not.

Engine coverage. Results from one engine say little about the others. A tool that reports each engine separately gives a truer picture than a blended score. Our census of the engines each tool tracks shows which tools cover which.

Source data. Does the tool record which pages each answer cited? That is the part you can act on, because those pages are where a brand can earn a mention.

Location and settings. Answers change by country, language and whether the user is logged in. Ask how the tool controls for that.

Trend handling. A useful report shows ranges over several weeks. A single week-on-week jump can be noise rather than change.

Prompt sets change the result

Two tools can also disagree about the same brand because they ask different questions. A tracker that monitors 25 broad prompts (“best CRM software”) will tell a different story from one tracking 200 narrow ones (“CRM for a three-person real estate team in Texas”). Each measures a different slice of demand.

Narrow prompts are usually closer to how buyers talk to AI assistants: longer, more specific and full of context. How many answers a question allows matters too. In the SparkToro study, answers about cloud computing providers, where there are few options, were far more consistent than answers about science fiction novels, where there are many. When choosing a tool, look at the prompt set it builds and ask whether real customers would phrase things that way.

Reading a report

Treat any AI visibility figure the way a pollster treats a single poll. One reading is a data point. The trend over a month or two, across several engines and a fair number of prompts, is the finding.

It also helps to separate two questions: whether you are mentioned more often, and why. The first comes from how often you appear. The second comes from the citations: which trade publications, review sites, comparison pages and forum threads the engines rely on for your category, and whether you are in them. A report that answers only the first gives you a score. One that answers both gives you something to act on.

The two studies point to a simple change in how to read the numbers. Instead of asking where you rank in ChatGPT, ask how often you show up, in which engines, and on the strength of whose pages. That question has an answer, and it changes slowly enough to act on.