Why the Same AI Question Can Return Different Sources

Same question does not establish the same search conditions

Interpretation: different sources in two AI answers are not, by themselves, evidence of a ranking change. Compare the question and its context, the retrieval process where disclosed, and the resulting answer separately. Matching the latest question is only the start of a useful comparison.

OpenAI says ChatGPT search typically rewrites a query into one or more targeted queries when working with search providers, and may send additional queries after reviewing initial results. ChatGPT consumer documentation describes this behavior; it is not a universal specification for other engines.

Before comparing answers, save the surface, timestamp, exact question, conversation context, available model and search settings, disclosed location and memory conditions, and the complete output. Record unavailable information as unknown, not as a presumed default.

Source-check date: 10 September 2026, using the supplied source packet. This is a documentation-based explainer with a hypothetical example, not an account test or API experiment.

Separate three stages

1. Question and context: keep the exact user wording separate from earlier messages, supplied files or instructions, and disclosed personalization conditions. Do not treat a short final question as the complete input record.

2. Retrieval queries: preserve any search queries actually exposed by the product or API, with their provenance. Keep them separate from the original question and from your own reconstruction of what might have been searched.

3. Answer and sources: save the answer text, citation links and any separately exposed source list. Compare each category with the same category in another observation.

What the providers actually document

ChatGPT may use approximate IP-based location for local results, while device location sharing is optional and off by default. OpenAI location documentation.

If memory is enabled, ChatGPT may use relevant saved memories when rewriting a search query. OpenAI memory and search documentation.

Google describes query fan-out in its Search generative AI documentation as concurrent, related queries generated by the model to fetch additional relevant search results. Google Search documentation.

Google describes grounding as retrieving relevant pages from its Search index and reviewing information from those pages to generate a response with supporting links. Google's grounding explanation.

Interpretation: these mechanisms explain why the user's words need not be the entire retrieval specification. They do not establish which mechanism caused a particular source difference, or promise deterministic source selection when visible settings match. Do not transfer Google's fan-out description to ChatGPT, or ChatGPT's memory behavior to Google Search or Gemini Apps.

Create a comparison record before interpreting changes

Use one record per observation. The following is an editorial recording protocol, not a list of controls every product exposes.

Keep unknown, not exposed, explicitly disabled and not applicable distinct. In particular, do not use an unavailable memory setting as evidence that memory was off.

Hypothetical example: an emergency plumber question

This example is invented to illustrate comparison design; the queries and source sets below were not observed. Assume two ChatGPT consumer runs with the same final question: “Which emergency plumbers near me should I contact?”

Interpretation: the final questions match, but the location inputs do not. Different retrieval queries and source sets in this scenario do not establish that a plumber's page rose or fell in ranking. Nor would the resulting city names prove the actual hidden queries.

For a follow-up comparison, explicitly name the same city in both questions, use the same conversation setup and surface, and record the remaining conditions. Retain this original pair as a context-mismatch example rather than silently rewriting its record.

API controls are not a consumer-interface replay

In OpenAI's Responses API, enabling web search does not require the model to use it on every request. OpenAI API documentation.

The API guide distinguishes Chat Completions search models, which always search before responding, from Responses search, which is a tool. Integration differences.

For Responses API web search, OpenAI documents domain filtering and control over live versus cached/indexed access. Search controls.

The API's search_context_size controls how much search-result context is available to the model, not a guaranteed number of sources or citations. Search context size.

OpenAI says API search actions usually, but not always, include the queries searched. Output and citations.

Interpretation: retain the endpoint, model, tool type, tool-choice setting, search configuration and returned trace when evaluating an API-based monitor. Treat those observations as API observations, not as an exact reproduction of consumer ChatGPT. Do not classify a response as a searched answer solely because the request enabled a search tool.

Compare like-scoped observations, then assess the vendor's claim

1. Define the unit before counting. For this protocol, a mention is a name in answer text; a citation is an attributed source link; referral traffic requires separate evidence of a visit. Keep these categories separate, and label API responses and consumer-interface observations separately.

2. Define “source set.” Decide whether you mean answer citations, a consumer source panel or an API retrieval list. Do not merge them under one unlabeled source count.

3. Check scope before interpreting differences. Separate changed questions, cities, context, surfaces and configurations. For repeats intended to measure change over time, record the time difference while holding other known conditions as consistent as possible.

4. Preserve the denominator. Ask a monitoring vendor for the question set, schedule, repeat count, inclusion rules, errors, non-search responses and treatment of retries—not just its selected screenshots.

5. Report the observation before the explanation. Prefer “this URL was cited in these saved responses under these recorded conditions” to “the engine now prefers this domain.” Mark possible explanations as hypotheses.

In consumer ChatGPT, the Sources view can include cited sources and other relevant links. Consumer source display.

OpenAI's API documentation distinguishes the sources field, containing the complete list of URLs consulted, from inline citations showing the most relevant references. API sources field.

Editorial limits: do not infer a private user prompt from a grounding phrase, extrapolate universal market share from a small repeated sample, or treat source-list order as a measured ranking without supporting evidence. A mention is not a citation, and neither is evidence of referral traffic under this protocol.

OpenAI says ChatGPT search placement is not guaranteed. Website inclusion documentation.

Accordingly, do not treat crawl permission as a promise of inclusion in a particular answer. This guide is about interpreting observations, not prescribing how to get cited.

What the record cannot settle

OpenAI warns that ChatGPT search results and citations can be incomplete, outdated or incorrect. Source-review guidance.

A useful comparison record is not a guarantee of answer accuracy or a complete account of internal retrieval. Review cited passages separately when factual support matters. When visible conditions match but the sources differ, report a source-set difference with an unresolved cause unless additional evidence supports an explanation.

Maintenance

Review this canonical guide monthly, with the next conceptual review due 10 October 2026. Review sooner when the linked documentation changes query handling, personalization, search controls or source reporting. Update affected explanations and the comparison template; do not create a daily page for answer variability. This first source check is a baseline, not evidence of a newly announced product change, and a new retrieval date alone is not a substantive correction.

Sources

• OpenAI: Searching the web with ChatGPT — consumer search, query rewriting, location, memory and source display.
• Google: Optimizing your website for generative AI features on Google Search — grounding and query fan-out in Google Search.
• OpenAI API: Web search — API integrations, search controls, tool behavior and output fields.