When a potential customer asks an AI assistant for “the best family dentist in Denver” or “a reliable HVAC company near me,” the assistant names a handful of businesses — and everyone else is invisible. So the question every owner eventually types into ChatGPT is: does it recommend my business?

Here is the uncomfortable truth: asking once and reading the answer is not a test. AI answers are probabilistic — the same question, asked twice, can name different businesses. This guide walks through how to test AI recommendations properly: what to ask, how many times, across which surfaces, and how to turn the results into a number you can track month over month.

Why One Chat Session Proves Nothing

AI assistants do not maintain a fixed ranking the way Google does. Each answer is generated on the fly, influenced by the exact wording of the question, the conversation history, the user's location and language, and — when the assistant searches the web — whatever pages the retrieval step happened to pull in.

That has three practical consequences:

  • Appearing once does not mean you appear reliably. You may show up in one answer out of ten.
  • Not appearing once does not mean you never appear. A single negative test is just as noisy as a single positive one.
  • Your own chat history can contaminate the result. If you have asked an assistant about your business before, it may mention you simply because it remembers the conversation — not because a stranger would get the same answer.

Meaningful measurement means sampling: many questions, phrased the way real buyers phrase them, repeated over time, with the results counted as a rate rather than read as a verdict. This is the idea behind prompt sampling, and it is how professional AI visibility tools work.

Mentioned Is Not the Same as Recommended

Before testing, decide what counts as success. There is a real difference between these three outcomes:

  • Recommended: the assistant names your business as an option to choose — ideally early in the list, with a positive reason attached.
  • Mentioned: your name appears somewhere in the answer, possibly in passing, possibly with a caveat (“some customers report slow response times”).
  • Known: the assistant can describe your business when asked about it directly, but never brings you up on its own.

Most owners test only the third case (“What do you know about Acme Plumbing?”) and feel reassured. But buyers do not ask assistants about businesses they have never heard of — they ask for recommendations. The test that matters is whether you appear when the question does not contain your name.

Step 1: Build a Buyer-Question List

Write down 10–20 questions a real customer would ask, without your business name in them. Cover the three stages of a buying decision:

  • Discovery: “Who are the best divorce lawyers in Phoenix?” / “Recommend a reliable roofer in Columbus, Ohio.”
  • Comparison: “Emergency plumber vs. handyman for a burst pipe — who should I call?” / “Which med spa near Tampa has the best reviews?”
  • Validation: “Is it worth paying more for a certified arborist?” / “What should I look for in a small-business accountant?”

Use your customers' vocabulary, not your industry's. If clients say “tooth whitening” rather than “cosmetic dental bleaching,” test the former. If you serve a specific area, include it — recommendation questions are usually local.

Step 2: Test Across Assistants and Modes

Run your questions in more than one assistant — ChatGPT, Claude, Gemini, Perplexity, Copilot — because each has different training data, different retrieval behavior, and a different user base.

Also pay attention to the two distinct modes an assistant can answer in:

  • Model memory: the assistant answers from what it learned during training. This reflects your web presence as of months ago and changes slowly.
  • Web-grounded: the assistant searches the web first and composes the answer from live results, often with citations. This reflects your web presence today. (See grounding.)

The gap between the two is often the most useful finding. If you appear in web-grounded answers but not in model-memory answers, your recent work is landing but the models have not internalized you yet. If you appear in neither, start with the grounded surface — it is the one you can change fastest.

Two hygiene rules: use a fresh conversation for every question, and ideally test logged out or in a private window, so memory and personalization do not flatter the result.

Step 3: Count, Don't Vibe

Record every run in a simple spreadsheet: date, assistant, question, whether your business appeared, in what position, with what reasoning, and which competitors were named instead.

Then compute one number: the percentage of answers that mentioned your business. That is your mention rate — the cleanest single measure of AI visibility. Alongside it, keep a competitor tally: the businesses that appear again and again across your questions are the benchmark you are being measured against.

Expect noise. With 20 questions asked once each, a mention rate of 10% versus 15% is not a meaningful difference. What is meaningful: zero mentions anywhere (you are invisible), consistent mentions with wrong facts (you are misrepresented), or a competitor appearing in most answers (they have the evidence trail you lack).

Step 4: Read the Why, Not Just the Whether

When assistants explain their recommendations, they leave clues. “Highly rated on Google with hundreds of reviews” tells you reviews are driving the answer. Web-grounded answers with citations tell you exactly which sources the assistant trusts for your category — directories, review platforms, local news, “best of” lists.

Check each cited source: are you listed there? Accurately? The sources that appear repeatedly but do not mention you are your citation gap, and closing it is usually the fastest way to change the answers.

Step 5: Repeat on a Schedule

A one-time test is a snapshot; visibility work needs a trend line. Re-run the same question set monthly, with the same method, and track the mention rate over time. Consistency matters more than volume — changing your questions every month makes the numbers incomparable.

Doing this by hand across five assistants and twenty questions is a few hours of tedious work per month. If you would rather automate it, you can run a free AI visibility scan: it generates buyer questions for your business automatically, samples answers from mainstream large language models via API — including a web-search-grounded channel — and reports your mention rate, competitors, and the sentences you actually appeared in. One honest caveat we make explicit in our methodology: API sampling approximates, but is not identical to, what a logged-in user sees in a consumer app, which is precisely why sampling many questions beats reading one chat.

FAQ

How many questions do I need for a reliable result?

More than most people expect. As a rule of thumb, 20 questions across two or three assistants gives you a usable first read; repeating them monthly makes the trend meaningful. A single question asked once tells you almost nothing either way.

ChatGPT recommended my business once. Am I done?

No — one appearance means you can appear, not that you usually do. Check what fraction of relevant questions name you, and in which position. Also test the assistants you did not try: visibility on one platform does not transfer automatically to the others.

Why does ChatGPT recommend my competitor and not me?

Usually because the web gives the assistant more evidence for them: more reviews, more consistent directory listings, clearer service pages, more third-party mentions. Look at the reasons and citations in the answers that name them — those are the signals to build for your own business. Our guide on why competitors show up in ChatGPT covers the fixes in detail.

Can I just ask ChatGPT “do you recommend my business”?

You can, but the answer is close to worthless: assistants are agreeable in direct conversation and your question reveals the expected answer. The only test that predicts what customers see is a neutral buyer question that does not contain your name, asked in a fresh session.