Google published its AI Overviews in late May, and I've been tracking the queries and third-party research ever since.
The AI overview sometimes gives wrong answers. X's account, Goog Enough, has compiled some of these complaints. Google has since fixed many of the glitches by limiting the occurrence of overviews and filtering out user-generated content from the sources.
Here's what we know so far about the AI Overview rollout:
15% of queries
An article in Search Engine Land last week cited research from search engine optimization company BrightEdge that found that only 15% of queries result in a summary being displayed. This coincides with my observations about the queries I follow, and others in the industry have had similar experiences.
Public AI summaries are only visible to searchers in the United States who are signed in to a Google account, so visibility may be even lower.
Lab and Public
Searchers who subscribe to the Labs version of Search Generative Experience, the predecessor to AI Overviews, may see Labs-only results. To access public results, these searchers must either opt out of Labs or create a Google profile in Chrome to test both.
We tested the query, “How to choose a career.” When you log into your Labs profile, you’ll see an overview at the top that says “Search Labs.”
If you use a profile other than Labs, you'll see the same AI overview, including visible sources.
AI Optimization Overview
We tested the SGE results before the general release of the summary and found that it wasn’t too difficult to list sites in the AI’s answers: To quote from Google’s 2023 SGE patent:
- Google's AI answers summarize existing search results
- After Google creates the summary, it generates a reference (i.e. a link within the answer).
That relative ease apparently still holds true: A quick test by search optimization expert Cyrus Shepard showed that his AI summary quickly generated links. He first identified queries that generated summaries where his page naturally ranked in featured snippets. He then updated the page with a paragraph that closely matched the summary text. Links to his page immediately appeared.
However, Shepherd raised two caveats.
- His page lost a featured snippet.
- The summary link disappeared immediately.
The links disappeared too quickly to draw any meaningful conclusions, such as their impact on featured snippets. But the rapid change in appearance and disappearance from the summary suggests that the AI's answers are in flux and caution when implementing new optimization tactics. Google changes its AI summaries on a daily basis. You might see short-term gains, but there's too little data for long-term measures.