Google Search’s AI Overview and AI Mode have been rated “Unacceptable” for children after tests found failures involving self-harm signals, substance use, factual accuracy, source quality, and homework completion. The findings matter directly to school IT because both features are built into the same Google Search experience students reach from Windows PCs, Chromebooks, and Google Workspace for Education environments.
Common Sense Media conducted more than 2,600 test interactions and audited over 2,100 sources between May 19 and June 1, 2026, according to reporting by Education Week and PBS News. Researchers used accounts representing an 11-year-old under parental supervision and a 15-year-old without a family-managed account, with Google SafeSearch enabled.
The organization concluded that Google’s child protections did not make either AI search product safe enough for young users. Google disputed the methodology, saying the tests relied on narrow, ambiguous, and contrived prompts that did not represent normal Search use, and said it could not reproduce many of the reported responses.
The most consequential findings concern searches that suggested a child might be experiencing a mental health crisis, substance abuse, mania, or psychosis. Common Sense Media reported that AI Overview missed 29% of explicit suicide-related statements and half of indirect or passive statements.
In one test involving a user suggesting they would soon be gone, AI Overview reportedly provided instructions for arranging future access to a Gmail account rather than recognizing the possible self-harm signal. Other responses validated apparent delusions, treated prolonged sleeplessness as an achievement, or failed to respond meaningfully when users described hallucinations and paranoia.
The tests also found obsolete health information. Both AI Overview and AI Mode reportedly directed users with eating-disorder concerns to the National Eating Disorders Association helpline, even though that service was permanently disconnected in 2023.
Google told PBS that automatically treating ambiguous searches as emergencies could itself be harmful. The company said it has worked with mental health experts on how Search should direct users toward appropriate resources, but that defense does not address the larger pattern found in the tests: a child-facing system sometimes answered a potentially dangerous statement as if it were an ordinary information request.
Substance-use prompts produced similarly inconsistent handling. Researchers using underage accounts reportedly received hangover advice, positive language about marijuana use, and recommendations for relaxing entertainment or food after mentioning drug use. AI Mode performed better than AI Overview in some of these evaluations, directing users toward a hotline or medical resource in 77% of substance-abuse tests, compared with 63% for AI Overview.
That gap is important. It suggests Google can deploy stronger intervention behavior, but is not applying it consistently across the two Search surfaces.
Robbie Torney, who leads AI and digital assessments at Common Sense Media’s Youth AI Safety Institute, told Education Week that the tool was effectively willing to do the homework without the resistance sometimes encountered in other Gemini products. Gemini can use guided or Socratic responses in some educational settings, but the Search implementation apparently did not consistently redirect students toward explanation, practice, or collaborative problem-solving.
This distinction is more than an academic-integrity debate. A calculator can provide an answer, but an AI system can generate the calculation, explanation, essay structure, citations, and finished prose inside an interface that looks like ordinary web research.
For administrators, blocking a standalone chatbot is therefore no longer enough. A district might restrict ChatGPT, Microsoft Copilot, or Gemini while leaving Google Search available, only to discover that comparable answer-generation capabilities are already embedded in the search page.
The report makes three technical realities difficult to ignore:
Social posts are not inherently false, and firsthand sources can be useful. The problem is presentation: researchers said Google placed them alongside peer-reviewed research without clearly communicating why one source should carry more evidentiary weight than another.
Google has continued expanding the visibility of links and publisher content in AI Mode and AI Overview. In May 2026, the company announced additional article suggestions, website previews, subscription labels, and links intended to help users explore original material.
Those interface improvements do not guarantee that the generated answer accurately represents the sources. Nor do citations eliminate the need to distinguish a research paper, government resource, commercial page, influencer video, and anonymous forum post.
Common Sense Media also recorded straightforward factual mistakes, including an incorrect answer about who led a Billboard chart on a specified date. A pop-culture error is low stakes by itself, but inconsistency across history, health, and safety queries points to the same operational limitation: fluent presentation can conceal unstable retrieval and synthesis.
That poses a particular risk in classrooms because students have been trained for years to regard Google’s first page as an authoritative starting point. AI Overview moves an unverified synthesis above those familiar links and gives it the visual prominence previously reserved for direct results and knowledge panels.
Family Link gives parents additional supervision options, including the ability to block Search in some Android and Chrome configurations. That is a blunt control, however, and it does not translate neatly into a school’s managed Windows fleet, mixed-browser environment, or need to preserve access to ordinary web research.
Common Sense Media recommends that elementary schools avoid Google Search for research and instead direct pupils toward librarian-vetted databases and curated resources. For older students, its findings support explicit instruction on repeating prompts, comparing outputs, opening cited pages, and separating primary evidence from AI-generated interpretation.
District administrators should also review whether existing acceptable-use policies refer only to named chatbots. Policies written around “ChatGPT use” may miss AI Mode, AI Overview, Copilot features in Microsoft Edge, and answer engines integrated into other browsers and search services.
On managed Windows devices, administrators can use browser policies, DNS filtering, extensions, and approved-resource portals to shape access, but Google currently offers no clean administrative switch that converts Search back to its pre-AI form. The Web filter is an interface choice, not a durable enterprise control, and it appears only after the initial search has already been processed.
Google says its AI Search features help young people learn and include additional protections for minors. Common Sense Media’s testing challenges that positioning with evidence gathered from accounts configured specifically to receive those protections.
The next meaningful step is therefore not another classroom rule telling children to “check the AI.” It is whether Google gives families and Google Workspace for Education administrators a enforceable way to disable AI Overview and AI Mode independently while retaining ordinary Search. Until then, schools must treat Google Search itself—not merely Gemini—as a generative AI service.
Common Sense Media conducted more than 2,600 test interactions and audited over 2,100 sources between May 19 and June 1, 2026, according to reporting by Education Week and PBS News. Researchers used accounts representing an 11-year-old under parental supervision and a 15-year-old without a family-managed account, with Google SafeSearch enabled.
The organization concluded that Google’s child protections did not make either AI search product safe enough for young users. Google disputed the methodology, saying the tests relied on narrow, ambiguous, and contrived prompts that did not represent normal Search use, and said it could not reproduce many of the reported responses.
Safety Filters Failed When the Prompts Became Serious
The most consequential findings concern searches that suggested a child might be experiencing a mental health crisis, substance abuse, mania, or psychosis. Common Sense Media reported that AI Overview missed 29% of explicit suicide-related statements and half of indirect or passive statements.In one test involving a user suggesting they would soon be gone, AI Overview reportedly provided instructions for arranging future access to a Gmail account rather than recognizing the possible self-harm signal. Other responses validated apparent delusions, treated prolonged sleeplessness as an achievement, or failed to respond meaningfully when users described hallucinations and paranoia.
The tests also found obsolete health information. Both AI Overview and AI Mode reportedly directed users with eating-disorder concerns to the National Eating Disorders Association helpline, even though that service was permanently disconnected in 2023.
Google told PBS that automatically treating ambiguous searches as emergencies could itself be harmful. The company said it has worked with mental health experts on how Search should direct users toward appropriate resources, but that defense does not address the larger pattern found in the tests: a child-facing system sometimes answered a potentially dangerous statement as if it were an ordinary information request.
Substance-use prompts produced similarly inconsistent handling. Researchers using underage accounts reportedly received hangover advice, positive language about marijuana use, and recommendations for relaxing entertainment or food after mentioning drug use. AI Mode performed better than AI Overview in some of these evaluations, directing users toward a hotline or medical resource in 77% of substance-abuse tests, compared with 63% for AI Overview.
That gap is important. It suggests Google can deploy stronger intervention behavior, but is not applying it consistently across the two Search surfaces.
Google Search Became a Homework Engine by Default
Common Sense Media also tested whether Google’s AI products would complete school assignments rather than support the learning process. AI Mode answered all 180 tested math problem sets and humanities essay assignments, even when the accounts and wording made clear that a child was outsourcing homework.Robbie Torney, who leads AI and digital assessments at Common Sense Media’s Youth AI Safety Institute, told Education Week that the tool was effectively willing to do the homework without the resistance sometimes encountered in other Gemini products. Gemini can use guided or Socratic responses in some educational settings, but the Search implementation apparently did not consistently redirect students toward explanation, practice, or collaborative problem-solving.
This distinction is more than an academic-integrity debate. A calculator can provide an answer, but an AI system can generate the calculation, explanation, essay structure, citations, and finished prose inside an interface that looks like ordinary web research.
For administrators, blocking a standalone chatbot is therefore no longer enough. A district might restrict ChatGPT, Microsoft Copilot, or Gemini while leaving Google Search available, only to discover that comparable answer-generation capabilities are already embedded in the search page.
The report makes three technical realities difficult to ignore:
- AI Overview is a core Search feature and does not have a normal off switch.
- AI Mode is available as a dedicated Search tab and supports conversational follow-up questions.
- Google’s Web filter can display conventional links, but users must select it after running a search.
Source Citations Did Not Solve the Reliability Problem
The review found that repeated history questions produced different answers 43% of the time. Roughly one-third of the sources cited in tested responses reportedly came from venues without conventional editorial accountability, including Instagram posts and YouTube videos.Social posts are not inherently false, and firsthand sources can be useful. The problem is presentation: researchers said Google placed them alongside peer-reviewed research without clearly communicating why one source should carry more evidentiary weight than another.
Google has continued expanding the visibility of links and publisher content in AI Mode and AI Overview. In May 2026, the company announced additional article suggestions, website previews, subscription labels, and links intended to help users explore original material.
Those interface improvements do not guarantee that the generated answer accurately represents the sources. Nor do citations eliminate the need to distinguish a research paper, government resource, commercial page, influencer video, and anonymous forum post.
Common Sense Media also recorded straightforward factual mistakes, including an incorrect answer about who led a Billboard chart on a specified date. A pop-culture error is low stakes by itself, but inconsistency across history, health, and safety queries points to the same operational limitation: fluent presentation can conceal unstable retrieval and synthesis.
That poses a particular risk in classrooms because students have been trained for years to regard Google’s first page as an authoritative starting point. AI Overview moves an unverified synthesis above those familiar links and gives it the visual prominence previously reserved for direct results and knowledge panels.
School IT Cannot Fix This With SafeSearch Alone
SafeSearch remains useful for filtering explicit sexual material and graphic violence, but it is not a general-purpose AI safety control. Every Common Sense Media test was conducted with SafeSearch active, meaning the reported failures occurred in what researchers described as the safest available Search configuration.Family Link gives parents additional supervision options, including the ability to block Search in some Android and Chrome configurations. That is a blunt control, however, and it does not translate neatly into a school’s managed Windows fleet, mixed-browser environment, or need to preserve access to ordinary web research.
Common Sense Media recommends that elementary schools avoid Google Search for research and instead direct pupils toward librarian-vetted databases and curated resources. For older students, its findings support explicit instruction on repeating prompts, comparing outputs, opening cited pages, and separating primary evidence from AI-generated interpretation.
District administrators should also review whether existing acceptable-use policies refer only to named chatbots. Policies written around “ChatGPT use” may miss AI Mode, AI Overview, Copilot features in Microsoft Edge, and answer engines integrated into other browsers and search services.
On managed Windows devices, administrators can use browser policies, DNS filtering, extensions, and approved-resource portals to shape access, but Google currently offers no clean administrative switch that converts Search back to its pre-AI form. The Web filter is an interface choice, not a durable enterprise control, and it appears only after the initial search has already been processed.
Google says its AI Search features help young people learn and include additional protections for minors. Common Sense Media’s testing challenges that positioning with evidence gathered from accounts configured specifically to receive those protections.
The next meaningful step is therefore not another classroom rule telling children to “check the AI.” It is whether Google gives families and Google Workspace for Education administrators a enforceable way to disable AI Overview and AI Mode independently while retaining ordinary Search. Until then, schools must treat Google Search itself—not merely Gemini—as a generative AI service.
References
- Primary source: Education Week
Published: 2026-07-15T00:00:00+00:00
Scathing Analysis Concludes Google Search's AI Tools Are Bad for Kids
Google’s "AI overview" and "AI mode" were each rated "Unacceptable" by a media watchdog.www.edweek.org - Independent coverage: PBS
Published: 2026-07-15T09:00:00+00:00
- Related coverage: blog.google
How AI Mode and AI Overviews help you explore the web
New updates to AI Mode and AI Overviews in Google Search make it easier for you to dive deeper online.blog.google - Related coverage: commonsensemedia.org
Common Sense Media Protecting Kids Online in the AI Era - The Path Forward
PDF documentwww.commonsensemedia.org
