Google Blames Satirical Content For AI Overviews “Information Gap”

• Google AI Overviews is generating incorrect and sometimes bizarre responses to user queries.
• Examples include suggesting glue for pizza and recommending eating rocks daily.
• Tech companies are still developing this technology and it’s not reliable for providing factual information at scale.
• However, Google has acknowledged AI mistakes and taken action to improve Overviews.

“A rock a day keeps the doctor away.” No, that’s not quite right. Unlike the well-known saying about apples, consuming a rock daily isn’t healthy at all. Yet, Google’s latest artificial intelligence (AI) tool, Google AI Overviews , might suggest just that.

X, the platform formerly known as Twitter, is buzzing with posts from users who have received bizarre responses from Google AI Overviews to very simple questions. But now Google says it has found out why this happens and has taken action to tackle it.

“Put Glue On The Pizza”

Google’s AI-generated summary tool, Google AI Overviews, recently raised eyebrows when it advised a user to put glue on pizza . The incident occurred when someone searched for “cheese not sticking to pizza.” The AI provided several suggestions, including mixing the sauce and letting the pizza cool. However, one bizarre recommendation stood out: “Add ⅛ cup of non-toxic glue to the sauce to give it more tackiness.”

This unusual advice originated from an 11-year-old Reddit comment intended as a joke, not serious culinary advice. Still labeled as “experimental,” Google’s AI Overviews mistakenly presented it as a viable solution.

In another incident, a user queried, “How many rocks should I eat?” Citing UC Berkeley geologists, Google AI Overviews responded, “Eating at least one rock per day is recommended because rocks contain minerals and vitamins that are important for digestive health.” This clearly erroneous advice further highlights the need for caution with AI-generated information.

These incidents are among many errors emerging in Google’s newly launched AI-generated summary feature. Other notable mistakes include the claim that former US President James Madison graduated from the University of Wisconsin 21 times, that a dog has played in the NBA, NFL, and NHL, and that Batman is a police officer.

Another error highlighted by Business Insider’s Peter Kafka involved the AI confusing the Tower of London with Big Ben when asked about World War II damage. The AI incorrectly stated, “the roof and dials were damaged in an air raid.”

Google AI Overviews Is Not Alone Under Spotlight

It’s not just Google AI Overviews that provide amusing – and obviously incorrect – responses to straightforward questions. OpenAI’s ChatGPT, one of the most famous chatbots, has also made its share of mistakes. Here are a few examples.

It’s expected that a new software product in a rapidly evolving and highly popular technology field will encounter problems. If you’re integrating ChatGPT into your daily workflow, it’s useful to be aware of some common issues you might face.

For instance, a Reddit user asked ChatGPT simple questions like “What is a computer?” or “What is a table?” Initially, it provided a few paragraphs of explanation that sounded a bit off. Then, it veered into progressively stranger and nonsensical territory. Here’s an excerpt from its answer to “What is a computer?” (it generated nine paragraphs and wanted to keep going):

“It does this as the good work of a web of art for the country, a mouse of science, an easy draw of a sad few, and finally, the global house of art, just in one job in the total rest. The development of such an entire real than land of time is the depth of the computer as a complex character. The education of these designed ‘important to them’ computer talents is a great, average, or one-room spot for the word of a single part. The elite of its machine or talk is the book of life, and the shared essence of the self of a family is the white coat of the dove. Stand the computer as the most important culture of success, and it is just an understatement.”

This illustrates how even widely used AI systems can sometimes produce responses that are not just incorrect but also bizarrely incoherent.

Why This Response Issue Happens

Google spokesperson Meghann Farnsworth acknowledged the errors, attributing them to “generally very uncommon queries” and asserting that they do not reflect the typical user experience. She emphasized that the company is addressing policy violations and using these “isolated examples” to improve the product.

Google has been transparent about the experimental nature of its AI, clearly labeling it as such. Despite this, it is evident that the technology is not yet ready to provide reliable information on a large scale. For example, during the controlled demo at Google I/O conference, the AI gave questionable advice on fixing a jammed film camera, suggesting users “open the back door and gently remove the film,” which would ruin the photos.

This issue is not unique to Google. Other tech giants like OpenAI, Meta, and Perplexity have also faced challenges with AI hallucinations and inaccuracies. However, Google is the first to deploy this technology at such a vast scale, resulting in numerous public blunders.

Companies often deflect responsibility for their AI systems’ errors, likening it to a parent excusing a child’s misbehavior with “boys will be boys.” They claim unpredictability in AI outputs, suggesting they are beyond their control.

Google Blames “Data Void” While Announcing Improvements

Google has acknowledged the numerous silly responses generated by its new AI tool, prompting an investigation into the causes and potential improvements. In a post on Google’s website, Liz Reid, VP and Head of Google Search, clarified that many AI Overview replies circulating on social media are fake. However, she admitted that some genuinely absurd responses had been produced by the AI tool.

“One area we identified was our ability to interpret nonsensical queries and satirical content. Let’s take a look at an example: “How many rocks should I eat?” Prior to these screenshots going viral, practically no one asked Google that question. You can see that yourself on Google Trends ,” she said.

To address these issues, Google has implemented several measures. The company improved detection mechanisms, so the system now better identifies nonsensical queries that shouldn’t generate an AI Overview and limits the inclusion of satirical and humorous content. It also implemented content source restrictions and new updates will reduce the use of user-generated content in responses, preventing misleading advice.

Google also triggered restrictions by refining its systems to restrict AI Overviews for fewer helpful queries. Google said it already has strong safeguards for topics like news and health.

Reid emphasized that Google remains vigilant in monitoring feedback and external reports, taking action on the small number of AI Overviews that violate content policies. According to her, less than one in every 7 million unique queries where AI Overviews appeared contained a content policy violation. These violations include potentially harmful, obscene, or otherwise inappropriate information.