Google Gemini vs ChatGPT Plus: Same Price, Smarter Product?

Key Takeaways

• Google has launched a paid-for chatbot service, Gemini Advanced.
• Gemini Advanced pricing puts the new chatbot in direct competition with ChatGPT Plus.
• The underlying Large Language Model (LLM) powering the latest service outperforms GPT-4 in most tests, but there is more nuance when it comes to comparing their real-world performance.

Google has completely overhauled its chatbot offering, rebranding Bard as Gemini and introducing a new paid-for tier – Gemini Advanced.

Priced at $19.99 a month, Gemini Advanced takes direct aim at OpenAI’s ChatGPT Plus, which offers a similar service for $20. Google has boasted  that its most advanced Large Language Model (LLM), Gemini Ultra, outperforms GPT-4 on most benchmarks. But that’s only half the story.

The Limitations of AI Performance Tests

To compare the performance of different AI models, developers use a series of benchmarks that test how well they complete certain tasks. 

While tests such as the MMLU (massive multitask language understanding) test have become the standard benchmarks for comparing different models, in a live setting, both Gemini and ChatGPT  use a variety of models depending on the task at hand. 

What’s more, when Google first published a research paper describing Gemini, it came under fire  for using an inconsistent testing methodology to show the new model in a better light. It also cited test results for the regular GPT-4 model, not the more advanced GPT-4 Turbo.

From a user’s perspective, such quantitative measures of performance are of limited value. Ultimately, how it performs in real-world settings will be far more important in determining whether Gemini Advanced is better than ChatGPT Plus. 

Battle of the Bots Continues

With Gemini Advanced, Google has taken aim at OpenAI’s current dominance of the chatbot market. To make its offer more enticing, the firm has offered users 2 months’ access for free. It has also promised to unveil more features in the coming days.

Hinting at new services for Gemini API users and a potential Google Cloud AI launch, on Thursday, Alphabet CEO Sundar Pichai disclosed  that “next week we’ll share more details on what’s coming for developers and Cloud customers.”

Depending on how successful the new model is in the coming weeks and months, Google’s new multimodal AI offering could put pressure on OpenAI to accelerate the development of its next GPT model.

In an interview with the Financial Times in November, OpenAI CEO Sam Altman confirmed that the company is working on GPT-5, but didn’t commit to a timeline for its release. More recently, Altman discussed the next-generation GPT model in an episode of Bill Gates’ podcast . He told the Microsoft founder that users can expect improved reasoning ability, enhanced multimedia capabilities and greater reliability from future iterations of ChatGPT.

Meanwhile, Pichai stated that Google is “already well underway” training the next iteration of Gemini.

As the competition for paying generative AI customers intensifies, a technology race between the 2 main players is fast emerging as the next iOS vs. Android or PlayStation vs. Xbox rivalry.

About the Author

James Morales

Although his background is in crypto and FinTech news, these days, James likes to roam across CCN’s editorial breadth, focusing mostly on digital technology. Having always been fascinated by the latest innovations, he uses his platform as a journalist to explore how new technologies work, why they matter and how they might shape our future.

See more

[email protected] LinkedIn Twitter