Google has completely overhauled its chatbot offering, rebranding Bard as Gemini and introducing a new paid-for tier – Gemini Advanced.
Priced at $19.99 a month, Gemini Advanced takes direct aim at OpenAI’s ChatGPT Plus, which offers a similar service for $20. Google has boasted that its most advanced Large Language Model (LLM), Gemini Ultra, outperforms GPT-4 on most benchmarks. But that’s only half the story.
To compare the performance of different AI models, developers use a series of benchmarks that test how well they complete certain tasks.
While tests such as the MMLU (massive multitask language understanding) test have become the standard benchmarks for comparing different models, in a live setting, both Gemini and ChatGPT use a variety of models depending on the task at hand.
What’s more, when Google first published a research paper describing Gemini, it came under fire for using an inconsistent testing methodology to show the new model in a better light. It also cited test results for the regular GPT-4 model, not the more advanced GPT-4 Turbo.
From a user’s perspective, such quantitative measures of performance are of limited value. Ultimately, how it performs in real-world settings will be far more important in determining whether Gemini Advanced is better than ChatGPT Plus.
With Gemini Advanced, Google has taken aim at OpenAI’s current dominance of the chatbot market. To make its offer more enticing, the firm has offered users 2 months’ access for free. It has also promised to unveil more features in the coming days.
Hinting at new services for Gemini API users and a potential Google Cloud AI launch, on Thursday, Alphabet CEO Sundar Pichai disclosed that “next week we’ll share more details on what’s coming for developers and Cloud customers.”
Depending on how successful the new model is in the coming weeks and months, Google’s new multimodal AI offering could put pressure on OpenAI to accelerate the development of its next GPT model.
In an interview with the Financial Times in November, OpenAI CEO Sam Altman confirmed that the company is working on GPT-5, but didn’t commit to a timeline for its release. More recently, Altman discussed the next-generation GPT model in an episode of Bill Gates’ podcast . He told the Microsoft founder that users can expect improved reasoning ability, enhanced multimedia capabilities and greater reliability from future iterations of ChatGPT.
Meanwhile, Pichai stated that Google is “already well underway” training the next iteration of Gemini.
As the competition for paying generative AI customers intensifies, a technology race between the 2 main players is fast emerging as the next iOS vs. Android or PlayStation vs. Xbox rivalry.