Home / News / Technology / How Does Mistral Large 2 Compare to LLaMA 3.1?
Technology
3 min read

How Does Mistral Large 2 Compare to LLaMA 3.1?

Published
James Morales
Published

Key Takeaways

  • Mistral has released the latest version of its large AI model.
  • The release comes just a day after Meta unveiled LLaMA 3.1.
  • Mistral Large 2 outperforms LLaMA 3.1 205B on some benchmarks despite being less than 2-thirds the size.

Just a day after Meta released LLaMA 3.1, on Thursday, July 24, Mistral unveiled  a new language model it says competes with Meta’s largest AI.

Head to head, Mistral Large 2 outperforms LLaMA 3.1 205B on several key benchmarks, especially those that measure models’ ability to generate code.

Mistral Large Does More With Less

At 123 billion parameters, Mistral Large 2 sits between the medium and large-sized LLaMA models, yet it was better at certain tasks than the 205 billion parameter version of LLaMA 3.1.

While LLaMA 3.1 205B retains the lead in the crucial Massive Multitask Language Understanding (MMLU) benchmark, Mistral Large 2 pulls ahead in areas such as coding, mathematics and instruction following.

chart showing Mistral vs. LLaMA 3.1 benchmarks
Mistral Large 2 vs. LLaMA 3.1. Source Mistral.

Considering its smaller footprint, Mistral’s model also performs well in multi-lingual text generation and reasoning.

Delivering More Concise Responses

Comparing the length of AI responses to different prompts, Mistral Large 2 output shorter answers than LLaMA 3.1, GPT-4o and Claude 3.

Emphasizing that shorter generations are more cost-effective for inference, Mistral said: “we spent a lot of effort to ensure that generations remain succinct and to the point whenever possible.”

chart showing average length of model outputs
Average length of model outputs. Source: Mistral.

This focus on conciseness highlights the growing emphasis on AI efficiency that has inspired the recent generation of smaller, more compact models. 

While longer answers tend to improve scores on many benchmarks, Mistral Large is designed to perform well without being unnecessarily verbose, a principle that could help it gain traction with business users.

Mistral Large 2 vs. Meta Llama 3.1 405B

Benchmark Meta-Llama 3.1 405B Mistral Large 2
MMLU

MATH

MBPP Base

HumanEval Code

87.3%

73.8%

88.6%

89%

84.0%

71.5%

80%

92%

Competition Heats up Among Open-Source AI Rivals

The rivalry between Mistral Large 2 and LLaMA 3.1 highlights the rapid advancements and competitive nature of open-source AI development. In terms of performance, for the first time, open-source AI poses a viable alternative to proprietary models offered by Google and OpenAI.

Both models have been made freely available for research purposes. However, Mistral and Meta diverge in their commitment to open-sourcing their AI.

While Meta has moved toward more permissive licenses, since partnering with Microsoft, Mistral has gone in the opposite direction, restricting commercial applications of its latest releases to paying customers.

Was this Article helpful? Yes No

James Morales

Although his background is in crypto and FinTech news, these days, James likes to roam across CCN’s editorial breadth, focusing mostly on digital technology. Having always been fascinated by the latest innovations, he uses his platform as a journalist to explore how new technologies work, why they matter and how they might shape our future.
See more