Meta recently achieved a high score in the LM Arena rankings with its new Llama 4 Maverick AI model. However, it later turned out that this score was obtained using an experimental – that is, a specially optimized version that was not yet available to the public.
LM Arena officially apologized for this situation and changed its policy. Now they only rate “regular”, that is, unmodified models. As a result, vanilla Llama-4-Maverick fell far behind its competitors.
It ranked lower than models such as GPT-4o (OpenAI), Claude 3.5 Sonnet (Anthropic), and Gemini 1.5 Pro (Google). This led to criticism that Meta was trying to increase its ranking using an experimental model specifically tuned for chat.
Meta said in a statement:
- “We are testing different options. Now we have released an open source version. We will see what solutions developers create.”
Note: LM Arena often evaluates AI using scores based on user preferences. This does not always fully reflect the effectiveness of the model in real life.
Leave a Reply