Controversy Erupts Over xAI's Grok 3 Benchmark Claims

Sign Up | Top AI Tools

Welcome AI Fans!
Top story: A dispute has arisen regarding the benchmark results published by xAI for its latest AI model, Grok 3, with an OpenAI employee accusing the company of presenting misleading data.

_ _ _ _ _ _ _ _ _ _ _ _

In today’s morning:

• ⚡ Controversy Erupts Over xAI's Grok 3 Benchmark Claims

• 🌐 How to automate web data extraction with Reworkd?

• 🎥 Google Unveils Pricing for Veo 2 AI Video Generation Model

• 🛠️ New Top AI Tools

• 🔮 More in the AI’s World

🕒 Estimated Reading Time: 5 minutes

LATEST IN AI

GROK 3
⚡ Controversy Erupts Over xAI's Grok 3

Overview: A dispute has arisen regarding the benchmark results published by xAI for its latest AI model, Grok 3, with an OpenAI employee accusing the company of presenting misleading data. The controversy centers around the omission of certain metrics in xAI's performance comparisons.

Essential Details:

• xAI published a graph showing Grok 3 outperforming OpenAI's o3-mini-high model on the AIME 2025 math benchmark.

• Critics point out that xAI's graph omitted the "cons@64" (consensus@64) metric for OpenAI's model, which allows 64 attempts at each problem.

• When comparing single-attempt (@1) scores, Grok 3 variants actually scored below OpenAI's o3-mini-high.

• xAI co-founder Igor Babushkin defended the company's methodology, while neutral observers have created more comprehensive comparison graphs.

• The incident has sparked broader discussions about transparency and standardization in AI benchmark reporting.

Why It Matters: This controversy highlights the challenges in fairly comparing AI models and the potential for misleading presentations of benchmark data. It underscores the need for standardized reporting practices in the AI industry to ensure transparency and accurate assessments of model capabilities.

TUTORIAL
🌐 How to automate web data extraction with Reworkd?

Overview: Reworkd.ai is revolutionizing web data extraction by offering no-code, fully automated web scraping solutions that eliminate the complexities of handling large-scale data.

Step-by-step:

• Sign Up – Create an account on Reworkd.ai and access your dashboard.

• Select Task – Choose web scraping, monitoring, or custom automation.

• Configure Agent – Enter URLs, set data fields, and adjust settings.

• Run & Monitor – Start extraction and track progress in real-time.

• Export Data – Download as CSV, JSON, API, or sync with tools.

Pro tip: Schedule automated extractions to keep your data updated without manual effort.

GOOGLE
🎥 Google Announces Pricing for Veo 2 AI Model

Overview: Google has revealed the pricing structure for its new AI video generation model, Veo 2, setting the cost at 50 cents per second of generated video. This announcement marks a significant step in the commercialization of AI-powered video creation tools.

Key Details:

• Veo 2 will cost users $30 per minute or $1,800 per hour of generated video.

• Google DeepMind researcher Jon Barron compared this pricing to traditional film production, noting that "Avengers: Endgame" cost around $32,000 per second.

• The model is designed to create clips of two minutes or more, as highlighted in Google's December announcement.

• Veo 2's pricing contrasts with OpenAI's Sora model, which is available through a $200 monthly ChatGPT Pro subscription.

• While the per-second cost may seem high, it could potentially reduce overall production expenses and timelines for certain projects.

Why It Matters: Google's pricing strategy for Veo 2 positions it as a premium service for professionals and enterprises. As AI-generated video becomes more sophisticated, it could revolutionize content creation across industries, potentially offering a more cost-effective alternative to traditional video production methods.

AI INSIGHTS

AI TOOLS
🛠️ Must-Try AI Tools & Websites:

• HowsThisGoing – Automates Slack standups with concise summaries.

• GroqChat – AI-powered conversational assistant for various needs.

• ImageToCartoon – Converts images into cartoon-style avatars.

• Inline Help – Provides contextual support within apps and websites.

• ImagineMe – Creates AI-generated self-portraits from text prompts.

📚 More in the AI’s World

Microsoft is expanding its server capacity in anticipation of OpenAI's upcoming GPT-5 model, which promises enhanced AI capabilities.

Anthropic's Claude AI is getting an update with improved reasoning and web search features, similar to ChatGPT's recent enhancements.

YouTube is introducing AI-generated video clips for its Shorts platform, powered by Google DeepMind's Veo 2 model, allowing creators to enhance their content.

Researchers have developed Evo 2, a massive AI model that can predict genetic mutations and design new genomes, marking a significant breakthrough in biomolecular sciences.

Elon Musk's startup xAI is set to release Grok 3, a chatbot that aims to surpass the capabilities of models like ChatGPT, further expanding AI's reach.

Thanks for sticking around…

That’s all for now—catch you next time!

Have any thoughts or questions? Feel free to reach out at community@nurowave.ai – we’re always eager to chat.

P.S.: Do follow me on LinkedIn and enjoy a little treat!

Jahanzaib