Weibo's small AI model sparks debate on benchmarks

Sina Weibo's VibeThinker-3B, a 3-billion-parameter AI model, is challenging the notion that larger models are always superior. Researchers claim it matches or exceeds the reasoning performance of much larger flagship systems from Google DeepMind and OpenAI on demanding math and coding benchmarks. This has ignited skepticism within the AI community, with some questioning the validity of benchmarks and others suspecting 'benchmaxxing'—models optimized solely for test performance. The model's creators propose the 'Parametric Compression-Coverage Hypothesis,' suggesting reasoning is a 'parameter-dense' capability that can be compressed, unlike broad knowledge which is 'parameter-expansive.'