Posted by David Gerard

https://pivot-to-ai.com/2025/11/06/oxford-pretends-ai-benchmarks-are-science-not-marketing/

https://pivot-to-ai.com/?p=6310

Chatbot vendors routinely make up a new benchmark, then show how well their hot new chatbot does on it. Like that time OpenAI’s o3 model trounced the FrontierMath benchmark, and it’s just a coincidence that OpenAI paid for the benchmark and got access to the questions ahead of time.

Every new model will be trained hard against all the benchmarks. There is no such thing as real world performance — there’s only benchmark numbers.

There’s a new conference paper from Oxford University’s Reasoning With Machines Lab: “Measuring what Matters: Construct Validity in Large Language Model Benchmarks.” [press release; paper, PDF]

Reasoning With Machines doesn’t work on reasoning, really. It’s almost entirely large language models — chatbots. Because that’s where the money — sorry, the industry interest — is. But this paper got into the NeurIPS 2025 conference.

The researchers did a systematic review of 46,000 AI papers. Well. What they actually did is they ran the papers through GPT-4o Mini. Using a chatbot anywhere in your supposedly scientific process is a bad sign if you’re claiming to do serious research.

The chatbot pointed the researchers at 445 benchmarking tests. You’ll be 0% surprised that most of these benchmarks were rubbish:

vague definitions for target phenomena or an absence of statistical tests. We consider these challenges to the construct validity of LLM benchmarks: many benchmarks are not valid measurements of their intended targets.

Wow, that’s terrible! How did these benchmarks get that way? Well, the paper never asks that question.

But pretty obviously, science-shaped text to make a product look good is precisely the job of marketing material. The purpose is to generate something to put in the press release.

So what’s the Reasoning With Machines answer to this problem? What’s the action item?

We built a taxonomy of these failures and translated them into an operational checklist to help future benchmark authors demonstrate construct validity.

Now, that’s the right answer — if what the benchmark authors are doing is actually science. But chatbot benchmarks are not science. They were never science. They’re marketing.

This paper never addresses this. This is an 82-page paper, and it never talks about what the AI benchmarks were created for, and what they’re used for in the real world. The word ”marketing” does not appear in the paper. The concept of marketing doesn’t appear in the paper. Not even as “we’re not addressing this right now,” it’s just not there.

It’s like when someone pretends you can talk about chatbots purely as technical artifacts — and somehow never mention what the chatbots are made for, who’s paying for them, why they’re paying all the money they have for chatbots, and the political programme they’re promoting the chatbots to advance. It’s glaringly dodging the issue.

That’s what this paper does — it artificially separates benchmarks from why the benchmarks are this bad. The researchers cannot have been this unaware.

What are the researchers envisioning here? The people creating the chatbot benchmarks — a lot of these are Ph.D scientists. Are these just poor distracted lab workers who somehow forgot how to do science, so it’s a good thing Reasoning With Machines is here to help?

No. Their job was to create marketing materials shaped like science.

This paper treats chatbot benchmarks as defective science that can be fixed. And that was never what chatbot benchmarks were for.

The Oxford Reasoning With Machines Lab is pretending not to understand something that they absolutely should understand, given most of the lab’s work is chatbots.

That’s because this paper is also marketing — to sell Reasoning With Machines’ services to the chatbot vendors, so they can do their marketing better. And make the benchmark lies a bit less obvious.

• Video — Podcast

https://pivot-to-ai.com/2025/11/06/oxford-pretends-ai-benchmarks-are-science-not-marketing/

https://pivot-to-ai.com/?p=6310