Cassidy: A multi-university paper says the benchmarks can be cooked — and the very same week
Cassidy: Meta decides the cure is to hand you the weights and let you check the math yourself.
Cassidy: This is the AI Daily Briefing for Wednesday.
Cassidy: Today: the Princeton-MIT paper putting every “state of the art” claim under a shadow
Cassidy: Meta’s open-source move through the inference-stack lens
Cassidy: and HubSpot’s 200 updates with no technical report in sight.
Bill: When an AI company drops a blog post saying their new model is “state of the art” because it topped some benchmark
Bill: how do we actually know if that number means anything — or if they just figured out how to ace the test?
Cassidy: Yeah, it’s a genuinely important question, and the short answer is: a lot of those numbers don’t mean much anymore.
Cassidy: A paper from a multi-university team — Princeton
Cassidy: MIT Cornell Brown and others — published on arXiv puts it bluntly: benchmarking is broken.
Cassidy: The problem has two parts.
Cassidy: First, saturation. According to analysis from Benchr
Cassidy: most of the benchmarks you see cited most often — MMLU
Cassidy: HumanEval GSM8K — have scores clustered so high that the top five frontier models are separated by less than three points.
Cassidy: A tournament where everyone scores in the upper nineties has, in their words, “stopped sorting.” Second, gaming.
Cassidy: Per that same Benchr analysis
Cassidy: HumanEval and MMLU are specifically flagged as benchmarks to retire because they’ve been so extensively trained against that a high score
Cassidy: may tell you more about whether a company optimized for the test than whether the model is actually capable.
Cassidy: So when you see a headline score, ask: is this benchmark saturated?
Cassidy: Was the methodology published, and could a third party reproduce it?
Cassidy: And is the company grading its own homework?
Bill: That last one — companies grading their own homework — what does that actually look like in practice, and why is it a problem?
Cassidy: The arXiv paper flags this directly in its title: “Don’t Let AI Be Its Own Judge” — meaning when a model is evaluated using another AI
Cassidy: system as the scorer
Cassidy: the incentives get circular and the results get unreliable fast.
Cassidy: So watch whether a benchmark score comes with an independent methodology — something a third party could reproduce without access to the
Cassidy: company’s internal data.
Cassidy: If the only source is the company’s own blog post
Cassidy: with no linked technical report
Cassidy: system card or external replication
Cassidy: treat the score as marketing copy until someone else can check it.
Cassidy: This one's from Gizmodo:
Quote: While the new releases will reportedly maintain some proprietary parts for alleged safety purposes
Quote: the company apparently plans to open-source the models
Quote: likely offering licensing agreements to firms that want to use the model instead of going full black box
Quote: like many of its competitors.
Bill: Six hundred billion earmarked for AI, and the headline move is open-sourcing the weights.
Bill: Hard to call that floundering.
Bill: Feels like Meta just showed us the strategy.
Cassidy: And Gizmodo wants you reading the “floundering” word.
Cassidy: I’d read it through the inference stack instead: open weights mean nothing if Meta still controls who fine-tunes and who serves at scale.
Bill: Right and tie it back to what we just hit on benchmarks — if the yardsticks are bent
Bill: shipping open weights is one of the few honest moves left.
Bill: The outside world gets to run its own evals.
Cassidy: If they ship clean. The report already says some parts stay proprietary for “safety.” Always the asterisk.
Bill: First release under Alexandr Wang, fresh off the Scale AI acquisition.
Bill: So the data-labeling guy is now running the lab that bought his company to fix its models. No pressure.
Cassidy: And the timing lands the same week the Senate killed the federal moratorium ninety-nine to one.
Cassidy: You want to compete under fifty different state regimes?
Cassidy: Open-source the model and let licensees carry the fine-tuning risk. Very convenient.
Cassidy: This one's from HubSpot:
Quote: “SMBs don't need more AI hype—they need technology that helps,” said Andy Pitre, EVP of Product at HubSpot.
Quote: “The products we’re launching in our Spring 2025 Spotlight are helping teams move fast on AI and solve their go-to-market challenges.
Quote: We've embedded AI throughout our entire platform so businesses of any size can start seeing value immediately
Quote: without massive teams or budgets.”
Bill: Two hundred-plus features, embedded AI across the whole platform, and the pitch is SMBs feel like they’re falling behind.
Bill: So the cure for AI overwhelm is...
Bill: two hundred more things to learn?
Cassidy: And notice what comes with this press release.
Cassidy: Nothing. No technical report, no eval, no failure rate — just a count of features and a quote from the EVP of Product.
Bill: Right, and the honest version of this is fine — they’re automating the stuff a sales rep already hated doing.
Bill: Logging calls, drafting follow-ups.
Bill: I want to know whether any of those two hundred updates changes what the rep actually has to know, or just kills the busywork.
Cassidy: Andy Pitre says SMBs don’t need more hype, they need tech that helps — said in a release announcing two hundred features at once.
Cassidy: Pick a lane, Andy.
Bill: And after the benchmark segment we just did, there’s no number here to even check.
Bill: At least a frontier lab gives you a cooked benchmark to argue about.
Bill: This is just a feature count.
Cassidy: If AI Daily Briefing is part of your routine, consider subscribing wherever you’re listening.
Cassidy: And if you have a moment, leave a quick review — it really helps other people find the show.
Cassidy: We’ve put the links for everything we covered today in the show notes.
Cassidy: If something caught your ear, that’s the place to dig in a little deeper.
Cassidy: That’s AI Daily Briefing for today.
Cassidy: This is a Lantern Podcast.