Today is dominated by the Claude Opus 4.7 launch — but the more interesting content sits underneath it: a debate about what the benchmarks actually measure, OpenAI's quiet pivot toward domain-specialized reasoning, and Dwarkesh Patel's detailed notes on why pretraining runs fail and whether distillation can be stopped.

---

Opus 4.7: Real Gains, Contested Tradeoffs

The headline numbers are unambiguous. Opus 4.7 hits 64.3% on SWE-bench Pro (+11 points over 4.6), 87.6% on SWE-bench Verified (+7 points), and claimed #1 on Vals Index at 71.4% — up from a 67.7% previous best. Cursor's internal benchmark jumped from 58% to 70%; Notion reportedly saw a 14% lift with one-third the tool errors. Swyx's AINews summary is the most comprehensive read: 4.7-low is strictly better than 4.6-medium, 4.7-medium strictly better than 4.6-high, and 4.7-high beats 4.6-max — with a new xhigh effort tier that Claude Code now defaults to.

The vision upgrade is the sleeper story. Opus 4.7 accepts images up to 2,576px on the long edge (~3.75 megapixels), more than 3x prior Claude limits, with no downscaling. For computer-use agents parsing dense screenshots or fine-grained diagram extraction, this is a material capability shift, not a footnote.

Where it gets messier is the tokenizer. Several observers flagged that 4.7 ships a different tokenizer from 4.6, prompting debate about whether this is a new base model, a tokenizer-swapped continuation, or a Mythos distillation. Anthropic's Boris Cherny acknowledged that the same input can map to 1.0–1.35x more tokens depending on content type, while simultaneously arguing that reasoning efficiency improved enough that total token use is still down up to 50% versus equivalent 4.6 effort levels. Cherny said Anthropic increased subscriber limits to compensate — but enterprise document pipelines face real cost pressure: LlamaIndex's ParseBench comparison puts Opus 4.7 at ~7¢/page for OCR-like work, versus ~0.4¢ for their cost-effective mode.

Document understanding shows the characteristic uneven pattern: charts improved massively (13.5% → 55.8%), while layout actually regressed (16.5% → 14.0%). Content faithfulness is strong, formatting is modest, tables negligible. The model got better in aggregate but not uniformly.

The most pointed technical pushback is on long-context performance. Multiple users noted MRCR (a needle-in-a-haystack-style metric) looks worse on 4.7. Cherny's response is that Anthropic is intentionally phasing out MRCR because it "overweights distractor-stacking tricks" and moving toward Graphwalks as a better applied-reasoning signal — where 4.7 goes 38.7% → 58.6%. That's a defensible position, but it also means you should treat the long-context story as genuinely contested until independent evaluation settles it.

Simon Willison adds a characteristically wry data point from his pelican-riding-a-bicycle SVG benchmark: Qwen3.6-35B-A3B running as a 21GB quantized model on a MacBook M5 outperformed Opus 4.7 on both SVG generation tests. His own caveat is that this doesn't mean Qwen is more capable generally — but it does illustrate something real: the correlation between benchmark leadership and practical task performance is noisier than it used to be at the top of the capability curve.

Cat Wu's operational framing from Anthropic is worth retaining: treat Opus 4.7 like an engineer you delegate to, not a pair programmer you micromanage — put full goals, constraints, and acceptance criteria up front, and encode testing workflows in claude.md. That's less a tip than a signal about what Anthropic optimized for.

---

OpenAI's Parallel Track: Domain Models + Superapp

While Opus 4.7 dominated the discourse, OpenAI put out two substantial things in 3 days. GPT-Rosalind, the first in a planned life sciences series, scored better than 95% of human scientists on a blind RNA prediction task from Dyno Therapeutics, with Amgen, Moderna, and the Allen Institute already in the test phase. Two days earlier, GPT-5.4-Cyber launched. The pattern is clear: a flagship general model plus purpose-built domain variants tuned for specific reasoning tasks and toolchains.

The Codex update is a separate move — background computer use, parallel agents, an in-app browser, memory across sessions, and inline image generation built in. At 3M weekly users and 70% month-over-month growth, Codex head Thibault Sottiaux called it "building the super app out in the open." The analogy to Claude Code is direct: both are now general-purpose agentic environments, not just coding assistants.

The domain model strategy is worth watching closely. The argument is that the best performance on high-value professional wedges — drug discovery, cybersecurity, financial analysis — may require models trained on domain-specific objectives, not just general RLHF on broad capability. If that's right, the competitive axis shifts from "who has the best general model" to "who has the best domain coverage map."

---

The Distillation Problem and the Gated Frontier

Dwarkesh Patel has a sharp framing of the distillation question that's worth sitting with. The core argument: frontier labs can't really stop open-source models from absorbing their capabilities, and the numbers make this uncomfortable. At $25/MTok for Opus 4.6 output tokens, 1T tokens of synthetic training data costs $25M. That's noise for a well-funded lab. The labs are hiding chain-of-thought, but Dwarkesh argues this is easy to route around — you can instruct models not to surface thinking, or reconstruct it as an RLVR target. More fundamentally, agentic tool use (file writes, bash commands) happens locally and can't be hidden.

His most interesting observation is about coding product companies inadvertently building better distillation than direct token sampling: by capturing the "gold diff" — what users actually accepted after multiple back-and-forths — and using that as an RL target, companies building on top of frontier APIs may end up training models better calibrated to real user preferences than the frontier models themselves.

This connects to what Swyx's AINews piece notes about Mythos: Anthropic is now running 2 parallel tracks — a fast 2-month public release cadence and a gated frontier line accessible only to select partners (reportedly including certain U.S. government agencies). The Mythos Preview scores 77.8% on SWE-bench Pro versus Opus 4.7's 64.3% — a 13-point gap that Anthropic is explicitly not closing for the public. This is the first major public instance of the "true frontier" being meaningfully ahead of what's commercially available, and the distillation dynamics Dwarkesh describes make it an unstable equilibrium.

On the cybersecurity question specifically: Dwarkesh's framing is that Mythos's qualitative leap in cyberattack capability isn't a smooth intelligence increase but a combinatorial unlock — prior models found individual vulnerabilities; Mythos can chain 5 together into a working exploit. He's cautiously optimistic on defense: software has gotten more secure despite accumulated human hacking talent, and AI-assisted patching of latent zero-days could strengthen defense more than offense. But a security expert he spoke to pushes back — AI is much better at finding vulnerabilities than patching them, because patching requires understanding all the downstream behavioral dependencies of a fix.

---

Why Pretraining Runs Fail

Dwarkesh's notes from a lecture by Horace He on training parallelisms are the most technically dense content of the day, and genuinely worth reading for practitioners. Two failure patterns stand out.

Breaking causality via expert routing: In mixture-of-experts (MoE) models, expert-choice routing (where experts pick tokens rather than tokens picking experts) gives cleaner load balancing during training but breaks the causal mask — token N's expert assignment can depend on token N+K. This may explain why Llama 4 and Gemini 2 underperformed expectations. The fix is known but the tradeoff is real: you sacrifice load balancing for correctness, or vice versa.

Floating point bias accumulation: The original GPT-4 training apparently hit a significant slowdown from a subtle FP16 bug in collective operations. When summing 10,000 small values into a large accumulator, the logarithmic density of FP16 means you can hit quantization cliffs — adding 1 to 1024 and rounding back down, repeatedly, causing the accumulated value to diverge 10x from ground truth. Bias compounds; variance averages out.

The broader framing Dwarkesh takes from his conversations: it's unlikely there are just "5 ways training runs fail" that labs can definitively solve. New failure modes keep emerging at scale. His source is bearish on AI fully automating kernel writing in the near term on these grounds — it's more "AGI-complete" than the "verifiable domain → easy RL loop" framing suggests.

---

The practical question today's content surfaces: the Mythos gap (13+ SWE-bench Pro points ahead of what's publicly available) and the distillation dynamics Dwarkesh describes create a structural tension. If gated frontier models can't maintain a durable lead over open-source distillation, the value of the gating degrades over time — but it may be durable long enough to matter for cybersecurity, biotech, and other high-stakes domains where a 6-month capability advantage has asymmetric consequences. Whether that gap is strategically meaningful or just a temporary moat is the question that will define the next year of lab positioning.

---

TL;DR - Opus 4.7 delivers real benchmark gains (+11 SWE-bench Pro, 3x image resolution) but reactions are divided on non-coding tasks, long-context performance is contested, and the new tokenizer creates cost complexity for document pipelines. - OpenAI is running a dual strategy of domain-specific models (Rosalind for biotech, 5.4-Cyber) alongside a Codex-as-superapp push, suggesting the flagship-plus-verticals architecture is becoming an industry pattern. - Dwarkesh argues distillation can't meaningfully be stopped — agentic tool use is observable, synthetic data is cheap, and coding product companies may be building superior training signal from user-accepted "gold diffs." - Two underappreciated causes of failed pretraining runs are causal mask violations in MoE expert routing (possibly behind Llama 4 and Gemini 2 underperformance) and FP16 bias accumulation in collective operations, with sources suggesting new failure modes keep emerging at every new scale.

Compiled from 4 sources · 6 items

• Simon Willison (3)
• Rowan Cheung (1)
• Swyx (1)
• Dwarkesh Patel (1)