Three trillion-parameter open-weight models landed in April, now scoring within 6 points of GPT-5.5 on standard benchmarks — and concurrent research is making the case that the competitive surface has shifted from base model IQ to agent runtime design.

---

Open Weights Close to Within 6 Points of the Frontier

The past month delivered what practitioners are calling one of the best periods ever for open models. DeepSeek V4 Pro (1.6T parameters / 49B active), Kimi K2.6 (1T / 32B active), and MiMo V2.5 Pro (1T / 42B active) now score 52–54 on Artificial Analysis's Intelligence Index — against 57 for Gemini 3.1 Pro Preview and Claude Opus 4.7, and 60 for GPT-5.5. All three are permissively licensed. The remaining gap is no longer diffuse — it's concentrated in HLE, CritPt, TerminalBench Hard, and hallucination-heavy evaluations, which means for the bulk of real-world coding and agentic tasks, the practical distance between open and closed has meaningfully collapsed.

The strongest hands-on report came from testing DeepSeek V4 Pro inside the Pi coding agent, where it was described as "the first open-weight model that genuinely feels comparable to Codex or Claude Code for multi-turn agentic coding" — notable from someone running it seriously. Architecture details worth noting: 1M context, hybrid CSA/HCA attention (compressed sparse vs. hybrid cross-attention), key-value cache reduced to 10% of standard size, and roughly 4x lower inference FLOPs at long context. That last number matters for serving economics.

At the smaller end, Qwen 27B is getting serious adoption as a coding model. Practitioners describe it as more reliable than Opus 4.6 for structured tasks like merge conflict resolution (less hallucination), capable of running at 44 tokens/second at 35B Q8 on a Strix Halo 128GB, and workable as a Claude Code substitute when tasks are properly decomposed and documentation is accessible. The model isn't winning on creativity — it's winning on instruction-following reliability, which turns out to be what iterative agentic loops need most.

Qwen also released Qwen-Scope: sparse autoencoders (SAEs) mapping internal features across all layers of Qwen 3.5 models from 2B through 35B mixture-of-experts (MoE). Described as potentially the largest open-source interpretability tool to date (GemmaScope topped out at 9B), it supports surgical feature ablation, concept steering, and model debugging via activation heatmaps — practically useful for diagnosing unexpected behaviors before they become entrenched.

Grok 4.3 rounds out the release batch with mixed reception: up 4 points on the Intelligence Index to 53, 40% lower input and 60% lower output pricing, a strong jump on GDPval-AA (up 321 Elo to 1500), but non-hallucination dropped 8 points — a capability/reliability tradeoff the community noticed immediately. The structural critique from practitioners is that Grok's low prices may reflect subsidized hardware utilization, and that cache economics, not model quality alone, increasingly determine agentic total cost of ownership.

---

The Runtime Has Become the Product

The Codex vs. Claude Code debate dominating practitioner discussion isn't really about which base model is smarter. GPT-5.5 is faster, more token-efficient, and more "direct" in fast-mode agentic use; Claude Opus 4.7 has better intent-reading and taste but higher latency (time-to-first-token / tokens-per-second) and more tool calls. The consensus from people using both seriously is that the choice is task-shape dependent, not a general capability verdict — and benchmark comparisons remain highly harness-dependent (GPT-5.5 does not beat Opus 4.7 in the Claude Code harness on PostTrainBench).

The more consequential story is what the research literature is revealing about retrieval and memory in agent systems. ReaLM-Retrieve argues that reasoning models need retrieval during inference, not just before it, reporting +10.1% absolute F1 over standard retrieval-augmented generation (RAG), 47% fewer retrieval calls than fixed-interval approaches, and 3.2x lower per-retrieval overhead. At a moment when retrieval latency is often the agentic bottleneck, that efficiency profile matters. OCR-Memory takes a different angle: storing long-horizon agent trajectories as images with indexed anchors rather than lossy text summaries, enabling exact recall of prior state under strict context limits. It claims state-of-the-art on Mind2Web and AppWorld.

A third research thread: agents communicating through shared latent recursive computation rather than natural-language exchanges reportedly yields 8.3% average accuracy improvement, 1.2x–2.4x end-to-end speedup, and 34.6%–75.6% token reduction across 9 benchmarks. If agent-to-agent communication cost becomes a dominant constraint at scale, this direction matters.

On the production infrastructure side, LangChain pushed concrete primitives for multi-user agent deployments — data isolation, delegated credentials, and role-based access control (RBAC) — plus human-in-the-loop (HITL) semantics where a human reply returns directly as a tool result, and durable pause/resume for consequential actions. Cloudflare shipped Dynamic Workflows for durable execution in agent plans. The pattern across stacks is consistent: sandboxing, checkpointing, and orchestration have become the hidden technical debt and the primary differentiation layer. Simon Willison built a practical illustration of where the toolchain now sits — assembling a Python CLI, a Git scraping repo, and a JavaScript frontend into a functioning iNaturalist observation browser, entirely from his phone via Claude Code while camping. The barrier to end-to-end agentic builds has dropped enough that solo developers are now doing in an afternoon what would have required a team.

---

Training Dynamics: Feedback Loops and Self-Improvement

OpenAI's post-mortem on "where the goblins came from" is a compact case study in reinforcement learning (RL) feedback loops. A reward signal for creative language in "nerdy" contexts caused goblin metaphors to proliferate in GPT-5.1. Subsequent models trained on prior outputs amplified the behavior, since it was indistinguishable from successfully rewarded creative responses. OpenAI retired the Nerdy personality and adjusted training protocols. The mechanism is the important part: RL-trained behaviors compound across model generations in ways that are hard to detect until they're entrenched, and the training corpus for next-generation models includes the artifacts of this round's mistraining.

Meta FAIR's self-improving pretraining addresses this class of problem more directly: a post-trained model rewrites pretraining suffixes toward safer, higher-quality continuations, then judges model rollouts during an RL-style pretraining phase. Reported gains: 36.2% relative improvement in factuality, 18.5% in safety, 86.3% win rate in generation quality vs. standard pretraining. The structural insight is that the pretraining phase is improvable via the same post-trained model it's supposed to precede — a recursive loop at the data level, not just at inference time.

Microsoft takes a complementary approach for computer-use agents: generate experiential data at scale by building 1,000 synthetic computers with realistic files and documents, running 8-hour agent simulations averaging 2,000+ turns each. The thesis is that for computer-use agents, the bottleneck is now scalable, realistic experiential data — not model capability. Synthetic worlds as a training data recipe is a credible response to the data wall that agentic training faces when you need long-horizon, multi-step trajectories.

OpenAI's Sebastien Bubeck has also claimed that their models are identifying mistakes in research papers and asking novel research questions — surpassing human researchers on at least those specific tasks. The community pushed back on the lack of verifiable evidence, and the framing drew predictable skepticism. The directional claim (models crossing from answering to generating research questions) is worth watching if substantiated, but treating a tweet as evidence would be a mistake.

---

Spatial Reasoning Gets Architecturally Explicit

Two separate releases this week converge on the same premise: text descriptions are a lossy medium for visual reasoning, and the fix is making spatial primitives first-class citizens in the reasoning process itself.

DeepSeek's "Thinking with Visual Primitives" framework (briefly public, now private, already mirrored by the community) embeds raw bounding box coordinates and coordinate points directly into chain-of-thought reasoning traces, letting the model "point while thinking." The reported architecture uses DeepSeek-ViT with compressed sparse attention and V4-Flash (284B total / 13B active). The target failure modes are where pure text description breaks down: counting, maze solving, path tracing — tasks where attention drift in complex images causes systematic errors. The key architectural bet is changing what the "minimal unit of thought" is for a vision-language model, from a word token to a spatial coordinate.

SenseNova-U1 takes a different angle — no variational autoencoder (VAE), no diffusion — integrating text rendering directly into image generation by processing semantic content rather than latents. The 8B parameter model at 2048×2048 supports image editing with reasoning and interleaved text-image generation in a single pass. Initial image quality reports are mixed on simple prompts, but the departure from diffusion-based pipelines eliminates the language pathway gap that makes traditional diffusion models weak at structured visual outputs like annotated diagrams and infographics.

Both approaches reflect the same growing recognition: spatial intelligence needs dedicated representational machinery, not better text-based approximations.

---

The unresolved question surfaced by today's content: if open-weight models are within 6 benchmark points of GPT-5.5 on standard tasks, and agent runtime design (harness, durable state, retrieval timing, HITL semantics) is the emerging differentiation layer, what exactly are frontier labs' closed models selling? The honest answer is probably reliability on the hardest tasks (HLE, TerminalBench Hard, hallucination resistance at scale) plus full-stack integration — but that's a narrower moat than it was 12 months ago, and it's narrowing faster than the labs' pricing models currently reflect.

---

TL;DR - Open-weight models have closed to within 6 benchmark points of GPT-5.5 on standard evaluations, with DeepSeek V4 Pro, Kimi K2.6, and MiMo V2.5 Pro all scoring 52–54, and the remaining gap concentrated in hard reasoning and hallucination-resistance tasks. - Agent harness design has overtaken base model IQ as the competitive differentiator — retrieval timing, durable state, HITL semantics, and multi-user deployment primitives are where the real differentiation is emerging across stacks. - RL feedback loops compound subtle training artifacts across model generations (OpenAI's goblins), while Meta FAIR's self-improving pretraining and Microsoft's synthetic computer worlds point toward recursive, data-level solutions that address the problem upstream. - Both DeepSeek and SenseNova bet that spatial reasoning requires explicit visual primitives (coordinates and bounding boxes) baked into the reasoning process itself — an architectural divergence from text-description approaches to vision-language tasks.

Compiled from 3 sources · 3 items

• Swyx (1)
• Simon Willison (1)
• Ben Thompson (1)