AI-run experiments could shrink big labs’ payroll advantage
Flint’s performance is impressive, but proving recursive self-improvement will require multiple generations…
The short version
- Recursive says Eureka automated connected architecture, quantization and kernel decisions while helping create Flint.
- Flint reports 55% Terminal-Bench and over 125 tokens per second on one M4 Pro configuration.
- True recursive self-improvement requires multiple generations where each model strengthens the research system that builds its successor.
Flint’s benchmark scores are catnip. Recursive’s bigger claim has my attention: the machine helped design itself.
The startup says its Eureka Machine proposed ideas, implemented them, ran experiments and chose what came next, spanning architecture search, quantization and hardware-specific kernels. Richard Socher said AI improved nearly every part of that pipeline. That matters more than another leaderboard win. Engineering gets expensive when work bounces between people guarding one slice and one Jira kingdom.
Flint is a serious receipt, but full recursive self-improvement needs another turn. Recursive must show Flint improving Eureka’s ability to build Flint’s successor, then repeat those gains across several generations. One clever development cycle proves automation. Repeated improvement of the research system would prove recursion.
Even without an intelligence explosion, this could wreck model-development economics. A small team automating the loop between research and infrastructure can run far more useful experiments per employee, weakening the organizational advantage large labs bought with giant payrolls. Skynet can wait; the org chart is already nervous.
Eureka searches across team boundaries
Eureka’s reported workflow starts with a target: strong coding performance on hardware people can own. It searches architecture directions in parallel, implements candidates and measures quality against speed. Promising results guide the next experiments, and the system can combine ideas from separate branches. For Flint, it favored a mixture-of-experts design that activates fewer parameters per token, then explored post-training quantization recipes for that architecture and generated specialized inference kernels for Apple Silicon and NVIDIA RTX Spark hardware. Because each choice reshapes the next stage’s search space, automating the connected loop can find gains isolated architecture search would miss.
Flint has about 40 billion total parameters, with roughly 2.5 billion active per generated token. The active body accounts for around 2.2 billion, with Recursive’s convention including the output head. Mixture-of-experts routing offers a large capacity pool while reading a much smaller slice during inference. Physics remains undefeated, but routing negotiates the invoice.
The quantization work is especially clever. Recursive says Eureka moved the output layer from 16-bit precision to 4-bit weights, then used the higher-precision model as a teacher for offline distillation. That relied on pre-sampled teacher responses from a coreset the system discovered. Because the output layer represents a large share of weights read per token when the active model body is relatively small, compressing it cuts memory traffic. Distillation tries to recover the accuracy lost in compression. Recursive reports the result finished within 0.7 benchmark points of its higher-precision teacher.
Then came the kernels. Recursive tailored low-bit operations and expert routing to Flint, with separate paths for attention, prompt processing and decoding on the target chips. Generic runtimes must tolerate an absurd zoo of model shapes; a model-specific runtime can arrange memory access around one architecture and its exact quantization recipe. Architecture search created an opening inference engineering exploited several steps later. Founders should study this: Eureka appears to optimize across team boundaries normally connected by meetings, tickets and at least one muted Slack channel.
Flint still underwent pretraining, mid-training, supervised fine-tuning and reinforcement learning. Eureka did not summon a coding model over lunch like pasta al pomodoro. It reportedly tightened a feedback loop that usually bounces decisions between researchers and systems engineers for weeks.
The Claude Code team describes its approach to review:
We didn't remove review. But we did move most of the reading to machines, optimized for human time and attention, and invested heavily in making it fast and easy to undo mistakes.
Anthropic identifies the persistent review bottleneck:
Coding agents keep increasing the pace at which teams write code, and the review bottleneck hasn't eased.
The checkpoint cannot prove who deserves credit
Recursive reports a 55% average on Terminal-Bench using its Terminus agent setup. Flint also reached 90% on SWE-bench Pro v2 with mini-swe-agent, fixed sampling settings and no internet access. A separate SWE-bench Verified run reached 74% under the same agent conditions. Recursive offers no like-for-like external baseline, and its model card warns that scores published elsewhere may use different evaluation rules. Unusually responsible behavior during leaderboard season.
A benchmark belongs to its whole test setup. The harness controls available tools and command execution. Timeouts determine whether slow but correct approaches survive. Sampling changes which branches the model explores; repeated attempts can make an erratic agent look dependable. Internet access can rescue a run, though Recursive counted Terminal-Bench solutions obtained through web retrieval as failures. Independent replication therefore requires matching harnesses, sampling settings, access rules and time limits. None has been published yet.
Even perfect reproduction would validate Flint only under those conditions. It would say little about Eureka’s contribution because Recursive has not published a counterfactual model built by the same team with conventional research tools. Architecture search remains tangled with data and training changes, while quantization and kernel engineering add variables. Anyone assigning neat percentages to each contribution is seasoning vibes and serving them as data.
The local-speed claim needs equal discipline. Recursive reports more than 125 generated tokens per second on an M4 Pro MacBook Pro with 48 GB of memory and 4-bit weights. Across tested context lengths, its `rec` runtime delivered 1.4–2.6 times llama.cpp’s token throughput, excluding prompt prefill. That is a company-run comparison on one machine using its runtime and quantization. I still want it on my desk immediately.
My measurements keep the laptop hype in perspective. In August 2026, I ran gpt-oss:20b on an M3 Max with 128 GB of memory, fully resident in RAM.
It generated about 74 tokens per second and processed prompts at roughly 756 tokens per second. Time to first token stayed below four seconds.
On the same Mac, gpt-oss:120b generated about 51 tokens per second and processed prompts at roughly 215. It remained fully resident, but the first token took almost six seconds.
Meanwhile, my RTX 5060 Ti was running ComfyUI, leaving Ollama 150 MB of VRAM and forcing the smaller language model onto the CPU. “Runs locally” can mean a smooth workstation experience or an expensive space heater having a difficult afternoon.
Recursion needs another generation
The documented chain ends with Flint. Humans define the objective and provide compute; Eureka searches for improvements; Recursive releases the model. Recursion requires Flint to improve Eureka’s research ability, then the upgraded system to produce a stronger successor that improves the research loop again. Several generations would show whether gains compound, flatten or create new bottlenecks. Recursive calls this loop its next bet. The future tense is doing important work.
Evaluation decides whether that bet pays. Eureka proposes a change, and tests determine whether it survives. If those tests capture the goal, useful changes guide later experiments. A missed regression gives an inferior candidate a flattering score, making the next search start from a worse foundation. Errors can compound as efficiently as progress. Coding models have convenient evaluators because programs compile and tests run. Open-ended research has fuzzier feedback, letting automated experimentation produce industrial quantities of immaculate nonsense.
Joe Bak-Coleman has made the strongest objection to runaway recursive-improvement stories. He argues that many takeoff accounts assume the self-amplifying loop they are meant to establish. Research may hit bottlenecks from Amdahl’s Law, increasing demands for human work or evaluators that fail beyond narrow tasks. Flint does not answer that critique. It does show that more model-development coordination can be automated than I expected. I focused too much on models replacing individual tasks and underestimated the value between them.
Operational control remains blank. Recursive has not documented whether humans review Eureka’s proposals before experiments run, how experiment selection is governed or which deployment permissions it receives. We also do not know whether it can request more compute itself. Boring details, until an agent burns six figures overnight because a benchmark ticked upward.
Richard Socher puts the speed-versus-control argument bluntly:
If you can’t control AI from committing felonies, maybe that organization should slow down. But it doesn’t mean everyone else has to slow down also doing good scientific work.
I agree, provided “control” has an engineering definition. Show me who holds the credentials and can raise spending limits. Document every promotion gate. A red stop button in a demo is theatre unless the documentation explains what it disconnects.
Small teams get much bigger experiment queues
Automated research changes startup economics through a simple chain. Lower labor per experiment lets a small team test more candidates against the same objective. More experiments increase the value of good evaluation because weak tests select garbage faster. Compute becomes the ceiling when an agent proposes training runs faster than hardware can execute them. Permissions decide which experiments may spend that compute or touch shared systems. The advantage goes to teams allocating those scarce resources well, even against much larger research groups. Headcount becomes a weaker proxy for experimental capacity.
I have lived through the cheap version in my automation pipelines. Once agents ran routine passes without me, my work shifted toward evaluation. I clicked fewer buttons and spent more time investigating why technically successful runs produced beautifully formatted nonsense. Automation removed chores while making my remaining mistakes costlier. Very founder-friendly. Terrible for sleep.
Near term, smaller model-development teams will run much larger experiment queues. Researchers will define objectives and inspect failures while agents handle more implementation. Systems engineers will build environments agents can safely modify and test. Humans will spend fewer hours producing keystrokes and more deciding which measurements deserve trust, a promotion with considerably more liability.
The open questions remain enormous. Can Eureka improve its research capability across several generations? Independent teams have not reproduced Flint’s results under identical conditions, and local performance may change across consumer hardware, runtimes, quantizations, long contexts or concurrent sessions. Recursive has also kept its human-review process for experiment selection private. Those gaps need plain answers before anyone draws exponential curves.
My bet for the end of 2027: one model lab will publish a full lineage where a model improves the research system, the upgraded system builds a successor, and an outside team reproduces part of the chain. If Recursive gets there first, Flint will look like the first employee badge issued to a lab made mostly of software.
Frequently asked questions
What is Recursive’s Eureka Machine?
Recursive’s Eureka Machine is an automated model-development system that proposes ideas, implements candidates, runs experiments and selects subsequent work. For Flint, it reportedly coordinated architecture search, post-training quantization and hardware-specific inference kernels, allowing choices in one stage to reshape the search space for later stages.
Has Recursive demonstrated full recursive self-improvement?
Recursive has not yet demonstrated full recursive self-improvement. It has shown a development cycle in which Eureka helped build Flint, but Flint must improve Eureka’s research ability. The upgraded system must then build a stronger successor across repeated generations, showing whether gains compound, flatten or encounter new bottlenecks.
How fast does Flint run locally on a Mac?
Recursive reports Flint exceeding 125 generated tokens per second on an M4 Pro MacBook Pro with 48 GB of memory and 4-bit weights. Its `rec` runtime delivered 1.4–2.6 times llama.cpp’s token throughput across tested context lengths, excluding prompt prefill, in a company-run comparison on one machine.
Sources
- This Startup Is All In on Recursive Self-Improvement. Its Latest Model Is Scary Good
- Introducing Flint
- Recursive Flint
- rec
- Flint documentation
- Recursive Self-Improvement through Multi-Agent Self-Supervision