← Back to leaderboard
omarsar0

elvis

Verifiedactive
@omarsar0

Building the frontier @dair_ai • Prev: Meta AI | Galactica LLM | Papers with Code | Elastic | PhD

RosterAI Model News & ReleasesAI Agents & Productivity Tools
325,716 followers · 1,308 following · Last polled 3h ago
Perf. Score
73.4
Rank #49
Value Score
43.8
#157 of 273 priced
Median views
12K
p25 7.8K · trailing 7d
Followers
326K
+0.3% 7d
ER (impr.)
2.79%
eng ÷ impressions
ER (foll.)
0.14%
eng ÷ followers
Est. CPM
$51.68
QT-rate basis
WoW views
+22.2%
▲ rising

Rates & value · est. from 12K median organic views

overpriced
Quote tweetbasis
$600
$51.68 / 1K views
Post
—
no CPM
Thread
—
no CPM
Retweet
—
no CPM
Value Score 43.8 · rank #157 of 273 priced$2489.63 / 1K engagementsrate age unknown (pre-audit import)

Trends

Follower growth
Median views / day
Engagement rate
View distribution · trailing-7d posts

Each blue dot is one organic post. The Performance Score ranks by the median (12K), not the mean (16K) — so a single viral post doesn't inflate a creator's reach. The p25 floor (7.8K) is the level 75% of posts clear.

Recent posts · ranked by views

PostPostedViewsLikesRepostsRepliesQuotesBookmarksER
This is impressive! I cloned my voice in the @cartesia playground from a ~15-second clip. The clone was ready in a couple of seconds. Then I had it speak Japanese, a language I don't speak. My clone now says in Japanese that I can find good papers by combining AI with my experience reviewing papers. I am impressed by how good this sounds. This runs on Sonic 3.6, their latest voice model, which supports 44 languages. You can do the same with any of them. This tech is getting really good, and I think it's worth exploring to make your content more accessible to more people in different languages. Try it here: https://t.co/HFfacNsnak More info about their model here: https://t.co/XWXvD4YSHsfrozen19d ago234K50313225103720.45%
Pay attention to this new wave of System One models if you are building custom harnesses. First Jev. Now, Contrastive Language Model (CLM). CLM is 9x faster than Jev. CLM seems to be a better verifier than Jev, particularly at long-horizon tasks. How do Jev and CLM differ? CLM is contrastive, and Jev is trained with Reinforcement Learning for Calibrated Decisions (RLCD). Jev receives a situation plus predefined questions, and returns typed decisions with probabilities. CLM embeds the situation and candidate actions, compares their similarity, then ranks or selects the best match. The point is that there are several ways to attack this problem, which is exciting. You can see my recent guide on combining System One and System Two models for building custom harnesses. https://t.co/Ccy5o4s6QU
frozen
28d ago
214K
1.6K
148
42
10
2.4K
1.99%
50K likes on this!? Most of what Karpthy has shared is things we have been doing for over a year. No surprises, as many replies are highlighting. But I still think this raises an interesting discussion. Is there a universal interface/langauge that allows more seamless communication between agents and humans? I believe there is. And I think that is what @karpathy is suggesting we think about. Right now, ideas are all scattered. As some of you know, I have been early on LLM Wikis, Artifacts, AI-generated visual explainers, dynamic interactives, etc. Over the past couple of months, I have been experimenting with a new universal interface for seamless human-agent collaboration. So I have a lot to share, but it's still too early, and I frankly haven't found the perfect solution. In short, my current optimal setup (see a simple snapshot attached) ties my high-level agent to highly flexible Notion-like pages that support embedding artifacts, text, videos, visual explainers, and just about anything an LLM can generate. And if it doesn't support it yet, I ask it to build it on the fly. It's crazy sometimes, the features it builds, but this is a co-evolved interface that satisfies both my agent's and my needs. The interface supports code review, note-taking, writing, research, prototyping, mock designing, and so on. In fact, I have now wired my agents to intelligently spit out these artifacts (for lack of a better word), depending on the task/instruction, instead of the default text responses, which are frankly overwhelming and hard to read at times. In other words, as I keep building this "universal interface," it feels less and less like I am chatting with an agent; I'm collaborating with it directly through the artifacts (through comments and suggestions). It allows me to move faster. I can consume and review faster. I am better organized. It feels like a language both the agent and I can speak fluently, so there is very little friction. Listen, this is all a personal experience. I don't think everyone will use interfaces like this. But it's been interesting to see Artifacts gain wide adoption, and now dots have Spaces that look a little like what I built here several months ago. More to share soon.frozen18d ago201K731388069340.89%
Learn to build evals, folks. Highly recommended reading on using decision models for evals. From our internal evals, System One models like Jev are a great fit for LLM Judges. If you want to go beyond Hamel's great write-up, I've also put together a few resources on applying decision models to LLM judges. (bookmark it) 1) Jev-as-a-Judge for Agent Evaluations - https://t.co/SkXGd4uA2J 2) Jev-as-a-Judge: Evaluating Agents with a Decision Model - https://t.co/SqGSzlMFut Bonus paper I would recommend, with a few other ideas that combine System One and System Two models: Jev-as-a-Judge paper - https://t.co/8Oi6oBEt2A2d ago124K1.7K1486473.3K4.19%
https://t.co/Pq7A9zA0Rf29d ago120K1.2K13044132.3K3.12%
https://t.co/j6lRVrWpAZ16d ago90K439642797681.45%
Own your intelligence stack, folks! Start by tapping into the rich knowledge in your agent traces. Then train small, specialized models to fix production issues. @OvermindLab is open source and makes this practical. It builds datasets from your agent's traces, evaluates them against your current model, and trains a smaller specialized model you own.frozen16d ago72K138101722530.58%
This is one of the strongest use cases I've seen for System One models like Jev. Pay close attention if you build custom harnesses. Jev is now available as a packaged router. What is it? You send the conversation to typesafe/jev-router, and OpenRouter picks the model and reasoning effort for each request. Choosing the right model and reasoning effort is an art, and doing it manually is inefficient, to say the least. It makes sense to offload reasoning effort, but you want to do it efficiently, smartly, and cheaply. System One Models like Jev are a great fit for this. Does it work? I tested Jev Router on a small support agent built with a custom harness I built with the Pi SDK. I ran the same 8 cases against a fixed GPT-6 Sol baseline, 32 real calls in total. Both got every case right. The router cost less than half as much ($0.008 vs $0.018) and had a lower median response time (1.5s vs 1.9s). The sample is small, but it's clear that, at scale, this could make a huge difference in cost and efficiency. One of my biggest concerns about Jev-as-a-Router is caching. While it's not fully solved, Jev Router gives me hope for smarter, more efficient routing patterns in custom harnesses that leads to better tradeoffs. I have shared an implementation of a routing pattern I'm excited about here: https://t.co/Ccy5o4s6QU The takeaway here is that Jev is unlocking interesting new patterns (routing, verification, guardrails, dynamic workflows, judges, and more) that you can tap into in your custom harness. I expect this trend to continue growing. And harnesses that are easier to customize, like Pi, will benefit even more.frozen26d ago62K280274253081.07%
This is one of Jev's most impressive use cases. I work a lot on agent evals, judges, and verifiers. I've found that Jev-as-a-Judge improves LLM judge reliability through consistency. Makes it ideal for judges, verifiers, and continuous monitoring.16d ago61K544524039502.59%
Real-time AI characters now move their full bodies and keep listening the whole time. This character was built in the Vivix A1 Playground. You can interrupt it mid-sentence or mid-walk. It changes course without resetting the scene. Objects it picks up keep their shape. @VivixLabs_HQ https://t.co/lmjfgckOr1frozen22d ago60K66571928052.61%
LLMs were just the beginning. Bullish on world models, and I think they are the next frontier in AI. Amazing to see a dedicated world model for engineering this early. SKY-001 against GPT-6 Astra matches accuracy on 3D construction at about 150x lower cost per sample. On BIM, it runs over 100x faster.3d ago50K702622335032.59%
Build your own harness, folks. Reading papers like this makes me realize how underexplored harness engineering really is. The authors find that on whole-repository migration, GPT-5.6 Sol goes from 6.5% to 31.0% when Codex is replaced with the HERMES harness, with the same model and effort setting. The gain comes from the harness. HERMES pairs each repository component with a resident LLM that knows its own code and dependencies. A dependency-aware step decides which components to activate, and a diagnosis step maps test failures back to the components that need changes. Across four software engineering benchmarks, it beats matched baseline harnesses by 12.4 points on average. With strong activation and diagnosis models, Qwen3-8B components come within 4.5 points of an all-GPT-5.6 Sol setup and cut Terminal-Bench 4.0 inference cost by 26.2%. Paper: https://t.co/gfEecJnDPm Chat with Paper: https://t.co/5B7EQjf7u6frozen14d ago48K827100111161.1K4.47%
Keep your agent harness minimal, folks. This paper from MIT CSAIL shows why. (bookmark it) While being very minimal it looks effective and promising for building self-improving agents. They present JAZ, an agent framework with one primitive, invoke. The LLM writes code, can call invoke recursively, and sees all of its inputs and history as variables in the code environment. With prompting only and no memory system, it beats Letta (MemGPT) by 8% at half the cost on the recall-heavy part of StuLife. On AppWorld it beats ACE, a self-improvement method, by 4% at lower cost. Memory and self-improvement are usually built as separate subsystems. Here both are code the agent writes inside the loop, and hooks handle the constraints and monitoring you want to enforce. Paper: https://t.co/qNk9mhY4vQ Explore Paper: https://t.co/rl6KWkpZ64frozen26d ago48K5156679167232.91%
Banger paper from Microsoft Research. (bookmark it) They run 1K+ coding agents at once to test a scalable self-organized multi-agent harness. This is an interesting test because most multi-agent systems today have some hierarchy or structure. Agensh has no central orchestrator. It coordinates parallel coding agents through a shared state instead of a central orchestrator. Each agent gathers context, claims a sub-task, does the work, shares what it found, verifies the result, and merges it, all asynchronously through a shared workspace and a message channel. On the five hardest ProgramBench tasks with GPT-5.6-sol, increasing agents from 1 to 128 raises the mean final test-pass rate from 19.31% to 28.78%. Larger teams also reach a given pass rate sooner. On pandoc, 1,024 agents take the test-pass rate from 33.89% to 55.06%. The authors also report forms of cooperation that the agents start on their own and that become standard practice as the team grows. Paper: https://t.co/gSASQJ2ThT Chat with Paper: https://t.co/Ywv8pkQGVUfrozen25d ago43K3925863175082.42%
MCP won! I started to use Pi heavily for its customisations. This will take it to another new level of what’s possible to build with it.frozen23d ago43K1439921250.68%
Build specialized harnesses, folks. This is a banger paper from Google Cloud AI Research on self-evolving agent harnesses. (bookmark it) They introduce OO-Harness, which raises AutomationBench from 61.6% to 77.4% with the same Gemini 3.5 Flash executor. It also beats Meta-Harness on all three benchmarks they test. More details below. Instead of evolving one global harness for every task, an evolution agent builds a class hierarchy of specialized harnesses from execution records. The design has three parts. > Subclasses inherit the base harness and override only the methods a task cluster needs > Design lessons record which changes helped and which ones failed > A router reads the task and the base run, then picks a subclass or keeps the base result On Workspace-Bench, the score goes from 25.5% to 39.5%, and on Toolathlon-GYM strict pass goes from 31.2% to 37.5%. Spending extra inference on best-of-K sampling or self-refinement does not reliably help. Inheritance also improves results. On Workspace-Bench, harnesses built as subclasses score 4.96 points higher than the same evolved code copied into flat classes. https://t.co/JKZ6y8OXhs Chat with Paper: https://t.co/Vg9NlOnfEZ1d ago39K6828487139554.70%
This is a bigger deal than it seems. I like this push on the Pareto frontier to squeeze as much as you can out of your tokens. Feels underexplored. All labs are quick to launch models, so certain aspects just aren't optimized. These are just a few of the great things you can start doing with frontier open models. Ember-1 produces shorter reasoning traces (40% fewer tokens) without sacrificing performance.frozen25d ago37K250121421371.11%
I agree. Learn the fundamentals, folks! If you plan to be a solid builder using agents, this is a really important point. There is no way around it. I have recently started digging deep into evals, infra, architecture, scaling search, and things like sandboxes @dair_ai using agents, and I can tell you that it's really easy to produce absolute slop. You could be using the most advanced models today confidently and not know that everything is broken because the agents somehow got it to work through some weird hack. Agents don't have the taste of someone with proper domain expertise. And I don't think that's coming anytime soon. But something you learn in algorithms (one important fundamental subject) is that solutions can be optimal or suboptimal. In other words, many ways to transform inputs to get the same outputs. Agents don't give a crap about that and sometimes feel like they are flipping a coin. I can tell you that much. Stay learning, folks. This is the way.frozen26d ago36K428353253892.45%
Recommended. LLM agents love structure, so it's no surprise that a corpus improves agentic search.frozen17d ago35K222151913461.71%
Use of coding agents via CLI has been dead for a while now. We are early on discovering better interfaces but interactions now happen directly with agents and will continue that way going forward. What is working for me is a mixture of high-level persistent agents (like dots) and specialised agents (like codex desktop sessions) that are themselves managed by the persistent agents or accessible at anytime when I need to do focused work. This provides me lots of flexibility and I can leverage a variety of experiences and interactions with agents that range from the simpleness of grok bot to the more involved sessions offered by codex desktop. These again are some of the advantages I enjoy because I have built my own agent orchestrator and harness.frozen19d ago35K17255761621.16%
Own your personal agent, folks. Just built my own little personal agent with @pidotdev's Pi Durable. I wanted to see how quickly I could build an OpenAI dots clone with open-source tools. I am amazed at how quickly I got this off the ground. It's proactive. It keeps working when you're not watching. It remembers you, runs jobs on a schedule, waits for your approval, and uses its own computer. Pi Durable makes that possible. Every step is checkpointed. I can kill the process mid-task, and My Pi picks up right where it left off. Memory lives in durable documents. What My Pi learns about me in one Space survives a crash and carries over to every other Space. Approvals are durable too. It also has its own computer. Each Space gets a full Linux desktop that My Pi controls with screenshots, mouse, and keyboard. The model runs through OpenRouter, so I can swap models without changing anything else. That's all configurable however you like it. If you've been wanting your own personal agent, now is a great time to build one. I am building this as an experimental ground where I can test new ideas and experiment deeply with proactive personal agents. But I plan to open-source it after testing it a bit and adding more interesting primitives. The goal is to make it customizable and easy to build on, so others can experiment with their own personal agent functionalities.frozen17d ago35K387336236033.15%
Recursive self-improvement is one of the most discussed ideas in AI right now. I am particularly interested in how it applies to science. I saw this really cool new paper with neat ideas, especially on hypothesis generation. A good read. Specifically, they propose DiscoveryRSI, which applies RSI to scientific hypothesis generation in four life-science domains. Its 27B model produced hypotheses that match research published after its training cutoff 6.3% of the time, compared with 2.5% for GPT-6 and 2.3% for Claude Opus 5. The system grows trees of hypotheses with multi-agent evolutionary search and turns the chosen investigations into executable tasks. After each run, it assigns credit to the hypotheses that contributed, replays the research harness to find productive strategies and repeated failures, and applies accepted improvements to new investigations. Automated hypothesis quality rose from 4.86 to 5.39 after two rounds, with the largest gains in falsifiability. https://t.co/eKrY7fjoal Chat with Paper: https://t.co/77me5KRe9X3d ago32K395676433102.65%
Interesting paper on agent memory stored as a linked markdown wiki. Lots of great ideas and insights if you work with LLM Wikis. Wikis are useful for agents because each page holds dense text and the links between pages hold structure. WFM is a Wiki Foundation Model trained to use both at once. It turns an LLM Wiki into a graph and retrieves from it with message passing conditioned on the query, so the text of each page and the link structure shape the result together. The team also built a GPU-to-GPU training protocol that trains 10.5x faster, and reports strong results on five agent memory and multi-hop reasoning benchmarks. If your agent's long-term memory is a folder of linked markdown files, WFM is designed for that format. Paper: https://t.co/t3iwlcCFB8 Chat with Paper: https://t.co/4QL4IyKPBWfrozen1mo ago32K487656756273.95%
Banger paper from Google. It's standard practice to sample several agent rollouts and trust the answers they agree on. This Google paper shows that agreement can hide shared errors, while disagreement often points to the correct alternative. VeriHarness turns the same base model into an agentic verifier with two jobs. One resolves claims where rollouts disagree by checking workspace evidence. The other challenges claims that every rollout agrees on and looks for requirements they all missed. Across five long-horizon benchmarks, it gives the best selection scores among the baselines tested. With evidence-backed revision, it adds 6.2 points over a single rollout with Gemini 3.5 Flash and 6.4 points with Claude Opus 4.8. The authors also release about 26,000 rollouts. Paper: https://t.co/jDKhXcz4JJ Chat with Paper: https://t.co/YNW66TlJAMfrozen18d ago31K447695674683.36%
Banger paper from Google DeepMind on long-horizon multi-agent coding. (bookmark it) They report that, to the best of their knowledge, this is the first end-to-end autonomous completion of Minecraft by AI agents. They report that four Gemini 4 Argon agents beat Minecraft end-to-end. They defeated the Ender Dragon in 27 hours and 14 minutes with no human intervention. Each agent controlled one player through the standard Mineflayer library. No one assigned roles, wrote game logic, or scouted a seed in advance. The agents planned, wrote, and debugged every program themselves, and coordinated through in-game chat and a shared memory. Over the run, they wrote 2,783 programs and died 605 times. They split the work, shared what they learned, and recovered each other's items after deaths. If you build agents for long software or enterprise tasks, this is a useful case study. The team had to keep one plan consistent across thousands of dependent steps for more than a day. https://t.co/h1Z87NYKqp Chat with Paper: https://t.co/beTXikB3S8frozen6d ago31K5858599124614.01%
Recommended paper from Sakana AI on recursive self-improvement. They propose an interesting way to scale recursive self-improvement through multi-agent self-supervision. In this line of research, self-improvement loops usually need an external verifier, so open-ended tasks without a checker are left out. MASS removes that requirement. One base model proposes multi-agent workflows, runs them and grades them, and an evolutionary search keeps the workflows that score best. The model is then fine-tuned on its own traces, and the improved model starts the next cycle as a better optimizer and grader. Two cycles on Qwen3.6-27B raise performance per output token from 1.2 to 1.6x on four open-ended benchmarks. A student trained on multi-agent traces also beats a single-agent student trained on 1.4x more tokens. Paper: https://t.co/r0vGMpPMGn Chat with Paper: https://t.co/JQgMHnwgp7frozen12d ago31K4937358105463.83%
Must-read paper from Google on self-improving agent harnesses. If you auto-optimize your agent's harness, your eval score can go up while the agent gets worse on real tasks. This paper shows how to prevent that. Of five harness-evolution methods compared on agentic workspace tasks, RRSI scored the lowest on the tasks it evolved against and highest on all three out-of-distribution benchmarks. Automated harness evolution proposes edits to prompts, control flow, tools and memory, keeps the ones that raise the score, and repeats. The authors show this overfits the training tasks. Meta-Harness reached 93.0 on the Harvey LAB evolve split but gained only 0.3 to 1.5 points on JobBench, GDPval and APEX-Agents. RRSI adds regularization on both sides of the loop. The proposer gets an edit budget that shrinks over time and is pushed toward directions it has not tried. A critic rejects benchmark-specific edits, and a pruner removes edits that are too small, too costly or no longer useful. RRSI scored 90.5 on the evolve split and gained 3.5 to 4.7 points on the three held-out benchmarks. In the ablation, unregularized evolution used 3.80M tokens per trial against 2.42M for RRSI. With Gemini 3.5 Flash, RRSI raised Terminal-Bench 2.1 from 64.6 to 78.7 and carried a 2.2-point gain over to SWE-bench Verified. Paper: https://t.co/SlfjDg96VI Chat with Paper: https://t.co/gBotiH6Jfqfrozen29d ago31K400624895173.38%
Banger paper from Meta and colleagues. (bookmark it) They discuss benefits of giving your agent write access to its own context. In other words, they investigate how effective it is to allow a language model to natively manage its context. Context Language Models keep the context as a file the model edits with Bash, so the model decides what to keep, rewrite or remove. It's a bit different from RLM, for those who are wondering but it pulls an interesting theme. RLMs place a large input in an external variable that the model can read and process recursively, but they do not let the model directly edit its own live interaction context. CLMs instead expose the live context as a read-write file, including information accumulated during execution. Applied zero-shot, this gets 11.4% higher accuracy with 21.5% fewer FLOPs on BrowseComp-Plus than existing context-management strategies. On a 24-hour task where a swarm of agents works across six repositories, it gets 65% more improvement for the same compute. The strategy can also be trained. Online RL improves Qwen3.5-9B on BrowseComp-Plus by 47.6% while using 12% fewer FLOPs. Edits in the middle of the context break prefix caching, so the authors add Suffix Cache Reuse, which cuts server compute by 35% against standard SGLang. Paper: https://t.co/ISob0EHqQBfrozen21d ago30K363473674653.03%
Learn to build custom harnesses, folks! Here is an impressive paper showing the impact of a good custom harness on long-context work. GLM-5.3-Flash goes from 4.4% to 61.1% exact match on OfficeQA Pro V2 with the same model. The authors report that the gain comes from the harness. The model-agnostic harness name is LongHarness, built for corpora of 100M tokens and more. Why it works: Standard agent harnesses keep evidence and conclusions mixed into one growing chat history. Retrieval keeps expanding without closing the actual gap, and corrections stop reaching the answer. LongHarness stores both as explicit state. Open conclusions decide what to retrieve next, and each new piece of evidence confirms or revises the conclusions that asked for it. On the same benchmark, it beats Claude Code by 12.1 points on V1 and Codex by 18.9 points on V2. https://t.co/kkLcbh6dyT Chat with Paper: https://t.co/s0U2Gb8Pfafrozen9d ago29K60295105137455.39%
Banger paper from CMU on harness learning. (bookmark it) Also, pay attention to this important new AI engineering skill of improving agents by editing their harness code instead of their weights. Seeing a huge shift towards this. The authors train a proposer model with RL to read a task, the current harness and an execution report, then write a code edit to the harness. The reward is the score of the revised harness. The solver model never changes. A trained 4B proposer beats its 35B teacher at single-step revision on Reasoning Gym, including task families it never saw in training. A proposer trained on HotpotQA keeps improving harnesses on MuSiQue and 2WikiMultihopQA. Paper: https://t.co/wpOoUqPF6r Chat with Paper: https://t.co/1JHXwPwbhMfrozen19d ago29K469863835844.09%
Great prompt from Anthropic. It helps steer long runs when using Opus 5.5 in Claude Code. Apparently, on long-running tasks, the model sometimes stops to report instead of continuing. I've noticed that this happens a lot with frontier models. https://t.co/mD5MrwO7hEfrozen1mo ago28K438293736083.93%
Own your harness, folks. I feel like System One models take custom agent harnesses to a whole different level. It's not a surprise. This comes down to needs that haven't been met with System Two models alone. A need to be in control of your intelligence stack. A need for more reliable agents. Combining System One and System Two models is way better than just relying on one alone. A need for richer agent experiences (more personal and proactive agents). A need to squeeze out more value per token used. You can check my most recent guide on building a custom harness with Pi and Jev to learn more: https://t.co/Ccy5o4s6QU But let's not stop here. I expect a new wave from frontier labs pushing decision models further. And even better tools to build even more customized decision models on top. There are more layers to this. This is great news for us builders and an incredible opportunity for those who specifically build custom harnesses.frozen27d ago27K205202712551.87%
Highly recommended read. This is the type of interpretability work that we need more of. "We can now understand most neurons in models up to trillions of parameters, and think the outlook for mechanistic interpretability may be far better than we thought." https://t.co/GWZLmk7Ebrfrozen7d ago27K436541543223.11%
Banger paper from Google Research on multi-agent proof discovery. (bookmark it) It's really interesting to see this emerging multi-agent pattern: not enforcing too much execution structure and pairing it with dedicated agents for advising and verification. I think it is generally applicable as well. Great read. Here is how it works: Cogentic runs on Gemini and works on open problems in theoretical computer science, starting from the problem statement with no expert hints. The system works in rounds, and the orchestrator decides how many provers to run in each round. Every prover gets one direction to work on, such as a specific bound or a counterexample search, plus a short briefing that a summarizer agent writes from earlier attempts and verifier feedback. Each summarizer writes its briefing independently, so provers in the same round read different summaries of the same history. Each draft goes through two adversarial verifiers. One checks the draft on its own, and the other reads all of the round's drafts side by side to catch shared mistakes. A draft is accepted only if both pass it. The agents share state through two disk documents. A record logs every attempt with the objection it failed on, and a ledger stores verified lemmas and ruled-out directions. An auditor extracts correct lemmas from rejected proofs, verifies them again independently, and adds them to the ledger. A separate process advisor reads the verification logs across rounds and updates the instructions given to provers and verifiers. The orchestrator and the advisor can't give mathematical opinions, so the provers provide all the math. It produced new results on five open problems in online learning, auction theory, and mechanism design, each checked by domain experts. Most problems took around 100 Gemini calls, and the hardest took around 1,000. Paper: https://t.co/QdCVWgn54S Chat with Paper: https://t.co/vJsdwneoQefrozen20d ago26K221262632962.19%
Banger report on building faster memory for AI Agents. If your agent's memory layer is slow, this design is worth a look. Great weekend read. It proposes a new agentic memory architecture, Jev-Mem, inspired by System-One/System-Two cognition. This builds agent memory 6.6x faster and cuts query latency by 36.7% while scoring higher on LoCoMo. In this system, a lightweight controller makes the memory decisions. It assigns memory types and relations when storing, and handles routing, retrieval budget, graph traversal, scoring and stopping when reading. The LLM is called only for final reasoning and answer writing. On LoCoMo, it scores 0.777 overall with an LLM judge, an 11.0% relative improvement over the strongest baseline. Memory construction takes 158 seconds, and average query latency drops to 0.93 seconds. Paper: https://t.co/mnGFnHIeuqfrozen27d ago26K243224184012.74%
This is brilliant. Recommended reading. It's a great example of how you could potentially build more effective harnesses using System One and System Two models. It explores Jev's potential as a fuzzy linter. Before you dismiss it, check out the setup. I like this post because it provides a few details of a good setup for testing these ideas. It scopes the problem well, like filtering out rules that are easy to judge (i.e., don't require extra context/resoning). That's important because System One models like Jev aren't built for reasoning-heavy tasks. But it doesn't mean that they aren't more deterministic operations where Jev could help scale efficiency and performance. You just have to look closer at the harness. In the harness, a System Two model typically controls all the components and decisions that you could potentially offload to a System One model. It's aligned with what ideas I shared here: https://t.co/Ccy5o4s6QU It's becoming extremely obvious that you don't need frontier models for everything. You are likely paying a premium for something you don't need. I really hope we get more evals and benchmarks to measure these things because there is a thread worth pulling here.frozen25d ago26K127161822211.49%
Lots of JevRAG variants showing up. Interesting to say the least. You can’t really make any conclusions with these scoped tests but it raises discussions and exciting directions to find optimization in current RAG and agentic systems. I have been testing a JevRAG of my own for paper exploration. So far, I have had more success with Jev for reranking and some very interesting ways to search papers combining semantic search and Jev. More on that soon.frozen24d ago25K157202121541.41%
Learn how to build evals, folks! It's one of the pillars of the intelligence stack. Agents are hungry for data and can get it in many ways, but you can't build anything useful in the real world without proper evaluation. Evals make agents useful and let you build truly differentiated products. Invest heavily here. Or reach out if you need help. We are also building evals to measure the frontier @dair_ai.frozen4d ago25K372187783323.24%
Like Jev, I believe RL will unlock several more like this, slashing the cost of critical agent operations. flow-1 is competitive in performance, but a huge cost-saver for finding failures in agent traces. RSI doesn't only apply to general intelligence. It will equally accelerate specialized intelligence.frozen17d ago23K235182212332.18%
Super interesting paper from Google and colleagues. It studies where it's possible to distill an agent harness. With the specialized harness removed, macro task success goes from 23.3% to 44.3%. That is higher than the 41.7% the base model reaches with the harness attached. Harness-Zero uses the optimized harness only during training. The optimized harness and the deployment harness have different action spaces, so a harnessing agent guided by the optimized harness corrects the student's responses in the deployment action space before they run. Those corrected runs become the training demonstrations. Across 28 harness-induced behaviors in knowledge work, tool use and science, 82.3% are recovered on average. For frontier models using the same evolved harness, the agent-as-harness form also beats the code-as-harness form. It remains to be seen how robust the approach is, but it's very interesting to see potential in harness distillation. Paper: https://t.co/AiBm1p17Lp Chat with Paper: https://t.co/Nda4XsZgrhfrozen28d ago23K2994240102572.80%
Banger paper from Stanford, Together AI, and colleagues. Why is this an interesting paper? Today's LLM agents must write out every reasoning step as text before they can search, run code, or take action. Those steps cost tokens and can take time. This paper proposes Hybrid Reasoning Agents, which reason internally in latent space and switch to text only to call a tool or give an answer. A small module called HybridLink moves the model between the two modes, and a working memory keeps both kinds of context across turns. Tool calls stay in plain text, and the latent steps can still be inspected. Training happens in two stages. SFT teaches the model to work with HybridLink, then RL teaches it when to think in latent space and when to act in text. Across 11 benchmarks in math, science, medicine, search, and agentic reasoning, it outperforms frontier LLM agents and agentic frameworks, both text-based and latent, by 10.4% on average. It also uses 35.5% to 54.7% fewer tokens, runs 1.8x to 2.6x faster, and costs 2.7x to 3.7x less. The 14B model beats the strongest agentic baseline by 17.4% on agentic reasoning tasks. https://t.co/l4ImXKNTXo Chat with Paper: https://t.co/NWvHKrWA0ifrozen8d ago23K4345751123884.12%
https://t.co/1w0oH6XJwR2d ago23K21113833332.49%
Banger paper from Tencent on recursive self-improvement for terminal agents. (bookmark it) This is a great read if you are tracking research on training terminal or coding agents with RL Starting from Qwen3.8-27B, they alternate two phases over five stages. One phase fine-tunes on rewritten versions of trajectories that passed the verifier. The other runs PPO with a critic inherited from the previous stage. The four-suite Terminal-Bench average goes from 20.68% to 50.58%, with 86.52% on Terminal-Bench 2.1 and 79.00% on Terminal-Bench Hard. The reward audit is just as useful. Under a dense progress reward, failed runs collected 50.09% of the reward, and 48 failures passed every executed check but still missed the final test. Separating strict success from partial credit cut that share to 28.72%. https://t.co/86s9VwauAT Chat with Paper: https://t.co/mEDr00zTyVfrozen5d ago23K329416433743.56%
Pay close attention to custom harnesses. This is a super interesting paper showing the impact of training inside the harness. Qwen3-14B goes from 22.2% to 54.8% on Spider 2.0-SQLite when it is trained in the same execution harness it uses at deployment. Text-to-SQL models are usually trained to write one static query. Deployed database agents inspect schemas, run probe queries and revise, and the harness for that only appears at inference time. HarnessSQL builds isolated executable databases with hidden answer checks. Teachers run inside the target harness; we keep only verified trajectories for SFT, then apply execution-reward RL. In addition, Qwen3-8B rises from 15.5% to 45.2%, and both models transfer to BIRD-Interact and LiveSQLBench. Paper: https://t.co/FybRkH2gne Chat with Paper: https://t.co/1r2wfudPCofrozen12d ago23K3293960123783.60%
Anthropic just destroyed all proactive agent startups. That's what you'll see on your timeline. That couldn't be further from the truth. Personal agents, unlike other agent types, will exist in different flavors and personalities. It's nice that Claude Duet runs on Opus 5.5. Unlike other agent types, people will really think carefully about which personal agent to pick. Here, people think more carefully about trust, personality, intent understanding, continual learning, etc. The best personal agent is the one you can trust, not just the fastest one that gets the work done effectively. But I think people will use multiple personal agents for different types of tasks/work. I think we are about to see an explosion of vertical personal agents. Which one do I use? I am still testing a bunch of them (dots, Muse, and Grok Bot). But I have actually just built my own with Pi Durable. I like mine better because it is hyper-personalized to how I use personal agents.1d ago23K13115463991.29%
Pay attention to this new wave of VLMs. Glimpse is the first model from @ElorianAI, the brains that led Google's Gemini and Apple's MM-1. Quick test: 10 game boards from an overhead camera. Same prompt, same image, two models, hand-checked. Glimpse: 8/10 boards fully correct. Gemini 3.8 Flash: 5/10. This is what data labeling looks like. Getting the whole frame right, not just most of it, turns raw robot footage into training data. This is powerful Visual Thinking, not just visual recognition translated into text.frozen6d ago22K9415201490.80%
Banger paper from NVIDIA. One exciting trend I am seeing is building verifiable environments for your agents and training the harness alongside the model. Co-evolving the harness and the model is a big part of owning the intelligence stack. And many frontier AI companies have started doing that. This paper shows how this works: VERA turns benchmark trajectories into more than 9,000 restartable sandboxes with rubric scoring, and keeps only environments that run and can be scored from observable evidence. It then updates both the model weights and the harness. A harness edit is kept only if it passes self-tests and adds at least 5 points on the development set. The system rejects a model checkpoint if its score drops by more than 20%. At 9B, the co-evolved agent beats the strongest single-axis baseline by 10.3 points on AutoCoWorkBench and 13.0 points on AutoMedBench. At 27B, it scores 71.6 on AutoCoWorkBench, above Claude Opus 4.8. The environment corpus is open-sourced. Paper: https://t.co/Lt9DXDRQp2 Chat with Paper: https://t.co/fXlmBpm4m6frozen15d ago22K306483942933.15%
Dear Anthropic, please don't nerf Opus 5.5. I have been impressed by Opus 5.5 (using it via API). So I resubscribed to Max and was just mind-blown by how durable it has become. If you used Fable models, you would know how terrible and useless that Max subscription had become. Opus 5.5 is obviously one of the best models as a daily driver, but I didn't expect to get so much from the Max subscription with this new model. Say what you want about Anthropic, but Opus 5.5 is extremely efficient and a really good model. It's exciting. I hope models get more efficient and we can squeeze more value out of every token.frozen19d ago21K2355381211.41%
Brilliant paper on scaling laws for agentic RL data. It's a great paper worth reading if you are synthesizing data for agentic RL. (bookmark it) They fit scaling laws for agentic RL data along three axes: tasks, environments, and trajectories. Across model sizes, families, domains, and both PPO and GRPO, the exponents order the same way, with environments above tasks and tasks above trajectories. The environment advantage seems to come mainly from world diversity, meaning genuinely different settings. Varying the structure of the same world helps about as much as adding tasks. The laws also give each synthesis method a discount factor and a synthetic-to-real ratio at which training starts to collapse. Those ratios differ by an order of magnitude across axes. In addition, allocating the budget with their recipe gives up to 1.7x compute efficiency over the common practice of generating more trajectories. https://t.co/QXcqdIu1D5 Chat with Paper: https://t.co/YcyRKpTtTI2d ago21K235364812882.92%
Just published a few ideas for using Jev and Pi to build a custom harness. This is the first of the series. Some of the ideas include gates, routing, and verifiers. But in follow-ups, I plan to go deeper into newer ideas and benchmark cost and efficiency.29d ago20K140111921721.68%
You can't scrape a dataset that doesn't exist yet. Think about training a model on people performing a specific task, in a specific environment, from a specific camera angle. Someone has to go and capture those examples. This is what makes Kled V3 interesting. Labs can specify the data they need, and Kled can deploy collection tasks to its network of 500,000+ opt-in contributors within 72 hours. 108 configurable templates across image, video, audio, text, and annotation. Contributors capture the data on their phones, with instructions and examples of what qualifies. The opportunity here is a much tighter feedback loop: Identify where a model fails. Turn that failure into a collection task. Get new examples from the real world. Train and evaluate again. The ability to repeatedly collect the exact data a model is missing could be very powerful.frozen20d ago20K10381011061.14%
Learn to build a custom harness, folks. Mods in Claude Code, DeepSeek Harness, and now Pi Durable. What do these all have in common? Increased extensibility. Builders need coding agents and harnesses to be more malleable. Say you are building a simple agentic app for your users, a personalized headless experience, a software factory, or a domain-specific harness; you will quickly find that the generic harness won't cut it. Sometimes you need a different behavior in the UI. Sometimes you just need to easily add a new feature/plugin. Or sometimes you just want to build a completely new experience, like multiplayer support. You can build it from scratch, as I recommend. I mean, it's not that hard. But if you want to hit the ground running, where do you go? Pi Durable and DeepSeek Harness seem to want to solve that problem. Users want more customization in their harnesses. And those builders who understand how to build and maintain one will win big time from this next phase of more malleable harnesses. The battle of the best harness will be very different from the first wave of generic harnesses.frozen20d ago20K175174131912.17%
First AutoHarness, then AutoContext, now AutoCompact. I am seeing a rising trend of work that trains models to natively support more of what the harness does. This work specifically trains agents to decide for themselves when to compact. Reminds me of the new paper from Meta that trains models to manage context natively. But how good is this approach? AutoCompact trains the agent to decide when to compact, what working state to keep, and how to resume. A judge first reviews the base agent's compaction decisions and replaces flawed ones before they execute. The corrected trajectories are used for SFT, then RL with task-success rewards trains coding and compaction together. Pass rates improve by 9.2 points on SWE-bench Verified and 5.0 points on SWE-PolyBench Verified. The gain holds even with a 256K window that never overflows, so learned compaction helps when context space is not the limit. It remains to be seen how this works at scale and how robust it is across harnesses. One interesting note from the authors is that this type of proactive compaction is a form of model-harness co-design: the harness provides the compaction mechanism, while the model learns when to invoke it, what to preserve, and how to continue afterward. Even more interesting is how to combine the rule-based compaction techniques already packaged in harnesses with more model-invoked proactive ones. Paper: https://t.co/6Qyfp1cCrd Chat with Paper: https://t.co/rB69yyjRIPfrozen19d ago20K244394452312.86%
Learn to optimize your own harness, folks. You can squeeze much more performance and value from a harness. This work claims that a self-modified harness matched Codex for $4.03. Specifically, a coding agent rewrote its own harness until it solved 82.0% of Terminal-Bench 2.1 with DeepSeek V4 Flash, without any task reward during the search. The search cost was $4.03. That matches Codex, the top harness in a public nine-harness comparison run under the same settings. SelfSearch has agents modify their own instructions, tools, and procedures using records of earlier self-modification attempts. Each record holds the reasoning, tool actions, and outcomes. The modified agent then becomes the next improver. Population-mean success rises in all six model-benchmark settings, with single agents gaining up to 11.2 points. On SWE-bench Multilingual, one evolved agent gains 5.0 points and spends 38.5% less on tasks both versions solve. Paper: https://t.co/3lVjNentV2 Chat with Paper: https://t.co/HQckqpZAh3frozen17d ago19K264346163583.72%
Must-read paper from Sakana AI on orchestrating coding agents for scientific research. (bookmark it) This is an important read with lots of useful insights if you are building multi-agent systems. I like this work because it presents a concrete multi-agent framework and an end-to-end distributed pipeline that lets human researchers conduct computational research alongside modern coding agents. What they did: A development version of their system helped write a paper that was accepted as a main-conference talk at ALIFE 2026, after human verification and edits. How they did it: The Agentic Scientist splits a project into stages. Task drafting, ideation, planning, sandboxed experiments, baselines and ablations, analysis, and report writing each run on a coding agent that checks its own work. A meta-controller runs many of these agents in parallel and checks every stage against a rubric. The ablations show that reliable autonomy depends on that orchestration layer as well as on the base model. https://t.co/jOzTNOJB13 Chat with Paper: https://t.co/AZhJKXjcVLfrozen7d ago19K334405563644.20%
Banger paper from Google DeepMind on co-evolving an agent's harness and its weights. (bookmark it) If you already log your harness changes and results, those logs can become training data for the model running inside the harness. This is a nice paper from Google showing how to do this co-evolution. Specifically, they propose Evo2, a framework for weight-harness co-evolution. How it works: One policy plays two roles. As a Proposer, it writes edits to its own harness, and as a Reflector, it diagnoses what each edit changed. Those edits and their outcomes then become training data for both roles, using SFT followed by interleaved GRPO. The result: Evo2 reaches 48.3% on SWE-bench Verified and 82.1% on Polyglot, against 41.0% and 62.6% for the strongest baselines. The harness part alone moves a GPT-5-mini agent from 24.4% to 48.3% on SWE-bench Verified. The evolved harnesses also transfer to other benchmarks without further evolution. Paper: https://t.co/ea6xgqY4HY Chat with Paper: https://t.co/ueh8i82GcVfrozen9d ago19K334585443274.10%
Recommended, must-read paper on building recursive, self-improving agent harnesses for open-world tasks.frozen7d ago19K249231403323.30%
Great paper from Amazon AGI on generating RL environments for agents. (bookmark it) Also, pay attention to this important new AI engineering skill of creating RL environments. Seeing a huge shift towards this. The authors propose AutoGym which writes the task, the executable environment and the verifier together, starting from a small domain seed or past model trajectories. Paper: https://t.co/vw11mPBJnn Chat with Paper: https://t.co/mp5TRf7fqvfrozen24d ago19K240372583123.32%
Most of the personal agents I see today are heavily optimized to make us consume more (e.g., buy stuff online, book flights, plan a vacation, etc.). I get that every company building these has its own incentives. To be clear, nothing wrong with consumer agents. They are useful. However, I'd also love to see more human-centric agents that unlock real opportunities, like starting a business, getting a job, running a company, or learning something new. Sadly, today's agents just aren't sufficiently trained and aligned for this. But I think it's worth solving. To be fair, I do recognize a few companies that are trying hard to build more human-centric AI models. And I hope they continue doing so. But who is building personal agents to solve this?frozen26d ago18K865352400.94%
Great paper from Meta Superintelligence Labs on compounding inference for long-horizon agents. One clear approach that's working with large multi-agent systems is persistent memory. Meta researchers describe a concrete approach for this in this paper. You'll find that inference-time compute usually goes to separate attempts, so most runs ignore partial work and failed ideas. Here is how they solve this problem: GitSwarm gives hundreds of asynchronous agents one shared Git repository. Each contribution is a commit that records which earlier commits, on any branch, it builds on. There is no central planner. Each worker decides whether to explore, verify, refine, or combine existing work. With GPT-5.5 Codex workers, it solves all 30 IMOProofBench-Advanced problems in one run. On ProgramBench, it scores 79.4%, against 65.1% for a single Codex agent prompted to keep going at a similar budget. Score keeps rising with more workers, from 73.1% at 100 to 79.4% at 500. Later workers build on 94.7% of published contributions, and 70.1% of failed research commits are cited by later accepted work. Paper: https://t.co/5HrLQncE8v Chat with Paper: https://t.co/2gycpbcaElfrozen11d ago18K2283460111552.76%
Recommended read if you are building AI products. Title doesn’t say much. There is a lot of great insights on what drives AI distribution and how to build defensibility in a super competitive AI landscape.frozen24d ago18K7891011011.13%
Cool paper on catching alignment failures with Jev. The ideas is to ask Jev one generic yes/no question about a model's response, and use its probability as a score. With no extra training, that score separates failures from good responses well, with a median AUROC of 0.886. It is also cheap. On 19 benchmarks, a Jev pass cost $0.30, while the LLM judges those benchmarks use cost $18.96. Researchers built RLCDAlignBench from 44 existing benchmarks across ten failure types, including sycophancy, jailbreaks, deception, prompt injection and reward hacking. Jev, TypeSafe AI's calibrated decision model, answers many typed questions about one input in a single call, each with a probability. On StrongREJECT, it agrees with human labels as well as the GPT-4o-mini scorer does, and ranks responses better (AUROC 0.971 vs 0.929). Where Jev confidently disagreed with benchmark labels, it found label errors in three benchmarks. They find that the question wording matters little. The thresholds do not transfer between benchmarks, and fitting one on 10 labelled items raises median F1 from 0.706 to 0.793. Paper: https://t.co/vUZV4k786o Chat with Paper: https://t.co/3BLEIhdpYyfrozen25d ago18K177173101712.26%
Personal agents gone wrong. As we embrace more personal agents to carry out personalized tasks in the real world, interesting dynamics and behaviors will emerge. I think personal agents as they stand still require careful steering and tuning to ground them in our expectations. In this interesting new work, a personal agent read a user's emails about a $680K 401K and a vested stock grant, then recommended a $601 business-class ticket when a $91 economy fare was available. They ran 325K experiments on 13 models across flights, health insurance and graduate programs. Eight models chose more expensive options for users they inferred were wealthy, with the request held identical. The gaps reach $198 per flight and $284 per month for insurance with Claude Opus 4.8. When a wealthy user asked for the cheapest flight, Gemini 2.5 Flash still picked options $208 above it. Hiding non-financial fields in the profile does not remove the gap, and hiding employment raised GPT-5.5's insurance gap by 40%. Larger models do no better. If your agent has memory or inbox access, the context you give it changes what it recommends. Paper: https://t.co/28aRMxrHKk Chat with Paper: https://t.co/m5G9jrFSnYfrozen24d ago18K10321246721.29%
Recursive self-improvement (RSI) is one of the hottest topics in AI. We are still very early on this. It's a great time to catch up on the latest RSI research. Great companies and products will emerge from it. If you are looking for great RSI papers to understand the history and learn about the latest techniques, look no further. I've also put together a collection of great reads on RSI if you want to track the trend. Here you go: https://t.co/xJ3AvR22wJfrozen14d ago17K157311522032.35%
Banger paper from Meta Superintelligence Labs on self-improving agents. (bookmark it) It's hard to know exactly what drives self-improvement, since so many variables are at play (notes, skills, tool calls between runs, etc.). Meta researchers explore and discuss a way to measure whether self-improvement pays off. They call it agent plasticity. It is the gain on held-out tasks per dollar spent on learning, with model weights frozen and every run starting from a fresh context. They find that the model that performs the best is often a different model from the one that learns most efficiently. In chess, Go, and Hex, Claude Fable 5 reaches the highest final score, while GPT-5.6 Sol gains the most per dollar. In NetHack, only Claude Opus 5.5 improves significantly, by 66 normalized points for about $1,073 of learning. Another interesting finding is that slow learners often ignore artifacts they already wrote. Faster learners reuse their artifacts and still fail when an artifact is low quality. Paper: https://t.co/u4QM1lFNIM Chat with Paper: https://t.co/IKRBNKf0p4frozen13d ago17K269404543013.82%
Great application of small language models from Salesforce and colleagues. AI-scientist pipelines are well known to struggle to propose novel and diverse research ideas. This team trains an 8B reasoning model to write several research ideas in one generation from a set of source papers. The reward scores the whole set, weighting each idea's quality by how far it sits from the ideas written before it. The model learns to move each new idea away from the earlier ones. On their contamination-free benchmark, the 8B model scores higher on creative utility, a measure that combines quality and diversity, than Claude Opus 4.8 and GPT-6 Astra. https://t.co/U75ba31DV8 Chat with Paper: https://t.co/sMFyV5EZuMfrozen5d ago17K194284181832.72%
Highly recommended read from Google DeepMind and colleagues. There is a lot of interest in self-evolving agents, but they're hard to get right. This work highlights how important and difficult memory and retrieval are in agents. The big question in this paper is whether self-evolving agents keep learning without forgetting. They tested ReasoningBank, GEPA, and Meta-Harness on four task streams that drift in topic, workflow, or domain, built from BIRD, BFCL, and ScienceWorld. The weights stay frozen, and the agents adapt by rewriting prompts, memory, and skills. Self-evolution improves overall scores on most streams. But more adaptation consistently means agents forget more. Methods that rewrite a shared prompt or workspace make global edits, so they interfere with unseen tasks and overwrite older knowledge when the stream shifts. Memory retrieval keeps updates more local, but it still pulls in memories from other domains and collides when similar queries need different workflows. Static benchmarks hide both problems, so test self-evolving agents on a stream that changes over time. https://t.co/eO28Lcsj9M Chat with Paper: https://t.co/Jhe28VZ8dj2d ago17K334585602974.51%
This is what the future of coding looks like. Multiple AI models working in the same Codex session, with Jev routing them to tackle different parts of the task. Here, I had it build a full-stack demo app: > Opus 5.5 planned it. > GPT-6 Astra built the backend. > Kimi K3 built the frontend. > DeepSeek handled testing. > GLM 5.3 Flash wrote the docs. Jev handles the switching. I never touched the model picker. 40% cheaper, 15% faster for the same build.24d ago16K111102421321.72%
Banger paper from Microsoft and colleagues. We talk about human taste being important in this AI era. But for recursive self-improvement, an agent's taste also matters. The big question is: How often do frontier agents pick the better direction at a decision point in a long task? It's apparently just under 60% of the time, according to Taste-Bench. In this benchmark, each question shows a point in a trajectory where several directions are open, and one leads to a better outcome. The forks are mined automatically from parallel attempts and detours in engineering and research runs. The best model answers 59.7% correctly. Forks whose deciding evidence appears later in the trajectory are much harder, and a larger reasoning budget does not raise accuracy. The authors then distill a teacher's judgment of the outcome into a student model, which improves end-to-end success on held-out SWE-bench Pro tasks. Paper: https://t.co/uLoZWXiepY Chat with Paper: https://t.co/KUUToNZ8kBfrozen27d ago16K185232471872.65%
// Latent Semantic Memory for LLMs // This paper presents some really interesting findings about agent memory. They find that sixteen memory tokens beat full-context inference on FactKG while using 108x fewer tokens for the source documents. Approach: Documents are encoded once into reusable embeddings. For each query, a learned module gathers the relevant information from the whole collection and writes it into 16 tokens in the input space of a frozen LLM. Results: It beats Mem0, MemoRAG, and other baselines on five of eight benchmarks. On FactKG, it scores 84.33% against 80.43% with the full context. It also works as agent memory. In TextWorld, using the agent's own history as the collection raises success from 18.6% to 24.2%, and the agent revisits fewer rooms. It also seems that it could help frozen LLMs accumulate and use knowledge effectively across long-context interactions. This is applicable to building continually learning AI systems. https://t.co/C3fypjXicf Chat with Paper: https://t.co/GkrKKw6HpZfrozen4d ago16K213365532383.42%
Generally, good advice here. Systems thinking is at the top of my list too. That’s how you interact and build effectively with agents today, by thinking of the subparts of the system. I would add understanding how to measure outcomes and build evals. It comes with experience but it’s something you want to apply early on. If you are not objectively measuring what you build it’s hard to know if you are making real progress. Use agents to help you get started but always look at the data and traces.frozen3d ago16K236204222203.27%
Don't sleep on using Jev-as-a-Judge for agent evaluation. This is one of the most impressive Jev use cases I have found so far. Jev is a natural fit as a Judge, but it doesn't mean you use it everywhere. Similarly, you shouldn't use frontier models for evals everywhere. I'm running lots of tests on this atm, but early results point to an optimized flow (balancing accuracy and cost) that combines Jev and frontier models. Concretely, use Jev in high-confidence situations, and escalate to a frontier model (GPT-6 or Opus 5.5) in low-confidence verdicts. Entire write-up coming soon. Let me know if you have questions as I build the full guide.frozen29d ago16K165164021512.37%
Nice paper on scaling data environments from scratch. They built an engine that creates data-analysis environments from scratch. They apply RL, which adds 20.9 points on DABstep-Hard. Data World first writes down the mechanisms that generate a dataset, then produces the files, tasks, and grading rubrics. Because the generating process is known, open-ended findings can be checked, and independent audits confirm each answer can be recovered from the files. Training Qwen3.6-35B-A3B with RL on 2,135 of these tasks, with no distilled trajectories, also adds 14.28 points on DAComp and 12.13 on JobBench. The benchmark finding is just as useful. Ten frontier models compute the right numbers more often than they draw the right conclusion from them. https://t.co/topvSPDLz9 Chat with Paper: https://t.co/ZqLEWkAw4Yfrozen4d ago16K167163251652.47%
1M-token, multimodal decision models are here! But there is more to this one. You can classify, extract, and summarize with 1M tokens of text and image input. ScaleDown's Constrained Decoding Models read the input once and point straight to the answer, with no token-by-token generation. 200x faster, 467x cheaper than frontier LLM APIs.frozen9d ago16K13892231492.06%
Cool to see classification models resurge in the era of agents. The effects of Jev are insane. Also very cool to see our emotion dataset built at @dair_ai used to test this new model, Julia-1. I spent my entire PhD building efficient classifiers from scratch (from graph-based to deep learning), so it's exciting to see this trend again. So much nostalgia. More importantly, if you build agent harnesses today (where all the edge is now that model capabilities have clustered), it's worth spending time running experiments that combine System One models and System Two models. You'll realize how many problems a great classifier (what I now call a System One model) can solve. Hybrid systems have historically performed better on domain-specific problems, and that still holds, even with general-purpose frontier models. Build your harness, create the evals, and run the experiments yourself. You will learn so much and gain great insights to take advantage of these advancements. This is a great example of what open-source enables. This is a work in progress. The cloud GPU spending on Julia 1 training and experiments was about R$540 (US$104.08). Will we see a new type of frontier lab focused on these models?frozen25d ago15K11812301591.42%
Small models are getting really good at reasoning. It's exciting because SLMs can unlock so much at the harness layer. TwIL-LM3-Pro from @thewebAI has 3.6B parameters and runs locally on everyday computers. It scores 95.4 on BIG-Bench Hard, well ahead of Qwen3-8B at 63.7. I like their post-training recipe. They fine-tune on formal logic, merge the weights back toward the base model, and then run RL against a programmatic verifier. Logic scores go up, and general reasoning holds steady. Great to see more of this work released as open source.frozen21d ago15K237201531812.95%
More agents don't mean higher performance. There is a coordination bottleneck to consider. Not to mention the unnecessary costs. So how many of a multi-agent team's actions actually help it finish the task? In this AgentWorld paper, fewer than a third. AgentWorld puts 3 to 20 LLM agents with different roles into a game sandbox for tasks that run 50+ rounds. Agents can't see each other's internal state, so they have to coordinate through messages and shared plans. Gemini 3 Flash has the highest task success at 52.0%. Coordination tasks are the hardest category, at 12% success, and common failures include communication breakdowns, role confusion, and lost shared plans. Paper: https://t.co/nJCElOemGC Chat with Paper: https://t.co/vRSVPhkPYCfrozen21d ago15K184353891472.72%
Recommended paper. Research on effective agent collaboration/communication is lacking, but self-organizing agents seem much more capable than previously thought.29d ago15K8110110981.32%
NEW: Microsoft also releases its System One model, Microsoft-Decision-1. What a crazy effect Jev has had in the space. Also, a very cool application of decision-making models to power LLM judges and improve scientific discovery pipelines (e.g., screening candidate hypotheses). I'll add this model to my ever-growing list of eval runs. In my ongoing evals, consistency and more complex decision-making are two areas where I see these decision models struggle. I'm also exploring a bunch of science applications at @dair_ai and will share when ready.frozen13d ago15K13892231021.84%
Recommended paper integrating decision models like Jev into harnesses.frozen5d ago15K819821601.75%
Great overview of context compression in LLM Agents And one interesting, unexpected finding. If your compaction policy is tuned to cut tokens, it may be making your agent run slower. Great study from UT Austin on context compression in coding agents. They ran nearly 35,000 agent runs on SWE-bench Verified and Terminal-Bench and varied three decisions separately. These are how context is compressed, when compression triggers, and how much is removed. On Terminal-Bench with Qwen, policies that use about a third of the tokens can take 20% to 80% longer than keeping full context. Step-triggered policies cut the most tokens per step but need 10% to 27% more model calls. Threshold-triggered policies cut tokens by 22% to 55% with call counts close to full context. Results also differ by model. A policy that works well for Qwen drops Devstral to 38.7% and makes it slower, so measure latency and cost per model before choosing one. Paper: https://t.co/1v9eR7DLAn Chat with Paper: https://t.co/tGi6A2YIU8frozen18d ago15K172316371412.80%
What comes after loops? Will try to live tweet this, at least the key notes. https://t.co/SHXo9Fw7Vgfrozen23d ago15K757123531.02%
Pay close attention to agentic search. LLM agents get much better at specialized tasks when you give them structured, expert knowledge. Xerj Corpus Hub lets you do this for any agent with no retraining. You connect a ready-made corpus of docs, guidelines, or real source code, and the agent searches it while it works. It's free, open source, and runs locally. https://t.co/GuQtX1ThuDfrozen9d ago15K162132402372.99%
Opus 5.5 feels more like Opus 4.5. Is it just me, or did it get a lot better at writing? Claudese feels like it's mostly gone. Responses also feel easier to consume.frozen1mo ago15K97331190.97%
Introducing MCP tools for @dair_ai. Connect the MCP to your Codex/Claude/Grok Bot to discover and explore the top AI papers. We carefully curate the papers, so you can expect mostly bangers. The index has all the papers I have featured on X over the last couple of years. Get started: https://t.co/Sg7OaryRY5 > find and summarize awesome papers > explore our harness engineering collection > find new methods to experiment with > find sota results around AI topics > generate literature review/surveys > build your personal collection > visualize papers > and much more The demo shows how to install it, including some ideas inspired by @karpathy for visualizing a paper. We are just getting started. Reach out if you have feedback. We are releasing a few benchmarks with this in the next couple of weeks and will ship regular additions and improvements. Stay tuned!frozen15d ago14K9116264781.50%
Recommended NVIDIA paper on memory-efficient post-training RL. A very interesting technique for scaling RL for LLMs. The main takeaway is that you can update all of a model's weights with RL and still use less memory than LoRA. At 1.5B, LoGRA averages 7.2 GiB of training memory, while rank-256 LoRA needs 13.2 GiB. Pass@4 is about the same (81.8% and 81.4%), and Pass@1 is about 2 points lower. If GPU memory is what stops you from running RL on a bigger model, this method is worth a look. LoGRA never builds a full gradient for the weight matrices it compresses. As microbatches accumulate, each weight gradient is compressed with a rank-256 random projection that is redrawn every step, and the update is rebuilt from two small matrices. Before each step, it predicts how much the policy will change and scales the step to stay within a KL budget. Average training memory per GPU drops 45.7% at 7B (31.8 to 17.3 GiB), with peak accuracy close to dense Adam (72.3% and 72.5% Pass@1). It also trains a 27B model for more than 1,100 steps on a single eight-GPU node, where dense Adam runs out of memory. Paper: https://t.co/I8y1S1Pn4x Chat with Paper: https://t.co/CNjgGLiJS6frozen10d ago14K182313051532.81%
New paper from Microsoft and colleagues on evolving agent harnesses. It's a really cool idea to evolve a harness from published research. Something I have also been testing for the past couple of months. ScholarEvolve proposes harness changes based on published agent research rather than the agent's own failure logs. It splits the harness into modules for tool use, memory management, and task execution. It runs topic modeling over recent papers to identify distinct improvement strategies for each module, then implements and tests combinations. New papers can be added over time. With the model held fixed, Qwen3.5-27B goal completion on AppWorld Challenge rises from 49.6% to 63.6%, and GPT-5.4-mini on Tau2-Bench Telecom rises from 72.7% to 81.9%. Paper: https://t.co/ZC280LPsV0 Chat with Paper: https://t.co/CLtaoFYn1Ifrozen19d ago14K193332971753.16%
Surprised that so few people are talking about this. Persistence has become a core capability of recent models (Opus 5.5, GPT 6 Astra, Gemini 4 Argon,...). I was just reading through the Gemini 4 Argon Minecraft paper details and noticed that no roles were assigned to the agents. Somehow the four agents completed the goal after 27 hours, 2,783 programs, and 605 deaths. They recovered items, revised strategies, and shared lessons. I think this is a good example of persistence helping them eventually succeed. Some more thoughts for discussion: So this is not just about hooking an agent up to an external memory system. It's also not about loop-prompting it with /goal, although that is one of the most effective techniques right now. This is about the model's native ability to choose what to persist, and how and when to use it. It unlocks long-horizon capabilities, as agents often execute more complex tasks more effectively through persistence. So it's less that the models got more intelligent and more that they got better at coordinating and planning for more complex tasks. We are in the early days of this. But this is an important capability to optimize to really unlock AI for scientific discovery. This is also important for building more personalized agents with identities and roles that can track and follow human preferences well and learn continually over time. We also don't entirely know how to quantify and measure it as a field. Reward hacking combined with persistence can also enable weird, unsafe agent behaviors. Just a few quick thoughts, and I hope we can start deeper conversations about this. I am tracking this space closely and will be sharing research on the topic as it surfaces.frozen6d ago13K10714353821.80%
Build for the agentic era. The future of how we build is being redefined, and it's exciting. I haven't seen much innovation in this space, but Base Code (from @Base44), with its focus on cloud and collaborative tooling, makes a lot of sense. And it's model-agnostic too. Recommend reading if you are a software engineer.frozen24d ago13K52280400.77%
Not so impressed with the performance, tbh. I suspect many will feel the same given what we have recently seen with Kimi, Qwen, DeepSeek, GLM, etc. But you know what. That's okay! What we need is more efficient, diverse, customizable frontier open-weight models. It's great to see Mistral back in the game. I like what I see in the multimodal capabilities. That's where I think Mistral Large can stand out. Cyber could also be an interesting direction for them.frozen16d ago13K914240181.03%
Must-read paper on the effectiveness of multi-agent systems and persistent memory. We have recently seen many viral examples of multi-agent systems scaled via persistent memory. This work takes a closer look. They test one three-tier chain-of-agents system. Splitting long-context work across agents cuts peak KV cache to 14.3 MiB per query, against about 35 MiB for single-pass and RAG baselines. The persistent trace-recall tier adds 0.368 MiB of peak cache and changes accuracy by +0.015, with a confidence interval that crosses zero. The authors explain why. Single-question benchmarks give each item its own evidence and score it independently, so stored traces have nothing useful to recall. Paper: https://t.co/E0kYC2PFec Chat with Paper: https://t.co/50n5Bst9XAfrozen10d ago13K157284821703.11%
OpenAI announces dots. I have been testing it for the last few days and it’s something special. Proactive capabilities are really great. I used it to organize my entire DevDay from travels to session/meetings setup. It has its own computer. Has come great note taking tools for organising work and it’s only going to get better for collaboration. A lot of work ahead but it’s already fun and useful.frozen23d ago13K6710222140.91%
Interesting work from Anthropic. They reframe MLP neurons as a stronger basis for interpretability than commonly assumed. It is standard practice to study circuits with transcoders, because individual MLP neurons are considered too polysemantic to interpret. They find that many active transcoder features in late layers are close copies of single MLP neurons. They measure this across 13 open-source transcoders. These neuron-aligned features are also the only ones with a stable link to the underlying MLP. The other features change more between training runs, don't match the MLP's Jacobian, and move less from their random initialization. That makes MLP neurons a stronger starting point for interpretability than usually assumed. It also means the non-neuron transcoder features need better validation before they are used to explain model behavior. https://t.co/JBiBt9yeVh Chat with Paper: https://t.co/0Rwb1GVLH82d ago13K247351411823.78%
Brilliant paper on scaling research agents from UC Berkeley. (bookmark it) They propose an unexpected and interesting way to scale automated research with agents. It's common to see research agents keep their full history and iterate on it to solve a task. After a while, each agent anchors on its own past attempts, makes smaller edits, and extra budget stops improving things. So this work induces forgetting in the agents. Periodically, they discard each agent's session and the candidate history, and then restart from the best candidate found so far. Across heuristic programming, kernel optimization, and model training tasks, this gives up to 15% over parallel agents and up to 39% over a single agent with the same budget. https://t.co/90xDBVtcrW Chat with Paper: https://t.co/ZZnRj0A39zfrozen7d ago13K148253331452.80%
Build your own harness, folks! One of the biggest benefits of building my own meta harness is how smoothly I can switch between models. I feel like this is becoming more important as model release times shrink. RSI will only scale releases. This has been one of the best decisions I made since last November. Switching models can really break things, but it shouldn't. If it does, your setup isn't ideal for this insane release cycle we are in. You need to fix that now.frozen21d ago13K994382611.63%
Is harness engineering dead? Absolutely NOT! Like the "prompt engineering is dead" and "RAG is dead" claims, this is just another misinformed claim used for clicks and ragebaiting. Mostly, it stems from a few recently published papers with poor experimental setups that focus only on SWE on already benchmaxxed SWE benchmarks. If we are talking about SWE, of course a lot of the harness stuff is getting absorbed by the model. And that's great. Thanks to better approaches for co-evolving harnesses and models. But in reality, you still need a stable, minimal harness to get high-quality results reliably and affordably. The big issue I see is that we assume SWE won't evolve. It already has, and as a result, teams are finding they need specialized harnesses to improve code review, cybersecurity, and work in more complex environments (multi-agent systems, sandboxes, etc.). The harness isn't just about prompts; it's the scaffolding that makes AI models work in the real world, covering security, safety guardrails, authorization, coordination, memory, scalable and new infra, efficiency, costs, and more. If you have been in computer science for long, none of these stay fixed. So how can anyone claim the scaffold/harness isn't needed? And worse, how can anyone claim these AI models will know all that, much of it proprietary and custom to every problem and domain? It gets more complex when you apply agents beyond SWE; domain-specific harnesses are popping up everywhere. These require entirely new ways to optimize context, memory, guardrails, security, and more. To make my point even clearer, look at the new wave of proactive and personal agents (Muse, Grok Bot, dots, Instinct, etc.). They involve clever, complex harness engineering. I think we will see vertical personal agents too, so the harness will get even more specialized. I can go on and on. I'd love to hear what other serious builders here are seeing.2d ago12K687534331.33%
Pay attention to System One models if you are building custom harnesses and agentic workflows. Great to see DSPy integrate Jev and now support System One models for calibrating agentic workflows. I'm amazed at how naturally System One models like Jev fit into these frameworks. I am using Jev to build my own custom harnesses like here: https://t.co/Ccy5o4s6QU So far, I've had great success with guardrails, routing, and verifiers. However, I am starting to see other interesting applications in other parts of the agent harness, like: > structuring skills and better optimizing them > more efficient tool calling for agents > dynamic workflows (i.e., dynamically generating harnesses with orchestrator + subagents patterns) that leverage Jev-like models for more optimally structuring outputs/information traveling between these workflows > enhance coordination between agents > dynamic and smarter loading (like tools and MCP) of system prompts, and really, all areas of context engineering that benefit from smarter and faster decision-making capabilities > RLM + System One Models And the list goes on and on. Just a wonderful time to explore new ways to work with agents.frozen27d ago12K8411181891.64%
Same here. I used the terminal for a long time, and I loved it. Agents have changed all of that. I didn't imagine that that day would arrive, but I have completely moved away from the terminal. You can delegate so much more of this stuff to agents now. And today there are undeniably better interfaces for working with agents. But that experience with terminals has made me a better systems thinker, which I have used in areas like designing useful sandboxes and interesting abstractions on top of them.frozen18d ago12K662240391.06%
Banger paper from Google DeepMind on open-ended math discovery with agents. This is one of the early, exciting works I have seen on autonomous AI-powered discoveries. (bookmark it) A Gemini-based agent found and formally verified new acceleration rules for eleven unresolved Turing machines in BB(6), the current frontier of the Busy Beaver problem. Busy Beaver is uncomputable, so no fixed list of problems exists. The agent has to pick machines, spot patterns in their execution traces, and propose compact rules that summarize many steps at once. Each proposed rule then goes through informal reasoning and a Lean proof. Only proved rules are kept, so the result does not depend on trusting the model's reasoning. The authors provisionally call these machines Cryptids. Their runs from a blank tape now have exact, compact descriptions, but whether they halt is still open. This is a good example of an agent producing new, formally checked mathematical results on problems it chose itself. https://t.co/uZVecWK8t0 Chat with Paper: https://t.co/1nzoUaCkEJfrozen3d ago12K166214001493.05%
Real talk. PRs suck in the age of coding agents. There must be a better way. I agree with this post, agents are so capable now. Let them do the heavy lifting. They can generate (on demand) the right personalised views and artifacts that make things move a lot faster.frozen6d ago12K743371261.17%
AI is creating new jobs, and I expect many companies to start hiring Automation Engineers. The role needs someone who understands how a team really works and can build the automations it needs. Jake explains it well in this article. At @getserval, Automation Engineers embed with IT, HR, Finance, and Legal teams and build AI agents directly inside each customer's Serval setup.frozen21d ago12K652110731.25%
Bullish on this trend of making post-training more accessible. A new post-training era is upon us. If you work on agentic RL, long-context tasks (a big focus today) are expensive, inefficient, and don't scale well. I've been diving into RL envs and evals for long-context tasks, and I can see this being useful. In agent RL, rollouts use most of the tokens. Every turn re-reads the whole growing context, including tool outputs, files, and earlier turns. Tinker just cut the price of those tokens. Long-context prefill and sampling now cost the same as short context. This means that evaluating your trained models on long inputs also gets cheaper. Huge win here. I believe RL will keep unlocking specialized models that slash the cost of critical agent operations. Cheaper long rollouts make them more practical to build. Own your intelligence stack!frozen13d ago12K11216302822.02%
Great paper for all you pre-training and post-training nerds. The surprising finding is that pre-trained LLMs, equipped with a light inference harness, can serve as capable agents and can even surpass post-trained counterparts with a sufficient test-time budget.frozen19d ago12K13510171892.11%
Agent-to-agent communication with a personal agent on top is wild. Wow. I underestimated how well models like Opus 5.5 can coordinate already. This opens a whole new set of experiences with agents. But it takes a bit of tuning (automations, webhooks, system prompts, tooling, etc.) and retrieving context at the right time. I don't have a perfect setup yet, but I feel like I'm already living in the future. My agent team can do work (higher quality than I could imagine) at speeds I can't even keep up with. It's crazy how fast things have moved in about a year. I started with individual Claude Code sessions. Then, just a couple of months later, I moved to multi-agent systems with subagents, parallelizing work. Then, a couple of months ago, I built a persistent, higher-level agent team, similar to Grok Bot. Of course, with my own agent orchestrator and a team of like 8 specialized bots (CFO, CSO, CTO, CMO, CCO, CoS, etc.) Then, about a month ago, I started playing around with agent-to-agent communication with a personal agent. And I can tell you, most of the apps like Code and Claude Desktop are way behind. But they will catch up. This is why I believe we should all be building our own agent orchestrator/harness. You need to personalize to what works for you, which creates an important edge. You can start with a fork; who cares. Start somewhere. It's just mind-blowing to see agents coordinate and do things on their own that would take a village or an entire org in some cases. Just sharing my raw thoughts here, but things will dramatically change again in the next couple of months, leading into next year. Sharing a snapshot of one of my personal agents, Clio (Chief Content Officer), doing research all day and putting together high-quality content to inform me on things I am interested in and things I am writing about on X.frozen15d ago12K103124151272.43%
Really nice paper on harness-aware distillation for small language model agents. (bookmark it) It's a super interesting distillation framework for small language model agents that are deployed with a harness. In simple terms, you run the big model with and without harness info, then train the small model on cases where its action changes. Researchers show that adding the harness to on-policy distillation raises how often the student uses harness information on ALFWorld (65.7% to 73.1%) but leaves success flat (43.1% to 43.5%). Their method, Harness-Aware Distillation, queries the same teacher with and without the harness information and trains the student to prefer the action chosen with it. A filter drops pairs whose preferred action contradicts the harness records. The method uses no task rewards or success labels. The student reaches 63.4% on unseen ALFWorld tasks, compared with 47.0% for the best baseline, and exceeds its 8B teacher. It also escapes 59.7% of stalls, while the baselines stay near the untrained student's 46.8%. Paper: https://t.co/cU5Beyu9Pi Chat with Paper: https://t.co/xTjXpmJAiIfrozen16d ago12K188301612063.75%
New impressive work from Microsoft and collaborators. And it's one of my favorite recent topics: RL environments. They show that Qwen3.6-35B-A3B goes from 41.1 to 53.3 on Terminal-Bench 2.1 by training on environments it designed itself. AutoEnvScaling gives environment design to a proposer agent that works in a terminal, looks at the rollouts of the model being trained, and builds new RL environments from them. The trained model's next rollouts guide the next round. Building environments in a terminal raises the share of valid environments up to 25 times over prompt-only generation, at up to 75% lower cost per valid environment. With GPT-5.6-sol or Claude Opus 5 as the proposer, Qwen3.5-35B-A3B scores 4.3 and 3.8 points above matched static environment pools. The gain shrinks when the proposer cannot see solver feedback. The pipeline also produces environments for software engineering, computer use, and GPU kernel work. https://t.co/o4d8tfErYD Chat with Paper: https://t.co/jC3MRnbAbkfrozen8d ago12K142222621082.57%
Don't sleep on domain-specific harnesses. Coding agents are great because of their harnesses, but they aren't built for creative work. Creative work needs its own harness. Voyager looks great. It's an open harness for video, graphics, and games. The agent works with your files and drives apps like Blender, DaVinci Resolve, and Unity right on your desktop. Bring Opus, Astra, or DeepSeek. Excited to try this one.frozen14d ago12K9952021262.17%
Goodbye, context engineering; hello, cache engineering. Jokes aside, this paper takes an interesting approach to the problem.1d ago12K174302221733.45%
Mind-blowing to see these Pareto improvements in performance and cost. More than anything, it's crazy how quickly these improvements are landing across providers. This matters a lot to builders. https://t.co/zkjDvkaTLpfrozen5d ago11K10412212321.49%
Once coding agents can write the whole app, deploying and running it becomes the bottleneck. InstaCloud from @insforge lets agents deploy straight to a serverless cloud that scales with traffic and down to zero when idle. Branching is great for agent workflows. An agent can copy an entire service, data included, in seconds and test changes there before touching the live app.frozen23d ago11K414111220.69%
Great feature in Codex. I have had my own composer prediction tool in my agent orchestrator for months. It's tunable and adapts to my preferences as I use it more. In fact, I use smaller models for this, like Haiku and Luna. It's a nice quality-of-life little feature that makes agents a bit more proactive and boosts productivity.frozen13d ago11K642240431.18%
Another great application of System One models. I think much of what we call hallucination in web agents comes from misreading the page. When a page is flattened to text, a small ✗ in a pricing table disappears, and the agent answers incorrectly with full confidence. Expand Fetch reads the rendered page, returns the underlying data as clean JSON, and keeps a replay of exactly what your agent saw.frozen8d ago11K565195991.63%
Good paper on credit assignment for agent RL. The main finding is that you want an LLM judge to choose where to check a trajectory, and the rollouts to decide how much credit that step gets. GRPO gives every token in a trajectory the same advantage, so the training signal cannot tell the decisive step from the rest. ProVer has a judge compare successful and failed rollouts and name the segment it thinks caused the difference. It then samples continuations from just before and just after that segment and uses the change in success rate as the segment's advantage. Across ALFWorld, WebShop and SearchQA, this gives relative improvements over GRPO of 9.91% for Qwen3.5-2B and 7.12% for Qwen3.5-4B. It still helps when the judge is a smaller model. Paper: https://t.co/Nk2DIIMQrF Chat with Paper: https://t.co/tfVyS0SRfQfrozen21d ago11K12315202892.21%
This is impressive! Face-to-face conversation is where AI interfaces are heading. @tavus gave me early access to Griffin, a video-to-video model that watches and listens as you talk. It can interrupt you, take an interruption, and react to what is happening in your room. On NVIDIA's Video Full-Duplex Benchmark, it scored 3.83 out of 5 on generation, against 3.92 for a real human. Worth checking out all the details below.frozen21d ago11K45180180.64%
Recommended reading and a super-interesting benchmark for measuring continual learning capabilities through memory. Agent memory is largely unsolved. As agents advance in capabilities like long-horizon work, it's unclear which models or harnesses work best, or which memory systems and types to use. Opus 5.5 at the top is not a surprise. But it's exciting that open-weight models like GLM-5.3 perform so well on memory generation. My big question is how this (better memory systems and persistent agent capabilities) translates into more performant, efficient RSI agents and harnesses.23h ago11K9311412701.95%
Hours researching the rise of Microsoft. 5 min feeding that research into Pexo. 10–15 min later, a finished 30s video. The collage work is what really got me. Pexo took the research and built the whole visual treatment around it—cutouts, stickers, supporting images, all of it. Nothing felt randomly added. It all followed the story. And this was the first take. If you make history or explainer content, I think the workflow just changed. @Pexoai_officalfrozen27d ago11K60540861.40%
I've been researching and writing a lot about persistent agents, how to build them, and the tools I've been using. Here is the latest. More to come.2d ago11K93122101252.27%
I think the consumer breakthrough for agents comes when they start the conversation themselves. Lucas lives in iMessage and WhatsApp and texts you first when something needs handling, like a refund on a canceled flight or a booking you keep putting off. Proactive capabilities are great.frozen22d ago11K351160140.60%
Sharing some notes from OpenAI DevDay in parts. The first part matters, since it's on everyone's mind. How do you optimize for cost and performance? If you are building agentic applications today, these are 4 of the most important levers you absolutely must try: - prompt caching - reasoning effort - programmatic tool calling - batch request The OpenAI dev docs have plenty of information, but these alone are good starting points for optimizing your agentic applications. Familiarize yourself with these and measure them with evals.frozen20d ago11K436192260.88%
It is already happening at a subscale indeed! And the more you own your intelligence stack (model, harness, evals), the more scale you can unlock. At this pace, recursive, self-improving AI companies will start showing up everywhere in different forms. But don't wait around. Start now by recursively self-improving a feature or function of your business/company, and build the processes/evals you'll need when those more advanced models and agents arrive. This lets you scale further, ideally closer to, like, frontier labs (at least that should be the ambition).frozen25d ago11K37290410.82%
Pay attention to decision models. From my recent tests, decision models can be extremely transformative for businesses. Many calls in an agent workflow are structured decisions, so paying frontier prices doesn't make sense. SubQ Snap from @subquadratic is trained with RL for those calls. It holds 384K tokens of context and leads Jev on 13 of 16 enterprise benchmarks, so it can decide with a whole repo or a stack of contracts in view.frozen10d ago11K684120671.39%
Pay close attention to harnesses, part 2. Last week I shared a paper where a model more than doubled its score after training inside the harness it runs in. The same idea applies at a higher level. Claude Code and Codex are harnesses for a model. Once a team runs several agents, the company also needs a harness for its people and agents. Kylon is building that layer. You bring the agents your team already uses, like Codex and Claude, into shared rooms where they work alongside everyone.frozen9d ago11K9041411232.14%
This will be a fun one. @MilksandMatcha runs DevX at Cerebras and @vivianmshen is a General Partner at YC. Two people who work deep in AI every day are now talking about what life inside Silicon Valley is really like. SOTA Wives just launched. Watch the intro below.frozen11d ago11K24190100.41%
This is a good example of what a company brain unlocks and how it compounds. You want it so that when someone on your team figures out a great prompt, skill, or workflow, the whole company can build on it. @tryramp just launched Glass to help with this. It’s connected to tools like Linear, Notion, Slack, and Gong.frozen9d ago11K545170711.36%
The applications of new world models like Agora-2 are mind-blowing. Agora-2 now lets up to 20 humans and agents share one simulated world in real time. That means you can train robots, self-driving cars, and cyber defense agents together, and each one improves as the others get better. It also gives you a safe place to study agent collusion before it reaches real systems. Watch this space closely!frozen28d ago11K65960521.23%
Banger paper from Microsoft Research and colleagues. It studies the potential benefits of agents that share progress while they work. (bookmark it) Communication is still a challenge with multi-agent systems. In this setup, agents have no predefined roles and communicate via a shared directory. They report that a team of k agents that write their findings to a shared directory matches the success rate of 4k agents working independently on ARC-AGI-3. The gap grows with k, and teams reliably solve some tasks that no single agent solves. The same setup beat best@k on polyomino packing and exceeded the prior best-known score. On MNIST compression, a four-agent team wrote a 1,957-byte classifier with 99.4% test accuracy, smaller than the best-known human solution. Independent agents still do better when compute is tight or when there is no clear measure of progress, so the paper also tells you when it might be a good idea to skip communication. Paper: https://t.co/eeM3nApkW4 Chat with Paper: https://t.co/rgsWvpKUamfrozen29d ago11K122182411212.66%
AI has sparked a new era of learning. Jamie earned 12 advanced degrees while growing Crimson Education. But learning while building is a skill in itself. Traditional school doesn’t offer much of this, but with AI being this good, nothing should stop you.frozen27d ago11K19350150.39%
Does giving a multimodal model tools make it worse at refusing harmful requests? New work from NVIDIA, accepted at NeurIPS 2026, says yes for every model it tested. Refusal failures rise by up to 68.7% relative, and by 17.7% on average. The drop appears in Claude Opus 4.6 and 4.7, Gemini Agentic Vision, Qwen3.5-122B-A10B, and agent-tuned open models across MM-SafetyBench, HoliSafe, and VLSBench. The authors trace it to two causes. Tool outputs fill the context and bury the original request's harmful intent. The model also shifts its attention to describing what the tools returned instead of making the safety decision. Re-inserting the original request and image right before the final response restores part of the lost refusals. If your safety evals run only in plain chat, they may overstate how safe your agent is. Paper: https://t.co/qOORjeWcMU Chat with Paper: https://t.co/KVXaYHGWG9frozen15d ago11K7811394411.62%
Nice paper showing how to re-evaluate a production agent at a fraction of the cost. 200 questions, 38.5% of the full benchmark, reproduce the full score to within 1.03 points. The authors studied an analytics agent that serves tens of thousands of monthly users, using 574 historical benchmark runs split by date into calibration and held-out periods. They compared random sampling, cached results, fixed representative subsets and adaptive testing based on item response theory. Multidimensional 2PL adaptive testing gave the best fidelity. The team deployed difficulty-stratified fixed subsets instead because they are simpler to run. Those subsets transferred to five other agent families without recalibration and stayed stable with calibration windows as short as one day. Paper: https://t.co/CyH01y4Lvg Chat with Paper: https://t.co/Vdzr56ulKgfrozen29d ago11K628230671.50%
Own your intelligence stack, folks! This chart could mean so many different things without raw details. I suspect more companies/devs are starting to realize the business opportunities RL unlocks, given the impressive models we already have today. For many real-world tasks, you don't need AGI; you need a proper specialized model, harness, and data flywheel. A new, fruitful post-training era is upon us. It's hard to see, but it's starting to happen with full-stack AI companies, and it's only going to keep growing.frozen15d ago11K475110340.91%
LLM-based agents love structured data. I've seen this work really well, but mostly in complex codebases. But I think context graphs have broader applications, as shown in this Harvey report on legal knowledge tasks. We are working on this at @dair_ai for research-related tasks. More to share soon.frozen7d ago11K667330721.68%
I expect more AI companies to run the service themselves and sell the outcome. @arceuslegal does that for legal work. Clients send contracts over Slack, AI gathers the context, and a licensed attorney reviews every piece of work before it goes back. Contract reviews take 3 to 5 hours on average, at a flat fee agreed before the work starts. Personalization has massive implications in the business world too.frozen21d ago11K37170260.67%
WER dropped from 85% to 7.65% on Bengali with one fine-tune. Numbers like that are why 80 labs asked to license Monsoon in a week. What I respect most is the rule behind it: @voicearena_ai only builds a dataset if it moves the needle. Most data vendors can't say that.frozen14d ago11K30270230.58%
Decision models are going multimodal. Drex Max looks really strong. It can make fast decisions straight from screenshots, screencasts, scanned documents, scientific charts, and photos. This model is a really good fit for computer-use and browser-use agents, where every step starts with reading the screen. Nace reports 88.3% on Mind2Web (outperforms Jev 1.14). Drex Max also scores 67.6 on the Decision Index (Jev 1.14: 60.70).frozen3d ago11K887232741.83%
So back! Let's connect if you are at DevDay and want to chat about all things startups, harness engineering, and RSI.frozen23d ago11K3116160.43%
Or just build your own. That's how you make personal agents fully yours. You can sit and wait for the next bug fix or for one of these providers to address your safety concerns, or you can start co-evolving a personal agent as needs arise. I have chosen the latter. A personal agent is something you should fully own, from the traces to memory to all outputs. So much resistance to building your own harness and agents. I understand the fear of maintaining yet another thing. But AI makes it easy to build and maintain this stuff. It's not that hard to build agents today, folks. You seriously should consider building your own solution. This is a great time to start. Start from scratch, keep it minimal, and evolve it as you go. Or you can start with an open-source solution like Pi Durable and build from there (btw, Pi Durable is crazy good for this stuff).frozen17d ago10K413153350.92%
Not sure what model Space Bunny is, but I am impressed. I had a chance to test it this week. It's free in OpenCode for the next week, with a 1M token context window and image input. I focused my tests on visual work, mostly starting from a single image or sketch. Here are my results. Overall, its visual understanding is excellent, and its design taste is strong. One behavior that stood out is that it checks its own work. When it has a browser, it opens the page, looks at screenshots, and fixes what it sees before it says it's done. These verification capabilities matter in long-horizon agents. From a rough wireframe with seven handwritten notes, it built a full landing page and followed every note. From one photo of a moka pot, it built a 3D model that matches the original down to the eight-sided body, the brass valve, and the camping stove underneath. It built a 3D anatomy of a mirrorless camera with 67 labeled parts, an 11-element lens, a 9-blade iris, and an exploded view. Very excited about these results. I asked it for a 60-second animated proof of why a circle's area is πr². It built the whole thing, with slices that rearrange into a rectangle, a live slider, captions, and narration. Not perfect, but still impressive. I gave it a rough hand-drawn sketch of a game level. It built a playable 3D game in Three.js that follows the sketch and all ten rules I wrote on it. Then it playtested its own jumps in Chrome and tuned the physics until every lava stone was reachable. Based on this, I would reach for it for image-to-code work, interactive prototypes, and creative coding in 3D. It’s also quite fast, which makes it great for rapid prototyping. Space Bunny generated everything in the video in OpenCode. I wrote the prompts and supplied the images.frozen28d ago10K487143451.12%
We need more efforts like this. Every agent benchmark should audit its verifiers. Parsewave went through all 600 public tasks in Zapier's AutomationBench. Agents wrote realistic wrong answers to try to fool each verifier, and human review confirmed 206 real bugs. AutomationBench Verified fixed all 206. Regarding 1,235 Kimi K3 runs, the fixed verifiers changed 27.9% of the grades. I just started looking into this benchmark for some independent eval work I am doing, so this is good timing to see this audit.frozen16d ago10K488273321.13%
This is a big deal. Sabi just raised $50M to build a brain-AI interface you wear as a cap, outfitted with 100,000 sensors, that will enable people to talk to their AI agents just by thinking. I did some BCI research during my PhD, but the tech wasn't that great. It's great to see investment in improving the hardware, since current AI can unlock interesting research and personal agentic applications.frozen13d ago10K34280160.57%
This is a great product. Recommended if you are building personal agents and need reliable, fast browser-use tools. Agent Browsers by @gumloop opens a browser in its own cloud sandbox when there is no connector, then logs in and downloads the export. You do the task once; it saves the steps as a recipe and repeats the job every week after that. Great, reliable, reusable browser you can hook all your agents to.frozen16d ago10K33520380.75%
Indeed. I would add that if you know how to build a good eval on top of it, you will be on the frontier in your domain/task in no time. That is the kind of edge that evals can unlock for you. Learn to build good evals too. It's worth it.frozen16d ago10K595152631.38%
I think faster inference is one of the next big unlocks for coding agents. Volantis is using optics to give each chip far more memory and much higher memory bandwidth. They're targeting up to 10,000 tokens per second per user on models over 10T parameters. That's crazy! At that speed, a coding agent that takes hours today could finish in minutes. Definitely one of the more exciting raises I have seen recently.frozen21d ago10K351200190.72%
Agents need their own identity. This is long overdue. I build a lot of personal agents, and today they either borrow my logins or get blocked at sign-up. I already use @agentmail, so this is a no-brainer. AgentID gives each agent its own verified inbox to sign in with, linked to a real human owner. Apps can let agents in and know exactly who is behind each one.frozen16d ago10K26480370.72%
Agents are quickly becoming your API and docs' main users. The @readme team says agents now make up 72% of traffic to API docs. And when an agent gets stuck, it rarely goes looking for the docs. Restless puts the fix steps and a doc link right in the error response. If the agent is still stuck, it can report what is missing, and that turns into doc updates or issues. I build a lot of agents that call APIs, and I want every API to have this.frozen10d ago10K381202270.85%
Interesting results here. This is why I expect more agent workloads to run on blended models. Pareto 26.9 from @TheUnbiasedCo sends requests to several frontier and open models and keeps the best answer. In the new eval of 30 agent tasks, Pareto tied GPT-6 Astra for first place at about 1/3 the cost per successful task. It also finished tasks faster than DeepSeek V4 Pro and GLM 5.3 Flash.frozen28d ago10K285131120.57%
I think AI voice is missing a quality-check layer. In most apps I build, the voices already sound great. What still breaks production for me is a model reading "Q3", "v1.2", or a brand name wrong. It makes a huge difference in how people perceive voice agents. @OnepinAI is working to solve this. They check every line for naturalness, clarity, and pronunciation before you hear it. When a word comes out wrong, it fixes that word directly instead of regenerating the whole line.frozen21d ago10K26180140.47%
I keep saying the real upside of AI agents is giving your most technical people their time back. SeatGeek automated 50% of its IT requests in 60 days with @getserval. Its IT engineers now work inside business teams as embedded partners. Zero layoffs, and hiring this year is already ahead of all of last year.frozen23d ago10K21251140.41%
Highlight of DevDay for me is @thsottiaux shouting out @pidotdev. It aligns perfectly with how serious OpenAI seems to be about building a more open ecosystem. And it’s not just feel-good words, there is a big opportunity here to become that platform everyone builds on top of. From those just getting started to the top builders in the worlds. There is a long-term vision at play here just saying. Makes sense in a world where everyone builds.frozen23d ago10K2219050.36%
SF is different man. Vibe here is just on another level.frozen23d ago10K22210070.40%
Pay attention to this if you are building with agents. Simulated companies are a big deal for improving agentic products. This is because good agent evals need environments that behave like real companies. Era by Eon can generate a complete company across Salesforce, Zendesk, Slack, Jira, and other systems. Each system behaves like the real vendor, down to rate limits, pagination, and error codes. You can reset the company and rerun the same tasks after changing the model or the prompt.frozen17d ago10K38491280.77%
Go-to-market is quickly becoming an engineering problem. Good outreach depends on clean lead data and on giving your AI agent the right context about who buys and why. Pancake builds that context from your website and sends it to Claude or Codex through a single MCP server. Your agent finds and scores leads, and you approve each one before anything is sent. Looks very useful for growing a business.frozen9d ago10K333140320.80%
Ambitious stuff right here, but much needed to better leverage your intelligence stack. Companies will need a system of record for how their AI works, just as they do for customers and finances. CData's new Connect AI Gateway handles every request between your models, agents, and business systems. It routes each one to the right model or tool, checks identity and policy on every call, and saves what each interaction learns as context other agents can reuse.frozen23d ago10K26390150.52%
Beyond agent harnesses, I believe agent interfaces are an important direction for the development of self-evolving agents. This is connected to the idea of building for agents. You can pack so much into a good harness and interface that agents understand well. Think about the efficiencies you get. Costs you can reduce. More importantly, this lets agents work more effectively and reliably with interfaces that make sense for the task at hand. With Artifacts, we've already seen huge gains in productivity. It's a more natural interface for humans and agents to collaborate. We'll discover more of these interfaces over time, and agents will be able to build them on the fly, iterate on them, and evolve as needs arise. Excited about this emerging trend.1d ago10K12973811212.90%
Agents are coming to healthcare. Doctors spend much of their day on admin and research. @almanac_health lets them delegate it to an agent and get that time back. Almanac runs each task in its own sandbox, pulls from specialist medical knowledge, and asks permission before it acts.frozen29d ago10K21530140.42%
Support calls are a hard real-world test for AI agents. When the agent can't solve the issue, the human who takes over needs the full context. Cresta carries it across, so nobody starts over. Also, Marshawn in Beast Mode is fire 😂frozen29d ago10K1315060.25%
Great paper from Google on program repair agents inside CI. (bookmark it) They propose FlowAgent, which runs a ReAct-style generate-and-validate loop on pre-submit test failures and shows the fix inside Google's code review tools. Two abstention filters decide whether a fix is worth showing. One runs before execution and one after, so weak suggestions never reach the developer. In a manual review of 195 real failures, 67.18% of its fixes were correct. After the Google-wide launch, it suggested fixes on 295,508 changes. Developers previewed 65,069 and applied 28,554. Paper: https://t.co/sCo4UxChgl Chat with Paper: https://t.co/dQnJtRCkYHfrozen14d ago10K8218346581.96%
A big milestone for AI in healthcare. @nollahealth can now use AI to prescribe acne treatment in Utah. The AI handles intake, a skin scan, assessment, prescription, and follow-up, and a clinician steps in when needed. Starting with acne makes sense to me because it is common and relatively low risk. Very excited about AI-powered personalized healthcare. I have personally benefited from AI for health, so it's awesome to see this progress.frozen17d ago10K511111210.84%
Bullish on automated AI research. Early signs of self-improving AI, no doubt. Flint is the first model from @recursive_si. Their Eureka Machine contributed to nearly every part of the stack, from data and training recipes to the inference kernels, and the team built it from scratch in under four months! It is a 40B MoE with 2.5B active parameters and open weights under Apache 2.0. It runs at 125+ tokens/s on an M4 Pro MacBook Pro at short contexts. Highly recommend checking it out. We will be testing this in our internal autoresearch evals.1d ago10K99951691.82%
I can usually tell when a deck was made with AI. Generating slides with AI is easy now. They are just missing that personal touch. Gamma 5 from @GammaApp was rebuilt to address this. It's really good; the slide decks now feel like my own, with my vision, and closely follow my brand style.16d ago10K18130170.39%
These days, I spend more time verifying agent-written code than writing it. This is a big bottleneck in so-called software factories. Ship from @ContextQa takes a bug reported in Slack or Linear, reproduces it, and hands Claude Code or Codex the context to fix it. Coding agents need this kind of feedback loop to get better at fixing their own bugs.17d ago9.9K252221120.63%
I’ve never seen anything like this. AI voice technology is getting out of hand. Drama 3 is the most control a model has shown over the direction of language. It can change tone in the same sentence and has vocal control similar to human speech. I tried testing this feature and couldn’t believe how well it worked. Do yourself a favor and try @FishAudio out.frozen14d ago9.9K44470441.00%
Interesting new research from Meta Superintelligence Labs It discusses how reward hacking affects coordination in multi-agent coding systems. Coding agents on a team share code, memory, and plans, so a reward hack one agent learns gets passed to the others. The paper traces this through three channels. Hacks move through shared code, through shared memory, and across roles through natural-language plans. Agents on small open-weight models can induce hacking in stronger closed-weight agents. On hard tasks, the hacking can grow along a chain even after the original agent has left, because the code collects more convincing justifications on the way. It also biases task routing toward the hacking agents and survives candidate aggregation. So how do you mitigate this problem? The authors suggest that code review by the coder or by a separate verifier reduces the hacks that get delivered. If your agents share a repo, the review step is the first thing to add. https://t.co/5kEYLL9BPV Chat with Paper: https://t.co/DDxYFnYGaEfrozen8d ago9.8K11212736802.89%
Ninety minutes of daily use per user is wild for Status. My instinct was that cost just comes with the territory at that scale, but Inworld cut it by roughly 95% for them.frozen8d ago9.8K11030130.28%
This is my new favorite way to keep up with AI papers. I turn my curated papers into a 2-minute spoken technical briefing with @FishAudio Drama 3, straight from Codex. In the demo, I took one of my favorite recent NVIDIA papers and built a small skill that reads the full paper, writes the script, and sends it to the Fish Audio API with a voice I picked. What I like about Drama 3 is that you direct the delivery in plain language, like [calm news-anchor delivery] or [slight excitement]. It makes a dense technical summary easy to follow. Now I catch up on papers on the road or while exercising. Just incredible how good this is. It has the right levers I can tune to stay engaged with the audio summaries. The voice you hear in the clip is AI-generated with Drama 3. Let me know if you would like access to the skill.frozen9d ago9.7K54960691.42%
Step 5 Preview from @StepFun_ai hit #1 on OpenRouter Trending, a day after launch. It's available in Kilo Code, Cline, Hermes Agent, and OpenCode. Keep your workflow and just switch models. I've tested the model. It’s a great model and has some nice properties. I've been using it as a coding agent since early access. It checks its own work and stops when the task is done. I would use it for long agent runs that I don't watch closely. StepFun built it for real engineering work. That covers fixing bugs in unfamiliar codebases, implementing features across files, and refactoring, with changes that run and pass checks. Frontend is the other big focus. You can give it a screenshot, a mockup, or a PRD, and it can return a working web page. It can also build a playable browser game from a set of assets. Beyond code, it reads annual reports and operating data and turns them into financial reports, editable spreadsheets, and slide decks. The context window is 1M tokens, with up to 1M output tokens. That is enough for large codebases and long multi-step tasks. Under the hood, Step 5 Preview is a sparse MoE with 600B total parameters and 27B active per token. Open weights are coming on Oct 15. Very much looking forward to its release.frozen13d ago9.7K34821090.74%
Very cool to see this idea resurface. I wrote about a similar tool I had built months ago and how I integrated into my agent orchestrator. https://t.co/03YohbmoQE The models have gotten a lot better at understanding all kinds of complex information. There is lots of room to explore interesting ways to interact with agents. This is just scratching the surface. From my experience with building and using something similar for months, this works great for designing work. It’s not as useful for other types of work so I think there will be different optimal ways to interact with agents and not just one way.3d ago9.6K321140541.05%
Huge release from @odysseyml. Odyssey-3 Pro sets a new top score on Physics-IQ Verified, a benchmark that asks models to continue videos of real physics experiments. The robotics results stood out to me. With tens of hours of demos, the robot arm recovered from a missed grasp, a behavior that never appeared in those demos.frozen14d ago9.6K334113130.67%
AI video creation is a really huge market. It's really taking off in areas like marketing, education, and customer support. This team @clueso is on a mission to dominate this space. Clueso agents autonomously plan and create videos at scale, capture screen recordings with browser agents, and create on-brand animations with AI. Congrats on their raise.frozen9d ago9.5K570122371.14%
I've recently worked on agentic enterprise integrations, and things can get messy fast. You quickly find that agents are only as useful as what they can do inside a customer's Salesforce or NetSuite. Every customer sets those up differently, with their own custom objects, fields, and permissions. That means there are many personalized solutions. @WithAmpersand is integration infrastructure built for that problem. You define each customer's objects and mappings in one config, and Ampersand handles auth, retries, backfills, and monitoring behind the scenes.frozen16d ago9.5K23140150.45%
Every week I go through hundreds of new AI papers to pick the top ones for our Top AI Papers of the Week digest. Besides reading a lot of papers, I have started to use agent teams a lot. Here is what works for me. The most important thing is making sure I can make the final call on tasks agents are assigned. The volume is what makes the weekly triage difficult. Before Viktor, I tracked papers with saved searches and a spreadsheet. That stopped working once the weekly volume grew past what I could read myself. Now I use 12 triage rules, all in the image below. Each rule shows how Viktor, my favorite AI employee, handles it in Slack. First, I keep context close. Every proposal Viktor sends me comes with the reason, the source, and the next step. That way I can check where a recommendation came from before I act on it. One rule I use often is escalating exceptions. When a paper releases a dataset, Viktor checks the license before I feature it. He handles the routine ones and sends me any with a non-commercial clause. Nothing in the digest is published until I approve it. Viktor proposes, and I decide, so I can stand behind every issue that goes out. Twelve rules in the image. Which of them would you add? Try it free at @viktor_com. $100 in credits, no card. Full link in my first reply.frozen28d ago9.5K395130270.89%
I just ran a paper-research agent on an open stack and had this week's top 5 papers on agent memory in my terminal in about 10 seconds. How does it work? I used Tavily to pull the last 7 days of arXiv. NVIDIA Nemotron 3 Ultra, served on Nebius Token Factory, reads and ranks them. It's 24 lines of Python on an OpenAI-compatible API. It cost me nothing to try. You can build this too! Here is how: The new Nebius AI Builder Program gives you $400+ in credits and discounts on day one, across Token Factory, Tavily, and launch partners, plus runnable blueprints and free courses built with NVIDIA. What I like most is that every layer is swappable. Change the model or the search layer, and the rest keeps working. Walkthrough in the video. Join for free here: https://t.co/i8pWCWYWXa Thanks to Nebius for collaborating on this post.frozen24d ago9.4K326111320.87%
Recommended read. And I agree that the RSI is also a systems engineering problem. Self-improving agents need better research environments. RSIGym gives a research agent training, inference, evals, and sandboxes as services it can call. The agent spends its budget on experiments instead of rebuilding infra. With Opus 5 as the researcher, the improved system went from 17.67% to 50.33% on SWE-bench Verified. Also cool to see a way to measure the quality of co-evolution between harnesses and models, which is how full-stack AI companies stay on the frontier.frozen14d ago9.4K55890461.26%
Reading eval results is now the slowest part of building agents. I'm Elvis, founder of @dair_ai. I lead research, build, and teach about AI agents. I run harness experiments every night, but reading the results was eating my mornings. Every change to my harness gets evaluated overnight, whether it touches memory, tool use, or context compaction. The morning after is the hard part. I check which tasks my agent got right yesterday but wrong today. Then I open the logs for each failure, one by one, to figure out which of my changes caused it. I tried a dashboard first. It showed the pass rate dropped. It couldn't tell me why. That is the job Viktor, an AI employee in Slack, is built for. He reviews the results overnight. Here is how that plays out. Say 23 tasks that passed yesterday fail today. Viktor checks all 23 logs, traces them to the one change that caused them, and suggests undoing it. I check the logs and make the call. Viktor does the digging. I decide what goes into the harness. He is also proactive. He flags problems before you ask, which helps you stay on track with complex eval runs and other research tasks. Harness engineers, do you check every eval run, or only when the pass rate drops? Try free at @viktor_com. $100 in credits, no card. Full link in my first reply. Thanks to the team for partnering with me on this postfrozen15d ago9.3K413182321.03%
Agents will do more and more of their work inside systems of record, so those systems need to be built for them. Attio's new hybrid architecture is one of the cleaner approaches I've seen. Agents can query all of Attio's data in SQL, and the business logic, policies, and compliance rules still apply behind every query. The release also adds data connectors, support for hundreds of millions of records, and Activities in beta.frozen8d ago9.3K363181691.36%
Code review has to change for coding agents. Why? For me, it’s about scaling coding reviews while ensuring quality. If you give a coding agent one task, it can return several pull requests across different repos. Review tools still look at each PR in isolation. Reviewers have to guess which PR matters and how the PRs connect. @QodoAI just launched Qodo 3.0 with PR Triage to solve this. Here is how it works. PR Triage groups related PRs into one work package and scores it on codebase impact, SLA, and open findings. A cross-repo map shows every repo the package touches, and every score shows its own reasoning. Qodo writes one review brief for the whole package, so you review the PRs together. Your coding agent can then work through the findings. Humans still approve the code. Now they see the whole package before approving the first PR. https://t.co/a4ZjqlR2XQ Thanks to the Qodo team for partnering on this post.frozen21d ago9.3K386141210.86%
Recommended benchmark. I expect voice to become one of the main ways people interact with robots and physical AI. That makes speech recognition in real conversations much more important than it seems today. Real conversations are hard to transcribe. People pause, talk over each other, and switch languages mid-sentence. Robots also need to understand the languages people actually speak. More than 5.5 billion people across the Global South are non-English speakers. The community needs a good way to measure these frontier capabilities. @humynlabs built BRIDGE ASR 2.0 to measure exactly this. It tests 23 speech recognition models on real two-person conversations, each 10 to 15 minutes long, in 18 Indic languages plus Spanish, Portuguese, and Vietnamese. The metric I find most useful is code-switch F1. It checks whether English words mixed into an Indic sentence stay in English. A model that writes "data backup" in Devanagari script scores zero. You can also filter the leaderboard by language and by metric. The methodology and evaluation data are public, and the team wants researchers to test the benchmark and find where it breaks. Check out the benchmark here: https://t.co/CO8LoiMNh3frozen27d ago9.3K264221160.74%
AI image tools have gotten very good at giving you exactly what you ask for. Knowing what to ask for is still hard. @envato just launched Burst Mode to help. You start from a rough idea or a reference image, get up to 6 directions side by side, then steer them with moodboards and a creativity control that goes from focused to wild. A really good experience and a great way to get strong results fast.frozen15d ago9.2K1608150.32%
Very good to see more competent labs training open-weight models. This Reflection's new Beam model seems to have strong reasoning efficiency, which I think is a big deal for long-running agents. 3-4x better inference efficiency than rivals like GLM 5.2, positioning it as a strong Western open-weight option.frozen17d ago9.2K608121171.07%
AI voice products need to know who spoke and what they said. That is harder than transcription or diarization alone, especially when speakers talk over each other. Pulse by @smallest_AI is now #1 on the Diarization + ASR track of the @voicearena_ai's Diarization Bench, with 24.4% DER. DER is an error rate, so lower is better. The next system in the track scores 40.7%.frozen15d ago9.1K555123291.14%
Agents need a better interface. In my own agent orchestrator, I often render more actionable, valuable outputs as small interfaces. @openuidev makes this easy for any AI app. It is an open-source, model-agnostic library that streams a UI built from your own components. I can easily see this becoming a standard way to interact with agents in the future. Go check it out!frozen7d ago9.1K453100541.23%
It's an engineering problem! This topic has gotten too political. Let's get back to the fundamentals. "Trust and innovation are not in conflict. Safety is how trust is earned."frozen24d ago9.1K40560100.67%
Build for agents, folks! Have said for a while now that models are already very "smart", but need better harnesses and environments. And you don't want to do this because it's cool. The cost implications are massive here. I highly recommend reading the report and comparing how Pine Computer can help your team. I'll do some testing myself and share more soon.frozen13d ago9.0K54782281.10%
Richer context is what makes personal agents personal. Today, they see your email, calendar, and accounts, and miss everything happening in front of you. NewEyes AI from @CollovLabs is built as a personal agent with eyes. Camera glasses see what you see, and open-ear earphones answer in your ear. The more of your day an agent can see, the more it can tailor its help to you.frozen10d ago8.8K330232190.87%
ICYMI, I set up a Jev-powered automation to track and index all the great use cases of Jev shared on X. Great list of inspirations there. https://t.co/QHZWEbtGD1frozen13d ago8.8K46793491.30%
Damn! Looking at the results, Meta really cooked here! More progress on frontier open models is always a big deal. Great to see Meta going back to its roots. The only way to know this checkpoint's quality is to test it yourself. https://t.co/07dsCnNCcwfrozen7d ago8.7K558131111.01%
Own your intelligence stack, folks! If you train or fine-tune your own models, you also need a way to prove they are yours once the weights are out in the world. Model fingerprinting does this. The Fingerprinting Leaderboard from @Sentient_found is an open benchmark for these methods. It scores each one on how much model quality it keeps, how well it survives removal attempts, and how reliably the owner can detect it.frozen8d ago8.4K463140411.23%
Your agent traces are your best eval data. The most important failures only show up in production, and they rarely make it into your test set. DigitalOcean's Signals flags sessions where users got frustrated or tools failed and turns them into evals with one click. Simulations then replay them against your fix on your agent's real harness before it ships. If you're running agents in production, you need this loop. https://t.co/cWqMgwrEhLfrozen10d ago8.4K656280461.72%
No better time to learn how to build with agents and open models. I encourage students to apply context engineering techniques, then build their own domain-specific harnesses. And don't forget to build evals along the way. If you want a place to start, the Nebius AI Builder Program is expanding to Europe, with $10M across compute grants, capital, and community.frozen3d ago8.4K365160250.98%
The next agent interface might not have a screen. Personal agents already hint at what's coming. Simpler interfaces are where it's all going. Agents are good enough now that the slow step is handing them work. Pulling out a phone to type a request breaks your focus every time. Interface by @NaturaAI is a ring that lets you control your agents from your hand. Press and hold, say what you need, then release. The request goes to Claude Code, Codex, Hermes, or whichever agent you connect. Getting myself one for sure.frozen14d ago8.3K27291150.65%
Very cool to see more specialized models. This is the first dedicated AI design model I've seen. I am bullish on specialized models, as they are easier to encode things like taste and understand intent. This makes models like Vyb easier to use for design, to avoid the slop and generic outputs of today's generalized AI models. More of this, please.2d ago8.1K24051210.63%
Viggle's JST-2 comes with a PINOC MCP server, so one prompt to Claude Code can make the characters, animate them from text, and put the scene together. Characters and motion share one skeleton, so there's no retargeting step for the agent to choke on halfway through. Would love to see someone build a small game start to finish this way.frozen3d ago7.8K233141180.76%
Very excited for this. Ed’s point about compute access in the era of agents is spot on. Plus, AI infrastructure needs changes to meet the demands of agentic engineering. Agents may need GPUs for inference and CPUs to run code in the same workflow (e.g., co-evolution of model and harnesses). I want rental options that fit those changing needs. Follow @edsgoode and @shadeformai this week for cool launches around this.frozen3d ago7.7K20114050.52%
There is a huge opportunity for specialized personal agents. Here is a great example: @OdessiaTravel just launched one for travel and experiences. You text it what you want, and it plans and books the trip, the table, or the tickets. It also keeps watching after you ask. You can set daily watches on tables, shows, and fares, and it pings you when something opens up. The thing is, like specialized and general models, I think we will see the same for personal agents. Love seeing more of these pop up recently. https://t.co/t6D2R81J5Wfrozen8d ago7.6K333121391.16%
// MemoryTax: The Bitter Lesson of Agent Memory // Great insights for agent builders; bookmark it. As models increase in capability, they also improve at things like memory generation. Agent memory frameworks are everywhere right now. This study measured whether they still help once the model can search raw history itself. The results suggest that a strong model needs less designed memory and better tools for searching the raw record. On one 419-turn LoCoMo conversation, Mem0 beats plain full context only with GPT-4o-mini. Building Mem0's memory costs more than an entire full-context rollout. With Claude Opus 5.5, the Pi coding agent searching the raw history scores 99% for under half the cost of full context. Its one miss comes from a wrong answer key. Memory also has a ceiling set by the model that wrote it. A stronger reader cannot recover what a weaker writer left out. The author clearly defines the scope: one conversation and one run per cell. It is still a cheap experiment to repeat on your own agent before adding a memory layer. https://t.co/l7U9ey8CVG Chat with Paper: https://t.co/cwriPTOP0L7h ago7.1K113185831014.13%
Own your intelligence stack, folks! Open models, open data, and open harnesses are where a lot of the real agent building is happening right now. Here's a great opportunity to catch up on the latest. @digitalocean's Open Intelligence Summit today features talks on building an agent with an open harness, when it's worth post-training an open model for your agents, and how to keep a post-training loop running without idle GPUs. Great lineup from Nous Research, OpenHands, NVIDIA, Hugging Face, and more. The keynote is at 1 PM PT.frozen9d ago6.9K40480221.07%
Crazy to see how fast agentic video creation is improving. Syren learns your style from your library, including preferred graphics, motion, and editing rhythm. Then you build new ones from a prompt and refine them in a chat. These will only get better as models and harness improve. MCP tools included. We're exploring this for education, so we're excited to try it out.frozen13d ago6.7K26281190.84%
We need to rethink security for AI attackers. Security tools were built to stop human attackers. A human might give up on a route after a few hops, but an agent swarm keeps going. Cogent Attack Path Analysis uses @cogent_security's own cyber models to find those longer routes into a company. Then it tells the team which single fix closes each one.frozen14d ago6.3K23341130.70%
I think robots that live alongside people will need a different kind of intelligence than robots built for factories. Empathy and personalization matter here. Hugo is a human companion robot from @theermann and Farbod Farshidian, who spent 15+ years teaching robots to run, jump, and work at Boston Dynamics and the RAI Institute. Curious how customizable this could be. I see great educational applications for this.frozen6d ago6.2K24240110.66%
Build for agents, folks! Agents will do more and more work inside business software, so every app needs to work through APIs and permissions. @ERPdotAI builds that in from the start. Every app you generate from a prompt is available through an API, and Proto, their agent, can build workflows and reports on top of it.frozen7d ago6.2K21020250.77%
This is super cool. Texting a business should be as easy as texting a friend. At least that should be the standard. @usebland turns any business website into a branded text line with an agent behind it. It pulls prices, hours, and inventory live from the site, and it can open the page in a real browser and text you a screenshot of what it found. Text any website to +1 (415) 481-7537 to try it.frozen7d ago6.1K16230150.59%
AI agents are useful in outbound when they handle the entire email workflow. Ami from AiSDR researches each prospect, writes a personalized email, sends it, follows up, and handles replies. It also manages the email infrastructure and warm-up. AiSDR now charges $1.50 per researched prospect with unlimited touchpoints.frozen6d ago6.1K2119070.63%
As owning your intelligence stack matters more than ever, reliable access to AI infrastructure and agent compute is key to realising that vision. I’d like to see more of that capacity easily available to builders running experiments and post-training. That’s what interests me about @shadeformai’s Flex VMs announcement. Congrats @edsgoode and team!2d ago6.0K2036080.62%
This is a cool product. In my experience, the best AI images start with a real photo. Prompting from scratch usually gives me something generic. Editing a good photo keeps whatever made it work in the first place. Nice to see @envato lean into this. You pick a photo from their library and edit it with AI from there.frozen7d ago5.9K1724050.47%
Multiplayer AI is what powers successful businesses today. It enables self-improving businesses in which human and personalized agents collaborate. @zenzap_chat puts people and AI agents in the same group chat. Each agent connects only to the tools it needs, a human approves key steps, and the team's conversations build a shared company brain over time.frozen8d ago5.8K2416080.67%
Fast inference is becoming increasingly important for agents. As proactive, always-on agents rise and demand for multi-agent systems for complex tasks grows, heterogeneous compute matters. @general_compute is solving a critical problem here, and this @positron_ai partnership makes a lot of sense.2d ago5.7K20030140.65%
What AI Safety needs is better tooling and engineering. For instance, building stronger red-teaming tools. Great to see this open-source effort to red-team image models. It seems to learn better ways to write attack prompts. It is open source and runs on an open Qwen model with Gemini Flash as the judge, so defenders can run the same tests on their own image models.frozen8d ago5.6K243160171.07%
Home robots need taste too. Personalized agents are a super interesting new trend I am bullish on. If a robot is going to live in my home, I want one I like having around. @roboticscorp is building robots for that. The first 999 units are numbered and open for reservation today.2d ago5.4K1904150.54%
https://t.co/fD3hjHxmID5h ago5.1K39322391.66%
Recommended reading. I've been testing decision models such as Jev and Drex Max for research tasks (e.g., paper classification). Just published our initial exploratory work on it. Drex Max is a strong fit for our use cases, and we are already planning production efforts around it. Multimodal decision models also open up a whole range of exciting areas. We'll share more results and resources soon.5h ago2.6K312122242.76%
Brilliant work on using many harness configurations to scale agentic RL. Many approaches use mainly one fixed harness, which specializes the model but doesn't generalize well. This work uses many harness configurations as a scaling dimension. But the big question is whether it can generalize to harness configurations never seen in training. They pick a compact training subset with balanced factorial coverage, and vary one part of the harness at a time on the same task. Across benchmarks, model sizes, and RL algorithms, this improves results on harness configurations never seen in training. Models trained this way also transfer to unseen external black-box harnesses better than models trained through one complex black-box harness. If your model will run inside harnesses you did not train it in, treat harness diversity as its own scaling axis. https://t.co/lcQQ5y2qQ9 Chat with Paper: https://t.co/fqr3AZswBq4h ago2.0K25480182.75%
I like the vision of distributed superintelligence. But I do worry it's becoming less distributed. @distcog is exploring collective intelligence through networks of specialized models that work together. I like this research direction. I'd be interested to see which tasks benefit from specialization and how much coordination costs as the network grows. Great vision and a nice article with interesting ideas. Recommended read.4h ago1.8K904020.85%
Data remains the workhorse of frontier AI models. Private, high-quality, domain-specific data is especially in demand. Great to see new marketplaces like HUD where individuals and companies license their data to AI labs. Listings include private codebases with full Git history and PRs, RL environments with their tasksets, and raw company records.4h ago1.7K303130.60%