In Brief
Teel is an open source Python runtime that overlaps independent IO-bound calls in code you wrote sequentially. No async , no await , no futures, no gather . Mark expensive functions with @cacheable , mark orchestration with @jit , and a magic shadow interpreter reads ahead through your bytecode, finds calls whose arguments are already known, and dispatches them on background threads before the main thread arrives. On Princeton’s Tree of Thoughts, unmodified: 83% less runtime on average, and 8.07x on the longest run. 19% faster on average than LLM-generated async JavaScript inside a Deep Agents coding agent. Check the repo:github.com/AI21Labs/teel
Why this matters for agents
Agent quality right now comes mostly from spending more compute at inference. Run an ensemble. Generate five candidate patches instead of one. Split a task across N subagents and merge what they come back with. Different techniques, one execution shape underneath: fan out a lot of independent, IO-bound work, then join it back.
Our recent results on BrowseComp-Plus, SWE-rebench and DeepResearch Bench II came from three unrelated techniques and drew the same picture at the compute level: many independent rollouts, retrievals, tool calls or whole agent runs, overlapping, then merged.
And more and more, the agent writes that orchestration itself. In LangChain’s Deep Agents, the model writes a short script that dispatches its own subagents, instead of calling them one tool call at a time. Whoever writes it, all that fan-out comes with a price. Agent swarms take it furthest: Moonshot’s Kimi Agent Swarm runs up to 300 sub-agents in parallel and reports finishing tasks about 4.5x faster than a single agent.
The problem
When we profiled our agents’ runs, most of the wall clock was a thread sitting on a socket: model calls, retrievals, tool calls, a shell exec in a sandbox. Shaving the main agent code barely moved the total. The leverage is in the waits that do not depend on each other.
In Python, overlapping those waits means rewriting your code first. To run two calls at once you make them async. Then every function that calls them has to become async too, and so does every function that calls those. A change you wanted in one place climbs the whole call chain, until orchestration logic and concurrency plumbing are the same lines of code. Every new fan-out means doing it again. And when a model writes that code, it has to get all of it right on every run.
Here’s a small code-review agent three ways: as written, rewritten with asyncio, and as written plus Teel’s decorators. It asks a model for the worst bug in each changed file, asks whether those findings should block the merge, then either requests changes or approves.
Versions 2 and 3 finish in about the same time, 3.0s and 2.9s against 7.1s sequential. Both get there the same way: they start both possible follow-ups before the verdict is in, and keep whichever one it picks. Teel does that on its own, by following both branches of the if while the verdict is still pending. The asyncio version does it by hand, with three tasks, two cancellations, a finally block, and an async twin of every function it calls. The difference isn’t the concurrency. It’s who has to find it and manage it.
That’s why we built Teel. Concurrency shouldn’t have to be part of the code a developer, or an agent, writes. It can be the runtime’s job. You write the sequential version. Teel reads your function’s bytecode ahead of the line currently running, spots calls that aren’t waiting on anything, and starts them early on background threads. By the time your code reaches them, the results are already there or on their way.
Results
Workload Baseline With Teel Speedup Tree-of-thoughts-llm (longest run)307s 38s8.07x RLM in LangChain Deep Agents (sync Python)150s 58s2.58x RLM in LangChain Deep Agents (async JS)71s 58s1.22x
Tree of Thoughts, unmodified. We ran Princeton’s tree-of-thoughts-llm on Game of 24 samples with GPT-4 without editing the repo: @jit, @cacheable and @inline were applied from the outside. It is a textbook fan-out written as textbook sequential code (generate candidates, evaluate, keep the best, recurse), and the original walks independent siblings one at a time. Average runtime reduction was 83%, from 148s down to 24s at the low end and 307s down to 38s at the high end.
Code the model writes at runtime. LangChain’s Deep Agents ships a dcode agent with a js_eval tool: the model solves a task by emitting a snippet that runs in a QuickJS sandbox. The snippet can call tools.web_search() and spawn subagents via task(), so the model is authoring a fan-out on the fly. But it authors it in async JavaScript, which hands the concurrency model to the model itself, and models are inconsistent at reaching for Promise.all.
We forked Deep Agents and swapped the snippet language for synchronous Python under Teel, expecting simpler generated code at some cost in speed. It came out faster instead: across ten mixed web-search-and-LLM prompts, 58s average against 71s for the async JS and 150s for the same Python run sequentially. Teel parallelizes the independent calls the model forgot to overlap, and speculatively runs fallback branches ahead of the decision that needs them. (The same prompt yields different code each time, so this compares distributions of generated programs, not two implementations of one.) Our fork is at github.com/AI21Labs/deepagents.
The API
There are three decorators and a context manager. Teel currently needs Python 3.12.13 exactly; Known limits explains why.
1from teel import cacheable, jit, Teel
2
3@cacheable # a discrete, expensive, parallelizable step:
4def fetch(url): ... # an LLM call, an HTTP request, a retrieval
5
6@cacheable
7def summarize(text): ...
8
9@jit # orchestration Teel is allowed to analyze
10def research(a, b):
11 pa = fetch(a) # no dependency between these two
12 pb = fetch(b) # Teel runs them at the same time
13 return summarize(pa + pb) # waits only for what it truly needs
14
15with Teel(): # inside this context, independent
16 answer = research("a.txt", "b.txt") # calls overlap - no async in sight
@cacheable marks the heavy functions worth dispatching early. Results are memoized on a key built from json.dumps of the arguments, so keep those arguments JSON-serializable and boring.
@jit marks the functions Teel is allowed to look inside. This is your orchestration layer.
@inline is optional. It marks cheap helpers (string formatting, dict digging) that Teel may actually execute during lookahead, which usually uncovers more @cacheable calls to dispatch ahead of time.
with Teel(): turns it on. Outside the context every decorated function runs as plain Python, untouched.
How it works
Why bytecode? Teel’s implementation is strange enough that the path there is worth 30 seconds, because each step explains the next. Every step kept the same execution idea, independent calls running in parallel, and changed only the interface you wrote orchestration in.
The first version ran orchestration on Celery. Canvas signatures gave us a real DAG the cluster could schedule: utilization up, runs resumable, orchestration finally separate from computation. It was also miserable to write, because orchestration is business logic and Celery makes you write business logic in a pipe-and-signature dialect that is not quite Python.
So we brought it back into Python, or into a restricted subset of Python. We parsed that subset with ast, compiled it into an execution graph and wrote our own executor. The graph paid off on its own terms: independent functions ran in parallel, if bodies could be entered speculatively before the test resolved, and a call like func(x, y) could push into its subcalls while x and y were still pending. All of it available only for the slice of the language we were willing to parse.
Giving up the restriction meant giving up ast. Bytecode is the level at which every Python program looks the same.
How does a runtime know what you are about to call? When a @jit function runs, Teel duplicates the execution state into a limited shadow interpreter and steps through the bytecode your main thread hasn’t reached yet. It works on its own copy, so your program’s state is left alone.
Does Teel build a dependency graph? No. The ast version did. Teel never needs the whole graph, only one answer per call: are this call’s arguments ready yet? Any value it can’t compute yet, like the result of a @cacheable call still in flight, is marked pending, and anything computed from a pending value is pending too:
``` 1pa = fetch(a) # arguments known: dispatched now, pa is pending