# Why we built a benchmark before training for instruction following

Date: 25 Oct 2026
Released by: AI21 Labs

## MediaRelease.co Summary

AI21 Labs has released LIFT-Bench, a benchmark for instruction following in which each constraint is scoped to one section of a document and the task content is scored alongside formatting. The company says training on the open IFTrain dataset let a model maximise its reward while median answers shrank from 114 words to 18. LIFT-Bench has 784 test tasks, and a companion LIFT-Train set of 34,258 tasks, used as the RL reward for Qwen2.5-7B, avoided that collapse. LIFT-Bench, LIFT-Train and the pipeline are published on GitHub.

## Original release

In Brief Train a model on standard instruction-following (IF) rewards, and it learns to satisfy the constraints and skip the task. One of our runs answered a request for a history chapter with a single word, and got full reward. In this post, we walk through how we trained it right. That meant building LIFT-Bench first, a benchmark where every constraint is scoped to one section of a document and the task itself is scored, and then using the same score as the RL reward. Trained that way, the model didn’t collapse, and it held steady on outside benchmarks where IFTrain slipped. The run that looked perfect Say you decided to train a model that drafts documents. Your users give it a topic, an outline, and a few rules: this section in German, a comparison table here, the summary under 100 words. You picked the best base model you could find, took IFTrain, the go-to open dataset for instruction following, and ran RL with its checkers as rewards. It works! The reward climbs to 1.0, IFEval goes up 5 points, IFBench 16. Then you actually read the outputs (like you should). Asked for a chapter of alternate history about the Xinhai Revolution, with no word repeated more than four times, your model answers: “China”. This is not a thought experiment; it happened when we ran this (real) experiment. It taught us the lesson behind everything that follows: an instruction-following reward has to check whether the task was done, not just whether the rules were followed. Getting there meant building a benchmark first. What went wrong: the reward never saw the task “China” wasn’t a fluke: by step 390, the median answer had shrunk from 114 words to 18. The reason is simple. IFTrain’s reward, like IFEval’s, checks constraints with code (no commas, a keyword three times) against the whole response. Nothing checks whether the task was done, so dropping it costs nothing, and RL finds that out. The usual patches don’t close the gap. IFBench adds a reward-model classifier, but it scores general quality, not this task. AdvancedIF gets closer: it trains on content rubrics from a fine-tuned rubric generator. But the generator, the training data and the RL code are all closed, so you can’t build on any of it. So before training anything, we needed an eval that could see the task. Step 1: Build an eval that sees the task Drafting has one big advantage here: structure. Instead of checking every constraint against the whole response, we attach each one to a section of the outline and score it only against that section’s text. That’s the difference between “a paragraph in French somewhere” and “section 2.1 in French”. We call these scoped constraints, and they’re the core of LIFT-Bench. Each section carries two kinds of constraints. Verifiable ones are checked by code (e.g. write in German, use “tournament” at least three times). Judged ones are checked by an LLM and describe what the section has to cover (e.g. trace the championships back to their first official tournament). The judged constraints are the task, written down section by section, and they count in the same score as the formatting. Scoping also gives us a knob no existing IF benchmark has: density, the number of constraints on a single section. We set it when generating tasks, separately from the total number of constraints in the document. Everything is synthetic. Topics come from Wikipedia, outlines and constraints from an LLM, and a sampler controls how many constraints land on each section. The same pipeline also produces flat tasks, with no outline and every constraint applied to the whole response, the way IFEval does it. LIFT-Bench has 784 test tasks in ten splits: short and long documents (about 4 and 9 sections) at four density levels, plus two flat splits. Step 2: Make sure the judge works Judged constraints need a judge, and a reward is only as good as its judge. The default setup hands the judge the full prompt, the full response, and one criterion. We tested that against 305 human-labelled examples, and it said yes to almost everything. Gemma-4-31B accepted 292 of them when only 227 were actually satisfied. Unscoped Qwen2.5-32B scored 0.44 macro-F1, about the same as always answering yes (0.43). So we scoped the judge too. Instead of the whole document, it gets a short request for one section and only the text the constraint applies to: Scoping added 11–19 points, the biggest effect of anything we varied, including the judge model itself. Our read: the unscoped judges weren’t failing at judging so much as at finding the right paragraph in a long document. We went with scoped Gemma-4-31B. This matters more than usual here, because the same judge also hands out the RL reward. A judge that says yes to almost everything would pay for the same collapse we started with. Step 3: Rethink the base model Back to the start: you picked the best base model. Best according to what? Most IF leaderboards use flat, IFEval-style prompts, where every constraint applies to the whole response. LIFT-Bench lets us compare that setup with scoped tasks in one pipeline. We evaluated 17 models, 8 samples per task. On structured tasks, the top models are close to ceiling: GPT-5.5 averages 0.96. On flat tasks, nobody passes 0.85, and the leading closed models average 0.65–0.72. That isn’t about load. Flat tasks carry far fewer constraints, and when we restrict both to tasks with 10 constraints or fewer, all but one model still scores lower on flat. The outline points that only structured tasks can earn are 10% of the score, too little to explain a gap this size. The rankings change too. Among our nine main models, GPT-5.5 is first on structured tasks and fifth on flat, and the two best overall averages belong to open-weight models, Kimi K3 and DeepSeek V4 Pro. The best model on a flat leaderboard isn’t necessarily the best model for your drafting task. Load elsewhere in the document matters too. Keep a section’s own constraints fixed and add more to other sections, and frontier models barely move (Kimi K3 loses 0.4% per density step, GPT-5.5 1.5%), while GPT-5 nano loses 5.5%. A model that handles one constrained section well can slip once the rest of the document is loaded too. Step 4: Train on what you measure With an eval we trust, the next step follows: use the same score as the RL reward, unchanged. That’s LIFT-Train, 34,258 tasks from the same generator. We trained Qwen2.5-7B with GRPO on LIFT-Train and on IFTrain, and looked first at what happened during training. The two runs went in opposite directions. IFTrain’s reward hit 1.0 by step 180, while its median answer fell from 114 words to 18 by step 390, with 38% of answers scoring at least 0.8 in under 30 words. On LIFT-Train, the median answer stayed around 390–530 words, and not one high-scoring answer came in under 30 words. The reward curve won’t warn you about this, so if you’re running RL for instruction following, track answer length next to it. Now, some of this is by construction: a reward that scores content can’t be maxed by deleting it. The real question is whether content is what makes the difference, since LIFT-Train also differs from IFTrain in format and topics. So we ran a control. A flat-only slice of LIFT-Train matches IFTrain’s surface, with flat prompts and IFEval-style constraints, but keeps content in the reward. It trained cleanly too. Topics still differ (Wikipedia vs. WildChat), so it’s not a perfect control, but it points to the content in the reward, not the format, as the part that matters. Not collapsing is the minimum bar. The harder question is whether our reward helps beyond our own benchmark, so we compared both runs on five benchmarks, including two outside ones that also score content: Each recipe does best on its own benchmark, as expected. On AdvancedIF and ComplexBench, IFTrain slips and LIFT-Train holds steady. The deltas are small, so we read this as LIFT-Train avoiding a regression rather than as a general win, but it’s the pattern you’d expect when one reward sees the task and the other doesn’t. The flat/structured split from Step 3 matters for training data too. We trained Qwen3-32B on each scope and on a mix. Each single-scope run helps the other scope, but less than the mix: structured-only reaches 0.805 on flat tasks and flat-only 0.798 on structured ones, against 0.896 and 0.919 for the mix. An easy-to-hard curriculum made no difference. Summary The lesson isn’t just “build a better benchmark”. The reward decides which parts of the task a model learns to care about. If it measures the rules and ignores the task, a rising reward can hide a model that’s getting worse at what you asked. What we’d tell another team training for instruction following: - Measure the task, not just the rules. Scope each constraint to the text it governs, and score content next to formatting. - Check your judge against human labels. It also hands out your reward. - Pick the base model on that eval, not on a flat leaderboard. - Use the eval as the reward, and track answer length, not just the reward curve. LIFT-Bench, LIFT-Train and the pipeline are available here. Read the full technical paper here. Source: https://www.ai21.com/blog/lift-bench-blog/

## Key details

- Issued by: AI21 Labs
- Published: 25 Oct 2026
- Publisher country: United States
- Topics: AI

Original source: https://www.ai21.com/blog/lift-bench-blog/
MediaRelease.co canonical URL: https://mediarelease.co/us/mr02286-ai21-labs-lift-bench-instruction-following-benchmark-rl-training-25102026.html