Why we built a benchmark before training for instruction following
AI21 Labs has released LIFT-Bench, a benchmark for instruction following in which each constraint is scoped to one section of a document and the task content is scored alongside formatting. The company says training on the open IFTrain dataset let a model maximise its reward while median answers shrank from 114 words to 18. LIFT-Bench has 784 test tasks, and a companion LIFT-Train set of 34,258 tasks, used as the RL reward for Qwen2.5-7B, avoided that collapse. LIFT-Bench, LIFT-Train and the pipeline are published on GitHub.