Turning database grading into RL rewards: post-training agents with ThinkingBox
Microsoft and Hugging Face describe ThinkingBox-Training, which uses automated checks of an agent's final database state as reinforcement-learning rewards. After 50 GRPO steps on Qwen3.8-27B, average success rose from 51.70% to 60.78%.
What the article says
This joint Microsoft and Hugging Face post, published October 22, 2026, follows an earlier ThinkingBox post. It introduces ThinkingBox-Training, a reinforcement learning setup that reuses the benchmark's executable checks as rewards. ThinkingBox-Bench has 507 tasks; each compares the agent's final database state with the expected outcome, and the check returns 1 or 0.
Training tasks, taken from outside the benchmark, are selected by offline policy-relative difficulty estimated with a Jeffreys Beta posterior, and the pool stays fixed. The reference run fine-tunes Qwen3.8-27B with GRPO for 50 steps on 187 tasks across 24 H100 GPUs. Average pass@1 rises from 51.70% to 60.78%, and tasks passed on all 20 attempts from 38 to 107. Estimated cost per dependable task falls from $24.36 to $8.00, a comparative index rather than actual billing. The code is released; the authors note the workflows are synthetic and a minority of checks use an LLM judge.
Read the official article AnnuIA profile
Summary written by AnnuIA in its own words from the official page of Hugging Face. The content and rights belong to the publisher: please read the original article.