Is one layer enough? Training a single transformer layer can match full-parameter RL training

This paper explores a surprising structural property of large language models: most reinforcement learning (RL) gains are concentrated in a very small subset of transformer layers. By isolating and training individual layers, researchers discovered that optimizing just a single middle layer can match or even exceed the performance of full-parameter RL training. This phenomenon was remarkably consistent across multiple model families like Qwen3 and Qwen2.5, various RL algorithms, and diverse tasks including mathematics, coding, and agentic decision-making. The study reveals that layers near the input and output ends contribute significantly less to post-training improvements than those in the 40%–60% depth ra...