Technical Trigger
The TRL library has implemented more than 75 post-training methods, with a focus on making these methods easy to try, compare, and use in practice. The library’s design has been shaped by years of iteration, with the first commit dating back more than six years. The trl package now includes both stable and experimental methods, with explicitly different contracts. For example, from trl import SFTTrainer imports a stable trainer, while from trl.experimental.orpo import ORPOTrainer imports an experimental one.
Developer / Implementation Hook
Developers can use the TRL library to implement post-training methods in their own projects. The library’s experimental surface is broader and moves faster than the stable surface, with new methods being added regularly. The TRL documentation provides an up-to-date view of the available methods. Developers can also use the trl package to create their own custom trainers, using the SFTTrainer or ORPOTrainer as a starting point.
The Structural Shift
The post-training landscape is shifting from a focus on individual methods to a more adaptive and chaotic approach, where methods are constantly being updated and replaced.
Early Warning — Act Before Mainstream
To take advantage of the TRL library’s adaptive design, developers can:
* Use the trl package to implement post-training methods in their own projects
* Explore the TRL documentation to stay up-to-date with the latest methods and features
* Create custom trainers using the SFTTrainer or ORPOTrainer as a starting point