Trajectory, in collaboration with UC Berkeley's Sky Computing Lab and Anyscale, has open-sourced SkyRL—a multi-LoRA reinforcement learning training platform that lets large language models evolve in hours using real-time production data. The key innovation is a concurrent training architecture called Multi-LoRA Training.
Here's the problem with traditional fine-tuning: developers have to schedule separate GPU nodes for each experiment, constantly loading and unloading the same massive model. Try training a 397-billion-parameter model the old way, and you'll burn through clusters loading hundreds of gigabytes of weights over and over. It's a colossal waste of compute.
Multi-LoRA eliminates that cold-start hell. The system keeps a "shared base model" resident in GPU memory and treats each fine-tuning experiment as a lightweight "adapter module." During training, it swaps adapter states between CPU and GPU on the fly, running forward and backward passes serially without reloading the giant base model. Combined with concurrent inference processing, overall experiment throughput gets a massive boost.
Test results show up to 2.81x improvement in end-to-end experiment throughput and about 3.25x throughput per node within a fixed time window, all without degrading model performance. The training code is now open-source in the SkyRL repository, aiming to help developers evolve their large models in hours with minimal GPU memory costs.