UFO

A fully open-source development framework for humanoid unsupervised reinforcement-learning control (Unsupervised RL Control), covering training infrastructure, data pipelines, algorithm research, and inference deployment.

The framework is dedicated to lowering the barrier to unsupervised RL control research — enabling researchers to quickly reproduce SOTA methods, explore novel behavior representations, adapt across robot platforms, and achieve integrated development from training to real-robot teleoperation deployment.

Fast Training Infrastructure

UFO uses the lightweight MJLab backend and supports both single-GPU and multi-GPU parallel training. It completes BFM-Zero training in under 12 hours on eight RTX 4090 GPUs without relying on a single GPU with very large memory; on eight H200 GPUs, training takes approximately 6–8 hours while consistently outperforming BFM-Zero.

General and Extensible Framework

The unified codebase is not tied to a specific robot and adapts seamlessly across morphologies, dramatically reducing migration cost to new platforms. It supports mixed training with data from different sources and provides flexible scheduling and mixture controls. With a well-designed data distribution, unsupervised reinforcement learning can acquire stable general-purpose motion while also learning highly dynamic behaviors such as cartwheels.

New Representation Integration

Beyond the classic BFM-Zero (FB Representation), the framework supports research into multiple unsupervised behavior representations. We have explored TeCH (Temporal Distance Modeling via Contrastive Representation Learning for Humanoid Whole-Body Control) and other new representations, achieving strong control results and providing a unified experimental platform for more general unsupervised control algorithms.

Teleoperation in the Real World

For the first time, the teleoperation code and complete validation scheme for unsupervised RL control are open-sourced, supporting real-robot deployment. The robot naturally performs squats, half-squats, kneeling, rolling, fall recovery, and disturbance resistance — providing a complete reference implementation for unsupervised RL in real-world scenarios.