UC Berkeley's GSBR Pushes Robot Success Rates From 33% to 95%
GSBR regularizes online RL by Euclidean distance to a base policy's support, not by likelihood. Actions inside the support cost nothing, so the actor can drop suboptimal modes freely. Support region is estimated from base-policy samples using a nearest-neighbor quantile radius.
Takeaways
- GSBR regularizes online RL by Euclidean distance to a base policy's support, not by likelihood.
- Actions inside the support cost nothing, so the actor can drop suboptimal modes freely.
- Support region is estimated from base-policy samples using a nearest-neighbor quantile radius.
- Across 19 tasks on four benchmarks, success rate climbs from 33% to 95%.
- About 23% higher normalized throughput than the next-best RL baseline, RLT.
- Code at github.com/charlesxu0124/gsbr , paper at arxiv.org/abs/2611.05295 .
Pretrained diffusion and flow-matching robot policies reproduce broad demonstration datasets, including their suboptimal behaviors. Online reinforcement learning can improve success and speed, yet unconstrained actor updates may exploit critic errors and drift into poorly covered actions. GSBR , developed by researchers at UC Berkeley and Physical Intelligence, adds a sample-based geometric constraint that the team evaluated across 19 robotic tasks.
Why familiar constraints stall
Behavioral-cloning and KL regularizers pull the fine-tuned actor toward the full base-policy distribution. That pressure preserves every demonstrated mode, including slow, imprecise, or unsuccessful ones the actor should discard. Hard support constraints allow mode selection but reject actions outside the covered region, including small refinements that could improve a grasp, insertion, or trajectory.
Diffusion and flow-matching policies generate actions by transforming sampled noise and may lack a convenient probability-density function. GSBR therefore uses samples from the frozen base policy. The paper operationalizes support as the action region covered by neighborhoods around those samples.
Draw the boundary from samples
GSBR surrounds each sampled base-policy action with a ball of radius
Reporting is indexed from AlphaSignal. Rights remain with the original publisher and cited sources.
Read original report
