SRI International's FAMDA Shrinks SAM and Depth Anything Into One Edge Robot Model
FAMDA uses frozen SAM and Depth Anything as teachers in a self-training UDA loop for multi-task dense prediction. SAM masks refine teacher segmentation pseudo-labels via majority voting; DAM directly supervises depth with scale-invariant RMSE.
Takeaways
- FAMDA uses frozen SAM and Depth Anything as teachers in a self-training UDA loop for multi-task dense prediction.
- SAM masks refine teacher segmentation pseudo-labels via majority voting; DAM directly supervises depth with scale-invariant RMSE.
- SOTA on SYNTHIA to Cityscapes (60.0 mIoU, 6.49m RMSE) and Virtual KITTI 2 to Cityscapes benchmarks.
- MiT-B2 student is 10x smaller than DAM, 27x smaller than SAM, runs at 7 Hz on Jetson Orin Nano.
- Distillation gains grow as student shrinks: MiT-B0 gains +5.8% mIoU from SAM refinement vs +1.5% for MiT-B5.
- Extends to three tasks (segmentation, depth, normals) and a new day-to-night low-light driving benchmark.
Robotic perception models often lose accuracy after a domain shift, such as moving from synthetic streets to real roads or from daylight to darkness. Segmentation models may drop classes, while monocular depth models can misjudge distance. In a paper accepted at WACV 2027, researchers from SRI International and Seoul National University present Foundation Model-Assisted Multi-Task Unsupervised Domain Adaptation, or FAMDA. The framework uses large pretrained vision models as supervisors, then distills their outputs into one compact network for edge deployment.
FAMDA jointly predicts semantic segmentation, which assigns a class to each pixel, and monocular depth, which estimates distance from one camera image. Its Mix Transformer B2 student, MiT-B2, delivers the following reported deployment profile:
| Property | Reported value |
|---|---|
| Training data | Labeled source images and unlabeled target images |
| Foundation supervisors | Segment Anything and Depth Anything |
| Model size | About 1/10 of Depth Anything and 1/27 of Segment Anything |
| Inference latency | 53% lower than Depth Anything alone |
| Edge throughput | About 7 frames per second on an NVIDIA Orin Nano |
| Deployed components | Student network only |
Why Two Tasks Break Familiar UDA Recipes
Unsupervised domain adaptation starts with labels from a source domain and unlabeled images from the target environment. Modern single-task segmentation systems commonly use self-training: a teacher predicts target labels, a student learns from those pseudo-labels, and an exponential moving average, or EMA, gradually updates the teacher from the student’s weights.
Depth disrupts that recipe because source and target images can use incompatible depth scales. Segmentation methods also rely on class-mixing augmentations that paste regions between images, but the resulting depth targets may combine inconsistent scales and viewpoints. Large camera and scene shifts further reduce the reliability of teacher-generated depth maps.
Reporting is indexed from AlphaSignal. Rights remain with the original publisher and cited sources.
Read original report
