Northeastern's SA-MARL Beats Underwater Drone Swarm Coordination by 22 Points
New scene-adaptive embodied intelligence architecture for AUV swarms doing multi-target tracking under tight acoustic bandwidth. SA-MARL uses dual critics, one scene-specific and one global-safety, fused by a dynamic weight to kill gradient conflicts.
Takeaways
- New scene-adaptive embodied intelligence architecture for AUV swarms doing multi-target tracking under tight acoustic bandwidth.
- SA-MARL uses dual critics, one scene-specific and one global-safety, fused by a dynamic weight to kill gradient conflicts.
- Beacon-based protocol encodes cooperative intent and control gradients into periodic signals instead of raw telemetry streams.
- Three-layer design splits perception, decision (CTDE MARL), and execution (PID thrust/torque mapping) across USV, Local Control AUV, and followers.
- Hits 92% tracking accuracy on 4v2 and 86% on 12v4, converging within 1,000 episodes versus MADDPG, MAPPO, MASAC, MATD3.
- Code released at github.com/PingshengRen0901/SA-MARL ; evaluation is simulation-only with particle-abstracted AUVs.
Acoustic links give autonomous underwater vehicles only kilobits per second, add long delays, and disappear as vehicles move out of range. Those conditions can destabilize multi-agent reinforcement learning when collision avoidance, formation control, and mission rewards push a shared policy in different directions. Researchers from Northeastern University and Hohai University describe a scene-adaptive architecture in an arXiv preprint that models each vehicle as a perception-decision-execution loop and trains it with a dual-critic algorithm called SA-MARL. The authors also released a GitHub repository .
Acoustics rewrite the coordination problem
Underwater networks provide sparse, delayed observations of neighboring vehicles and targets. Their topology changes whenever vehicles drift, dive, or encounter obstacles. A policy trained on fresher and more complete state can therefore act on obsolete information after deployment.
The paper identifies two related failure modes:
- Conflicting training signals: General constraints such as collision avoidance and formation stability can produce gradients that interfere with scene-specific objectives such as tracking or encirclement.
- A gap between policy and control: Abstract action vectors must become physical thrust and torque, while multiple vehicles remain coordinated without saturating the acoustic channel.
Three layers, two control tiers
The proposed architecture gives every AUV a three-stage control loop:
| Layer | Input | Output |
|---|---|---|
| Perception | Active and passive sonar | Relative positions and velocities |
| Decision | Encoded observations and scene estimates | Policy action vectors |
| Execution | Policy action vectors | Thrust and torque through PID control |
Centralized training with decentralized execution, commonly abbreviated CTDE, allows critics to use shared information during learning. Each vehicle’s actor later selects actions from local observations and received coordination state. In this design, an unmanned surface vessel provides global coordination, and a designated Local Control AUV manages regional task allocation and critic training for its cluster.
Reporting is indexed from AlphaSignal. Rights remain with the original publisher and cited sources.
Read original report
