Learning interactive multi-agent behaviors from scratch is often sample-inefficient
and fails to exploit reusable skills learned in simpler settings.
While latent skill representations enable efficient single-agent reinforcement learning,
their extension to multi-agent interaction requires conditioning behaviors on other agents
without destroying pretrained structure. We formulate multi-agent interaction as a latent adaptation problem and propose
the Latent Motion Adjuster (LMA),
a lightweight conditional module that modifies latent actions produced by
a pretrained single-agent policy based on other agents’ states.
Rather than relearning policies from scratch,
our method performs structured residual adaptation in latent space,
enabling efficient skill reuse under both cooperative and competitive scenarios.
Experiments on physics-based control benchmarks demonstrate that
latent-space adaptation improves sample efficiency and interaction performance
over fine-tuning and strategic baselines.
These results suggest that conditional latent modulation
provides a principled mechanism for transferring single-agent skills
to multi-agent reinforcement learning.
LMA Structure
The training process of LMA consists of the following three stages.
Pre-training stage where the latent motion representation of the motion prior is acquired by imitating the motion clip dataset.
Task stage where the high-level policy is learned to control the latent motion of the motion prior according to the given task.
Multi-agent interaction stage where the latent motion adjuster is trained to adjust the latent motion output by the high-level policy according to the opponent's state.
Results
Cooperative tasks
LMA achieved higher cooperative motions at a comparable or faster learning speed than conventional methods.
Table tennis
Green: Agent / Orange: Opponent
(i) PULSE (from scratch)
(ii) PULSE (fine tuning)
(iii) PULSE (fine tuning w/ expansion)
(iv) PULSE + residual RL (action)
(v) PULSE + residual RL (hidden-layer)
(vi) PULSE + KL
(vii) PULSE + LMA (ours w/ freeze)
(viii) PULSE + LMA (ours w/o freeze)
Tennis
Green: Agent / Orange: Opponent
(i) PULSE (from scratch)
(ii) PULSE (fine tuning)
(iii) PULSE (fine tuning w/ expansion)
(iv) PULSE + residual RL (action)
(v) PULSE + residual RL (hidden-layer)
(vi) PULSE + KL
(vii) PULSE + LMA (ours w/ freeze)
(viii) PULSE + LMA (ours w/o freeze)
Comparison of learning performance in cooperative tasks.
Competitive tasks
Although LMA achieved higher competitive motions than conventional methods in table tennis task, no significant difference was observed in the tennis task.
This is because the policy learned during the single-agent stage did not achieve sufficient performance.
A future challenge is the development of a continual skill improvement framework.
