Sparse Policies in Noncooperative Games
Information sparsity, regularized dynamic programming, feedback Nash equilibrium
News
- 2024-10-22 We submitted this work and published a preprint version!
About
TL;DR: We propose a dynamic programming approach for computing sparse feedback policies that selectively depend on a subset of agents in noncooperative games. We prove the convergence of the approach and show the advantages of employing sparse policies.
Abstract
Common feedback strategies in multi-agent dynamic games require all players’ state information to compute control strategies. However, in real-world scenarios, sensing and communication limitations between agents make full state feedback expensive or impractical, and such strategies can become fragile when state information from other agents is inaccurate. To this end, we propose a regularized dynamic programming approach for finding sparse feedback policies that selectively depend on the states of a subset of agents in dynamic games. The proposed approach solves convex adaptive group Lasso problems to compute sparse policies approximating Nash equilibrium solutions. We prove the regularized solutions’ asymptotic convergence to a neighborhood of Nash equilibrium policies in linear-quadratic (LQ) games. We extend the proposed approach to general non-LQ games via an iterative algorithm. Empirical results in multi-robot interaction scenarios show that the proposed approach effectively computes feedback policies with varying sparsity levels. When agents have noisy observations of other agents’ states, simulation results indicate that the proposed regularized policies consistently achieve lower costs than standard Nash equilibrium policies by up to 77% for all interacting agents whose costs are coupled with other agents’ states.
Paper
