Satellites swarm cooperation for pursuit-attachment tasks with transformer-based reinforcement learning

The on-orbit intelligent planning of satellites swarm has attracted increasing attention from scholars. Especially in tasks such as the pursuit and attachment of non-cooperative satellites, satellites swarm must achieve coordinated cooperation with limited resources. The study proposes a reinforcement learning framework that integrates the transformer and expert networks. Firstly, under the constraints of incomplete information about non-cooperative satellites, an implicit multi-satellites cooperation strategy was designed using a communication sharing mechanism. Subsequently, for the characteristics of the pursuit-attachment tasks, the multi-agent reinforcement learning framework is improved by introducing transformers and expert networks inspired by transfer learning ideas. To address the issue of satellites swarm scalability, sequence modelling based on transformers is utilized to craft memory-augmented policy networks, meanwhile increasing the scalability of the swarm. By comparing the convergence curves with other algorithms, it is shown that the proposed method is qualified for pursuit-attachment tasks of satellites swarm. Additionally, simulations under different maneuvering strategies of non-cooperative satellites respectively demonstrate the robustness of the algorithm and the task efficiency of the swarm system. The success rate of pursuit-attachment tasks is analyzed through Monte Carlo simulations.