SpringerLink
Log in

Time-varying formation optimization tracking of multi-agent systems with semi-Markov switching topology

  • Original Paper
  • Published:
Nonlinear Dynamics Aims and scope Submit manuscript

Abstract

This paper investigates the problem of achieving time-varying formation (TVF) optimal tracking control for second-order nonlinear multi-agent systems (MASs) with switching topology. The main objective is to enable the follower agent to track the leader while achieving a given TVF. Considering that MASs often operate in complex environments and encounter diverse changes, this paper introduces a semi-Markov switching strategy. The purpose is to enable the system to achieve the desired time-varying formation even under anomalous conditions. Furthermore, when it comes to Hamilton–Jacobi–Bellman (HJB) optimization, directly dealing with unknown equations becomes a difficult task. However, this challenge can be effectively addressed through the implementation of an actor-critic structural network. In existing approaches, the neural network parameters are updated in a more intricate manner by employing a gradient descent algorithm on the square of the approximate HJB equation, also known as the Bellman residuals. In the optimization scheme proposed in this paper, the neural network parameters are updated using a concise method derived from the negative gradient of a simple positive function. This approach offers a more streamlined alternative compared to existing update methods. By employing this method, an optimal control scheme is provided to tackle the TVF control problem with switching topology. Finally, the validity of the theoretical approach is substantiated through the utilization of Lyapunov stability theory and numerical simulation, thereby demonstrating its effectiveness in the field of MASs optimization.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.

References

  1. Bai, H., Wen, J.T.: Cooperative load transport: a formation-control perspective. IEEE Trans. Robot. 26(4), 742–750 (2010)

    Article  Google Scholar 

  2. Ju, C., Son, H.I.: Modeling and control of heterogeneous agricultural field robots based on Ramadge–Wonham theory. IEEE Robot. Autom. Lett. 5(1), 48–55 (2020)

    Article  Google Scholar 

  3. Nigam, N., Bieniawski, S., Kroo, I., Vian, J.: Control of multiple UAVs for persistent surveillance: algorithm and flight test results. IEEE Trans. Control Syst. Technol. 20(5), 1236–1251 (2012)

    Article  Google Scholar 

  4. Liu, G.-P., Zhang, S.: A survey on formation control of small satellites. Proc. IEEE 106(3), 440–457 (2018)

    Article  Google Scholar 

  5. Chen, L., Li, M.: A nonlinear formation control of wheeled mobile robots with virtual structure approach. In: 34th Chinese Control Conference, Hangzhou, China, pp. 1080–1085 (2015)

  6. Zhen, Q., Wan, L., Li, Y., Jiang, D.: Formation control of a multi-AUVs system based on virtual structure and artificial potential field on SE(3). Ocean Eng 253, 111148 (2022)

    Article  Google Scholar 

  7. Wen, J., Yang, J., Li, Y., He, J., Li, Z., Song, H.: Behavior-based formation control digital twin for multi-AUG in edge computing. IEEE Trans. Netw. Sci. Eng. (2022)

  8. Ouyang, Q., Wu, Z., Cong, Y., Wang, Z.: Formation control of unmanned aerial vehicle swarms: a comprehensive review. Asian J. Control 25(1), 570–593 (2023)

    Article  Google Scholar 

  9. Ranjbar-Sahraei, B., Shabaninia, F., Nemati, A., Stan, S.-D.: A novel robust decentralized adaptive fuzzy control for swarm formation of multiagent systems. IEEE Trans. Ind. Electron. 59(8), 3124–3134 (2012)

    Article  Google Scholar 

  10. Shen, H., Hu, X., Wang, J., Cao, J., Qian, W.: Non-Fragile \( H^\infty \) synchronization for Markov jump singularly perturbed coupled neural networks subject to double-layer switching regulation. IEEE Trans. Neural Netw. Learn. Syst. 1–11 (2021)

  11. Zhu, Y., Wang, Z., Liang, H., Ahn, C. K.: Neural-network-based predefined-time adaptive consensus in nonlinear multi-agent systems with switching topologies. IEEE Trans. Neural Netw. Learn. Syst. 1–11 (2023)

  12. Tian, L., Hua, Y., Dong, X., Lü, J., Ren, Z.: Distributed time-varying group formation tracking for multiagent systems with switching interaction topologies via adaptive control protocols. IEEE Trans. Ind. Inform. 18(12), 8422–8433 (2022)

    Article  Google Scholar 

  13. Liu, X., **e, Y., Li, F., Gui, W.: Sliding-mode-based admissible consensus tracking of nonlinear singular multiagent systems under jointly connected topologies. Trans. Cybern. 52(11), 12491–12500 (2022)

    Article  Google Scholar 

  14. Dong, X., Li, Y., Lu, C., Hu, G., Li, Q., Ren, Z.: Time-varying formation tracking for UAV swarm systems with switching directed topologies. IEEE Trans. Neural Netw. Learn. Syst. 30(12), 3674–3685 (2018)

    Article  MathSciNet  Google Scholar 

  15. Zhao, Y., Jiang, Z., Luo, J.: Optimal control of multi-agent systems with Markovian switching topologies. Automatica 54, 298–305 (2015)

    Google Scholar 

  16. Wei, J., Fang, H.: Multi-agent consensus with time-varying delays and switching topologies. J. Syst. Eng. Electron. 25(3), 489–495 (2014)

    Article  Google Scholar 

  17. Dai, J., Guo, G.: Event-triggered leader-following consensus for multi-agent systems with semi-Markov switching topologies. Inf. Sci. 59, 290–301 (2018)

    Article  MathSciNet  Google Scholar 

  18. Liang, H., Zhang, L., Sun, Y., Huang, T.: Containment control of semi-Markovian multiagent systems with switching topologies. IEEE Trans. Syst. Man Cybern. Syst. 51(6), 3889–3899 (2021)

    Article  Google Scholar 

  19. Guo, X., Liang, J., Lu, J.: Scaled consensus problem for multi-agent systems with semi-Markov switching topologies: a view from the probability. J. Frankl. Inst. 358(6), 3150–3166 (2021)

    Article  MathSciNet  Google Scholar 

  20. Bellman, R.E., Corporation, R.: Dynamic Programming. Princeton University Press, Princeton (1957)

    Google Scholar 

  21. Lewis, F., Vrabie, D.: Optimal control of multi-agent systems with applications in autonomous vehicle guidance. Proc. IEEE 96(1), 77–110 (2018)

    Google Scholar 

  22. Ren, W., Beard, R.: Distributed Consensus in Multi-Vehicle Cooperative Control: Theory and Applications. Springer, Berlin (2008)

    Book  Google Scholar 

  23. Peng, H., Akella: M. Optimal control of multi-agent systems: a review. IEEE Trans. Autom. Control 62(9), 4642–4657 (2017)

  24. Khan, A.U., Basar, T.: Game theory in multi-agent systems: a review, recent developments, and future directions. IEEE Trans. Control Netw. Syst. 5(2), 785–805 (2018)

    Google Scholar 

  25. Liu, Y., Geng, Z.: Finite-time optimal formation tracking control of vehicles in horizontal plane. Nonlinear Dyn. 76(1), 481–495 (2014)

    Article  MathSciNet  Google Scholar 

  26. Huang, M., Liu, D., Huang, B.: Cooperative optimal control of multi-agent systems with application to formation flying. Int. J. Robust Nonlinear Control 22(18), 2047–2063 (2012)

    Google Scholar 

  27. Si, J., Wang, Y.: Online learning control by association and reinforcement. IEEE Trans. Neural Netw. 12(2), 264–276 (2001)

    Article  Google Scholar 

  28. Zhang, H., Jiang, H., Luo, Y., **ao, G.: Data-driven optimal consensus control for discrete-time multi-agent systems with unknown dynamics using reinforcement learning method. IEEE Trans. Ind. Electron. 64(5), 4091–4100 (2017)

    Article  Google Scholar 

  29. Zhang, C., Ji, L., Yang, S., Li, H.: Optimal antisynchronization control for unknown multiagent systems with deep deterministic policy gradient approach. Inf. Sci. 622, 946–961 (2023)

    Article  Google Scholar 

  30. Li, J., Ji, L., Zhang, C., Li, H.: Optimal couple-group tracking control for the heterogeneous multi-agent systems with cooperative–competitive interactions via reinforcement learning method. Inf. Sci. 610, 401–424 (2022)

    Article  Google Scholar 

  31. Wen, G., Chen, C.L.P., Liu, Y.-J., Liu, Z.: Neural network-based adaptive leader-following consensus control for a class of nonlinear multiagent state-delay systems. IEEE Trans. Cybern. 47(8), 2151–2160 (2017)

    Article  Google Scholar 

  32. Wen, G., Chen, C.L.P., Feng, J., Zhou, N.: Optimized multi-agent formation control based on an identifier-actor-critic reinforcement learning algorithm. IEEE Trans. Fuzzy Syst. 26(5), 2719–2731 (2018)

    Article  Google Scholar 

  33. Wen, G., Chen, C.L.P., Li, B.: Optimized formation control using simplified reinforcement learning for a class of multiagent systems with unknown dynamics. IEEE Trans. Ind. Electron. 67(9), 7879–7888 (2020)

    Article  Google Scholar 

  34. Wen, G., Li, B.: Optimized leader–follower consensus control using reinforcement learning for a class of second-order nonlinear multiagent systems. IEEE Trans. Syst. Man Cybern. Syst. 52(9), 5546–5555 (2022)

    Article  Google Scholar 

Download references

Funding

This paper was supported in part by the National Natural Science Foundation of China under Grant Nos. 62276036 and 62006031, in part by the Major Project of Scientific and Technological Research Program of Chongqing Municipal Education Commission under Grant No. KJZD-M202100602, in part by the Surface Project of Natural Science Foundation of Chongqing under Grant No. cstc2021jcyj-msxmX1043, in part by the Anhui Provincial Research Programming Project under Grant No. 2022AH051039, and in part by the Doctoral Talent Training Project of Chongqing University of Posts and Telecommunications under Grant No. BYJS202210.

Author information

Authors and Affiliations

  1. Chongqing Key Laboratory of Image Cognition, Chongqing University of Posts and Telecommunications, Chongqing, 400065, China

    Cuijuan Zhang, Lianghao Ji, Shasha Yang & **ng Guo

  2. Key Laboratory of Intelligent Perception and Computing of Anhui Province, Anqing Normal University, Anqing, 246133, China

    Cuijuan Zhang

  3. College of Electronic and Information Engineering, Southwest University, Chongqing, 400715, China

    Jun Li & Huaqing Li

Authors
  1. Cuijuan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lianghao Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shasha Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. **ng Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jun Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Huaqing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lianghao Ji.

Ethics declarations

Conflict of interest

The authors declare that they have no Conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, C., Ji, L., Yang, S. et al. Time-varying formation optimization tracking of multi-agent systems with semi-Markov switching topology. Nonlinear Dyn 112, 10095–10108 (2024). https://doi.org/10.1007/s11071-024-09599-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11071-024-09599-4

Keywords

Search

Navigation