Introduction

This simulation is designed to verify the effectiveness of our proposed method " (RL) Reinforcement-Learning-Based Control Framework for Leader-Following Cascade Formation of Multiple Perturbed Surface Vehicles" by comparing its performance with other existing methods:

1. (FXESO) Fixed-time extended state observer-based trajectory tracking and point stabilization control for marine surface vessels with uncertainties and disturbances
2. (AISMC) An enhanced tracking control of marine surface vessels based on adaptive integral sliding mode control and disturbance observer

Method

This is a cascade control scheme with two loops:

• The outer loop is the high-level trajectory generation, which generates the trajectory reference for the inner loop to achieve the target formation.
• The inner loop is the low-level RL-based optimal controller, which minimizes the performance index.

Result and Comparison

To begin with, we have the trajectory of all five SVs achieving the target formation. The picture presents snapshots of the simulation at different time instants.

Subsequently, the following pictures depict the comparison of the control input in the x-axis, y-axis, and yaw angle of SV 4, from three different control schemes (AISMC, FXESO, and RL, respectively):

Then, we have the tracking error performance in the x-axis, y-axis, and yaw angle, respectively of SV 4. It is observable that our control strategy is the only method that can guarantee the prescribed performance:

Finally, we present the evolution of both Actor & Critic training weights of SV 4:

It should be noted that we only show the results collected from SV 4 because the results of each SV are similar to those of other SVs.

Step-by-step guidance

• FIRST, all the m files and simulink files are in MATLAB R2023a version. Make sure you have at least the R2023a version for the best performance.
• Run AISMC simulation, remember to change the direction in the "save as txt file" section to your desired direction.
• Run FXESO simulation, also remember to change the save-data direction
• To run Actor-Critic simulation, run Parameters.m. Then, to have the data in txt file, run convertData2txt.m.
• Now, run drawGraph.m to have graph.

Contact

Feel free to contact me via [email protected] if you need anything related to this repo.