MARINE 2023
MARIN will be presenting at the 10th International Conference on Computational Methods in Marine Engineering, MARINE 2023, on the subject of Reinforcement Learning:
Session IS02 (I) -Artificial Intelligence Applied in Marine Engineering
Combining deep reinforcement learning and computational fluid dynamics for efficient navigation in turbulent flows
https://marine2023.cimne.com/event/contribution/8ca61bbf-7fe0-11ed-8458-000c29ddfc0c
Navigation of autonomous underwater vehicles (AUVs) in turbulent environments is a difficult control problem due to the unsteady external forces acting on the vehicle. This becomes particularly challenging when carrying tasks such as visual inspection of offshore structures, which generate large turbulent wakes that can increase the risk of collision and damage, as well as decrease the success ratio of recorded video frames. To address this issue, reinforcement learning (RL) is combined with computational fluid dynamics (CFD) in order to develop control strategies more suitable to handling such complex navigation problems. RL has been demonstrated to allow optimal control in turbulent environments, including giving the manoeuvring actors the ability to scavenge energy from the flow. By interfacing RL methods implemented in the Stable Baselines 3 library with the computational fluid dynamic (CFD) solver ReFRESCO it has become possible to model realistic turbulent flows around AUV-like geometries and control them using an RL agent. Using the soft actor-critic (SAC) algorithm, an agent has been trained to manoeuvre the vehicle during a mock inspection simulation. The article will discuss the challenges associated with combining RL and CFD, discuss the details of the implementation of a successful decision-making policy needed to handle such complex control problems, and showcase results from various configurations of the agent and training environment.
Session IS02 (I) -Artificial Intelligence Applied in Marine Engineering
Combining deep reinforcement learning and computational fluid dynamics for efficient navigation in turbulent flows
https://marine2023.cimne.com/event/contribution/8ca61bbf-7fe0-11ed-8458-000c29ddfc0c
Navigation of autonomous underwater vehicles (AUVs) in turbulent environments is a difficult control problem due to the unsteady external forces acting on the vehicle. This becomes particularly challenging when carrying tasks such as visual inspection of offshore structures, which generate large turbulent wakes that can increase the risk of collision and damage, as well as decrease the success ratio of recorded video frames. To address this issue, reinforcement learning (RL) is combined with computational fluid dynamics (CFD) in order to develop control strategies more suitable to handling such complex navigation problems. RL has been demonstrated to allow optimal control in turbulent environments, including giving the manoeuvring actors the ability to scavenge energy from the flow. By interfacing RL methods implemented in the Stable Baselines 3 library with the computational fluid dynamic (CFD) solver ReFRESCO it has become possible to model realistic turbulent flows around AUV-like geometries and control them using an RL agent. Using the soft actor-critic (SAC) algorithm, an agent has been trained to manoeuvre the vehicle during a mock inspection simulation. The article will discuss the challenges associated with combining RL and CFD, discuss the details of the implementation of a successful decision-making policy needed to handle such complex control problems, and showcase results from various configurations of the agent and training environment.
