Ship Course-Keeping in Waves using Reinforcement Learning
Keeping a ship on course during its voyage is important to make the journey as quick and efficient as possible. Conventional controllers are often used to solve this problem, but these must be tuned accordingly to different conditions. At MARIN, we investigated the application of Reinforcement Learning (RL) for this problem. A RL agent learns a control strategy through the process of interacting with its environment, receiving feedback in the form of rewards or penalties, and using this feedback to adjust its actions to maximize expected sum of rewards.
Model-Based Reinforcement Learning
One of the limitations of reinforcement learning is the need for a large number of agent-environment interactions in order for the agent to optimise its control strategy. However, by learning a separate model on the dynamics of the environment, we can leverage the learned model to generate hypothetical experiences, reducing the number of actual interactions with the environment needed. This is called Model-Based Reinforcement Learning. This reinforcement learning method was successfully applied to the problem of course-keeping in waves. Not only did the agent manage to stay on course, but it also reduced its rudder usage considerably by ignoring the first-order wave disturbances, all while reducing its training time significantly.
More detailed information can be found in the following paper:
Ship course-keeping in waves using sample-efficient reinforcement learning
More detailed information can be found in the following paper:
Ship course-keeping in waves using sample-efficient reinforcement learning
Contact
Justin Greep
Applied Data Scientist