In this session we will share our experience and describe our approach to learning-based control. We do this for underwater (marine) environments where we want to approximate some of the hydrodynamic factors in 6 degrees of freedom. Our work addresses “learning to swim” via the automatic synthesis of swimming controllers for the AQUA platform: a six legged autonomous underwater vehicle. First, we will describe our approaches for simulating the underwater dynamics of the AQUA robot. This description includes our modelling choices and the integration into the Gazebo simulator. Second, we will describe the software interfaces we developed, based on the ROS framework, for testing learning algorithms in the simulation environment. Finally, we will show how ROS facilitated the use of our software on physical robots, and discuss the current research that our software has enabled.