FLOW-MEDIATED NAVIGATION AND COORDINATION OF ARTIFICIAL SWIMMERS USING DEEP REINFORCEMENT LEARNING

Aquatic organisms are able to achieve swimming efficiencies that are much higher than any underwater vehicle that has been designed by humans. This is mainly due to the adaptive swimming patterns that they display in response to changes in their environment and their behaviors, i.e., hunting, fleeing, or foraging. In this work, we explore these adaptations from a hydrodynamics standpoint, using numerical simulations to emulate self-propelled artificial swimmers in various flow fields. Apart from still or uniform flow, the most likely flow field encountered by swimmers are those formed by the wakes of solid objects, such as roots of aquatic vegetation, or underwater structures. Therefore, a simplified bio-inspired design of porous structures consisting of nine cylinders was considered to identify arrangements that could produce wakes of varying velocities and enstrophy, which in turn might provide beneficial environments for underwater swimmers. These structures were analyzed using a combination of numerical simulations and experiments, and the underlying flow physics was examined using a variety of data-analysis techniques.
Subsequently, in order to recreate the adaptations of natural swimmers in different flow regimes, artificial swimmers were positioned in each of these different types of flow fields and then trained to optimize their movements to maximize swimming efficiency using deep reinforcement learning. These artificial swimmers utilize a sensory input system that allows them to detect the velocity field and pressure on the surface of their body, which is similar to the lateral line sensing system in biological fish. The results demonstrate that the information gleaned from the simplified lateral line system was sufficient for the swimmer to replicate naturally found behaviors such as K´arm´an gaiting. The phenomenon of schooling in underwater organisms is similarly thought to provide opportunities for swimmers to increase their energy efficiency, along with the other associated benefits. Thus, multiple swimmers were trained using multi-agent reinforcement learning to discover optimal swimming patterns at the group level as well as the individual level.