Dynamic Channel Access for Underwater Sensor Networks: A Deep Reinforcement Learning Approach

Abstract

We studied the dynamic channel access problem in distributed underwater acoustic sensor netowkrs (UASNs) by using deep reinforcement learning algorithm. First, a multi-agent Markov decision process was applied to formulate the channel allocation problem in UASNs. In an underwater environment, each underwater sensor is considered for the purpose of maximizing the total network data throughput without sending and receiving data or coordinating with other underwater sensors. Then, we propose a deep Q learning-based reinforcement learning algorithm in which each underwater sensor learns not only the channel access behavior of other underwater sensors, but also features such as the channel error probability of the available underwater acoustic channels to maximize total network data throughput. Afterwards, extensive performance evaluation was performed to confirm whether the performance of the proposed algorithm was similar or superior when compared to the performance of the reference algorithms even when implemented in a distributed manner without data exchange between sensors.