abstract | bibtex

Autonomous navigation and formation control of multi-UAV systems pose a significant challenge for the robotic systems that operate in partially-observable, dynamic and continuous environments. This paper addresses the problem of multi-UAV cooperative sensing and coverage of a flood-struck region to identify serviceable paths to critical locations for waterborne vehicles (WBV) in real-time. A serviceable path is defined as a location that is obstacle free and has adequate water level for possible movement of WBVs. We develop a deep reinforcement learning model to learn a cooperative multi-UAV policy for real-time coverage of a flooded region. The coverage information gathered by the UAVs captures the presence of obstacles present in the path connecting the start and critical locations given by the shortest Manhattan distance. This coverage information is utilized by the path planning algorithm, i.e., MEA*, to minimize the number of expansion nodes and identify a serviceable path quickly. To conserve energy, UAVs initially follow a guided path to explore the optimal route. If obstacles are encountered, the UAVs search nearby areas for an alternate path to reach the critical location. Experiments are performed over real-world inspired simulated flood environments. The proposed approach, MEA*MADDPG, is compared with other prevalent techniques from the literature. The results highlight the significance of the proposed model as it outperforms other techniques when compared over various performance metrics.

@INPROCEEDINGS{garg_air_2023,
  author={Garg, Armaan and Jha, Shashi Shekhar},
  booktitle={Advances In Robotics - 6th International Conference of The Robotics Society (AIR 2023), July 5--8, 2023, Ropar, India}, 
  title={Real-Time Serviceable Path Planning using UAVs for Waterborne Vehicle Navigation during Floods}, 
  year={2023},
  doi={10.1145/3610419.3610433},
  ISBN={978-1-4503-9980-7/23/07}}
              

abstract | bibtex

During floods, access to real-time ground information is of critical importance for disaster response teams. Unmanned Aerial Vehicles (UAVs) can be quickly deployed in such disaster scenarios for gauging the ground situation. In this paper, we present a decentralized multi-UAV control algorithm based on deep reinforcement learning for flood area coverage. The UAVs are tasked to access the risk levels of the flooded regions and autonomously distribute themselves in order to gather ground information from flood areas in a time-sensitive manner. The task is time-sensitive due to the limited battery of the UAVs and the human lives at risk. In our proposed approach, we follow the paradigm of decentralized training and decentralized execution with opportunistic communication wherein each UAV makes individual decisions based on the information captured locally and the information received via intermittent communication with other UAVs. Further, to learn the best-performing control policy, a flood-water flow estimation algorithm called D8 is employed. With D8, we utilize the domain knowledge to generate better exploration strategies for boosting the initial policy gradients in the right direction. Experiments are performed over real-world inspired simulated flood environments. The proposed decentralized multi-UAV control model, dec-DQNC8, is compared with other prevalent techniques from the literature. The results highlight the significance of the proposed model as it outperforms other techniques and moreover, has the optimal performance when evaluated over a test environment.

@INPROCEEDINGS{garg_ala_2023,
  author={Garg, Armaan and Jha, Shashi Shekhar},
  booktitle={In 15th Adaptive and Learning Agents Workshop (ALA) at the International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2023)}, 
  title={Autonomous Flood Area Coverage using Decentralized Multi-UAV System with Directed Explorations}, 
  year={2023},
  url={https://alaworkshop2023.github.io/papers/ALA2023_paper_7.pdf}}
              

abstract | bibtex

The disaster relief operations during floods require time critical information of the flooded area to save lives. Finding critical regions of the disaster struck area in a limited time frame is crucial for effective relief planning. In this paper, we propose a multi-UAV based system with directed explorations of flooded area to gather time-critical ground information using deep reinforcement learning based controls. We learn an exploration policy for the multi-UAV system with limited battery for autonomous coverage of the flooded region. Further, we integrate D8 flow algorithm that approximates the water flow direction based on image pixel information of a sub-region in the UAVs' exploration strategy. The results show that our proposed method for multi-UAV exploration of flooded area outperforms other methods from the literature. Moreover, the learnt multi-UAV exploration policy is able to generalize to unseen flooded regions without any retraining.

@INPROCEEDINGS{garg_case_2022,
  author={Garg, Armaan and Jha, Shashi Shekhar},
  booktitle={2022 IEEE 18th International Conference on Automation Science and Engineering (CASE)}, 
  title={Directed Explorations During Flood Disasters Using Multi-UAV System}, 
  year={2022},
  volume={},
  number={},
  pages={2154-2161},
  doi={10.1109/CASE49997.2022.9926454}}
              

abstract | bibtex

Autonomous navigation and formation control of multi-UAV systems poses a significant challenge for the robotic systems that operate in partially-observable, dynamic and continuous environments. This paper addresses the problem of multi-UAV formation control while cooperatively tracking a set of moving objects. The objective of the multi-UAV system is to maintain the moving objects under their joint coverage along with aligning themselves in an optimal formation for maximizing the overall area coverage. We develop a multi-agent reinforcement learning model to learn a cooperative multi-UAV policy for the multi-object tracking and formation control. We design a reward function to encode the objectives of tracking, formation and collision avoidance into the model. The proposed deep reinforcement learning based model is deployed and tested against a baseline controller using the Gazebo simulator. The result indicates that the proposed model is robust against the tracking and alignment errors outperforming the baseline model.

@INPROCEEDINGS{garg_indicon_2021,
  author={Kaushik, Prakarsh and Garg, Armaan and Jha, Shashi Shekhar},
  booktitle={2021 IEEE 18th India Council International Conference (INDICON)}, 
  title={On Learning Multi-UAV Policy for Multi-Object Tracking and Formation Control}, 
  year={2021},
  volume={},
  number={},
  pages={1-6},
  doi={10.1109/INDICON52576.2021.9691567}}