Finding Malfunctions in UAV’s Physical Sensors using Reinforcement Learning; Sensor Fusion v. Raw Data
Faculty Supervisor:
Dr. Gokhan Kul, University of Massachusetts Dartmouth
Committee Members:
Dr. Joshua Carberry, University of Massachusetts Dartmouth
Dr. Yuchou Chang, University of Massachusetts Dartmouth
Abstract:
Unmanned aerial vehicles or UAVs are becoming frequently used in the military and in law enforcement applications. For the military, they not only provide additional surveillance but aid in recon, combat, and protection. Drones in law enforcement are considered force multipliers by organizations like the Federal Law Enforcement Training Center, and that gives officers multifunctional tools that can assist in daily duties. However, it should be noted that when UAVs fly, the environment they are in can be unpredictable. UAVs are vulnerable to the environment and to autonomous path determination attacks which can lead to deviation from its path or crashing. While many anomaly detection methods exist, a significant portion relies on limited amount of raw data, more so does not account for the physics of the world and the relationships between that and the drone’s physical sensors. The method presented in this paper is an anomaly detection framework that uses a reinforcement learning (RL) deep Q-network (DQN) to learn from real flight data to find normal and anomalous behaviors. In this paper, we will compare the effectiveness of using raw data and sensor fused data to train the RL. The main contribution of this paper to the existing research is the various sensor fusions created to detect malfunctions and anomalies of the physical sensors. Sensor-fused data involves cross-verifying data through sensor v sensor checks, sensor v physics checks and physics v physics. To further elaborate, sensor v sensor fusions involve comparing two values to each other while sensor v physics fusions compare a sensor value to a mathematical computation using the data. Physics v physics sensor fusions involve comparing two laws of physics to each other. Unlike many existing machine learning (ML) solutions which rely on raw datasets, the solution presented compares this method to normalized sensor fused data based on drone-specific aerodynamics before evaluation. After running evaluations, we found that the sensor fused (and normalized) model consistently achieved higher rewards during training compared to the raw data. The sensor fused model was also superior when it came to anomaly detection. For the rewards for the DQN, the reward total for the sensor fused data was over two times more than the raw data (32605 vs 74060). Furthermore, there was no case in which the accuracy score or the F1 score was higher for the baseline raw data than it was for the sensor fused. The lowest accuracy for the sensor fused was 82.78% while for raw it was 67.07%. What was significant, however, was seeing that the combo fused data performed poorly in comparison to the sensor fused, and in some cases worse than the raw data. For the F1 scores, there was no case in which the combo fused had detected any true anomalies leading to an average of 0% across the board for both testing datasets. This research has applications in military defense, law enforcement, and commercial uses. Its main purpose is malfunction detection, so it’s useful for anyone who needs highly secure, tamper-proof autonomous navigation. The goal for this research is the eventual integration of this framework into UAVs so it can be used in real-time. The main goal is to integrate multi-UAV communication networks such as blockchain smart contracts where drones can monitor each other and tell operators about potential malfunctions before the drone crashes.
For further information please contact Dr Gokhan Kul at gkul@umassd.edu.
Dion 311 and Teams (https://teams.microsoft.com/meet/2719811093827?p=KtSBWMxZOo55nt9oIu)
Dr. Gokhan Kul
gkul@umassd.edu
https://teams.microsoft.com/meet/2719811093827?p=KtSBWMxZOo55nt9oIu