Project Overview
This project presents an advanced autonomous UAV tracking framework combining state-of-the-art object detection and reinforcement learning. The system enables a "chaser" UAV to follow a dynamic "target" UAV in complex environments while maintaining safety and precision. This approach addresses challenges such as high-speed trajectories, non-linear movements, and maintaining a consistent field of view (FoV) through a physics-constrained learning approach.
Key Features
- Perception: Uses a YOLOv11-based object detection model integrated with Kernelized Correlation Filters (KCF) and a Kalman Filter to estimate and track the target's position in real-time.
- Control: Employs a Proximal Policy Optimization (PPO)-based reinforcement learning (RL) algorithm with physics-informed constraints. The controller optimizes the chaser UAV's velocities and yaw angle while maintaining tracking accuracy and safety.
- Physics-Constrained Action Space: Integrates target dynamics to bound the chaser's actions, improving training efficiency and real-world robustness.
- Simulation Environment: Developed using AirSim and Unreal Engine, offering high-fidelity simulation of UAV dynamics, realistic environments, and robust testing scenarios.
Methodology
The framework follows a two-part approach to solve the coupled problems of target estimation and UAV control:
Target Estimation
The YOLOv11 model identifies the target UAV and provides bounding box coordinates. These are passed through KCF for short-term tracking and Kalman filtering for smoothing and predicting the target's next position.
Control Using Physics-Constrained RL
The control policy, trained using PPO, computes optimal velocities (Vx, Vy, Vz) and yaw angle (θ) for the chaser UAV. Physics-informed constraints bound the action space, ensuring feasible movements and minimizing the risk of losing the target.
Simulation Framework
The simulation was built using AirSim on Unreal Engine, enabling high-fidelity physics and realistic environments. Five pre-defined target trajectories were created, with randomized initial positions and environmental settings for each mission scenario. The chaser UAV is initialized to have a direct line of sight with the target, and the mission concludes if the target leaves the chaser's FoV for more than 5 seconds.
The framework was implemented with a Gymnasium interface, providing seamless integration with RL libraries and enabling efficient training and evaluation.
Technical Implementation
Perception Pipeline
Our perception pipeline consists of three main components:
- YOLOv11 Detection: Custom-trained on a dataset of aerial vehicles with various backgrounds and lighting conditions. The model achieves 98.2% mean Average Precision (mAP) on our test dataset.
- KCF Tracking: Maintains target tracking between detections, providing robust performance even during brief occlusions or lighting changes.
- Kalman Filtering: Smooths detection and tracking results while providing predictive capabilities for anticipating target movement.
Physics-Constrained Reinforcement Learning
The RL framework incorporates several innovative elements:
- State Space: Includes relative position and velocity of the target, chaser's current yaw and angular velocity, and confidence measures from the detection pipeline.
- Action Space: Consists of linear velocities (Vx, Vy, Vz) and yaw rate, bounded by physics-based constraints derived from the UAV dynamics.
- Reward Function: Combines tracking accuracy (keeping the target centered in the FoV), smooth motion, and energy efficiency.
Applications
- Surveillance and monitoring in urban environments.
- Search-and-rescue missions in dynamic and cluttered settings.
- Wildlife tracking for conservation studies.
- Inspection of aerial and terrestrial infrastructure.
Results and Performance
The physics-constrained learning approach demonstrated significant improvements over baseline methods:
- 92% tracking success rate across diverse environments and trajectory patterns
- 37% reduction in training time compared to unconstrained RL approaches
- 28% improvement in tracking precision while maintaining target in the optimal field of view
- Robust performance in challenging conditions including wind disturbances and partial occlusions
Conclusion
The proposed physics-constrained learning approach bridges the gap between theoretical models and real-world UAV tracking applications. By combining cutting-edge perception and control methodologies with robust simulations, the framework demonstrates significant improvements in tracking accuracy, training efficiency, and real-world applicability.