Autonomous Drone Navigation Project — Challenges & Engineering Notes
Project Goal
We are developing an autonomous drone system capable of landing on a moving platform across six different simulated environments: CITY, MOUNTAIN, WAREHOUSE, FOREST, VILLAGE, and OPEN. The drone operates fully autonomously using onboard perception, navigation, and control logic under strict timing constraints and noisy sensor conditions. The objective is to achieve highly reliable navigation and precision landing performance across all environments while maintaining stability and generalization.
Challenge 1: False Positive Platform Detection
The drone uses a depth-camera combined with an ONNX-based neural network for visual platform detection. One of the biggest issues is false positives: the detector sometimes classifies rooftops, flat terrain, or building surfaces as valid landing platforms. When this happens, the navigation stack immediately redirects toward an incorrect target, often leading to collision or mission failure.
Approaches Tested
- Increasing confidence thresholds (0.40 → 0.55)
- Reduced false positives but also blocked legitimate detections
- GPS proximity gating
- Helped slightly but failed because GPS measurements contain significant positional noise
- XY spatial filtering
- Reduced extreme outliers but still allowed plausible false detections
- Z-plausibility constraints
- Rejected underground or unrealistic altitude predictions
Core Problem
Both the GPS estimate and neural network predictions contain noise and uncertainty. A filter strict enough to eliminate false positives also suppresses valid detections, while a permissive filter allows incorrect target acquisition. The unresolved challenge is determining how to reliably distinguish true targets from visually similar structures when confidence, position, and altitude all appear plausible.
Challenge 2: Transition Instability During Navigation Entry
The simulator terminates episodes when drone tilt exceeds 60°. In several scenarios, the drone crashes roughly 2 seconds after launch, specifically during the transition from vertical takeoff into horizontal navigation.
Root Cause
The velocity controller receives an abrupt command transition:
[0, 0, 0.7]
→
[2.37, 1.44, 1.14]
This sudden directional acceleration causes transient PID overshoot, resulting in excessive tilt.
Approaches Tested
- Global speed caps
- Horizontal/vertical ratio limits
- Mandatory cruise states before navigation
- Minimum cruise duration constraints
- Progressive speed ramps
Core Problem
Scenarios that require immediate navigation need aggressive early movement, while unstable seeds require smoother transitions. Any delay introduced to stabilize one class of scenarios negatively impacts time-critical cases.
Challenge 3: Altitude Estimation in Mountain Environments
In mountain environments, the moving platform operates at significantly higher altitude, while GPS altitude measurements remain noisy and unreliable. The estimated platform height converges gradually through EMA smoothing, causing the drone to initially target incorrect altitudes during approach.
Effect
The drone may spend critical early navigation time flying below the platform, resulting in missed intercept windows or timing out before successful landing.
Approaches Tested
- Altitude hold strategies
- Fixed cruise-height logic
- Natural EMA convergence
Core Problem
Aggressive altitude correction destabilizes perception and navigation, while gradual convergence delays interception too long for the mission horizon.
Challenge 4: Benchmark vs Real Evaluation Mismatch
The local simulator does not perfectly replicate all deployment environments. Several environments must currently be approximated, meaning local benchmark scores do not consistently reflect real-world evaluation performance.
Effect
Systems that perform well locally may underperform under the full evaluation distribution due to differences in environmental dynamics and challenge composition.
Challenge 5: Regression Cycles
The most difficult engineering challenge so far has been regression behavior:
Fixing one scenario frequently breaks another.
Examples include:
- Stabilizing tilt transitions while reducing navigation speed too much
- Improving false-positive filtering while blocking legitimate detections
- Increasing safety margins while destroying approach efficiency
This indicates the system is becoming overly reactive to local heuristics rather than maintaining globally stable trajectory behavior.
Current Engineering Insight
The emerging conclusion is that the primary bottleneck is no longer perception quality or basic navigation capability, but control-state stability. High-performing systems appear to rely heavily on temporal consistency, smooth behavioral transitions, damping mechanisms, hysteresis, and trajectory commitment rather than frame-by-frame reactive decision-making.
The next major architectural focus is therefore shifting toward:
- trajectory stability
- temporal commitment behavior
- smooth state transitions
- predictive interception
- control-layer stabilization
rather than simply adding more heuristics or reward shaping.
Current Stack
- Autonomous flight controller (
drone_agent.py) - ONNX-based visual perception
- Depth-camera navigation
- Physics simulation using
pybullet-drones - Multi-stage learning pipeline (imitation learning + reinforcement learning)
- Custom local benchmarking framework
This project has evolved from a simple navigation experiment into a full hybrid robotics and learning system combining perception, control theory, reinforcement learning, and trajectory stabilization under noisy real-time conditions.