Perception Aware MPC

Hello all,

I am sharing a small NMPC project I've been building as part of a larger thesis on drone target-tracking. Check here https://github.com/brunopinto900/argus/tree/master

This is simulation of 12-state quadrotor (position, velocity, ZYX Euler angles, body rates), 4 inputs (collective thrust + commanded body rates, matching a PX4-style body-rate setpoint interface). Formulated as a NONLINEAR_LS OCP in acados, code-generated to C and called from C++.

Alongside the usual position/velocity tracking terms, there's a soft constraint keeping a ground target inside the camera's FOV cone.

The OCP's prediction model uses a first-order lag for the inner rate loop, but the plant simulator runs the actual second-order dynamics.

Result in the gif: 2m-radius circle at 1.5m altitude, 3m/s, camera locked on a static ground target, ~0.1m steady-state XY tracking error.

u/bruno_pinto90 — 1 day ago
▲ 23 r/devpt

Projeto pessoal: Um drone que constrói o mapa enquanto persegue um alvo

Boas pessoal! Quero partilhar um projeto pessoal. Implementei um conjunto de algoritmos, onde o drone navega e evita obstáculos de forma dinâmica, sem mapa prévio. Podem ver um render aqui: https://github.com/brunopinto900/Vista-Tracker/blob/main/README.md

Funciona da seguinte forma: O drone começa às cegas. Só conhece o que a câmara vê no momento. Se o obstáculo está longe do alcance da câmera, ele não existe e o planeador adapta-se em tempo real apenas com o que foi descoberto e planeia um caminho que evita obstáculos. Depois controla os motores para seguir esse caminho.

O plano para os próximos tempos é introduzir mais realismo no sistema e passar do simulador para um drone real.

Usei extensivamente o Claude para visualização e renderings, leitura/escrita de logs e refactoring.

Muito obrigado.

u/bruno_pinto90 — 3 days ago

Autonomous Target Following with Obstacle Avoidance

Hello all,

Been spending the last week porting my submitted thesis to C++/ROS2. Check the repo here https://github.com/brunopinto900/Vista-Tracker

The problem: a drone must autonomously maintain a target inside its camera's FOV while navigating a cluttered environment. Altitude and tracking distance are computed directly from FOV geometry given target size and desired framing coverage.

Current baseline:

  • RRT* for global path planning with Catmull-Rom spline smoothing on the output
  • Cascaded PID Attitude and velocity loops. It is straightforward but it shows its limits fast. No constraint handling, no prediction, coupling between axes is ignored, visibility is not accounted for.

Why PID isn't enough here:
The tracking objective isn't just "follow a waypoint", the drone needs to reason about where the target will be, stay within sensing range, respect velocity/attitude limits, and keep the target in frame simultaneously. PID handles none of that jointly.

Next step: MPC
Plan to formulate it as a receding-horizon problem with the target's predicted position as a reference, FOV constraints baked into the cost, and UAV dynamics as the prediction model.

u/bruno_pinto90 — 12 days ago

Promotion or downgrading?

Hello all,

I’ve been working in automotive ADAS perception systems for about four years now.

So far my work focus on real-world perception issues: investigating why certain edge cases break the pipeline and how errors propagate through higher level tasks. A large part of the day-to-day involved making sense of system failures using logs and data, figuring out what was actually happening when the system behaved unexpectedly in the field, and then proposing and implementing corrective changes.

Now my manager proposed me a new position. To work across multiple customers and platforms, and most of my time is spent understanding requirements, analyzing gaps between requested KPIs and system capability (like latency, robustness,number of false positives, accuracy,etc.), and coordinating discussions between different internal teams.

It is still technical work, you need to be an expert and have substantial experience to assess these trade-offs properly, but the focus is increasingly on constraints, trade-offs, and communication across stakeholders rather than directly digging into system behavior.

It still feels technical on paper, but it doesn’t feel the same as working directly on system behavior or debugging actual failures in the perception pipeline.

Is this new position moving me away from hands-on technical work or is this the natural progression into more senior positions?

Thank you.

reddit.com
u/bruno_pinto90 — 12 days ago

Sharing my master thesis

Hello all,

I finished my thesis in robotics and wanted to share it.

The work focuses on perception-aware trajectory planning for autonomous aerial target tracking in cluttered 3D environments.

The core problem is not just tracking a moving target, but maintaining continuous visibility under occlusions while respecting UAV dynamics and constraints.

https://preview.redd.it/k2kbdsfvsh6h1.png?width=653&format=png&auto=webp&s=153c196ed8f17b2c9369b1d1573df2b355bd5c58

To address this, I designed a two-stage planning architecture:

  • A front-end that generates a reference tracking behavior using a leader–follower formulation.
  • A back-end based on Model Predictive Control (MPC) that refines this trajectory into a dynamically feasible and safe motion plan.

The MPC formulation explicitly balances multiple competing objectives:

  • maintaining a desired relative distance to the target
  • minimizing occlusion in the camera field-of-view
  • enforcing collision avoidance via repulsive costs
  • aligning UAV attitude to keep the target within view (pitch/yaw coupling)

A key aspect of the system is the trade-off between visibility and safety: in cluttered scenarios, the optimizer naturally selects higher-altitude or longer detour trajectories when direct paths would lead to occlusion.

The framework was evaluated in simulation under:

  • dense cluttered environments
  • noisy target estimates
  • variations in cost weights and prediction horizons

Results show consistent stable tracking and robust visibility maintenance even in challenging scenarios.

Next steps include stress-testing in more extreme environments and moving toward a full C++/ROS2 + PX4 implementation.

One of the simplest scenarios, just to showcase.

https://reddit.com/link/1u29cwc/video/i2xedyg8vh6h1/player

reddit.com
u/bruno_pinto90 — 26 days ago

I find the problems but never fix them

Hello all,

I've been in my current company for 3.5 years. I’m a radar/sensing engineer in automotive (embedded, edge-case debugging, data analysis).

My work looks like:

  • Investigating real-world failures (occlusion, multipath, false detections)
  • Digging through logs/data
  • Identifying root causes (sensor limits, model issues)

It’s technically interesting, but I’m stuck in a “diagnose, explain and move on” loop. I rarely get to implement fixes or influence design decisions.

I want to move towards perception systems and robotics / autonomy.

A few questions for people who’ve been through this:

  • How to break out of this into more ownership (internally or externally)?
  • What would be realistic next steps for someone in my position over the next 3–6 months?
  • Am I undervaluing this experience, or is it actually a solid foundation for moving into robotics/perception?

Appreciate any honest perspectives or concrete advice.

Thank you.

reddit.com
u/bruno_pinto90 — 1 month ago
▲ 13 r/DSP

I find the problems but never fix them

Hello all,

I've been in my current company for 3.5 years. I’m a radar/sensing engineer in automotive (embedded, edge-case debugging, data analysis).

My work looks like:

  • Investigating real-world failures (occlusion, multipath, false detections)
  • Digging through logs/data
  • Identifying root causes (sensor limits, model issues)

It’s technically interesting, but I’m stuck in a “diagnose, explain and move on” loop. I rarely get to implement fixes or influence design decisions.

I want to move towards perception systems and robotics / autonomy.

A few questions for people who’ve been through this:

  • How to break out of this into more ownership (internally or externally)?
  • What would be realistic next steps for someone in my position over the next 3–6 months?
  • Am I undervaluing this experience, or is it actually a solid foundation for moving into robotics/perception?

Appreciate any honest perspectives or concrete advice.

Thank you.

reddit.com
u/bruno_pinto90 — 1 month ago

Career in edgeAI

Hello,

At work, i was given the opportunity to take the embedded AI role of a computer vision project/team and i need to decide to take it or not.

I’m trying to understand if this field has real technical depth long-term or if most jobs end up being “use vendor SDK + quantize model & pruning + benchmark FPS.”

Is there still room for meaningful engineering/research contributions in EdgeAI?
Things like: compiler/runtime optimization, custom kernel, model architecture for embedded

Or does most real-world work become integration + SDK fiddling?

Would love honest takes from people actually deploying models on-device.

Thank you.

reddit.com
u/bruno_pinto90 — 2 months ago

How do you balance deep understanding vs fast ticket execution?

Hi, I’m a software engineer (4 YOE). I joined a new team ~8 months ago working in a legacy system, mostly customer issue handling / firefighting.

The expectation from management is pretty straightforward: analyze issues, debug, run existing scripts and quickly report findings. Basically very execution-focused.

But I’ve been trying to go deeper, understanding the scripts, data flow, and why certain failure modes happen.

The problem is time pressure. If I spend extra time properly understanding the system or digging into script internals, I naturally take longer on tickets. My manager has already shown frustration with turnaround time, since from their perspective the job is just to diagnose and move on.

Has anyone dealt with this in legacy/firefighting-heavy environments? How do you balance learning deeply with the need to close tickets quickly?

Thank you

reddit.com
u/bruno_pinto90 — 2 months ago
▲ 1 r/devpt

Crítica ao meu CV

Olá a todos,

Gostaria de receber uma opinião sincera sobre o meu currículo. Tabalho com radares (automóvel) há 2 anos, em análise de dados e na implementação de baixo nível em sistemas embebidos (C++ em SoC), com algumas tarefas de automação pelo meio.

O meu objetivo passa por vagas mid-level na área de robótica (drones, carros, submarinos). Não pretendo permanecer na área automóvel

Este é o meu currículo:

Junior Radar Signal Processing Engineer - Tier1 Automotive Company (2 Years)

  • Conducted root-cause analysis of tracking failures (multipath, ghosting) in low-SNR environments; Replicated edge cases via on a PyTorch-based simulator to mitigate tracking failures; Optimized EKF/UKF state estimation for fixed-point C++ deployment.
  • Optimized memory-critical pipelines via cache-aligned data structures and rigorous profiling, reducing runtime overhead and memory footprint to fit complex tracking logic within highly constrained SoC resources.
  • Architected a Hardware-in-the-Loop (HiL) CI pipeline that accelerated radar KPI validation from days to a 3-hour cycle, achieving 100% automated regression coverage on embedded targets.
  • Developed a Python-based diagnostic suite for MF4/CAN parsing; automated the triage of thousands of field logs to isolate interference and jitter, reducing manual analysis time.
  • Implemented offline tracking and clustering algorithms to establish a reference baseline for validating real-time multi-target tracking performance.

Será que precisa de mais impacto? Mas também não quero inventar e soar falso!
Acho que meu currículo se concentra mais em ferramentas e implementação do que em design, mas é o que é. É a minha empresa. Quero expandir para a área de robótica, o meu currículo está muito focado em radares?

Obrigado pelo feedback.

reddit.com
u/bruno_pinto90 — 2 months ago
▲ 10 r/DSP

Critique my CV Radar Signal Processing Engineer

Hello everyone,

I’m looking for some honest feedback on my CV. I’ve been working automotive radar for 2 years, moving between data analysis and low-level embedded implementation (C++ on SoC), with some automation tasks in between.

I’m aiming for junior-level roles at robotics (drones, cars, submarines). I don't want to stay in automotive RADARS.

This is my CV:

[Current Role] - Junior Radar Signal Processing Engineer - Tier1 Automotive Company.

  • Conducted root-cause analysis of tracking failures (multipath, ghosting) in low-SNR environments; Replicated edge cases via on a PyTorch-based simulator to mitigate tracking failures; Optimized EKF/UKF state estimation for fixed-point C++ deployment.
  • Optimized memory-critical pipelines via cache-aligned data structures and rigorous profiling, reducing runtime overhead and memory footprint to fit complex tracking logic within highly constrained SoC resources.
  • Architected a Hardware-in-the-Loop (HiL) CI pipeline that accelerated radar KPI validation from days to a 3-hour cycle, achieving 100% automated regression coverage on embedded targets.
  • Developed a Python-based diagnostic suite for MF4/CAN parsing; automated the triage of thousands of field logs to isolate interference and jitter, reducing manual analysis time.
  • Implemented offline tracking and clustering algorithms to establish a reference baseline for validating real-time multi-target tracking performance.

Does it need more impact? I think my CV leans more on tooling and implementation, rather than design and lack ownership. I want to expand to robotics, is my CV too focused on RADARs?

Thank you for the feedback.

reddit.com
u/bruno_pinto90 — 2 months ago