What we got wrong building a sensor board, and what we'd change

What we got wrong building a sensor board, and what we'd change

We've been building a small board that handles cameras and sensors for robots, so adding a sensor stops costing a weekend. The hardware was the manageable part. The things that shaped the product were the ones we couldn't see from the whiteboard.

What really cost us weeks:

  • The flash budget made our biggest decision for us. The early assumption was the board would speak ROS natively. Then we put micro-ROS on the target and the library ate about 60% of flash before our own code got a byte. That ended the debate: the device speaks Cyphal, ROS lives host-side behind a thin bridge, and the same image serves three transports. Felt like a defeat, now it's the part we'd defend hardest.
  • We learned which reboots are placebo. A link that died on every board swap ignored power cycles and even full host reboots, because the chip at fault never actually turned off, back-powered through its data lines, holding half-configured state through everything. The fix was one reset write. The lesson: "restart everything" only works if everything restarts, and you don't know that until you've traced where each chip's power really comes from. An older unsolved mystery on the same rig dissolved the day we got this.
  • Hardware timestamps earned their place the hard way. We treated per-sample timestamping as a nice-to-have, sensors stream, host receives, how far off can time be. Then you look at what fusion actually needs: SLAM doesn't care when the host received a sample, it cares when the sensor saw the world, and once readings come from different nodes over different transports, those are very different numbers. Stamping at capture, on the node, with sequence numbers to catch drops, went from footnote to load-bearing.

Not saying any of this is some unique insight, probably every hardware team hits some version of this.

u/NickShipsRobots — 2 days ago
▲ 28 r/robotics+1 crossposts

A camera and an IMU that disagree by a millisecond are describing two different robots

Most stacks work around this with interpolation and manual tuning, then spend months chasing an estimator problem that is really a clock problem underneath.

We ran into same thing building sensor nodes for edge VSLAM. Current firmware still leans on external clock to keep camera and IMU aligned. Next revision removes that dependency, each unit disciplines its own clock on the bus instead of trusting one outside reference.

Two units in the test setup below, same mechanism scales to more nodes on one bus.

u/NickShipsRobots — 1 day ago

Why integration work still gatekeep robotics more than idea itself?

In my experience, adding one sensor to small robot still take whole weekend:) New sensor mean new driver, new wiring decision, calibration pass, maybe serial protocol nobody bother to document properly. For lot of people who want build something simple this is where project stall before it even start.

We work on a project that sits between sensor and compute and takes the driver and comms part on itself: you plug sensor in, driver gets generated for it, and clean data shows up on the other side for your stack. You still need to understand your own sensor and robot, no module fixes that. But building it made me suspicious about how much of sensor bring-up pain is real, and how much is just tax nobody bothered to remove.

reddit.com
u/NickShipsRobots — 16 days ago

sensor compensation on the node or on the host, who's actually right?

We've built a sensor board that runs the compensation on-device: the node reads raw counts, does the datasheet math itself, and puts real Pascals, m/s², and Kelvin on the wire. The host never sees a raw register. That was a bet, and the pushback I keep imagining is from the estimation crowd: datasheet compensation is generic, and if you do your own per-unit calibration, temperature models, bias tracking, then finished units are information loss, you can't un-apply math you didn't choose. The integrator in me says nobody should need a per-sensor driver to learn that this int is milli-g. The fusion person in me wants the counts. (We ended up shipping both, the SI subjects and the raw stream side by side, which is either the correct answer or a refusal to answer, I genuinely can't tell.)

reddit.com
u/NickShipsRobots — 23 days ago

1,900 sensor boards are "supported." we've validated seven

Our sensor board has a mikroBUS socket, which means 1,900+ MikroElektronika Click modules fit into it: every category from IMUs to gas sensors. On paper, we support all of them. In practice, "supported" means physically compatible, and validated means someone plugged it in and watched real data come out. We've done that for seven. The rest is a queue. Going through it by hand means someone's whole job becomes plugging in Clicks, forever, so we're thinking about a harness that brings up an unknown board, figures out what it speaks (I²C, SPI, UART, analog), and confirms it's alive without a human at a scope.

If you've validated hardware at this kind of scale, device farms, protocol auto-detection, anything, how did you do it?

u/NickShipsRobots — 30 days ago
▲ 16 r/u_NickShipsRobots+1 crossposts

we built on the MikroElektronika Click ecosystem, now the problem is validating 1,900 boards

NXS, for those who haven't met it, is a small board that does sensor pre-processing on-device. Its job is turning whatever sensor you plug in into clean SI data, with a generated driver in between. Wire it however your robot is wired: CAN, serial, or the camera coax. ROS 2 picks the data up host-side through a thin bridge. When we started designing it, the first real decision wasn't about transports at all. It was which sensor ecosystem to build on.

We went with MikroElektronika's Click boards, little standardized modules on a mikroBUS socket, instead of designing our own breakout for every sensor. The appeal was obvious: 1,900+ boards covering basically every sensor category that exists, one pinout so the physical side never changes, and the analog frontend already done for us, by people who enjoy doing analog frontends. The catch is on the software side. These boards grew up in the Arduino and PIC world, so the drivers that exist mostly aren't drivers we can use. Which means "1,900 boards supported" really means "1,900 boards physically compatible," not "1,900 boards validated." So that gap is now our whole job.

So far we've validated the pipeline on a handful of boards: a few IMUs, an eCompass, and two GNSS boards including an RTK one, all streaming data end to end. The rest of the catalog is a board-by-board grind, and we haven't solved doing it at scale. Testing 1,900 boards by hand isn't a plan, unless we hire someone whose whole job is plugging in Clicks forever. We're looking at building a test harness that can bring up an unknown Click, figure out what it speaks (I²C, SPI, UART, analog), and confirm basic communication without a human at a scope. The human would like to do other things.

Has anyone done hardware validation at this kind of scale? Device farms, protocol auto-detection, anything in that direction, we'd love to hear what worked and what didn't. And purely for fun: you get one of these today, what Click do you plug in first?

u/NickShipsRobots — 1 month ago

if you've used Cyphal in production, did you stick with it?

I'm building on Cyphal right now (smart-sensor node, CAN-FD and serial) and I want feedback from people who've lived with it longer than a few months. What sold me is the self-describing node idea: data in SI units, types on the wire, a node you can interrogate with stock tooling. At least that's how it looks from here. Everything works so far, which probably just means I haven't found it yet.

So, for anyone past the demo stage: did it hold up, or did you end up back on raw CAN or DDS? And what do you actually drive it with day to day? yakut and yukon feel pretty raw to me, so I'm guessing everyone either scripts around them or builds their own, but maybe you all just have more patience than I do.

reddit.com
u/NickShipsRobots — 1 month ago

how do you handle address allocation for many identical sensor nodes on one bus?

We're building a smart-sensor board that can sit on either a serialized camera link (GMSL) or a CAN bus, and I've hit the classic problem of putting many identical boards on one bus: identity vs. address. Each board has a unique chip UID, but that's 96 bits and not a bus address. Hashing it down to a 7-bit I²C address just collides with other stuff on the bus, so that's out.

Right now I'm looking at two different mechanisms depending on the wire: on CAN, let nodes boot anonymous and have an allocator hand out node-IDs keyed on the UID (Cyphal-style plug-and-play). On the GMSL/I²C side, keep every board at a fixed address and use the deserializer's address-translation to give each a unique visible address, bringing links up one at a time.

For people who've shipped fleets of identical nodes: what did you actually use? DIP/solder straps (which I'm trying to avoid, since it's a human-maintained uniqueness invariant), PnP allocation, address translation, something else? And did you regret it?

reddit.com
u/NickShipsRobots — 1 month ago
▲ 6 r/ROS

Has anyone written a custom I²C transport for micro-ROS (or any MCU ROS stack)?

Small embedded team at a startup, weighing an idea before we commit to it. As far as I can tell, none of the MCU ROS stacks support I²C out of the box — micro-ROS does UART/UDP/CAN-FD, zenoh-pico does serial/UDP/TCP, Cyphal does CAN/serial. I²C isn't on any of those lists. So if we want a board to be a ROS node talking over I²C, it looks like we'd have to write that transport layer ourselves. micro-ROS at least exposes a custom-transport API (the open/close/write/read hooks + set_custom_transport), so in principle an I²C transport is writable. My question is whether anyone actually has.

Context for why I²C at all: our STM32 boards sit behind a GMSL camera link, which tunnels an I²C channel through to the host side. The wire is already there, so an I²C transport would cost us no extra wiring — that's the whole appeal.

So: Has anyone implemented a custom micro-ROS transport over I²C, or pushed zenoh-pico / Pico-ROS down onto an I²C link? How did it go — did the request/response and discovery patterns map onto a master-driven bus okay, or did that mismatch (the host has to poll, the MCU can't initiate) become the whole problem? And if you tried and bailed — what made you bail? Rather hear where it broke now than find out the hard way later. (Not asking whether I²C is a "real" bus for this — I know it's unusual. Asking specifically whether anyone's made a ROS middleware transport work on top of it.)

reddit.com
u/NickShipsRobots — 2 months ago
▲ 8 r/ROS

Multi-transport ROS nodes on MCUs — Zenoh, Cyphal, or micro-ROS?

We're a small embedded team at a startup, early in figuring out our comms architecture, and I'd love to hear how people who've actually built this made the call.

The goal: our STM32 boards should show up as real ROS 2 nodes, and talk over UART or CAN depending on how they're wired. Same firmware, multiple transports. On the host side we're leaning toward Zenoh, since ROS 2 (Kilted) speaks it natively now.

We've been reading, and the three options each seem to solve part of it but not all:

  • micro-ROS does UART and CAN-FD, but you're stuck running the XRCE agent as a bridge on the host, which is the thing we'd like to avoid.
  • Cyphal is great on CAN/CAN-FD with priority arbitration, but it's not ROS, so the board isn't a native node — we'd be bridging into ROS anyway.
  • Pico-ROS / zenoh-pico gets you a native ROS node with no bridge (and pairs nicely with Zenoh on the host), but zenoh-pico's transports are serial/UDP/TCP. No CAN.

So none of them obviously does "native ROS node and CAN" out of the box.

What I'd love practitioner answers to:

Has anyone built one MCU firmware that's a real ROS node and switches between UART and CAN cleanly? What did you put underneath, and what did you end up writing yourself?

For the no-bridge path specifically — if you went Pico-ROS/zenoh-pico to drop the agent, how did you handle CAN? Did you write a custom zenoh-pico CAN link layer, or give up and bridge? I keep imagining someone tried segmenting Zenoh frames over CAN-FD as a weekend cursed-project, and I really want to know where it broke — MTU, reassembly, the multicast-discovery assumption.

(We know micro-ROS is the safe answer. Trying to understand whether the no-bridge, multi-transport setup is real or still a research project.)

reddit.com
u/NickShipsRobots — 2 months ago

We swapped one sensor and spent next few weeks figuring out what else depended on it

Driver rewrite was expected part. What got us was everything downstream that was quietly depending on old sensor and nobody documented it. New sensor's X-axis points a different way, your TF was written around the old one, and now everything's subtly rotated but nothing throws an error. Rate doubles and you're retuning Kalman gains you thought were settled.

And then power rail - a  different draw, nothing to debug on software side, just had to find it by elimination.

Every single one of these was invisible until we actually swapped sensor.

Sensor swap is probably most honest test of whether architecture is actually modular or just looks modular in README, and I'm not sure if our codebase was just particularly messy or this is how it always goes.

reddit.com
u/NickShipsRobots — 2 months ago

We can have up to four 4-lane MIPI cameras fully synchronized with all AI compute offloaded from Jetson, but not sure it's worth the cost

Is there a robotics or autonomous systems use case where this is actually worth it? Thinking high-speed inspection, multi-camera SLAM, perception pipelines. Or is it over-engineered for most applications?

reddit.com
u/NickShipsRobots — 3 months ago
▲ 26 r/ROS

Before you switch to Cyclone, check your serial link

Saw another "just use Cyclone" comment today, got little tired of this honestly

For me it's not even real choice as I work with micro-ROS and Fast DDS is hardcoded in micro-ROS agent on host side. Switching to Cyclone just means extra integration work for nothing.

Also I think people are optimizing wrong thing in general. In our case serial link between microcontroller and host was the biggest latency contributor, not DDS at all. Took us embarrassingly long time to realize this lol

So before going deep into DDS comparison, first check where your actual bottleneck is. And if you do compare, test on your own hardware, don't trust numbers from internet. Setup is too different everywhere to make general conclusions.

If anyone's actually swapped the RMW in their micro-ROS agent, would like to know how that went. Doesn't seem worth it to me.

reddit.com
u/NickShipsRobots — 3 months ago
▲ 111 r/robotics

Are we overusing AI in robotics where simpler solutions would work?

Ok so I was debugging someone's code last week. They replaced PID loop with neural network. Why?? It was slower, harder to debug, and not even better. I think just looked cool in the presentation lol

I get it, ML is great for perception, manipulation, stuff you can't just write rules for. But for control loop? Come on. PID, LQR, MPC – predictable, you know what they do, you can fix them at 3am when everything is on fire.

Also somebody will need to maintain this code in 3 years. Good luck explaining neural network to that person:)

But maybe I am missing something here. Anyone actually replaced classical control with ML and was happy with result?

reddit.com
u/NickShipsRobots — 3 months ago

Went to a robotics event last month. Lost count of how many booths said "AI-powered" on the banner lol

Asked a few engineers what was actually running – classical controllers, pre-trained detection models, one guy who genuinely couldn't explain what the AI part was doing.

The collateral damage is what bugs me most. When everything gets the same sticker, the projects that actually did something novel get lumped in with the ones that slapped "AI" on a PID loop. Buyers get burned, the whole category pays for it.

Filter I've been using: take the AI component out. Does the thing stop working, or just get slightly worse? "Slightly worse" is a feature, not a foundation.

Maybe I'm just getting cynical... do you still find the label useful when evaluating something, or do you just go straight to asking the engineers?

reddit.com
u/NickShipsRobots — 4 months ago
▲ 3 r/ROS

I work on a small robotics hardware team. We build perception and connectivity modules - the kind of stuff that sits between sensors and your compute stack and is supposed to just work. ROS 2 is a big part of how we think about integration, so we spend a lot of time in this space.

Sensor integration is one of those problems that quietly eats weeks. Driver hunting, power routing, timing debugging. Then repeat the whole thing for every new sensor on every new project. At some point our team just got tired of it and decided to fix it properly.

We built an extension module that puts a MikroBUS socket on our platform and, more importantly, runs the ROS 2 node on the board itself. It publishes directly to a topic. Your main compute just subscribes. No driver work on your end at all.

The video shows my coworker plugging in a MikroElektronika IMU Click Board. Topic appears instantly in ROS 2. That's the whole demo because that's genuinely the whole process.

Two transport options are supported depending on the setup:

  • GMSL - high bandwidth, single coax, up to 15m, sub-ms latency. Cameras and sensors share the same link.
  • CAN - deterministic, longer reach, automotive-grade reliability.

The reason MikroBUS was worth targeting: MikroElektronika's Click Board ecosystem has 1,900+ boards: IMUs, GNSS, ToF, gas sensors, motor drivers, environmental monitors. The abstraction scales.

Happy to go deep on the ROS 2 implementation, how we're handling the node lifecycle on the module, transport layer trade-offs, whatever's interesting. What sensor would you actually want to run first?

https://reddit.com/link/1sx9as6/video/u48jd88ohrxg1/player

reddit.com
u/NickShipsRobots — 4 months ago