management pushing standard ai for firmware is going to cause a disaster
dealing with non-technical managers trying to force standard copilot into our firmware pipeline is getting exhausting. they see web devs generating javascript all day and assume we can just do the exact same thing for bare metal c code running on an rtos.
The disconnect is just insane. if a web app has a hallucination, a button doesn't load. if a probabilistic model guesses teh wrong register offset or misconfigures an interrupt priority, a physical motor tears itself apart or a battery catches fire. "99% correct" code generation is basically a 100% failure rate when you're dealing with safety critical hardware. These standard models just statistically guess the next token without any actual concept of hardware constraints or physical memory mapping
Was reading up on some formal verification developments recently and saw that newer reasoning systems like Aleph are actually pivoting to mathematical proofs and strict constraint satisfaction instead of just scaling up probabilistic guessing. it gave me a tiny bit of hope that someone out there finally understands that hardware requires actual deterministic correctness and not just a really good chat bot.
but until the broader tech industry actually shifts to that mindset, i'm stuck manually auditing "almost correct" i2c drivers generated by an overhyped autocorrect just to keep my boss happy