u/firestar4430

Remember Navigate on openpilot?

We brought a modern version of that idea to StarPilot, and it works with any driving model.

This is based on the work of discountchubbs, who many of you probably recognize from his amazing work with SunnyPilot. We took his incredible foundation, implemented it into StarPilot, and built additional functionality around it.

Set a destination from Galaxy on your phone or computer, choose between routes based on traffic/ETA, and send it straight to your comma.

StarPilot tracks your position along the route and uses upcoming maneuvers as navigational context for both lat and long behavior.

Upcoming turns, forks, ramps, and lane splits can feed desires like turn left, turn right, keep left, and keep right into the model. With Route Speed Control, StarPilot can also automatically slow down for upcoming turns, U-turns, and roundabouts.

It isn't replacing the model or hardcoding steering commands. Your normal E2E model is still driving - we're just giving it another piece of information:

"Here's where the driver actually wants to go."

It also includes:

• Automatic rerouting
• Alternate routes + live traffic
• Home, Work, favorites & recent destinations
• Remote start/cancel from Galaxy
• On-device turn-by-turn navigation

And because it works alongside the driving model, you can use it with whatever model you prefer.

Huge credit to Chubbs for the work this is built upon. We've had a lot of fun adapting it and expanding what it can do inside StarPilot.

For the people who used the original NOO: how close does this get to what you've been missing?

One of the most common pieces of feedback we hear about openpilot is that the end-to-end model doesn't always position the car where drivers expect it to. Some people love the E2E model's natural positioning, while others wish it would simply stay centered between the lane lines.

So we added something pretty cool.

Lane Centering, now available in StarPilot.

This doesn't replace the end-to-end model, but works alongside it.

When both lane lines are detected clearly and confidently, the system compares the model's predicted path to the center of the lane. If they differ, it applies a small, smooth steering correction toward the lane center while still letting the E2E model make the driving decisions.

If the lane lines become unreliable, the lane width looks abnormal, you're changing lanes, or other required conditions aren't met, it immediately and seamlessly falls back to the normal E2E path.

Even better, it's configurable.

Lane Center Offset lets you fine-tune where the vehicle sits within the lane. If you prefer the vehicle to hug close to the right line, you can do that!

E2E Authority lets you dynamically choose how much the model can override lane centering. Want the model to still be able to maneuver around potholes or cyclists? Give E2E more authority. Want a more traditional centered feel? Increase Lane Centering.

Huge credit goes to u/jc01rho for designing and implementing this feature!

How does the latest openpilot model position itself on your local roads? Does it change throughout your commute? Do you prefer the model's natural path, or would you rather it stay centered between the lines?

I'd love to hear your feedback in the StarPilot discord! https://firestar.link/discord

So I built a system that teaches users how to tune their own. 

I originally learned lateral tuning with the help of twilsonco, and after hand tuning 25+ vehicles since then and seeing the massive impact it made on their openpilot experience, I started thinking: more people should learn how to do this.

One of StarPilot's biggest goals has always been tuning the vehicles we support to the highest possible quality. As we get more vehicles supported, there has been a pile of vehicles filling up my backlog requesting personalized tunes. 

So I've taken everything I've learned in the past few years about where most lateral tunes fall short and created FLM - a tool that analyzes local routes from your comma, explains why your car does what it does, suggests changes, requests feedback (because humans still matter), and generates iterative tuning suggestions using over 25 lateral tuning parameters - all from the UI. 

I think very few vehicles have experienced what true A+ lateral tuning feels like. How's your lateral experience been? 

Let me know, or try out FLM in the latest version of StarPilot and give me feedback in the discord!

https://firestar.link/discord

This whole project started because I was lazy.

If you’ve driven an HKG or Toyota vehicle in recent years, you’ve probably seen the dashboard automatically pick up speed limits using the camera. Coming from a car that didn't have this functionality, I was very jealous. Those cars can feed those speed limits directly into openpilot forks speed limit controllers. Mine couldn't.

The “solution” has been to manually add speed limits to Mapbox or OpenStreetMaps. It worked, but it was tedious. I live in rural Kansas, where my commute is almost entirely devoid of nerds contributing open source map data. I began mapping my commute when I first joined the project and within an hour said, “I ain't doin this”

We have an AI model that's good enough to drive our cars running on a smartphone chipset from years ago. How hard could it be to build one that just reads speed limit signs? I wanted StarPilot to look out the windshield, see a speed limit sign, and just know what it said.

That sounded simple enough.

It wasn’t.

The first prototype was basically held together with duct tape: I used all the public speed limit datasets I could find (glare and Lisa), some OpenCV, a little OCR, and a lot of wishful thinking. It could occasionally read a sign, but it missed 90% of them and produced enough false positives that you definitely wouldn’t want to trust it.

The important part wasn’t that it sucked - it was that it worked just well enough to start collecting better training data.

I run a fork of openpilot called StarPilot. We're heavily focused on individual vehicle tuning and testing wild ideas, which made my users the perfect candidates for this project.

Instead of relying on public datasets that barely resembled comma camera footage, StarPilot started collecting its own. Community members submitted routes from all over the country using bookmarks that were generated automatically when the model said "I think this might've been a sign," and every promising detection went through manual review. Eventually that grew into hundreds of gigabytes of real driving footage and thousands of carefully labeled speed limit signs.

From there it became an endless cycle:

• Train a better model.
• Mine more routes.
• Find more missed signs.
• Label them.
• Repeat.

The funny part is that training the neural network wasn’t actually the hardest problem.

The hard part was everything around it.

One of the biggest challenges wasn’t accuracy, it was speed. We have a full driving stack running alongside this model and every millisecond spent processing was another frame the model didn't have time to see. The faster it could process frames, the more chances it had to catch a sharp, readable speed limit sign before it disappeared. This was particularly imperative at night, when naturally camera frame rates drop alongside your chances of picking up a clear sign.

It’s slowly evolved from a proof of concept into a model that’s trained primarily on real comma footage instead of generic traffic sign datasets. It now runs entirely on-device, publishes vision-based speed limits directly into StarPilot, and no longer depends on OCR or manually maintaining map data.

And best of all…

While I'm too lazy to hand write hundreds of speed limit signs into OpenStreetMaps, this tool can now be used to automate the entire process, giving back to the open source mapping community that helps those in larger areas so well.

Give my vision speed limits a try, or help continue to build a better one than mine! All info and training stack is available on the StarPilot repo.

Massive thanks to my team that helped collect these logs so I could access them in one easy place! Sherloaf and Mary, heading up the dream team!

View the full technical writeup on the StarPilot Repo