Display-only mode seems to be an often mentioned solution, I don't see how though.
I'm probably doing something wrong, but here's my experience. If my Pixel phone isn't connected to the CarPodGo via bluetooth, for example I connect my phone to the bluetooth adapter with the aux-output and mic, Android Auto does not start. Gemini will not initiate routes, nor play music. I can say "hey google" with my screen off/locked. I get a response from gemini, but it is not an gemini-android-auto session. It asks me to unlock the screen (which doesn't happen when androd auto has been initiated). I unlock the screen. I tell gemini to route me to a destination. It tells me it can't. Again, because android auto isn't running. I've tried different phones, and different bluetooth adapters, and different phone settings like disabling "phone calls" and/or "media handling" in the bluetooth connections specific to the CarPodGo device. Nothing I do will allow hands free android auto to work when in display-only mode because at the most basic level, android auto isn't even running in the background.