Kokoro Web UI

I recently switched Hermes from Edge TTS to Kokoro-FastAPI running locally in docker and I'm really happy with the results.

For me, the main advantages over Edge are:

• Better voice quality, with a pretty large selection of voices
• Voice mixing, so you can blend multiple Kokoro voices together to get the voice you want
• Very fast local generation — on my RTX 3080 I'm seeing sub-second generation for most of Hermes responses
• Streaming TTS support - Though there is currently an open bug on this part (see below)
• No internet dependency for speech generation once everything's setup

Docker setup

I'm running Hermes and Kokoro as separate Docker Compose projects on the same Docker network (webproxy). This lets Hermes access Kokoro directly using Docker DNS at http://kokoro:8880.

My compose.yml for Kokoro looks like this:

name: kokoro

services:
  kokoro:
    image: ghcr.io/remsky/kokoro-fastapi-gpu:latest
    container_name: kokoro
    restart: unless-stopped
    ports:
      - "8880:8880"
    environment:
      - TZ=America/Chicago
      - USE_GPU=true
      - DEVICE_TYPE=cuda
      - API_LOG_LEVEL=INFO
      - ENABLE_WEB_PLAYER=true
      - DOWNLOAD_MODEL=true
    deploy:
      resources:
        limits:
          memory: 3G
          cpus: "1.0"
        reservations:
          devices:
            - driver: nvidia
              device_ids: ["0"]
              capabilities: [gpu]
    networks:
      webproxy:

networks:
  webproxy:
    external: true

I'm using the NVIDIA GPU image here. If you're running CPU-only, AMD, or newer Blackwell hardware, check the Kokoro-FastAPI README because there are different image options.

Also note that in my setup above the webproxy network is managed by Nginx. You may have different networking configuration here.

To start it:

docker compose pull
docker compose up -d

A useful check afterward is:

docker exec kokoro nvidia-smi

This will verify that GPU acceleration is enabled in the container.

Kokoro also has a Web UI at:

http://YOUR-SERVER:8880/web

The Web UI was helpful for testing voices and experimenting with voice mixes. You just add as many voices as you want on the right and change the percentage of each.

Connecting to Hermes

Kokoro-FastAPI exposes an OpenAI-compatible TTS API, so Hermes can use its existing openai TTS provider with Kokoro as the backend.

Unfortunately, some of the settings required for this aren't exposed through Hermes Desktop or Web Dashboard right now (e.g. base_url) so I had to edit config.yaml manually.

The relevant portion is:

tts:
  provider: openai

  openai:
    api_key: local-kokoro
    base_url: http://kokoro:8880/v1
    model: kokoro
    voice: am_michael+am_puck
    speed: 1.2

❗IMPORTANT: If you aren't using OpenAI APIs anywhere else, Hermes will require api_key to contain a value here. Even though Kokoro doesn't need or use an API key. The value itself doesn't matter.

Obviously, you can change the voice and speed to whatever you prefer, and adding two voices together mixes them:

am_michael+am_puck

I don't know the syntax for mixing with percentages.

After saving config.yaml, restart Hermes.

Because both containers are on the same Docker network, Hermes connects directly to:

http://kokoro:8880/v1

If your containers aren't on the same Docker network, you'll need to adjust that URL appropriately.

A nice sanity check from inside the Hermes container is:

docker exec hermes curl -fsS http://kokoro:8880/v1/audio/voices

If that returns Kokoro's voice list, Docker networking between the two is working.

Current Bug: Hermes Desktop streaming

Kokoro supports input streaming, but input streaming is currently broken through Hermes Desktop's OpenAI TTS path.

There's an open Hermes bug for it here:

https://github.com/NousResearch/hermes-agent/issues/79859

The current behavior is basically:

Hermes generates entire response -> TTS audio begins playback

instead of beginning playback as soon as the first completed sentence/chunk is available. This isn't great, but it's still better than Edge because Kokoro generation is blazing fast and Edge currently has a similar issue. The good news is this is a known issue and is being worked on. Once fixed, Kokoro responses will feel even snappier.

Updating Kokoro

Since I'm using latest as my image, updating Kokoro is just:

docker compose pull
docker compose up -d

That's basically it. I hope someone finds this useful!

I'm hoping some folks in this community can help me understand the performance gap between Hermes using GPT-5.5 and ChatGPT using GPT 5.5. I have been very interested in using Hermes as a replacement for Alexa, but response times for basic queries are just too slow.

I understand that ChatGPT benefits from prompt caching and result caching across all their millions of users, but this still doesn't seem to explain the performance gap for novel queries.

For example:

>How did SpaceX stock do this week?

ChatGPT with GPT 5.5: Produced a chart in 0:03 seconds and a full report in 8 seconds.

Hermes with GPT 5.5: Produced a much weaker report in 0:24 seconds. 16 seconds (or 3x) longer.

I fully expected prompt / answer caching here, so this didn't shock me. But 3x was still significant.

So, I wanted to test a prompt that would be highly novel and unlikely to be a cache hit:

>Are there currently any donut shops in Tomball TX offering Buffalo Wing flavored donuts?

ChatGPT with GPT 5.5: Responded in 0:03 seconds with a no and even included menus from 5 donut shops in the area.

Hermes with GPT 5.5: Responded in 0:52 seconds and the answer was basically "No, I couldn't find anything like that."

This one was significantly longer. 49 seconds or 17x longer.

Interestingly, weather was a closer race for both:

>Is it going to rain tomorrow in Cypress TX?

ChatGPT with GPT 5.5: Simple answer + 5 day and hourly forecast in 4 seconds.

Hermes with GPT 5.5: Simple answer only in 7 seconds (no graphics).

For me, these response times are usually just too long to wait (especially when asking basic questions over a smart speaker). How quickly the weather question was answered gives me a little hope that with a basic 'quick answer' skill I could get some prompts to respond in a meaningful time. But why is there such a gap here?

I know I can make another Hermes profile where all tools are disabled, but that reduces capabilities. And I don't want to turn off the Home Assistant skill because I need those capabilities to replace Alexa.

I appreciate any thoughts and learnings the community has to share. Thank you!

https://preview.redd.it/p598cjgjhgch1.png?width=441&format=png&auto=webp&s=b8854dc0ffc75537e6872360c7b216bc6edf0308

Hey, I guess it's still resourceful...