My Jetson Orin–optimized engine, little-gemma V1.0, substantially outperforms llama.cpp. Even after exhausting every practical GGUF option, however, it still falls short of the ideal performance level for Gemma E4B on the Jetson Orin Nano Super 8GB.

I forked exllamav3 and ported key code from little-gemma V1.0. This delivered the prefill improvement I needed. Although the decoding rate took a hit, overall voice-chat latency is lower. EXL3 on the Jetson Orin Nano isn’t for everyone, but if you need the superior prefill rate, this is it.

Fork: https://github.com/cortexist/exllamav3
Weights: https://huggingface.co/cortexist/gemma-4-E4B-it-qat-exl3-4.0bpw

I finally got Orin Nano Super 8GB setup, benchmarked against Orin NX 16GB, here are the results. I'm using my own Little-Gemma engine, the model is E4B QAT Q4_K_XL. The full-head MTP is the Google original, selected-16K is my own 16K reduced vocab head.

$500 module outperformed $900 module. The upcoming nano 2 will be even better.

EDIT: Put Orin NX in Super mode, got 1,042 tok/s prefill and 30.5 tok/s decode, which is 9.7% and 8.9% up from Nano Super. But my carrier couldn't hold up for too long at the 40W power consumption. I'll put on a bigger heatsink and see how it goes.

Gemma 4 12B runs on an RTX PRO 4500 Blackwell. Gemma 4 E2B run on a Jetson Orin NX 16GB; similar performance is expected on a Jetson Orin Nano Super 8GB. Both systems use a reSpeaker Flex 4-mic array and a 3W speaker. Inference is handled by Cortexist Little Gemma, a small LLM engine written in C for CUDA devices. On Jetson Orin it is faster than llama.cpp, and no degradation after long voice prompt. The pipeline supports lip sync, expressions, and gestures. Everything is open source.

They talk to humans too.

The engine source code: https://github.com/cortexist/little-gemma

The Rust code is super fast, simply saves your notes in markdown files in directories structured by date, can be picked up by Obsidian or similar (repo: https://github.com/cortexist/simple-note). The screenshot shows 2 notes on top of Sigma File Manager (our wayland/sway fork).

Custom Icon Font for Waybar

My Waybar config is unlikely to work out of the box for you. It depends on a full set of shell scripts written for my specific environment. Once you understand the approach, though, you can adapt the same idea to your own setup.

Step 1: Build a custom icon font from SVGs

The first (and most important) step is to create application icons from SVG files you find online and compile them into your own icon font.

I used a small Python script with the fontTools package to do this, but you don’t have to. Plenty of GUI typography tools make the process easier if you prefer a visual workflow.

The figure shows the method I created two-color icons for waybar. I have 2 SVGs for the same icon, the first one is the background, pay attention to "advance 0", GTK will overlap the foreground on top without shifting.

I mapped the icons starting at U+100000 (Plane 16). This range is reserved for private use in Unicode, so it’s ideal for custom Waybar icons.

Step 2: Installation & CSS setup

1. Place the resulting font file (waybar_icon_font.ttf) in: ~/.local/share/fonts/
2. In ~/.config/waybar/style.css, set the font stack like this: CSSfont-family: "My Icons", "JetBrains Mono", "Font Awesome 7 Free";

Even though the custom font only contains a dozen or so icons, the full Nerd Fonts set (and Font Awesome) remain available as fallbacks.

We forked Sigma File Manager and extended it specifically for Sway. One of our first additions is a floating file picker—something that may seem at odds with a tiling window manager. Enable it under Settings → Experimental for use in all apps (VS Code is shown in the screenshot). https://github.com/cortexist/sigma-file-manager

We forked https://reddit.com/r/SigmaFileManager and adapted it for Linux, modernizing the media player while adding a subtle nostalgic touch. The fork: https://github.com/cortexist/sigma-file-manager

arch, sway, waybar, sigma-file-manager (my fork on github). my forks link (a lot of improvements for Linux over the upstream): https://github.com/cortexist/sigma-file-manager

I've never designed a water block. This could be my first attempt.

I found these RTX Pro 4500 Blackwell photos online. I measured the dimensions, it seems the dimension of the PCB is close to the 170x170 mm mini-ITX footprint. The 2nd photo shows there is barely anything but some wires behind the blower. Will a 170x100mm single slot custom water block handle the 200W power power consumption?