Is llama.cpp suitable and capable to run NVFP4 models

With Qwen3.8:27B arriving yesterday, including NVFP4 versions, can llama.cpp successfully run these models, or must I use a separate instance of vLLM for NVFP4 models specifically?

Searching on the internet initially made me believe that vLLM was the only way, but then some more recent posts hinted that llama.cpp is being developed for NVFP4 as well.

If llama.cpp is indeed capable of running NVFP4 models, I'd appreciate some pointers on how to get llama.cpp running NVFP4 models.

reddit.com
u/Right_Fun_4902 — 4 days ago

New Zealand’s big infrastructure debate with Stuff

Warning: It is slightly over an hours long, but well worth watching it.

All the representatives were well informed on the issues and solutions, and it was great to see how well they all worked together and collaborated across party lines.

Really a demonstration of how aligned all the NZ parties actually currently are on infrastructure development and role-out.

Congratulations to Eng NZ and Stuff Media for arranging this and not trying to be controversial.

This just shows there is indeed great prospects for New Zealand, in spite of all the negative sentiments on Social Media and traditional media.

Just a minor observation that not all parties in NZ were present, with for instance TPM and TOP absent (not sure if they were invited).

youtube.com
u/Right_Fun_4902 — 13 days ago

Christopher Luxon calls out NZ First's Stuart Nash on 'sexist' comment | RNZ

How many NZF supporters will not vote for them due to Stuart Nash?

rnz.co.nz
u/Right_Fun_4902 — 29 days ago

Multi-User Hermes Configuration

I would like to provide access to Hermes Agent to a number of different users. My initial intent was just to create different profiles for each user, with the default agent used to administer the overall system for all the profiles, including creating and setting up new profiles for each user. But the more I look into this and use Hermes Agent (and now also the Desktop), the more it looks like the system design of Hermes actually requires completely separate Hermes instances for each user, with each user then able to run and control their own ring-fenced agent, profiles and workspaces as well as local linux user account.

Thus, if multiple Hermes Agents are to be installed on the same server, most likely a docker instance for each user will be the answer. I might still be able to use the default agent profile of the first agent to administer the whole system, but would not be too surprised if there are some hard Hermes safety guards that will prevent this.

For those that configured Hermes for multiple users, is the "docker Hermes instance per user" the correct way of doing this for all users (including the administrator Agent), or to simplify administration and access rights, do a direct local install for the first agent and the rest then spun up in separate docker images for each user, or is there another way?

Also, are there any hard built in Hermes safety guard rails that will prevent using the agent to spin up additional docker instances and configure those for other users?

reddit.com
u/Right_Fun_4902 — 3 months ago

Mandatory Rule Compliance and Hermes Agent

I have been playing around with HermesAgent for almost a month, but the one thing that I can't get right is to get the Agent to follow a rule all the time.

My Hermes Agent doesn't always follow my cardinal rule for file editing. When queried, he apologises profusely. I have now included in SOUL.md, but the results are similar. I have asked the Agent, why he ignored the rule, and he indicated that the SOUL.md and USERS.md file are just guidance. How can I make something a hard and fast rule?

Gemini suggested a system prompt injunction, or an edit tool wrapper, but before I go down that route, I thought I'll ask the community what they have done to solve this problem.

See the extract of the SOUL.md below:

## Cardinal Rule for Script Modifications:
BEFORE modifying ANY existing file, you MUST create a backup copy in the .old/ directory with timestamp.
Required Format:
- bash
- BACKUP_DIR=$(dirname "$FILE_PATH")/.old
- mkdir -p "$BACKUP_DIR"
- cp "$FILE_PATH" "${BACKUP_DIR}/${BASENAME}-bak-$(date '+%Y%m%d-%H%M%S').${EXTENSION}"
- VERIFY backup exists BEFORE proceeding with edits
u/Right_Fun_4902 — 3 months ago

Class action lawyers seek $675 million in Bayer Roundup settlement | Reuters

Why are Kiwis so addicted to using weed killers everywhere when it is proven to be carcinogenic?

And what will it take to finally ban Roundup and all other glyphosate products in New Zealand?

reuters.com
u/Right_Fun_4902 — 3 months ago
▲ 139 r/aotearoa

Kiwibank - Why doesn't Government, Councils and Public Institutions use it?

Why doesn't Government and all Councils exclusively use Kiwibank?

With Kiwibank fully owned by the government, what are the reasons for the Government, all Councils as well as all public funded organisations not using Kiwibank?

I have not once seen a public funded organisation use Kiwibank for their banking.

Surely if they all exclusively use Kiwibank, it will make Kiwibank much more profitable, and keep it within the bank to improve services and service delivery for all.

Right now, the government, councils and public funded organisations are funding foreign banks, to the detriment of kiwis.

reddit.com
u/Right_Fun_4902 — 3 months ago

Jellyfin Media Player

I had Jellyfin Media Player installed via flatpak and it worked very well after experiencing problems with the newer Jellyfin Desktop(also flatpak).

But today, after an update, my Jellyfin Media Player disappeared.

I had a look at flathub, and Jellyfin Media Player doesn't exist anymore. Only Jellyfin Desktop is there.

Then I had a look at the GitHub pages, and I could find Jellyfin Media Player's site from Jellyfin Desktop's page. With Jellyfin Media Player's page now renamed Jellyfin Desktop QT. I tried to download the appimage, but it no longer works.

It appears as if Jellyfin Media Player, was renamed to Jellyfin Desktop as well, to force migration to Jellyfin Desktop.

So far I haven't experienced the same old problems with Jellyfin Desktop, but I've only used it for a short while.

I really hope all the bugs with Jellyfin Desktop have been resolved, else I'll have to search for the app image of Jellyfin Media Player or compile directly from the source code on GitHub.

I have a unique setup where I'm on CachyOS on Ryzen 7600, 64GB and RTX 5070ti running Jellyfin Docker, and media on a ZFS pool. As I'm running the player on the same machine, I don't want any transcoding to ensure no degradation in quality.

reddit.com
u/Right_Fun_4902 — 3 months ago

I have just managed to installed Bazzite after numerous issues during the installation process, which eventually I solved by installing Fedora Kinoite, and then rebaising it to Bazzite. The main problem was because of the partitions and volumes that weren't recognised by the Bazzite installer, and also could not be configured, but was resolved when using Blivet-GUI partition software as part of the Fedora Kinoite installer.

As I'm running an Intel 4790, 16GB and GTX1070, I used the recommended image:

rpm-ostree rebase ostree-unverified-registry:ghcr.io/ublue-os/bazzite-nvidia:stable

Note that it must be unverified for the first rebase.

then, after a reboot you can use the signed version by running:

ujust verify-image

and finally to install the missing flatpaks:

ujust _install-system-flatpaks

Everything seems to be working, but I'm missing the default Bazzite theme in KDE.

Any assistance and advice to restore the default Bazzite KDE theme and settings would be appreciated.

reddit.com
u/Right_Fun_4902 — 4 months ago
▲ 2 r/PikaOS

I have tested PikaOS recently on some of my laptops, and like it a lot. However, before I move my main system from CachyOS, I wanted to confirm how PikaOS plays with ZFS.

I have seen another post here, but it is very old and not sure about the current status:

https://www.reddit.com/r/PikaOS/comments/1kck9ny/zfs_on_pikaos_borked/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

My main system is a used as a combined Gaming Machine, Local LLM Server (Ollama with OpenWebUI in docker), Jellyfin Server (Docker) and direct player (Jellyfin Media Player Flatpak), Internet browsing and Youtube machine (Brave) that is located in our living room and connected to our family's main TV.

The OS doesn't have to be on on ZFS boot drive, as I'm normally using whatever the default for the OS is. But my OS must be able to play nicely with the ZFS Pools as the game libraries and Jellyfin media files are located there. Currently using 4 x 20TB in ZFS1 config, with the main hardware consisting of Ryzen 5 7600, 64GB ram and 5070ti. Boot drive is a 1TB NVME drive.

My main reason for moving from CachyOS is that after 20 years of Ubuntu & Mint, I really struggle to get used to the Arch way of doing things. I haven't had any issues on CachyOS running ZFS and it just worked from day 1. However, if PikaOS doesn't play nice with ZFS, I'll have to continue my search. I'm particularly worried that the regular updates, will break my ZFS system regularly.

If anyone has been running a ZFS PikaOS system, I'd love to hear from you before I make the big move.

reddit.com
u/Right_Fun_4902 — 4 months ago