r/SonicAnalysis

Hi all,
with this post I want to talk again of AudioMuse-AI, a free and open source selfhostable software to analyze your song and automatically create playlist on your supported music server like Jellyfin, Navidrome (or open subsonic api based), Emby and Lyrion:

• https://github.com/NeptuneHub/AudioMuse-AI

With this post I want to celebrate two big things, first of all AudioMuse-Ai born on May 2025, so it's stil live and fully mantained after 1 years, 217 issue closed and 182 PR closed !

We also want to celebrate the new AudioMuse-AI v1.1.0 release that introduce Lyrics Semanthics similarity throug different functionality.

I'm very proud of this release because multiple time we heard that yes the mood is similar but totally different lyrics, now you can search your song also semathically with:

• Axis-based search: Explore songs across 5 defined semantic axes, selecting one or more values that best describe the target mood or meaning.
• Text search: Simple natural language queries (e.g., “love”, “run”) focused on lyrical meaning, not musical groove (distinct from DCLAP search).
• Song similarity search: Use a reference track to find similar songs, weighted by default as 75% lyrical meaning and 25% audio similarity to preserve genre consistency.

Lyrics functionality off course need lyrics, the best way is to have already them in your music server OR configure in AudioMuse-AI your favourite API in the setup wizard:

Example API formats supported in Setup Wizard:

https://api.example.com/get?artist={artist}&title={title}
https://api.example.com/v1/{artist}/{title}

Anyway as a fallback is also supported the transcription with Whisper Small and if needed can be disabled in the setup wizard by setting LYRICS_ENABLED=true

Important: after the update a new analysis will do the Lyrics analysis on the already analyzed song (if enabled, enabled by default) or a full analysis (Musicnn + Clap + Lyrics) for new song. This new analysis is mandatory to use the new functionality.

I hope you will like both of this milestone and as usual, if you want to support AudioMuse-AI, please add a start on the github repository.
Thanks to be with us for our first year!

Hi All,
this post is to show you AudioMuse-AI resources usage on a Raspberry Pi 5 8GB with NVME SSD hat, during the analysis. All the number are made over the last v2.4.0 release of today.

First of all for whom don't know AudioMuse-AI is a free and opensource software that enable to analyze the raw file of your song (sonic analysis) and based on this analysis it enable to create automatic playlist.
It work with Jellyfin, Navidrome (and other Open Subosnic API compatible music servee) Emby and Lyrion. Also made avaiable Jellyfin Plugin, Navidrome Plugin and I hope soon also an Music Assistant AudioMuse-AI provider plugin that will enable to command it with voice!

..and of course is all selfhostable and privacy first: your computer, your analysis, your data! no one can block you in future behind a paywall!

The reason for this post is that multiple user tought about it as something heavy, but it can work even on a Raspberry PI 5.
In the attached image you can show it during the most heavy part that is the analysis, and you can look how in avarage (k9s screenshot) it use half of the CPU/RAM resources and on the pike it still don't saturate them.

And speaking about resources, eare is the avarage analysis time per track on a Raspberry PI 5:

• Average analysis per track time: ~31 s

Breakdown (per track):

• Download: ~1 s
• MusiCNN analysis: ~9 s
• CLAP load + segment processing + unload: ~10 s
• Lyrics API lookup: ~7 s (NO ASR, off course depending from the API response time)
• Embedding: ~1 s
• ONNX session recycling: ~3 s

This to say that we don't just have it working, but it work also on low hand hardware. For more speed, no problem, you can run multiple worker in parallel during the analysis. Just wake up a worker on your desktop or your laptop!

And what about the idle resources? CPU in idle is not used, and about RAM we worked to balance the time to respond to a first API request and the memory usage, the number for a 188k+ library are:
- Flask RAM in idle: 1282mb => it load up to 3.5-4Gb, and then unload after 5 minutes idle
- Worker RAM in idle: 198mb

and the time for a call, still stay in the order of ms!

About the functionality you can ho on github and look around, you can also navigate some screenshot here:
- https://github.com/NeptuneHub/AudioMuse-AI/tree/main/screenshot/example

The one for which I'm more proud is the Lyrics search by song: it get in input a song and is able to search similar not only by their grove but also by their lyrics.

Hope you can enjoy all of this and maybe convince some new user that AudioMuse-AI is for everyone! and if you like it, please don't miss the chance to leave a ⭐on the github repo!

This is an actual text from my wife this morning after showing her how to start a radio mix for her drive to work! In a year of homelabbing and cord-cutting, this is the first unsolicited praise I’ve received hahaha it was just so nice to see!

We have about 26,000 songs in our library, and I started the sonic analysis last month on my mini PC, which ran off and on during that time and finished last night.

I have a habit of shooting myself in the foot when showing off anything new I set up, and in classic fashion to show my kids how it worked this AM, I started a radiomix off of the completely singular song “Mahna Mahna“ from the Muppets… and it followed it with a Tom Waits song and a bunch of other gravelly voiced, silly sounding music. We were thrilled!!

Thank you u/Old_Rock_9457 for helping my family reconnect with our music after bailing on the streaming services!!