Custom SFX classifications
Idea: local AI tool to auto-sort your SFX library into your own categories — feasible?
I've got a large sound effects library that's a pain to search through, and I started looking into whether AI classification could help sort it automatically. Turns out most of the pieces already exist — just not glued together the way a solo dev/hobbyist would want. Wanted to sanity-check the approach with people who've done audio ML or asset pipeline work.
The idea: point a tool at a folder of SFX, and for each file get back a best-fit category from your own category list (not a generic AI taxonomy), plus some searchable descriptive tags — so instead of scrolling 2,000 files you can filter to "impacts, low-pitched, short" or whatever.
Why it seems doable without training anything:
Pretrained audio tagging models (AST-class, trained on datasets like AudioSet) already classify raw audio into 500+ generic sound types — impacts, mechanical noise, vocalizations, ambience, etc. — and run locally on normal hardware, no cloud needed. That part's solved and free. The actual work is just the layer on top:
Classify — run each file through the pretrained model, get ranked generic labels with confidence scores.
Map to your categories — a lookup table that clusters those generic labels into whatever categories you actually use (e.g. "metallic impact / clang / thud" → your "Weapon Hit" bucket). Anything that doesn't clear a confidence threshold gets flagged "Other / needs review" instead of force-sorted wrong.
Extract search qualities separately — this part isn't even AI, it's basic signal analysis: duration, pitch center (spectral centroid), brightness (spectral rolloff), attack sharpness, etc. Cheap, deterministic, no model needed.
Output — a sidecar file or index (JSON/CSV) per sound with category + confidence + quality tags + a review flag.
None of this needs custom model training — it's an existing classifier plus a mapping/thresholding layer plus some standard DSP feature extraction. Feels like a scoped weekend-ish tool rather than an ML research project.
Where I'd want to sanity-check before building the mapping layer: run the raw pretrained classifier against a sample batch first and manually check what labels it actually produces, before designing the category-mapping table around assumptions. Also unclear how much the confidence threshold will need tuning per-library — too loose and "Other" is useless, too tight and it rejects correct matches.
Has anyone actually built something like this, or run into the wall where the generic labels don't map cleanly onto real use-case categories? Curious if there's a known gotcha I'm not seeing.
