Agent Synth - Modular synthesizer with basic agentic capabilities (WIP)
Hey all,
My first ever Reddit post so hope i'm fine :)
I'm originally a fullstack software developer as well as a musician, I already had multiple shots at this project with various coding agents in the past as C++ is not my native programming language. Only in the recent months I found that LLMs got good enough to help me build this project.
Here's the link to the repo - https://github.com/Diabl0269/agentsynth
First demo - https://youtu.be/DszUuEpnfaA
There's still much work to do on the project, so I'll start with where this project is now then proceed to the full vision.
Currently it's a basic modular synthesizer, got some nice functions like D&D of the modules, mod matrix, theme system, preset management, and more standard stuff, while the most interesting and immature feature is the very basic AI integration which currently only supports Ollama, it has the functionality to edit and create patches, just not so accurate yet. UI is also still in polish and suggestions are welcome.
The synth and all its functionality is fully open source including the AI integration, however I do plan to offer a dedicated AI subscription with a dedicated agent that should return the most optimal responses. My agent's code will be in a separate project so the main synth and integrations stay open.
A little bit about the AI vision - Modular synthesis is complex, and it's quite annoying and can get messy managing and moving all modules, especially inside a computer. AI can help managing all of it, from basic stuff like adding modules or modulations in a chain, adjusting parameters to more complex stuff like fully generating a patch based on a user request (Maybe even by existing sound input in the future) and help manage future functions that would integrate in the program.
At this stage it's still just a synth, however I also plan to add a timeline, automation capabilities, and some DAW functionality so people could create entire songs with it in the future. Of course add support for external instruments as well to allow integration of other VSTs or external hardware.
As said, I am a developer for a few years already, but not in C++. I have reviewed the file structure, and code as best as I could understand it without fully learning the language. Also made sure to have tests and full CI/CD with caching to make sure nothing breaks between PRs and new versions are automatically published. However, any comments on the code, or any kind of contribution would be very appreciated :)