Rebuilding my custom agentic AI because the original architecture was too heavy
I’ve decided to rebuild a custom agentic AI I’ve been working on.
The previous architecture became too heavy for the kind of system I actually want to build. Instead of solving that by throwing more compute at it, I decided to rethink the architecture from the ground up.
One of the biggest changes is that I removed the call to a pre-made LLM.
I’m now working on building my own intelligence layer instead of relying on an external LLM API as the core of the agent.
The goal is to have a lightweight agent that can develop capabilities such as:
Perception
Memory and knowledge
Decision-making
Learning from data
Behavioral adaptation
Interaction with its environment
Taking actions rather than simply generating text
I’m not trying to claim that I’ve built a replacement for today’s large LLMs. This is still a work in progress, and rebuilding the architecture means I’m essentially experimenting with the fundamentals again.
The interesting challenge for me is seeing how much agentic behavior can be achieved with a much smaller and more efficient architecture, rather than simply increasing model size and compute.
I’m curious what others here think: when building an agent from the ground up, how much of the intelligence actually needs to come from an LLM?
