u/Southern-Holiday-437

Hi guys,

I would like to introduce the HPD-AI Framework. It is an all in one solution for specific parts of an ai application development: AI Agents, RAG, Workflows, Machine Learning and even TUI application development in .NET. I am still not done and everything is pre 1.0.0  but I wanted to let you guys know and I hope this brings value to any one is the community that might have needed something like this. 

Think of this like Tanstack but for mainly ai related stuff and its .NET

Check it out here if you are curious, if you have any questions let me know in the comments.

Documentation for all isnt ready yet but coming pretty soon.

https://github.com/HPD-AI/HPD-AI-Framework.git

Sub Frameworks

Hi everyone,

A while back I posted about our original HPD-Agent repo. Since then we've been heads down expanding beyond just agents. We realized we needed a full stack to build the AI applications we're planning to ship, so we closed the original repo, consolidated everything, and restructured it into six frameworks that cover a good chunk of the whole application surface.

What we built:

HPD-Agent - Middleware-driven agents and multi-agent orchestration with tool harnesses, eval support, and MCP integration.

HPD-RAG - Pluggable RAG pipelines with 8 embedding providers, 16 vector stores, and hybrid vector plus graph retrieval.

HPD-Graph - DAG workflow engine with checkpointing, parallel execution, and human-in-the-loop waits.

HPD-TUI - Allocation-conscious terminal UI framework that's native AOT compatible, with streaming markdown and diff-rendered ANSI output.

HPD-ML - Classical ML pipelines for classification, regression, and clustering. Deep learning training and testing coming soon.

HPD-Auth - Drop-in auth system built on ASP.NET Identity with JWT and cookie support, 2FA, passkeys, OAuth, and an admin API.

Context:

We spent about 18 months building these separately, then consolidated everything into a single monorepo a couple months ago. The reason we built all six is that we're shipping multiple products later this year and next year. We needed agents, retrieval, workflows, auth, and TUI all working together coherently, so we built the infrastructure ourselves rather than stitching together five different third party libraries.

This is pre-1.0. We're targeting stable 1.0 releases by the end of the year. The APIs and persistence contracts may still shift. And documentation for some of them are not fully done yet.

Why we're open-sourcing it:

We built this for our own products and we figured if we went through the trouble of building it, we'd share it, somebody can benefit form it and we might be surpsied with the outcome. This is also not a post for people to use it in production(yet) it is more of an introduction to a project on the works.

The repo:

https://github.com/HPD-AI/HPD-AI-Framework

If you have any questions, suggestions, or interest in contributing, drop a comment or DM me. Drop a star, follow or a fork if you want to.

Also, HPD stands for High Performance Driven. Someone thought it was Houston Police Department last time, so just wanted to clarify that.

Hi everyone,

I wanted to gauge demand for something my team and I have been exploring.

RAG has moved beyond the basic “chunk → embed → retrieve → generate” pattern. There are now many approaches: standard RAG, contextual retrieval, GraphRAG, hybrid retrieval, agentic RAG, reranking, contextual compression, and more.

One thing we noticed, including in our own work, is that many teams do not just need “RAG.” They need a RAG pipeline that fits the type of documents they work with.

For example, financial documents, legal contracts, healthcare records, engineering docs, research papers, support tickets, and internal company knowledge bases may all need different choices for extraction, cleaning, chunking, metadata, embedding, indexing, retrieval, reranking, graph construction, and context assembly.

So instead of building a fixed RAG product, we have been exploring a modular RAG framework.

The idea is to make ingestion and retrieval pipelines composable. Think of it as a graph/DAG-style system where teams can mix, match, replace, and optimize each part of the pipeline depending on their documents and use case.

I know there are already strong tools in this space, especially LlamaIndex and Haystack. They are highly composable and already support advanced ingestion, retrieval, query pipelines, and agent-style workflows.

The gap we are looking at is different: most of those tools are Python-first and are increasingly becoming broader AI/agent frameworks. What we are exploring is a .NET-native framework focused specifically on composable RAG ingestion and retrieval pipelines.

We are not trying to make this a full agent framework, because we already have a separate dedicated agent framework for that. The goal here is to make RAG pipelines modular, swappable, and optimized around the document domain and retrieval strategy.

So the question I am trying to validate is not “can this be built?” but whether .NET teams actually want this as a framework.

Would your team prefer:

1. a modular RAG framework where you can design your own ingestion and retrieval pipeline, or
2. a more opinionated RAG product that makes most of those choices for you?

Also, if you already use RAG in production, where do you feel the biggest pain is: extraction, chunking, retrieval quality, reranking, evaluation, observability, domain-specific tuning, or deployment?

Hi everyone,

I wanted to gauge demand for something my team and I have been exploring.

RAG has moved beyond the basic “chunk → embed → retrieve → generate” pattern. There are now many approaches: standard RAG, contextual retrieval, GraphRAG, hybrid retrieval, agentic RAG, reranking, contextual compression, and more.

One thing we noticed, including in our own work, is that many teams do not just need “RAG.” They need a RAG pipeline that fits the type of documents they work with.

For example, financial documents, legal contracts, healthcare records, engineering docs, research papers, support tickets, and internal company knowledge bases may all need different choices for extraction, cleaning, chunking, metadata, embedding, indexing, retrieval, reranking, graph construction, and context assembly.

So instead of building a fixed RAG product, we have been exploring a modular RAG framework.

The idea is to make ingestion and retrieval pipelines composable. Think of it as a graph/DAG-style system where teams can mix, match, replace, and optimize each part of the pipeline depending on their documents and use case.

I know there are already strong tools in this space, especially LlamaIndex and Haystack. They are highly composable and already support advanced ingestion, retrieval, query pipelines, and agent-style workflows.

The gap we are looking at is different: most of those tools are Python-first and are increasingly transitioning into becoming AI Agent frameworks themselves. What we are exploring is a .NET-native framework focused specifically on composable RAG ingestion and retrieval pipelines. There was Kernel Memory but that has transitioned to something else too.

We are not trying to make this a full agent framework, because we already have a separate dedicated agent framework for that. The only goal here is to make RAG pipelines modular, swappable, optimized, oh and also durable around the document domain and retrieval strategy.

So the question I am trying to validate is not “can this be built?” but whether .NET teams actually want this as a framework.

Would your team prefer:

1. a modular RAG framework where you can design your own ingestion and retrieval pipeline, or
2. a more opinionated RAG product that makes most of those choices for you?

Also, if you already use RAG in production, where do you feel the biggest pain is: extraction, chunking, retrieval quality, reranking, evaluation, observability, domain-specific tuning, or deployment?

Edit:

I felt compelled to add this diagram after reading the comments.

https://preview.redd.it/kieuuu2son7h1.png?width=2926&format=png&auto=webp&s=f43692bc3b284e975347b9360d3b747fba6809c3

Has anybody released any cool projects with the new Microsoft Agent Framework. If so please showcase what you have been able to do. I am very curious what the community has been able to do with it.