u/player0497

Hey everyone,

I’m working on a personal project to build a completely local, lightweight system (codename: Orin) that can process messy unstructured information and segregate/clean it into highly structured, tabular formats (CSV files). Essentially, it's meant to be a better, fully offline version of Atlas.

Here is the exact data structure and the pipeline I am trying to build:

1. The Target Data Schema

The model needs to take raw info and divide it into clear subtopics:

• Columns: Topic | Subtopic1 | Subtopic2 | Subtopic3 | Info
• Example Output:
  • Topic: Flying machine
  • Subtopic1: Airplane
  • Subtopic2: Passenger plane
  • Example Scenario: If incoming news data says "Qatar Airways wins starring award again", the model should automatically categorize it under the correct subtopic hierarchies and store the relevant data in the final Info column.

2. Proposed Pipeline & Architecture

I am planning a Retrieval-Augmented Generation (RAG) approach using a combination of specialized, local agents:

• A Fact Searcher / Main Topic Searcher: To find missing points and gather core data from the dataset.
• A Local Summarizer / Keyword Generator: Acting as a text quantizer to condense the given prompt or raw context.
• A Joke Generator (Optional Component): To add humor or personality to the generated answer output.
• The Core Logic Flow: Prompt → Gathers data for it → Finds missing points → Fills the spots (to Phrase) → Final Answer.

3. The Big Bottleneck: Hardware Constraints & Failed Attempts

Since this system must run locally, finding the right LLM engine and model has been incredibly difficult. Here is what I’ve attempted so far:

• llama.cpp: Would technically work, but performance is a massive issue (it took over 2 hours just to compile 8%).
• TinyStories: Super fast at stitching sentences together, but it only tells stories; it cannot handle this specific data formatting task.
• TinyLlama (llama.co): Unable to get it to work properly / wouldn't run.
• Ollama: Cannot use it seamlessly because it isn't properly optimized or built for my hardware (ARM chips).

I would like to ask the community how to make the better and how to develop it to efficient RAG model For my Project.

Hey everyone,

I’m working on a personal project to build a completely local, lightweight system (codename: Orin) that can process messy unstructured information and segregate/clean it into highly structured, tabular formats (CSV files). Essentially, it's meant to be a better, fully offline version of Atlas.

Here is the exact data structure and the pipeline I am trying to build:

1. The Target Data Schema

The model needs to take raw info and divide it into clear subtopics:

• Columns: Topic | Subtopic1 | Subtopic2 | Subtopic3 | Info
• Example Output:
  • Topic: Flying machine
  • Subtopic1: Airplane
  • Subtopic2: Passenger plane
  • Example Scenario: If incoming news data says "Qatar Airways wins starring award again", the model should automatically categorize it under the correct subtopic hierarchies and store the relevant data in the final Info column.

2. Proposed Pipeline & Architecture

I am planning a Retrieval-Augmented Generation (RAG) approach using a combination of specialized, local agents:

• A Fact Searcher / Main Topic Searcher: To find missing points and gather core data from the dataset.
• A Local Summarizer / Keyword Generator: Acting as a text quantizer to condense the given prompt or raw context.
• A Joke Generator (Optional Component): To add humor or personality to the generated answer output.
• The Core Logic Flow: Prompt → Gathers data for it → Finds missing points → Fills the spots (to Phrase) → Final Answer.

3. The Big Bottleneck: Hardware Constraints & Failed Attempts

Since this system must run locally, finding the right LLM engine and model has been incredibly difficult. Here is what I’ve attempted so far:

• llama.cpp: Would technically work, but performance is a massive issue (it took over 2 hours just to compile 8%).
• TinyStories: Super fast at stitching sentences together, but it only tells stories; it cannot handle this specific data formatting task.
• TinyLlama (llama.co): Unable to get it to work properly / wouldn't run.
• Ollama: Cannot use it seamlessly because it isn't properly optimized or built for my hardware (ARM chips).

I would like to ask the community how to make the better and how to develop it to efficient RAG model For my Project.

Hey everyone,

I’m working on a personal project to build a completely local, lightweight system (codename: Orin) that can process messy unstructured information and segregate/clean it into highly structured, tabular formats (CSV files). Essentially, it's meant to be a better, fully offline version of Atlas.

Here is the exact data structure and the pipeline I am trying to build:

1. The Target Data Schema

The model needs to take raw info and divide it into clear subtopics:

• Columns: Topic | Subtopic1 | Subtopic2 | Subtopic3 | Info
• Example Output:
  • Topic: Flying machine
  • Subtopic1: Airplane
  • Subtopic2: Passenger plane
  • Example Scenario: If incoming news data says "Qatar Airways wins starring award again", the model should automatically categorize it under the correct subtopic hierarchies and store the relevant data in the final Info column.

2. Proposed Pipeline & Architecture

I am planning a Retrieval-Augmented Generation (RAG) approach using a combination of specialized, local agents:

• A Fact Searcher / Main Topic Searcher: To find missing points and gather core data from the dataset.
• A Local Summarizer / Keyword Generator: Acting as a text quantizer to condense the given prompt or raw context.
• A Joke Generator (Optional Component): To add humor or personality to the generated answer output.
• The Core Logic Flow: Prompt → Gathers data for it → Finds missing points → Fills the spots (to Phrase) → Final Answer.

3. The Big Bottleneck: Hardware Constraints & Failed Attempts

Since this system must run locally, finding the right LLM engine and model has been incredibly difficult. Here is what I’ve attempted so far:

• llama.cpp: Would technically work, but performance is a massive issue (it took over 2 hours just to compile 8%).
• TinyStories: Super fast at stitching sentences together, but it only tells stories; it cannot handle this specific data formatting task.
• TinyLlama (llama.co): Unable to get it to work properly / wouldn't run.
• Ollama: Cannot use it seamlessly because it isn't properly optimized or built for my hardware (ARM chips).

I would like to ask the community how to make the better and how to develop it to efficient RAG model For my Project.