I built this multiagent AI system alone in my bedroom 😂
I started 4 months ago with 0 prior knowledge and taught myself everything by reading articles on the internet.
I thought: put this on Reddit, you'll definitely get tips from other people on how to do something like this "correctly/better"
I'm really just looking for people to exchange ideas and stuff.
I'm still building it, just looking for helpful tips
THIS is how i started. You're laughing your ass off! 📹
The Beginning
PANTHEON AI
Pantheon AI – Artificial Organism Architecture
Instead of modeling an AI as a collection of independent agents, Pantheon models it as an artificial organism. Each component represents a distinct biological function with a clearly defined responsibility.
Organs
Internal systems responsible for cognition and regulation.
Thor — Executive brain; planning, coordination, decision making. Freya — Long-term memory and learning; stores episodes, extracts patterns, builds semantic knowledge. Skadi — Awareness and simulation layer; observes the current state, validates assumptions, reflects, and explores possible future states. It owns no long-term memory. Homeostasis Systems (e.g. "Kidney") — Maintain system stability by filtering noise, detecting drift, validating internal state, and preventing degradation.
Limbs
Components that interact with the external world.
Ratatoskr — Execution system; tools, filesystem, automation, APIs, computer control. Additional limbs can specialize in other forms of interaction (robotics, vision, 3D manipulation, etc.).
Limbs do not own goals or long-term reasoning. Their responsibility is reliable execution.
Principles
Memory is not awareness. Execution is not cognition. Learning is not decision making. Regulation is separate from action.
Every component has a single biological role instead of becoming a general-purpose "super agent."
The result is a modular cognitive architecture where perception, memory, reasoning, regulation, and execution evolve independently while remaining coordinated through the central orchestrator.
Agent Life cycle:
https://pastebin.com/4bKgdQrg
PANTHEON SYSTEM CIRCLE:
https://pastebin.com/mxHf7DKm
Pantheon Architecture :
### Root (4 Dateien)
| Datei | Groesse | Rolle |
|-------|---------|-------|
| main.py | 13KB | Entry point, startet alles, _R08LogStream stdout-Wrapper, MoltbookWatcher Scheduler-Hook |
| calendar_service.py | 6.5KB | Kalender-Backend (SQLite) |
| check.py | 0.3KB | Quick check script |
| restore_last.py | 2.9KB | Rollback fuer CodeApply |
| run_analyst.py | 0.8KB | Startet Freya-Analyst |
>
**Hinweis:**
r08_metrics.py wurde approved (31.07. 08:11) aber ist NICHT im Root gelandet. Backup in backups/applied/. Muss noch kopiert werden.
### core/ (48 Dateien) — Das Gehirn
| Datei | Groesse | Rolle |
|-------|---------|-------|
| llm_client.py | 52KB | LLM-API-Wrapper (alle Modelle) |
| config.py | 16KB | Pfade, Settings, Loki Avatar Map — BONSAI als einziges Ollama-Modell (01.08.) |
| task_memory.py | 33KB | SQLite Task/Step/Worker/Orchestrator Status |
| tool_governor.py | 16KB | Tool-Filterung, Capability-Gating |
| memory_manager.py | 21KB | User-Facts, Kontext (JSON) |
| memory_core.py | 16KB | Memory Foundation |
| simulator.py | 18KB | World Model — Kaskade Stufe 1+2+3 LIVE |
| decision_layer.py | 14KB | Action Selection — TM + QM verdrahtet, Q-Memory Fast Filter SCHARF |
| ai_helper.py | 19KB | AI-Helper-Layer fuer Worker-Outputs |
| llm_router.py | 10KB | Model-Routing |
| loki_bridge.py | 13KB | Thor->Loki Delegation Bridge |
| canonical_state.py | 9KB | World Model State Snapshot |
| code_apply.py | 10KB | FixReviewPanel Pipeline (gruener Knopf) |
| patch_manager.py | 9KB | Patch-Verwaltung |
| capability_store.py | 10KB | Worker-Capability-Registry |
| error_metrics.py | 12KB | Error-Scoring |
| feedback_analyst.py | 13KB | Feedback-Analyse |
| gap_detector.py | 9KB | Erkennt Luecken in Worker-Outputs |
| proactive_gap_detector.py | 9KB | Proaktive Luecken-Erkennung |
| prompt_builder.py | 12KB | Prompt-Konstruktion |
| q_memory.py | 8KB | Q-Learning Memory — 14 Eintraege, 271 Updates, avg Q=0.649, Fast Filter SCHARF |
| transition_memory.py | 11KB | Transition Memory — 16 Eintraege, 276 Transitions |
| state_manager.py | 13KB | State Management |
| summarizer.py | 11KB | Zusammenfassungen |
| token_tracker.py | 11KB | Token-Tracking |
| vision_processor.py | 9KB | Vision-Processing |
| video_job.py | 18KB | Video-Job-Management |
| youtube_auth.py | 12KB | YouTube OAuth |
| youtube_setup.py | 4KB | YouTube Setup Helper |
| status_codes.py | 6KB | Zentrale Status/Result Codes |
| logger.py | 6KB | Logging-Setup |
| event_bus.py | 3KB | Qt-Signal-Bus |
| abort_guard.py | 2KB | Pipeline-Step-Abbrueche |
| action_normalizer.py | 9KB | Action-Normalisierung |
| agent2_memory.py | 10KB | Agent Memory v2 |
| agent_goal_state.py | 13KB | Goal State Management |
| context_state.py | 3KB | Context State |
| eval_classifier.py | 8KB | Evaluation Classifier — execution_error scoring LIVE |
| eval_store.py | 7KB | Evaluation Store |
| execution_contract.py | 5KB | Execution Contracts |
| fix_classifier.py | 8KB | Fix-Klassifizierung |
| gpu_arbiter.py | 4KB | GPU-Verwaltung |
| scene_binding.py | 8KB | Scene-Binding |
| sync_source.py | 5KB | R08_source Sync (DEAKTIVIERT 15.07.) |
| trigger_resolver.py | 5KB | Trigger-Aufloesung |
| test_decision_layer.py | 5KB | Test fuer DecisionLayer |
| test_simulator.py | 7KB | Test fuer Simulator |
### orchestrator/ (25 Dateien) — Die Pipeline
| Datei | Groesse | Rolle |
|-------|---------|-------|
| loki_planner.py | 72KB | Lokis Planungs-Engine (groesste Datei!) |
| pipeline.py | 39KB | Haupt-Pipeline |
| planner.py | 43KB | Allgemeiner Planner |
| retry_orchestrator.py | 43KB | Retry-Logik |
| decision_layer.py | 53KB | Orchestrator Decision Layer (andere als core/) |
| router.py | 28KB | Intent/Action Routing |
| media_orchestrator.py | 26KB | Media-Pipeline |
| loop_engine.py | 26KB | Agent Loop Engine |
| scheduler.py | 20KB | Task-Scheduler |
| output_classifier.py | 20KB | Output-Klassifizierung |
| scene_builder.py | 18KB | Scene-Konstruktion |
| agent_loop.py | 17KB | Agent Loop — kennt browser_tools_neu.py noch nicht (Phase 1.2) |
| plan_builder.py | 16KB | Plan-Konstruktion |
| mini_orchestrator_base.py | 16KB | Basis fuer Mini-Orchestratoren |
| templates.py | 15KB | Templates |
| tool_registry.py | 16KB | Tool-Registry |
| orchestrator_registry.py | 10KB | Orchestrator-Registry |
| intent_resolver.py | 11KB | Intent-Aufloesung |
| gap_analysis.py | 11KB | Gap-Analyse |
| execution_plan.py | 11KB | Execution Plan |
| caption_builder.py | 6KB | Caption-Erstellung |
| prompt_builder.py | 3KB | Orchestrator Prompt Builder |
| insta_orchestrator.py | 2KB | Instagram Orchestrator |
| scene_policy.py | 1KB | Scene-Policy |
### workers/ (24 Dateien) — Die Haende
| Datei | Groesse | Rolle |
|-------|---------|-------|
| remotion_worker.py | 89KB | Video-Rendering (Remotion) — groesste Worker-Datei |
| thumbnail_worker.py | 63KB | Thumbnail-Generierung |
| browser_worker.py | 44KB | Browser-Automation |
| sd_worker.py | 43KB | Stable Diffusion Worker |
| youtube_worker.py | 34KB | YouTube Upload/Management |
| pexels_video_researcher.py | 25KB | Pexels Video-Recherche |
| file_worker.py | 24KB | Datei-Operationen |
| research_worker.py | 21KB | Web-Recherche |
| video_cutter_worker.py | 19KB | Video-Schnitt |
| email_worker.py | 18KB | E-Mail-Worker |
| code_worker.py | 15KB | Code-Ausfuehrung |
| notepad_worker.py | 11KB | Notizen |
| loki_image_generator.py | 10KB | Bild-Generierung via Loki |
| loki_scene_picker.py | 8KB | Scene-Auswahl |
| vision_worker.py | 8KB | Vision-Worker |
| music_query_resolver.py | 7KB | Musik-Query-Aufloesung |
| pexels_worker.py | 7KB | Pexels-API |
| base_worker.py | 12KB | Basis-Worker-Klasse |
| guten_tag_worker.py | ?? | UNTRACKED — Test/Spass-Worker? |
| hallo_worker.py | ?? | UNTRACKED — Test/Spass-Worker? |
| ich_finde_worker.py | ?? | UNTRACKED — Test/Spass-Worker? |
| meinst_du_worker.py | ?? | UNTRACKED — Test/Spass-Worker? |
| nein_ich_worker.py | ?? | UNTRACKED — Test/Spass-Worker? |
### thor/ (9 Dateien) — Der Agent
| Datei | Groesse | Rolle |
|-------|---------|-------|
| thor_agent.py | 58.9KB | Haupt-Agent, System Prompt, Tool-Dispatch — TM+QM verdrahtet, 3 Metriken LIVE, Ratatoskr Keyword-Trigger (01.08.) |
| thor_tools.py | 59.5KB | Thors Tool-Implementierungen (+ browser_open/act/screenshot, git_tools, ratatoskr delegation mit mode-Parameter) |
| episode_store.py | 13.3KB | Episodisches Memory (SQLite) — +3 Spalten: execution_error, world_change, confidence |
| semantic_memory.py | 7KB | Semantisches Memory (Facts) — 197 Eintraege |
| thor_memory_init.py | 4.9KB | Memory-Initialisierung |
| browser_tools_neu.py | 9.1KB | Browser-Tools: browser_open/act/screenshot + Approval Gate |
| git_tools.py | 8KB | Git-Tools: git_snapshot, git_log, git_status (NEU 31.07.) |
| tool_governor.py | 17.9KB | Tool Governor — Capability-Gating (NEU 31.07.) |
| video_research_tools.py | 5KB | Video-Research Tools (NEU 31.07.) |
### freya/ (8 Dateien) — Die Analystin
| Datei | Groesse | Rolle |
|-------|---------|-------|
| freya_core.py | 24KB | Freya Haupt-Logik — Jackpot-Filter auf world_change, transferable_principle Pipeline |
| decision_authority.py | 24KB | Approve/Reject-Entscheidungen |
| freya_reflection.py | 16KB | Reflexions-Engine |
| idle_cognition.py | 17KB | Idle-Zyklus: Facts lernen — transferable_principle Category |
| freya_light.py | 15KB | Leichtgewichtige Freya |
| freya_status.py | 11KB | Status-Reporting |
| proactive_queue.py | 11KB | Proaktive Vorschlaege — 7 total, 0 undelivered |
| moltbook_watcher.py | ~6KB | Moltbook Agenten-Sensor — pollt /api/v1/notifications alle 5h |
### tools/ (11 Dateien) — Werkzeuge
| Datei | Groesse | Rolle |
|-------|---------|-------|
| file_tools.py | 57.8KB | Datei-Operationen (lesen/schreiben/listen) |
| ollama_client.py | 16.8KB | Ollama-Client — BONSAI 27B als einziges Modell (01.08. Migration), Warmup entleert |
| ratatoskr_agent.py | 12.4KB | Ratatoskr Agent — FEATURE COMPLETE (02.08.). 5 Modes, Schema 1.0, JSON-Parser 4-Fallback |
| vision.py | 9.8KB | Vision/Screenshot |
| vision_click.py | 7.2KB | Vision-basierte Clicks |
| mouse_keyboard.py | 6.5KB | Maus/Tastatur |
| spotify_client.py | 6.5KB | Spotify-Integration |
| northstar.py | 5.1KB | Veraltet — Monat-1-Relikt |
| music_client.py | 3.6KB | Musik-Client |
| web_search.py | 1.4KB | Web-Suche |### Root (4 Dateien)
### memory
| Datei | Rolle |
|-------|-------|
| moltbook_last_check.json | MoltbookWatcher last_seen_id Tracking |
| q_memory.json | Q-Learning State-Action Werte — 15 Eintraege, 339 Updates, avg Q=0.672 |
| semantic_memory.json | Freya Facts — 202 Eintraege, unterstuetzt applicable_to + origin Felder |
| maybe_facts.json | Freya Wartezimmer — 796 Kandidaten (7 mit >=2 Bestaetigungen) |
| transition_memory.json | Transition Memory — 17 Eintraege, 344 Transitions |
| episodes.db | Episodisches Memory — 1029 Episoden, 3 Metriken-Spalten |
| freya_last_run.json | Freya Zyklus-Tracking (light=04.08., idle=03.08., reflection=05.08.) |
| proactive_queue.json | Proaktive Vorschlaege — 7 total, 0 undelivered. + `read` Feld (neu 06.08.) |
| token_session.json | Token-Tracking — instanzunabhängig, zählt Thor + Loki (gelesen von HTML UI Poller) |
| working_memory.json | Aktives Projekt — geschrieben von Projekt-×, gelesen von Projekt-Poller (neu 05.08.) |
### memory/Thor_memory/ — Core Memory