RAG on Qualcomm's newest Snapdragon X2 Laptop, 200k documents
The video is available on another Reddit Channel
https://www.reddit.com/r/LocalLLaMA/comments/1te93s3/rag_on_snapdragon_x2_laptop_200k_documents/
๐๐ข๐ ๐ก๐ฅ๐ข๐ ๐ก๐ญ๐ฌ:
โข ๐๐๐ฌ๐ฌ๐ข๐ฏ๐ ๐๐จ๐๐ฎ๐ฆ๐๐ง๐ญ ๐๐จ๐ฅ๐ฅ๐๐๐ญ๐ข๐จ๐ง: ~200,000 files being indexed (~100,000 completed in this run)
โข ๐๐จ๐ฐ-๐ญ๐จ๐ค๐๐ง ๐ซ๐๐ญ๐ซ๐ข๐๐ฏ๐๐ฅ: only ~1200 retrieval tokens used in this experiment
โข ๐๐จ๐ฐ-๐ฆ๐๐ฆ๐จ๐ซ๐ฒ ๐๐๐: most data offloaded to disk with only a 128-shard active buffer
โข ๐ ๐๐ฌ๐ญ ๐๐ง๐ ๐๐๐๐ฎ๐ซ๐๐ญ๐ ๐๐๐ ๐ฉ๐๐ซ๐๐จ๐ซ๐ฆ๐๐ง๐๐ ๐จ๐ง-๐๐๐ฏ๐ข๐๐
๐๐๐ก๐ข๐ง๐ ๐ญ๐ก๐ ๐ฌ๐๐๐ง๐๐ฌ, ๐๐๐๐๐โ๐ฌ ๐๐ฅ๐ฅ-๐ข๐ง-๐จ๐ง๐ ๐๐ ๐๐๐ญ๐๐๐๐ฌ๐ ๐ฉ๐ฅ๐๐ฒ๐ฌ ๐ ๐ค๐๐ฒ ๐ซ๐จ๐ฅ๐.
Enterprise-scale AI systems typically require multiple databases working together:
โข Vector database
โข Graph database
โข Relational database
โข Key-value store
โข Search database
โข Document database
We developed an in-house AI database platform that integrates the core functionality of all six systems into a unified architecture for enterprise AI and agent systems.
This enables joint optimization across indexing, retrieval, graph traversal, storage, and memory management, helping achieve low-token, low-memory, fast, and accurate AI systems on both cloud and AI-PC deployments.