Important RAG (LM Studio + AnythingLLM)
Hello !
I am quite new to the world of Open source LLM.
New owner of a MacBook Pro M5 with 24 GB of RAM, I have a very large number of medical documents and books (~ 900 PDFs with a large majority of more than 500 pages, all medical topics combined (radiology, cardiology, neurology, forensic medicine etc etc...).
So here are my questions:
- Should I classify these themes in different workspaces on AnythingLLM or can I afford to send him all my documents (in order to have more transversality between the different medical areas in my answers)?
- Do I have to change the embedding model that is originally Nomic 1.5 on LM Studio?
- For this RAG, is it better to use a model ~9B of parameters in Q8 see BF16 or a model ~ 30B in Q4?
- I am only looking for Text to Text but my documents have many images illustrating the texts. Do I have to use a model with a vision encoder or is it not necessary in case of RAG?