u/Icy_Calligrapher4022

I am building a Flask based chatbot application using DO's Agent Platform. The idea is that the users are typing messages directly in the application, the queiry is sent through API to the agent, agent retrieves the data from the KB and LLM generates the response which is sent back to the user(app). Pretty straightforward and in most of the cases its working.

The problem is that for some queries the agent returns the generic response written in the Agent's Prompt that there is no available information, but the information is already in the KB. The KB is made by few Word documents formated and primized for RAG, separated into sections and uploaded through DO console(no APIs, no Cloud Storages, directly uploaed in the KB).

All the documents are in Bulgarian, 99% of the queries are also in Bulgarian language, but when I am testing the chatbot from the app or from the Agent's Playground, it sends back the generic response for missing information. I am observing two strange behavious:

Let's take an example question: "How to contact person X?" (as part of the KB I have few docs containing contact information - name, email, position, etc.)

1. If I ask the question in English, everything is just fine, the agent is able to retrieve the information from the Bulgarian written docs and generates a valid response

2. Debugging with the same question in the RAG playgorund in both Bulgarian and ENnglish, the revtieval is fine and I can see that the model correctly found the corresponding information from the KB

3. But...when using the agent's playgorund or the app asking queries in Bulgarian, the agent cannot retrieve the information from the KB, even if the documents are in Bulgarian.

I tried to change the retrievel method from rewrting to None -- No results

Also, I've tried to update the Prompt of the Agent in DO -- No results

I've tried to add English Keyword at the top of the document(as suggested by Claude) -- still no results

Since the agent's prompt in DO is written in English, I added the Bulgarian version as well - its a small improvement, like 1 out 5 times the agent was capable to retrieve the information

The last thing that I can try is to translate all documents to English and reupload them to the KB, but since I have quite a lot of docs already, that might take some time. I just wanted to know if there is something else I am missing or I can try before translating the documentation. Its very strange that even if the docs are in Bulgarian, I need to be very specific when asking a specific question, like I need to copy-paste some information from the original document and just formulated it as a question in order to get a response from the chatbot. But in English, is way more capable and just with 2-3 words is able to generate a proper response based on the KB.

Any tips and hints how I can optimize the retrieval will be highly appreaciated.