u/Ju1ceyyy

I'm trying to fine-tune a language model (qwen 2.5 7b) to understand and generate text in a local language found in the Borneo islands. This language is a distinct Malay dialect spoken primarily in Sarawak, Borneo, making it a genuinely low-resource and linguistically complex language.

Issues I faced :

1. It turns into a text completion bot instead of an assistant that can conversate
2. It can no longer hold basic conversations — even in English
3. Catastrophic forgetting
4. The model loses its instruction-following ability entirely after fine-tuning