Question about creativity and exploration
We train language models to predict next token in pretraining and later fine tune them, but what about exploration and creativity. You can’t really do anything about that because we train the models on specific datasets, right?
I want language models to focus on different solutions for the same problem where the gap between solutions is extremely different.
For example, if I am working on a novel problem and I use existing mathematical functions, would it be possible for language models to neglect them because they might be completely wrong even though they were trained on them? Meaning instead of next token prediction, we would give the model different allowed next tokens or group of tokens and incite them to explore different next tokens for the same training example?
Is there a theory behind what I am asking for?