Google releases EmbeddingGemma 2, a new open model that runs locally on 0.5GB RAM.
It supports text, code, images, video and audio, with 740M parameters: text (270M), vision (170M), audio (300M).
Run & train the model via Unsloth Desktop. We're going to make the text model the default embedding model now most likely.
I had been using UD for about a week with no issues at all. Genuinely wonderful App to use, especially for somebody who is new to running LLM‘s at home. However, I a few days ago, there was a long freeze and a bunch of stuff crashed. No idea what it was, but UD would never run again without crashing after. Could be a coincidence.
After reinstalling the first time, via the exe, I clicked the expand arrow to show the PS that was running in the background. I noticed it said not to run it in an elevated prompt, but that's my default for windows. So that means it ran the same when I first installed it with no issues. UD booted up for its first run after reinstall, I began downloading a few models, jumped into the settings to.... and crash. I would be left with the gray UD window showing the server crashed and to copy logs/retry. This happened a few times. Even just starting up UD, walking away and coming back to the same crash screen.
Used Revo Uninstall to try and remove as much as I could, also uninstalled Git, Python, the HF cache, and the \.unsloth\ bits it didn't remove.
Installed it again, with lowered permission, didn't see the elevated warning during the install and was hopeful it would work. Sadly it did not. Same crashes. Uninstalled everything again, rebooted, reinstalled. Same issues.
I know the drivers aren't the newest, but it didn't seem to care for a week.
Sorry if this log is jumbled, I saved a bunch of stuff in a few txt docs to try and comb over but later gave up in frustration. Would really love any help. Thank you
Hey guys rather than just using Hugging Face, you can now connect ModelScope to Unsloth. This is especially useful for folks in countries were Hugging Face is blocked!
There will also be a new release today with many new features! https://github.com/unslothai/unsloth
Our ModelScope: https://modelscope.cn/models/unsloth/
I couldn't find a concrete answer anywhere, do you just distill the instruct model back?
If that is the case, what is a quality european language question set to turn it back into a chatbot/agentic, can a model at that size even be agentic? (i chose this size to learn) if i finetune for my specific harness? (i have a lot of training data of opus running in my harness)
my harness basically has the model output python code and has a few built-in functions like:
- vector_search_laws()
- graph_search()
could i have the model at least internalize a "hunch" on what stuff to search?
also what is the latest RL technique for agentic/harnes specific workflows?
I have a lot of RAW training data, like court decisions or commentaries or legislature, but not a lot of golds. could i use these to LLM synthesize training data and maybe RL the model in my harness to find that data?
What would y'all's strategy in the CPT->SFT->RL pipeline be for my specific problem?
I know this is a lot of questions im trying to figure out which direction to go, any pointers? Also good resources are welcome, for example that alex karpathi video was amazing for me, but i'd imagine its a bit outdated in terms of latest RL and SFT?