▲ 0 r/LocalLLaMA
MTP Speed with 3090 Qwen 27B Q4
What speed are you guys getting? I get max 55tks gen speed on coding related tasks. DDR4 though but that should matter on low context
u/GodComplecs — 7 days ago
u/GodComplecs
▲ 1 r/LocalLLM
What would constitute as digital sentience? I'm asking here specifically since I'd like to hear fellow local llm users opinions on this, since the in my opinion at some point we could be crossing over into talks of involuntary work etc if systems become sentient.
I know this seems very far fetched, but believe me the future is closer than you think, and I'd like to see what people who use local llms think since according to the big boys they are all ready AGI ASI super feeling beings that generate infinite universal income xD
u/GodComplecs — 22 days ago
▲ 738 r/LocalLLaMA
Well or pretty close to it, they are excellent work horses. I run them in real work scenarios doing some of the work I used to do myself as an skilled expert in my field, billing 200$ an hour. Ofc the key is building a system around their weaknesses, and I've had already LLM systems doing expert work years ago when first ones came (shout out nous hermes 2 mistral!).
But yeah pretty neat, especially noonghunnas club 3090 and you can have 3.6 27B fly on a single 3090.
u/GodComplecs — 22 days ago