u/Creative-Type9411

I cant help myself its ~20% faster for me, I took the highest speed branch(for me), added the vision fix, and am just riding it out for now

Dual Xeon 8268, 1.5t 2666, Tesla T4

~122eval ~38t/s out

i tried using the release today and during some light coding lamma.cpp crashed and the model restarted, and I didn't experience any crashes on the pre-release versions personally so I jumped back into it

on the actual release branch now I get ~110eval ~30t/s out

just curious what everyone else is doing and if there were any major downsides on the early builds, anyone is aware of

I'm using OpenWebUI and and making tools/skills to improve my models functionality. I am currently using Qwen 3.6 35B A3B Q8 (F16) 256k

I grabbed `parallel tools` to be able to run multiple tool calls at once.. so far i have the following...

--------------

Document Creator: (WIP started it yesterday, testing now, almost done, rest of list is fully working and done) creates DOC/PDF/XLS/PPTX files, using linux prereqs and helpers, allows model to create documents

Send_Email: Allows the model to send an email from an smtp server, supports file attachments if the model wants to send an attachment

Inline visualizer (I found this here on reddit): allows visualizations to be produced directly in chat, its pretty awesome

Github Url converter: converts regular github links into raw links to make it viewable by the model

Weather: gets current and/or historical weather information for any location, lookup time is reduced for us based locations based on zip database (reduces call time significantly)

Sports: scores/schedules lookup

Filesystem: Allows a sandboxed directory (path checking on every command, outside of the models control) to create/modify/delete files, read files, append files, ZIP files (password protected if asked) - The model is aware of all files it has possession of including any generated images, anything its made can be referenced and it will be able to get it and do whatever you want with it

Browse_Page: enhanced web scraping leveraging different user agents, filters, this is paired with the default websearch included with OpenWebUI and works really well i hit 20-30 links usually during search where i used to only get 4-5 reliable ones

Wayback_archive: this tool is called whenever a site is blocked through normal access to be able to get the data anyway from the latest archived version of it (found this idea here as well on reddit but i made a custom version/tool to save on tokens instead of using it as a prompt)

read_microsoft_learn: a tool to enable to model to lookup anything on microsoft learn, to help itself C#/WPF/NET or anything else it's unsure of

-----------------------

Im trying to figure out what to build out next after im done the document creator... Im going to try and add SMS, since emails are usually more of a "silent" alert, at least for me..

But right now I can literally tell this thing to create a flyer for my services then go find emails of local business owners and send it to them and it will... While i go do something else...

This is crazy im going to make skynet ;P

I have been trying to figure out what to do with the RAM heavy box. Its a 1U Dell r640 w/dual xeon platinum 8268's, and 1.5tb of 2666 ram. it has 8x2.4Tb SAS 2.5" drives so not a lot in the way of storage.

No GPU, trying AI anyway, token count is horrendous..

But it works. Grok 2, 512K Context, -t 40 + NUMA, 4.73t/s prompt, 1.35t/s gen.... web search enabled..

Do the Tesla GPU's fit off the stock risers in 1U servers or am I going to have to cut the top of this? Anyone have a similar build? Any recommendations? I'll be adding a GPU ASAP but interested in what other people trying to claw their way in are up to..

All you need to do is make an EULA pop up before your virus runs and have a continue button underneath of it and whatever you do on someone's computer will be perfectly legal /s

Look at Microsoft they're doing it... they can literally take anything in your OneDrive folder and delete it off your computer without telling you but because of the EULA there's nothing you can do about it

because it's not about the spirit of what's happening, it's about the fine print 😈

Do your worst and have fun!

I have a TrueNAS SCALE box, I was excited to build it. It's in a Dell T620 dual CPU w/192gb of RAM and 80t raw storage 8x10tb 3.5 drives, RAIDz2

That was fun to setup I have jellyfin/tvheadend, a VM running a website, i have HBA passthrough enabled the card is flashed etc..

Today.... I got a new box and I have no idea what do with it I need advice

Dell R640, dual cpu 48c/96t, 8x 2.4tb drives, this one has a PERC h740p that I cant flash but it doesnt currently have enough storage to replace my older server because that is over ~60% full at present anyway.. and 2.5 drive prices are way higher.. But this newer server came with 1.5Tb of RAM.. a full set of 24x64gb ddr4 2666 sticks included

I threw llama.cpp in it to try and run some AI models, no GPUs at the moment but it handle some larger models a bit slowly i was wondering what you guys suggest I try with this..?