u/Last_Bad_2687

My relative is working on some medical research/literature review as their retirement project.

I was given a paper to check citations and to "use my AI" to find more citations for some of the gaps in the proposed model (glucose production w.r.t. daylight or something like that)

Despite several (8-10) rounds of feedback between chatGPT (Sol 6.1 extra high) and Claude (5 extra high, 5.5 refused due to safe guards), my relative preferred my "v1" draft from halogen-flash-next.

Not sure if the model is less hesitant about medical stuff or if it just ran longer because I didn't have to worry about usage limits, but this is a serious win for me

Need something (safe, non-nsfw, legal, no sensitive information) written and out of usage? Well good for you, I have 100% of my Claude Max plan left expiring on Oct 4th (moved to Qwen 3.8 Flash Next).

Tell me what you need generated.

Pls no super complicated stuff I still need to figure out a way to get you guys the output and I don't want to deal with Gdrive

I'm not sure what I'm doing wrong but even with the goal plug in, my agent stops at "let me now implement" and just ends the turn there.

What am I doing wrong?

Warning, this post isn't relevant or interesting to 99% of people with a supernote.

I have a rooted Supernote (I have "admin" access to the underlying operating system, do this at your own risk).

Today I was out at a job site for work where the customer had a Raspberry Pi computer for their farming robot (Farmbot) that needed a re-image. I didn't have my SD card reader with me, neither did my coworker. Defeated, we rescheduled to come back until I realized.... The supernote has an SD CARD reader!

I popped the SD card in, very carefully made note of which partition (cat /proc/partitions) then used dd to image the raspberry Pi image.

It worked!

Now I can say I used my notebook to re-image the OS of a farming robot. The future is here lol

This post was written mostly by claude

local claude code replacement on a ryzen ai max+ 395 (strix halo, 128gb): qwen 3.6 27b, tested a bunch of configs, heres what actually won

spent about a week turning my framework desktop into a local coding agent to get off claude code. amd ryzen ai max+ 395, 128gb unified, gfx1151. i scored everything on agentic coding (tool calls, multi file tasks), not chatbot benchmarks.

tldr: plain dense qwen 3.6 27b at unsloth UD-Q4_K_XL with MTP speculative decoding won. everything i thought would be clever (the 35b moe, dflash, RYS layer dup, higher quants, my own finetune) lost. all numbers below are from my own harness on the box.

the box

• framework desktop, ryzen ai max+ 395 (strix halo), radeon 8060s igpu (gfx1151)
• 128gb unified lpddr5x, ~215 gb/s. the machine is bandwidth bound and that basically explains every result here.
• 112gb to GTT, rocm 7.2.4, llama.cpp b9982 (kyuz0s prebuilt strix halo image)

how i scored it built my own harness, didnt trust generic benchmarks for this:

• 144 tool call tests (hostile json args, parallel calls, 8 call chains, calls at 32k and 100k depth). strict pass = valid structured call, no raw <tool_call> leak, args validate
• 16 hard agentic tasks (real refactors, fix failing test, deprecation sweeps) in an actual tool loop
• 30 aider polyglot exercises (py/rust/cpp)
• 40 turn replay that measures spec acceptance and t/s by context depth

what won, exact command

ROCBLAS_USE_HIPBLASLT=1 llama-server \
  -m Qwen3.6-27B-UD-Q4_K_XL.gguf \
  -c 131072 -ngl 999 -fa on -b 2048 -ub 512 --no-mmap \
  --jinja --chat-template-file qwen-fixed-v21.3.jinja \
  --cache-reuse 256 -np 1 --metrics \
  --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0 \
  --spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-p-min 0.75

model is unsloth Qwen3.6-27B-UD-Q4_K_XL. template is froggerics fixed qwen template v21.3, not optional, the stock one caused loop aborts and stuck tool calls. MTP is the models own multi token prediction head used as a self drafter, no separate draft model to load.

the numbers

config	tool suite (144)	hard agentic (16)	polyglot (20)	decode <8k to 100k	spec accept
27b Q4_K_XL + MTP (winner)	99.3%	93.8%	65%	21 to 16 t/s	93-95%
27b Q6_K + MTP	100%	68.8%	65%	21 to 16 t/s	92-94%
35b-A3B moe Q4	-	75.0%	55%	~53 t/s raw	no MTP
27b Q4 + dflash	98.6%	-	-	15 to 11 t/s	~28%
27b Q4 + RYS-XL (d8)	-	25.0%	40%	~22 t/s	~83%
27b qlora finetune v2	-	87.5%	-	-	-

hard agentic is 16 tasks, polyglot is 20, so small samples, treat single task diffs as noise and big gaps as real. no spec decode for reference: q4 ~12 t/s, q8 ~7.8, moe ~53.

MTP vs dflash MTP wins here and its not close. no spec q4 is ~12 t/s, MTP takes it to ~21 under 8k and holds 16 at 100k, ~94% acceptance the whole way. the long context collapse people worry about (llama.cpp #23322) didnt happen for me. dflash got ~28% acceptance and 11-15 t/s, so slower than MTP. the reason matters for this hardware: MTP rides along with the target models weight reads, dflash is a separate model fighting for the same 215 gb/s bus. on a bandwidth bound apu thats a bad trade. an independent strix halo test found the same thing. dflash is a nvidia win, not here. also MTP is HIP only, vulkan gives garbage.

things i thought would win and didnt

• the 35b moe. decodes ~53 t/s, 2.5-3x the champion, i wanted it. it lost hard agentic by ~19 points and polyglot by 10. for real multi step work the dense 27b just solves more. speed isnt capability.
• higher quants. Q6 and Q8 gave nothing over the dynamic q4. the unsloth UD quant already keeps the important weights at higher precision, so plain Q6 spent bits for nothing and actually dropped 2 agentic tasks. q4 is the sweet spot, q8 just cost 10gb and decoded slower.
• RYS layer duplication (the training free self merge trick people post here). killed it. the one config thats even valid on this arch scored 25% hard agentic vs 94%. reads fine, cant solve anything. the RYS wins people cite are all eq/creative/math, i found zero coding upside. one gotcha: qwen 3.6 is a hybrid gated-deltanet/attention model that types layers by position, so a dup block only loads if its length is a multiple of 4. most posted RYS configs wont even load on it.
• my own qlora finetune on my claude code transcripts plus agentic data. i shipped the base model instead. first run taught it to quit early (16k training windows end mid task, so it learned to wrap up). fixed that, then it regressed tool calling at 32-100k because the training windows never had deep context calls. net worse than base. lesson: if you finetune for agentic, your training depth has to match how you actually serve.

strix halo gotchas

• bandwidth bound, so pick methods that dont add bandwidth. MTP yes, separate draft model no.
• 112gb GTT needs bios plus amdgpu.gttsize=114688 ttm.pages_limit=29360128 ttm.page_pool_size=29360128 amd_iommu=off
• runtime ROCBLAS_USE_HIPBLASLT=1. build flags if youre not using kyuz0s image: -DGGML_HIP=ON -DGPU_TARGETS=gfx1151 -DGGML_HIP_ROCWMMA_FATTN=ON -DGGML_HIP_NO_VMM=ON
• training on gfx1151 is rough. variable length batches make hipblaslt re-autotune every shape (~875 s/step until i packed to fixed length). the gated-deltanet fast path wants a cuda only kernel so training falls back to ~50 t/s. qlora works, just slow.
• mine hard locks under sustained igpu load with no remote power cycle. checkpoint often.

is it actually a claude code replacement its the best local one ive run on this hardware. 99.3% tool reliability and zero malformed calls in a 50 call soak, so it drives opencode and claude code fine (llama.cpp b9982 has a native anthropic /v1/messages endpoint now, no proxy, just point ANTHROPIC_BASE_URL at it). but its a 27b, expect more hand holding than claude on hard multi file stuff. for fully local and private, one overnight to set up, im happy with it.

can share the harness and configs if anyone wants. more interested in counterpoints on the negative results.

Caution, this post is written by an armchair jockey who is not a founder

Came across Alex Hormozi 2-3 weeks ago and since have been doing a deep dive on all his content, and it's convinced me to quit my cushy IT job and start a business.

I have been racking my brain and using AI to help come up with my Offer based on my past chats across various AI services, asking AI to summarize my chats and come up with what I am interested + skilled at.

It either recommended stuff I am good at, but hate doing, or stuff I loved doing but didn't have a demand.

I came up with this formula to rank the core offer:

(Pain of Customer/Difficulty of Fulfillment) * Passion * Time to First Value

This is a rework of his equation:

Value = (Dream Outcome * Perceived Likelihood of Achievement) / (Time Delay * Effort & Sacrifice)

Pain of Customer - How urgent is the need your fixing. One idea was to fix broken IT workflows for business who have "keyman" risk - processes that fail when the main IT guy leaves or goes on break. Pain here is high. The other idea was to do OCR digitizing + AI summary for paper documents - this is personally fun but the pain is low-med

Difficulty of Fulfillment - How easy it is (mentally) to fulfill - since I am in IT and I tend to over complicate solutions (which Sharran Srivatsa says the is the downfall of smart and hardworking entrepreneurs). Alex recommends lawn care etc. as simple first businesses but as a pasty indoors guy that doesn't sound feasible. It is easy to pick a business that sounds fun and sexy, and AI is all the rage, but I wanted to force myself to pick something simple to deliver. So "study a business and propose a custom AI workflow tailored to them" = 10, write a dead simple script to automate emails = 1 (for IT pros)

Passion = Sexiness of the idea, or how personally interresting it is

Time to First Value - basically time delay, I left it out initially but it's super important. it's how quickly the customer realizes you helped them. I was tending to pick super complex offers with very long Time to First Value, so this forces me to pick things that they can see value in days instead of months.

Hope this was helpful to any IT people out there following his stuff and have no idea how to do lawn care or snow removal

I have been doing local AI models for home and for work and realized I want to try my hand at starting a startup based on what I have learned while I'm still "young" (31)

My plan is to prototype/soft launch while I stay at my job and then quit if it picks up.

I can't decide between:

Option 1: Starting a service to do custom AI workflows for business automation. I know this is all the rage, I don't know as many people who use local models, at least in my small city. I built a simple handwriting OCR - > AI summary pipeline using existing APIs that converts my messy cursive (guess I'm not that young) to neatly summarized text. My plan is to focus on business that have a ton of handwritten stuff that needs OCR + data processing

Option 2: Hardware/Deep tech - I have been working on a custom AI model to do screenshot to text because I feel like that on a chip that controls the computer ('KVM' device) is going to be useful for 'agentic' AI. I just watched a YouTube video of an AI agent installing Windows on a PC using a virtual keyboard + screenshot processing. This feels like it has a ton of potential, lots to go wrong, I am not even close to having an actual product here.

My thinking is soft launch option 1, set up an LLC, do some paid ads, warm outreach and use any revenue to work on option 2. If it really takes off, then quit my job, if it does not, then shut it all down and say I tried.

Any tips on what to do between these (or if these ideas suck and I need to go back to the drawing board) appreciated.

Thanks.

I have an nc (2006) miata and a caged Chevy sonic that I want to trailer. What is a good cheap truck/van/SUV specifically for towing these? Under 5k ideally

Maybe I'm influenced by the sci-fi story "The Last Question" by Issac Assimov but I've always got a tickle imagining a huge model like Kimi running on, say, disk. Even if it is 0.001 tok/sec to ask complex questions and get an answer in a week

Is there any use or community focused on this?

My dad's Gen 13 Framework 13 Mainboard has some short when he plugged it into a dock. This is a known issue and the goal isn't to beat on the design.

I had a Gen 11 Framework board laying around from my own upgrade. In 20 minutes I had swapped it over, installed drivers (including extracting the .exe and manually installing the touchpad i2c driver), and he was up and running.

For my I now have a gen 13 board as my server instead of a gen 11, so it's immaterial to me. My dad mostly does word docs so the performance difference is negligible (infact, it went from a gen 13 i5 with 32GB to a gen 11 i7 with 64GB of DDR4 ram). My docker containers will barely notice.

What other laptop can you swap a whole mainboard to a spare and spend $0 on an emergency fix?

I bought a 128 GB Framework Desktop at launch but now that category is really popular for local LLM.

A used Mac studio m3 ultra 512 GB goes for about 3k. A new Framework desktop 128GB goes for about the same (unless there's cheaper used ones somewhere?)

Now, apart from having to deal with Apple stuff (I daily Fedora, BTW) the $/GB for local LLM makes a used 512GB Mac studio seem like a steal over 3x Framework Desktops to try Kimi?

What am I missing??

Sometimes when I cook bacon I save the grease in a jar for later cooking.

With the harsh limits of Claude I have been working on ideas to make sure the output of any token spend is captured, cleaned and put away.

I use chatGPT/Claude quite a bit for assistance in coding and I realized that multiple tines I had asked ChatGPT the same question (for me it was where to save grub2-mkconfig in Fedora 43 after messing with kernel parameters). I had at least 3 chats where I had asked this before.

I was like... wait a minute - if a model gives me a solution for a situation and it works... that solution should immediately be aliased and stored somewhere to be re-run later.

I haven't figured out a concrete way to "cache" the results of successful scripts /receipes generated but this was just an idea.

Disclaimer: I don't run open claw I run open terminal with Qwen3-coder-next inside open webUI

Has anyone implemented Karpathy LLM wiki idea in Open Web UI with knowledge base?

I used open terminal and qwen3-coder-next to implement a folder structure there. I was asking a model to read the wiki when it used a *query knowledge base* tool. I wasn't aware Open Web UI had such a feature and now I'm kinda stuck debating between keeping my LLM Wiki in open terminal vs implementing it in Knowledge base

I work at a non profit doing experimental/STEM research + outreach. Deployed an AI server (local AI models for staff), designed a STEM van from scratch (selected the van, taught myself cad, mocked up interiors, got custom cabinets made based on design etc). Manage a team of low code developers.

Worked at a research university computer lab prior to this doing Internet of Things research. Masters in Computer Engineering.

I am being encouraged by everyone to move to private sector where my skills will be more compensated. I just don't know where or how. My path here feels maxed out.

I'm 31 now, I feel the window closing to pivot.

Because my career has been all over the place, I don't seem to meet any of the requirements of what I'm looking for. I am not "AI" enough to work in hardcore AI, I manage our server but I can't remember much from classes on how the models work, forgotten all my math at this point. I don't want to be in a pure tech role either.

I have 1 year management experience now but not sure if that helps. Done a lot of events management type work this past year too (including co-creating and running a 500 person tech event, regularly running 100 person events). Making close to 6 figures.

My big question is where do I fit? I don't mind taking a pay cut to pivot but I have no clue where I fit...

I have a Windows 11 VM in a Framework 16 laptop with dGPU passthrough. I had a pain last time setting up Looking Glass, so I opted to do Sunshine/Moonlight.

I have the weirded bug - The stream only loads from Tailscale (Yes, VM to host over tailscale is dumb but saves debugging network config) but not if I put the libvirt bridge IP address - I get a black screen and it says failed to connect. I have triple checked Sunshine and moonlight firewall rules, used netcat to test udp etc. Both adapters are set to Private.

Has anyone faced this? I can't for the life of me figure out why it only loads an image over tailscale.

Something I am working on my rooted Supernote is the ability to have an AI agent co-edit/summarize my notes with me. I already have syncthing/inotifywait based scripts to sync my "dirty" (open/ being edited) .note files (and clean ones) which I sync to copyparty. I also found I can stream the ABS_X and ABS_Y of the pen via /dev/input/wacom_real (or similar - my supernote is currently away from me).

My idea: Stream ABS_X and ABS_Y to my Framework AI server to do "live" OCR and occasionally sync the "dirty" .note file as a "ground truth". I would love to see a "live" .md appear on my laptop as I write in my note as a first milestone.

The second milestone would be to have an AI agent read this live .md and think about insights based on a RAG of my previous notes.

The third milestone would be for a multi-modal agent to insert notes INTO my .note live based on insights (highlight this, or auto link this note to your other note on a related topic. Or create a todo based on this). I tried using "sendevent" to /dev/input/event7 but no luck. I briefly see a cursor that disappears. If the actual pen is hovering, I see a point appear sometimes. This seems the hardest part.

There are already supernote-ocr-enhance (I think) and rectangular file and sn2md, and similar projects to convert "closed" .note files to .md, but I think the real value (for me) would be for this to be used by a co-agent "live".

Anyone have any advice on how to "inject" pen strokes using "sendevent"?