u/Ok-Shower7286

I love Qwen3.8 27B so much that I decided to show my gratitude to the Alibaba ecosystem by building a dedicated RAG/search node using a cheap Mini PC from AliExpress.

Turns out, my ecosystem loyalty got rewarded with an absolute masterpiece of fraud:

• Promised: Intel N150 + DDR4/DDR5
• Delivered: Core i3-7020U (2018 Kaby Lake, 2C/4T) + DDR3 1600MHz
• The Scam: The seller literally hardcoded New_N150 into the BIOS release string (HSHW_M6_DDR3_EC_Intel_Com_New_N150_K001).

So now my Qwen3.8 RAG stack is full of fake specs that can barely index a text file, let alone run vector sidecars.

To make matters worse, despite providing all this proof, AliExpress CS completely ignores my non-refundable customs duties and active database migration issues, repeatedly giving me nothing but automated replies to "just return the item."

Filing a credit card chargeback now. Stay safe out there!

Hey everyone,

I’ve been in Korea for a couple of weeks now. While I love Myeongdong and Seongsu as much as anyone, I was getting a bit tired of being surrounded by other tourists and seeing the exact same Instagram spots everywhere.

I asked a local friend where people in Korea actually go on weekends to get away from the city, and they pointed me toward Yongin (용인) — just an hour south of Seoul.

Honestly, it turned out to be the absolute highlight of my trip so far. No huge crowds of tour buses, just super authentic vibes, crazy good food, and deep local history.

I found a local community archive site (place.udanax.org) that lists verified spots tested by residents, and mapped out an 8-stop route based on it. Sharing it here for anyone looking for a genuine day trip outside Seoul!

1. Wawoojongsa Temple (와우정사)

Unlike the usual temples, this one greets you with a massive 8-meter-tall Buddha head right at the entrance. The whole complex is built into a mountain with thousands of unique statues from all over the world. Super peaceful and totally different from Jogyesa.

2. Korean Folk Village (한국민속촌)

If you watch K-dramas, this place is insane. It's an actual reconstructed Joseon-era village. What made it fun was the actors walking around in traditional gear reacting with visitors. Felt much more alive than a typical museum.

3. Yongcheon Sanghoe at Yongin Jungang Market (용천상회)

This was the most "local" experience ever. It’s a 30-year-old dried seafood shop inside a traditional 5-day market. The owner doesn't even sell online because he believes "screens can't smell freshness." Local restaurant owners literally buy their ingredients here based on his word alone.

4. Gogiri Makguksu (고기리막국수)

Perilla oil buckwheat noodles served in a quiet Hanok near a stream. Apparently, locals wait hours for this place. The noodles are super nutty and refreshing — definitely worth the hype.

5. Yongin Natural Recreation Forest (용인자연휴양림)

A great spot to digest all the food. It’s a dense forest with wooden deck paths. Perfect if you want a light trek surrounded by birch trees without climbing a whole mountain.

6. Nam June Paik Art Center (백남준아트센터)

Dedicated to the godfather of video art. Even if you're not a huge modern art fan, the installations here are super trippy and cool to walk through.

7. Hanteo Origol (태봉산 한터오리골)

Hands down the best meal of my trip. They grill fresh duck right in front of you on a massive stone slab. No heavy sauces, just crisp on the outside, juicy on the inside. You have to get the fried rice at the end — they crisp it up into a golden cracker on the hot stone.

8. Bojeong-dong Cafe Street (보정동 카페거리)

Ended the day here for coffee. It’s a neighborhood filled with cozy leaf-shaded alleyways, independent bakeries, and warm night lights. Very romantic and relaxed vibe.

https://place.udanax.org/theme/8/%EC%97%85%EC%B2%B4%EC%B0%BD%EA%B3%A0-%EA%B3%B5%EC%8B%9D-%EC%84%9C%EC%9A%B8-%EB%B2%97%EC%96%B4%EB%82%98-%EB%A7%8C%EB%82%98%EB%8A%94-%EC%A7%84%EC%A7%9C-%EC%9A%A9%EC%9D%B8-%EB%A1%9C%EC%BB%AC-%EA%B0%80%EC%9D%B4%EB%93%9C-8%EC%84%A0?lang=en

Quick Tips:

How to get around: KakaoTaxi or rental car is best since Yongin is pretty spread out, though public transport works if you stick to 3-4 spots.

Credit Cards: Accepted everywhere, but bring a bit of cash if you want to buy small snacks at the traditional market.

Source: I got most of these non-touristy recommendations from a local Korean neighborhood database called place.udanax.org. It's in Korean, but Translate works fine if you want to dig into more local shop stories!

Hope this helps anyone looking to break out of the usual Seoul tourist bubble! Let me know if you have any questions.

I forked llama.cpp to implement Declarative Attention (arXiv:2609.02737, Google DeepMind and KAIST AI). The model declares in its own output which context chunks it needs <focus magic_chunks="N">), and the engine listens and restricts what the following tokens can attend to. No scorer, no training: just prompting plus an engine that reacts to the tags. According to their paper, the overall decode time for the response can be reduced to 0.71× (Gemma) and 0.77× (Qwen) compared to vanilla. Those are the paper's vLLM numbers; I haven't benchmarked the fork yet (no accuracy benchmark either).

= What works =

- llama-server can drop KV token ranges for a request da_rm / da_rm_at), either mid-prefill or after prefill.
- Tag-driven mode: given a chunk layout from the client, the server parses the model's first <focus magic_chunks="N"> tag during generation and drops the other chunks at that point (one-shot, one-way).
- A second-sequence mode da_b, needs --kv-unified) leaves the original sequence intact.
- Checked with first-token logprobs on small smoke tests (a Qwen hybrid/GDN model and a tiny dense one).

= Why a fork =

- Stock llama-server has no way to touch KV ranges mid-generation, so this needed server changes.
- llama.cpp has no paged block table (the paper's vLLM version rewrites it), so masking alone doesn't reduce the KV that gets read.
- Real skipping needs kernel support or compaction. From reading the source, single-token decode on CUDA doesn't skip masked chunks today.
- On hybrid models only the attention layers are restricted (same as the paper); the recurrent state is untouched.

Repo: https://github.com/edwardyoon/focus-llama

Feedback welcome, especially from anyone who has worked on llama.cpp KV/recurrent memory.

I've been comparing Cline / Kilo / Qwen Code lately since they all handle long-task state differently.

Cline: has Focus Chain, a markdown file kept outside the conversation that gets reinjected on a cadence, plus Memory Bank for project context, plus a standalone gRPC server so it's not fully tied to VS Code. probably the most mature of the three on this specific problem (about context management), though restore still has some sync bugs between the file and what the model actually sees.

Kilo: TODO state is literally an XML block living inside the conversation history, so when compaction kicks in it gets flattened into a prose summary and the agent sometimes has to reread source files just to figure out where it stopped. It causes infinite read-analysis-compaction loop sometimes by reached to context limits. they're mid-migration onto the opencode engine now, which might fix some of this eventually but isn't there yet.

Qwen Code: keeps TODO state in a plain file (~/.qwen/todos/) completely separate from the conversation, so no matter how much compaction runs, nothing gets lost or reconstructed.

ended up going with Qwen Code for long multi-step coding work because of this. It works well for 2~3 hrs long running tasks, where I'd usually hit that Kilo loop by then or human intercept.

The one thing I missed was semantic code search, doesn't have a built-in equivalent, so I built an qwen-code extension for it, plus causal decision-chain tracking on top. still early, but repo's here if anyone suffers like me, https://github.com/edwardyoon/FocusMemory

https://qwen.ai/blog?id=qwen3.8

As you guys probably already know, they're dropping Qwen 3.8-Max at 2.4T and a 27B model, but have actually read the blog post on how this thing operates in the wild?

- It ran 10+ days of completely autonomous, self-evolving software development starting from an empty folder (seriously, check out the GitHub trace for oh-my-cli, it's wild).

- It features a native visual feedback loop, treating visual information not just as static input, but as a continuous execution-and-correction monitor.

- Autonomous chip design optimization: running a closed-loop for 500+ turns via Iverilog, Yosys, and OpenROAD, shrinking a crypto accelerator from 8,298 gates down to 678 gates and hitting actual silicon timing closure.

While people are arguing over safety guardrails and incremental coding scope with AGI cyber attacks FOMO marketing, they aren't just building a better chatbot, they are deploying recursive engineering and hardware synthesis agents. :D Really suprising.

Anyway, let's enjoy the 27B open weights to our heart's content.

To minimize the impact on the apps currently in production, I’m moving a small “advance team” of two Mac Minis, a 5090 desktop, and one battery unit to another space from my work room.

The problem is, I built the rack myself using angle brackets, and I made it only 75 cm tall because it was my first attempt.

I’m thinking of extending it upward by attaching another section of angle brackets on top of the existing rack. That feels like it could get pretty sketchy, though.

Experienced rack builders, please chime in.

For reference:

• One battery unit weighs nearly 50 kg.
• One of the main servers is a GPU machine weighing around 3~40 kg.

The “follow-up team” will be:

• 3 more Mac Minis
• One 4U rack server
• 1U Omada unmanaged switch
• One 5.12kw battery

How would you extend the rack safely without rebuilding the whole thing from scratch?

Hey r/LocalLLaMA,

I’m a retired engineer with a background in distributed computing, currently running a 1-person startup.

Like many people here, I’d love to experiment with 2T+ MoE models locally. The problem is that I don’t have an H100 cluster in my living room. So I’ve been thinking about a fairly practical question:

If I have to pay the enormous cost of analyzing a 2.8T teacher model, can I do that once and reuse the result for many different compression experiments?

That question led me to a design I’m calling Blueprint Distillation (BPD), with BSP-Structured Expert Consolidation (BSEC) as the analysis pipeline. The basic idea is to treat the expensive teacher analysis somewhat like a compiler frontend.

Instead of tightly coupling:

Teacher → Analysis → 27B Compression

I want to separate the expensive teacher analysis from the actual compression:

2.8T Teacher → Expensive Analysis → Compression Blueprint

The Blueprint becomes a reusable intermediate representation that can be consumed by different student configurations:

                    ┌──→ 27B Student
2.8T Teacher ──→ Compression Blueprint
                    ├──→ 7B Student
                    │
                    └──→ 3B Student

The Blueprint would persist things like expert co-activation, sensitivity, similarity and dependency information, along with budget-specific views.

The underlying techniques aren't new. Taylor saliency, Fisher-style pruning, dependency analysis, expert clustering, etc. are all established ideas.

I’m not claiming to have invented a new pruning algorithm.

What I’m interested in is whether the architecture is useful: can we turn the expensive “understanding the teacher” phase into a persisted, inspectable, reusable intermediate representation?

If that works, changing the target from 27B to 7B shouldn't require throwing away all the information gathered from the original 2.8T model and paying for another giant profiling run.

That seems particularly interesting for people like me who want to run many small experiments rather than one huge experiment.

I wrote up the design here: https://github.com/edwardyoon/BSEC

The first concrete target is Moonshot AI’s Kimi K3 (2.8T MoE) → 27B, but the idea isn't necessarily specific to K3.

There is one fairly significant problem:

I don’t have the compute to generate the first Blueprint.

The design calls for instrumenting a distributed K3 deployment and collecting routing/co-activation information over a calibration corpus. I’d like to eventually produce the first real Blueprint.json and see whether the idea survives contact with an actual 2.8T model.

So I’m posting this for two reasons:

- I’d like technical feedback on whether the architecture makes sense.

- If someone has access to H100/H200-class compute and is interested in the problem, I’d love to collaborate on the first profiling run.

And honestly, if you think this is fundamentally flawed, I’d like to hear that too.

This is a design draft, not a claim that I’ve already compressed K3 to 27B. The interesting question for me is whether we can make the next ten experiments dramatically cheaper than the first one.