u/mantisalt — reddlx

What's the best tool for offline Wikipedia RAG at the moment?

Theoretical use case: I have a compressed version of wikipedia, and some function or tool fooRAG. Then I set my system prompt (or make a skill, or hook, etc.) to include "You can search wikipedia with the fooRAG tool like so...". Then in whatever harness I'm using I ask Qwen "List every character in the play Hamlet", and it uses the tool to pull and read the wikipedia page for Hamlet.

So far I've just found abandoned projects— this is the sort of question that's good to poll the community for. Definitely something I might take a crack at if there aren't good options available...

reddit.com

u/mantisalt — 2 days ago

▲ 14 r/morsecode

Farnsworth is counterproductive as soon as you've memorized the letters

Just realized this and wanted to share. Having gotten quite familiar with the acoustic "shape" of each letter at 30wpm farnsworth, I was listening to some regular 30wpm dialogue (from a book, not ragchew) and noticed it was much easier to hear and recognize common words like the, this, is, was, etc. I checked with farnsworth (20wpm/30 for chars) to compare and sure enough, I heard the letters clearly but it was much harder to recognize their relationship with each other.

Of course, this isn't a novel thought (it's the whole reason wordsworth exists), but I thought it might be useful to share. Maybe farnsworth is still useful for callsigns, though.

reddit.com

u/mantisalt — 11 days ago

▲ 30 r/LocalLLaMA

I wrote a GGUF inferencer from scratch, AMA

*for a specific model, in the R language. veeery impractical.

60 seconds per token, yum 😋
not vibecoded, I just wanted to learn how the architecture works on our end lmao (and all of karpathy's guides are outdated 😢)
writeup here, though it's kinda old news if you're already familiar with the math

I'd really like to make a useful inferencer next (you know, with an actual GPU kernel and memory management, in a language people actually use) but I can't find a good niche for one... llamacpp already runs on everything :/

reddit.com

u/mantisalt — 2 months ago

▲ 191 r/rstats

LLM inference running in pure R

This was a fun one to get working. The package has to unpack the weights into a .rds file for each layer and has to load and unload the layers one-by-one for each token... surprisingly, none of the math is particularly complex or hard to do in R. Just a lot of delicate data and file handling.

all coded by hand, yada yada. It runs off an old 3GB open-source model from last year.

it's set up as a (minimal, undocumented) package in case you want to try it out yourself :)
not very useful, but perhaps interesting to look at or play around with the code.

repo: https://github.com/gbkorr/ggufr
big writeup: https://gbkorr.github.io/r-bites/ggufr/ggufr
A little dissatisfied with how the writeup turned out, but oh well... it's still got some pretty cool stuff in there.

u/mantisalt — 2 months ago

▲ 523 r/rstats

Live Videoconference in the R Console

Back again with another evil project (writeup). Managed to get the delay under a second, and the rendering framerate is passable (10fps). This project is particularly silly because it uses an (unnecessarily) awful streaming strategy...

I haven't gotten to test outside of localhost because eduroam blocks port forwarding (lol), but it should work between two computers. Would love to see if anyone gets this running.

u/mantisalt — 3 months ago

▲ 227 r/rstats

Snake in base R(Studio)

Back again with more shenanigans. This is part of a big package I made last winter to make and run live games in vanilla R. It uses a second instance of R saving readLines() to a local file to be able to take live input without needing a package like shiny XD

I also used the game engine to make a clone of RStudio... running in RStudio. The engine is capable of running much prettier games but I haven't made one yet :P (though I did make the obligatory Bad Apple)

Videos are disabled on the sub so I put it into a GIF... hope it shows properly.

u/mantisalt — 3 months ago

▲ 590 r/rstats

Solving the classic issue of needing powerful machine learning on the go... hmm...

Writeup. I also managed to knit an Rmd page and push a git commit :0

u/mantisalt — 4 months ago

▲ 544 r/unixporn

Turns out you can get pretty decent phones nowadays for a great price if they're carrier-locked. And linux* runs great on them : )

[Moto G Power 2024]

CPU: 2.2GHz octa-core MediaTek Dimensity 7020
RAM: 8GB (16 with RAM boost)
Screen: 6.7in 1080 x 2400 LCD @ 90hz (~60fps using the desktop)
Storage: 128gb UFS 2.2 expandable
MSRP: $49.99 carrier-locked (I've seen a few on ebay at almost 40)
Keyboard: Targus Ergonomic Folding Keyboard ($10 on ebay). It and the phone both fit in one pocket (!)
Emu: Termux running Xfce4

Good battery life as far as I can tell. Runs great and smoothly, I was really blown away. You get used to the small screen (or just make everything bigger)

apologies for no rice

u/mantisalt — 4 months ago