u/mantisalt — reddlx

I wrote a GGUF inferencer from scratch, AMA

*for a specific model, in the R language. veeery impractical.

60 seconds per token, yum 😋
not vibecoded, I just wanted to learn how the architecture works on our end lmao (and all of karpathy's guides are outdated 😢)
writeup here, though it's kinda old news if you're already familiar with the math

I'd really like to make a useful inferencer next (you know, with an actual GPU kernel and memory management, in a language people actually use) but I can't find a good niche for one... llamacpp already runs on everything :/

reddit.com

u/mantisalt — 15 hours ago

▲ 181 r/rstats

LLM inference running in pure R

This was a fun one to get working. The package has to unpack the weights into a .rds file for each layer and has to load and unload the layers one-by-one for each token... surprisingly, none of the math is particularly complex or hard to do in R. Just a lot of delicate data and file handling.

all coded by hand, yada yada. It runs off an old 3GB open-source model from last year.

it's set up as a (minimal, undocumented) package in case you want to try it out yourself :)
not very useful, but perhaps interesting to look at or play around with the code.

repo: https://github.com/gbkorr/ggufr
big writeup: https://gbkorr.github.io/r-bites/ggufr/ggufr
A little dissatisfied with how the writeup turned out, but oh well... it's still got some pretty cool stuff in there.

u/mantisalt — 4 days ago

▲ 523 r/rstats

Live Videoconference in the R Console

Back again with another evil project (writeup). Managed to get the delay under a second, and the rendering framerate is passable (10fps). This project is particularly silly because it uses an (unnecessarily) awful streaming strategy...

I haven't gotten to test outside of localhost because eduroam blocks port forwarding (lol), but it should work between two computers. Would love to see if anyone gets this running.

u/mantisalt — 1 month ago

▲ 227 r/rstats

Snake in base R(Studio)

Back again with more shenanigans. This is part of a big package I made last winter to make and run live games in vanilla R. It uses a second instance of R saving readLines() to a local file to be able to take live input without needing a package like shiny XD

I also used the game engine to make a clone of RStudio... running in RStudio. The engine is capable of running much prettier games but I haven't made one yet :P (though I did make the obligatory Bad Apple)

Videos are disabled on the sub so I put it into a GIF... hope it shows properly.

u/mantisalt — 2 months ago

▲ 590 r/rstats

Solving the classic issue of needing powerful machine learning on the go... hmm...

Writeup. I also managed to knit an Rmd page and push a git commit :0

u/mantisalt — 2 months ago

▲ 544 r/unixporn

Turns out you can get pretty decent phones nowadays for a great price if they're carrier-locked. And linux* runs great on them : )

[Moto G Power 2024]

CPU: 2.2GHz octa-core MediaTek Dimensity 7020
RAM: 8GB (16 with RAM boost)
Screen: 6.7in 1080 x 2400 LCD @ 90hz (~60fps using the desktop)
Storage: 128gb UFS 2.2 expandable
MSRP: $49.99 carrier-locked (I've seen a few on ebay at almost 40)
Keyboard: Targus Ergonomic Folding Keyboard ($10 on ebay). It and the phone both fit in one pocket (!)
Emu: Termux running Xfce4

Good battery life as far as I can tell. Runs great and smoothly, I was really blown away. You get used to the small screen (or just make everything bigger)

apologies for no rice

u/mantisalt — 2 months ago