u/0xdeadf1sh

In case anyone wanted to program hardware-accelerated OpenGL ES 2.0 applications on the old Asus X101CH netbook running Windows XP, I have written the instructions on my blog post.

Full disclosure: I did use Fable to help me install Windows XP and get the GPU driver working properly. But the code that is making the triangle spin in the video above is from my meat fingers.

Enjoy!

I'm a type 1 diabetic and since March I've been working on training my own transformer model to predict my blood sugar:

It's an encoder-only transformer with variable-width context size (8 - 24 hours) that uses past blood glucose (BG), and past + future carbs & insulin to condition its predictions for future BG. Both input and output BG values live in kovatchev risk space reparameterized to [40, 400] range. I pretrained the model on the outputs of my custom simulator and then fine-tuned the model on three real-world datasets (ohiot1dm, azt1d, shanghait1dm), and on my own blood sugar data.

Here's the link to the source code (MIT-licensed) + weights + evaluation data: Github Project Link

I've posted about this model to another subreddit before where it got called fat, so let me emphasize that I've trained multiple models ranging in size from less than 40K parameters to ~17M parameters.

It natively supports what-if predictions, i.e. you can ask the model how will eating 20 grams of carbs with GI = N affect my blood sugar K minutes from now giving everything else that I've done in the past 8 - 24 hours?

It also predicts time by looking at the context (which contains past blood glucose and meal & bolus information).

Here are some evaluation metrics:

median line (RMSE/MAE in mg/dL, MARD in %)

Pooled, 3 real cohorts (n=1958 windows)
  PH     RMSE     MAE    MARD
  30    16.68   11.36    8.06
  60    22.77   15.65   10.91
 120    32.76   22.76   16.09

OhioT1DM (263 windows, 6 patients)
  PH     RMSE     MAE    MARD
  30    19.41   13.16    8.37
  60    26.39   18.19   11.51
 120    40.77   30.21   18.55

AZT1D (1350 windows, 24 patients)
  PH     RMSE     MAE    MARD
  30    16.90   11.56    8.26
  60    21.56   14.96   10.43
 120    27.66   19.43   13.65

ShanghaiT1DM (345 windows, 12 patients)
  PH     RMSE     MAE    MARD
  30    13.19    9.18    7.04
  60    24.35   16.42   12.32
 120    42.79   30.13   23.75

T1DMSIM, synthetic (1440 windows, 30 patients)
  PH     RMSE     MAE    MARD
  30    16.77   12.51    8.22
  60    28.73   21.56   14.66
 120    45.89   36.28   25.86

I used DILATE loss for the median line and pinball loss for uncertainty bands. The two are combined using Kendall-Gal weighting. I used Defazio's AdamC schedule-aware weight-decay correction to keep gradients stable toward the end of the training.

The architecture itself is inspired by BERT and modern LLMs: it uses bidirectional attention heads, Muon optimizer, RoPE, QK-norm, SwigGLU FFNs, and autoregressive rolling for >2 hour predictions.

I also have built a custom app for myself to use & test the model on my own data. There are two variants: fp32 version which runs on the CPU (~30 ms average inference) and an fp16 version that runs on the GPU using Vulkan (which turned out to be not that useful since GPU warm-up takes a lot of time).

I have also tried to use the Mediatek NPU on my phone but last time I tried it required buying a license.

There are exporter scripts in the repo that allow exporting the model for fp32 CPU and fp16 Vulkan-based inference.

Let me know what you think! The video above is the gui.py script running the model in what-if mode.

It's an encoder-only transformer that consumes past(blood glucose + carbs + insulin) and future(carbs + insulin) and predicts future blood glucose for the next 2 hours. Announced meals and boluses/basal are used to condition its predictions. The context size is variable (8 - 24 hours), and model can work in autoregressive mode to predict the next >2 hours. It also predicts time by looking at the context, but it never consumes time. The architecture is BERT-style: bidirectional attention with future BG masked. DILATE loss was used to fit the median line; pinball loss to fit the uncertainty bands. The two are "mixed" via Kendall-Gal. All blood glucose is in kovatchev risk space reparameterized to [40, 400] range.

I have trained 4 model classes (nano, small, medium, large) and 3 variants for each (pretrained on simulator only, pretrained + finetuned on ohiot1dm, pretrained and finetuned on ohiot1dm + azt1d + shanghait1dm). The largest one has ~17 million parameters (16 heads across 16 layers). Pretraining for the largest model took ~48 hours. Finetuning took <10 minutes. There is also another version finetuned on my own data that I am currently running on my phone.

Source is available here, released under the MIT license. The repo also contains links to trained weights and evaluation data.

I've worked on this project since March. There are still things to improve (e.g. it always requires announced carbs + insulin, would be better if it could also predict without them), but I have decided to publish it here to get your opinion (and also answer your questions, if any).

Edit: my model is getting fat-shamed ;_; so I just want to emphasize that there is a nano version with less than 40K parameters.

I've been working on a suite of open source software since March that now I've begun to share. This is in spirit of Dana Lewis's "Software-Shaped Feelings" post: it is an N=1 software that is purpose built for my purposes. But I've done more than just build a monitoring app, I also have a SOTA-competitive model for BG prediction, a heuristic simulator that I used to generate a large synthetic training dataset, and a backend that acts like a database & notification system. I also want to be honest about the fact that I have heavily used AI, but I doubt that I could have done all this in l such a short period of time, despite the fact that I am a software engineer. I am not sure whether I can post the GitHub link here (self-promo rules), but if anyone's interested I can share it.

Really pushing the limits of my GPU.