Login

u/PandaBean18

r/FPGA r/developersIndia r/MachineLearning
Image 1 — Wrote two parallel INT8 MAC engines with 16 bit accumulation on an FPGA with 1120 LUTs and 32 kB BRAM for a diffusion model
Image 2 — Wrote two parallel INT8 MAC engines with 16 bit accumulation on an FPGA with 1120 LUTs and 32 kB BRAM for a diffusion model
Image 3 — Wrote two parallel INT8 MAC engines with 16 bit accumulation on an FPGA with 1120 LUTs and 32 kB BRAM for a diffusion model
Image 4 — Wrote two parallel INT8 MAC engines with 16 bit accumulation on an FPGA with 1120 LUTs and 32 kB BRAM for a diffusion model
Image 5 — Wrote two parallel INT8 MAC engines with 16 bit accumulation on an FPGA with 1120 LUTs and 32 kB BRAM for a diffusion model
▲ 20 r/FPGA

Wrote two parallel INT8 MAC engines with 16 bit accumulation on an FPGA with 1120 LUTs and 32 kB BRAM for a diffusion model

Recently I started messing around with the Shrike lite, I wanted to work on a project that was something cool so I started training a diffusion model that generates 32*32 pixel images which could run comfortably on the 256KB of SRAM that the shrike has.

To speed up the calculation I tried writing parallel INT8 MAC engines with 16 bit accumulation, however due to the less amount of RAM it spent a lot of time doing I/O leading it to actually be slower than the MCU only approach (~220 seconds per image vs ~70 seconds per image).

I guess this was expected but it was still fun to tinker around with the microcontroller and the FPGA. A lot of the images look noisy but some actually turned out to be pretty cool.

Full case study here.

edit: added link for case study

u/PandaBean18 — 2 days ago
▲ 125 r/MachineLearning

Trained an diffusion model that runs on 264KB of RAM [P]

I recently bought a Shrike lite which has got 264KB of SRAM. I decided to train an image generation model that generates 32*32 pixel images.

The microcontroller also has an FPGA onboard which I used to create two parallel INT8 MAC engines with 16 bit accumulation to speed up calculations, however the system soon hit a memory wall due to the high number of I/O operations, this meant that the system with parallel MAC engines ran slower than the MCU only model (~220 seconds per image vs ~70 seconds per image).

It was still a fun project that I enjoyed messing around with. A lot of the images looked weird and noisy because of the heavy quantization and memory limits but some of them came out cool.

Full case study here.

edit: added link that leads straight to the case study

u/PandaBean18 — 2 days ago
▲ 428 r/developersIndia

Created an image generation model that runs on 264 KB of RAM

Recently I worked with Shrike Lite and tried to run a small image generation model that generates 32*32 pixels images. I tried using the onboard FPGA to run 2 parallel INT8 MACs with 16-bit accumulation to make the multiplication faster but pretty quickly hit the "Memory wall" due to massive amounts of I/O taking place.

The chip takes about 220 seconds to generate one image (took about 70 seconds with the MCU only) but it was fun to mess around with the shrike lite.

Most of the images that it generates are quite noisy but some of them actually look pretty cool.

Full case study here

u/PandaBean18 — 2 days ago