SLIM: a lossless image and video codec built on QOIR
At work I sometimes need to collect terabytes of video very fast, and it is usually done in an extreme operating environment that prevents me from just buying more drives. A few months ago an experiment went a little long and overflowed all available drives with imagery, so I started looking for lossless compression options that are very fast to encode. That led me to the exceptionally elegant QOI format, and from there the more performant QOIR, which I've based my own solution on. (Apache 2.0 license.)
https://gitlab.com/csp256/slim - very much still a work in progress! and beware, QOI's elegance is long gone at this point
I was already working on a container called Slate, so I'm calling this SLIM - the Slate Image format. On that same data that overflowed my drives, SLIM reaches speeds in excess of 4 GB/s on my 2023 MacBook Air to achieve a 0.019 compression ratio. On some modes it beats 0.010 ratio, or 100x compression. optipng -o0 is 38% larger and 40 times slower. jpeg-xl e1 is 22% smaller, but 23 times slower.
Preliminary benchmarks are on the GitLab, but on large RGB photographs from the FiveK data set SLIM achieves 0.376 at 1,238.6 MiB/s encode, comparable to PNG and JPEG-XL but drastically faster.
SLIM supports all major operating systems, threading, delta frames, masks, any data type up to 8 bytes, support for 1 to 4 channel images, improved handling of images with significant alpha gradients or low entropy regions, and a few other bespoke features.
Decode speed is not a priority and has not yet been optimized. However, it will be similar to encode. Faster, for delta frames.
It's past my bed time, but tomorrow I will provide more specifics about the format and how it varies from QOIR.
Opcode Changes
SLIM makes only a few direct modifications to QOIR. The opcode changes are particularly modest:
Runs of length 1 are now cannonicalized to use
DIFFinstead ofRUNS. The bias ofRUNShas been adjusted accordingly.The high bit of the second byte of
RUNLis reserved.RUNL's bias has been adjusted such thatRUNL 0is a run of length 1 longer than the longest amount representable byRUNS.If the high bit of the second byte of
RUNLis 1, then the 2 byteRUNLopcode is interpretted instead as a 3 byteRUNLLopcode, which uses the third byte and bottom 7 bits of the second byte to support runs of length up to 2^15. Tiles are 64 pixels square, or 2^16 pixels, so in principle only twoRUNLLopcodes are necessary for indicating a tile is a single constant color.If SLIM detects that a tile is all black (with alpha 255 if present), then SLIM may emit the entire tile payload as simply
INDEX 0. BecauseINDEX 0will never otherwise be emitted as the first byte of the payload there is no ambiguity for the decoder. This is subject to change. The reason for this is to make the "no change" delta frame code path compress as well as possible.
These changes have only a tiny but consistently positive impact on the QOI test suite, and a more significant positive impact on my data set.
Pack3 handling of individual channels
The more meaningful change that SLIM provides is what I call the "pack3" strategy: for 1 channel images, groups of three rows are re-interpreted to be a single row of "pseudo RGB" pixels. This provides the spatial correlations QOI-style gradient compressors want to exploit. If needed the bottom of the image is logically padded with an extra row or two that repeats the bottom row.
RGBA images with prominent alpha gradients were a weak point for QOI. QOIR significantly improves on this by adding more opcodes that express alpha changes, but SLIM also adds the optional ability to treat the alpha channel totally independent of the RGB channels by using the pack3 strategy on only that channel, and QOIR's normal RGBX strategy on the RGB channels.
Two channel images have both channels compressed with pack3 independently.
More data types
Data type lengths longer than 1 byte compress each byte plane independently, often with pack3 but sometimes with RGB.
Signed integers are zig-zag transformed before encoding and after decoding. That is to say, they are remapped such that -x and +x are adjacent to each other. Analogously, floating point values have one byte cyclically shifted by 1 such that the sign bit is now the low bit. This behavior is transparent to the user but intended to provide better support for data that changes sign frequently.
Low bytes of larger data types may be essentially incompressible so SLIM allows the user to mark them as such. Those bytes will always be emitted raw. SLIM also has experimental support for dynamically determining which bytes are incompressible. This feature is not yet tuned and may not work very well, but it seems to mitigate the most pathological cases.
Support for signed integers and floating point values is for completeness. It is not a design focus and may not perform well.
Delta frames and tile format changes
QOIR emits tiles with one of 4 modes: raw or as opcodes, and either unmodified or subsequently lz4 compressed. SLIM extends this with another axis: as an intra frame or as a delta to the previous frame. Because storing raw deltas uncompressed never makes sense (you could just store the raw frame in the same size), SLIM reserves tile mode 0x04 to indicate that this tile had no change from the previous frame. The payload length is 0 in this case. Tile modes 0x05 through 0x07 are identical to 0x01 to 0x03 except on delta images.
Delta frames are logically zig-zag coded versions of the signed difference relative to reference, packed into the original byte size. That is to say: the case where a pixel is 0 in one frame and saturating 255 the next (or vice versa) is properly handled. It will not be neglected just because it is a cyclic distance of just 1, even if a lossy mode telling SLIM to ignore small changes is enabled.
By default SLIM does not try to compress raw frames if given a reference frame from which to form a delta image. However if you want better compression at the cost of halving the speed, SLIM allows you to specify that compression should also be attempted on raw frames. Smallest wins.
QOIR optionally allows tiles to be emitted out of order. This is intended to help in multithreaded workloads with uneven workloads. A 4 byte tile index is prepended to each tile in this case.
Lossy modes
SLIM supports a couple different lossy modes, specifically for unsigned integer data types. The first is the noise_floor: values below the noise floor are raised to the noise floor before encoding, and values equal to the noise floor are dropped back to the noise floor. This is to decrease the gradient QOIR sees. The noise floor may only be 1 byte, but there is support for all data types and it can be set per-channel. While this transform is only applied to the low byte, it ensures that higher bytes are all 0. The purpose of this setting is to suppress spurious counts caused by leaky currents, thermal effects, etc in what would otherwise be a near perfectly black environment.
The second lossy mode is similar, except it is applied to delta images. I don't recommend this mode, because long sequences of delta frames can drift arbitrarily from the last intra frame as long as the change is gradual.
The third (and perhaps final) lossy mode is a bad pixel mask, supplied either as a u8 image or sequence of pixel positions. It can be applied to all channels or per channel. The mask can optionally also be embedded within the SLIM file. Pixels marked by the mask (values >= 128) are treated as if they're identical to the previous pixel. Note: currently, if the first pixel of a tile is marked bad it will be replaced with black (opaque if alpha present). I intend to modify this case to scan for the next good pixel instead.
Misc
SLIM has a variable length header that can encode arbitrary length user data. SLIM files currently do not have support for image sequences, but once they do it will also support per-frame metadata.
SLIM does not yet support premultiplied alpha, but it will be added eventually.
There is currently no special handling of limited bit depths, but I'm open to ideas!
SLIM has a recovery utility that can attempt to recover corrupted files, especially those that might be caused by sudden power loss during encoding or writing.
SLIM files typically have an entropy of 7.5 bits per byte, so entropy coding could in principle shrink the file by about 7%, but SLIM does not emphasize compression ratio to an extent where that is currently in-scope.