u/redsedit

PSA: When using `cjxl.exe`, always provide a `d` value. Don't assume that omitting `d` means lossless.

Backstory

I have tens of thousands of engineering drawings that we scanned for archival purposes. Maybe 10% will ever be viewed again, but we have no idea which 10%, so everything has to remain online.

The scanners don't support modern image formats, so we had to settle for PNG or TIFF depending on the model. To save storage space (which means lower cost), reduce backup times, and hopefully never have to care about restore times, I started looking at converting the PNGs to JPEG XL.

I ran some tests using visually lossless compression (`d=0.19`) and gave both the originals and the converted files to the managers. They couldn't see any difference, even at 500% magnification. The conversion was approved.

During a few test batches, I noticed something strange. I was also testing what I thought was lossless compression by omitting the `d` value, and in some cases the PNG files were actually smaller than the [what I believed to be] lossless JXL files. That seemed wrong.

A Google search pointed me toward a possible explanation. JPEG XL supports two compression modes: modular (typically used for non-photographic images and lossless compression) and VarDCT (typically used for photographs). One can outperform the other depending on the source image, and it's obvious which category a particular scanned drawing falls into.

Since the conversion wasn't under any time pressure, I wrote a quick PowerShell script that recursively scanned a directory and tested three encodes for every PNG:

• "Lossless" (no `d` value specified)
• `d=0.19` using VarDCT
• `d=0.19` using modular mode

The script compared the resulting file sizes, kept the smallest JXL, and replaced the original PNG. Since I could let it run overnight and on weekends, encoding speed wasn't a concern, so I bumped the effort setting to `-e 8`.

As always, I tested the script before turning it loose on the production data. During testing, I noticed that the supposed lossless version kept winning. That didn't make sense.

It couldn't be a modular versus VarDCT issue because I was explicitly testing both at `d=0.19`. How could a file that throws away no data be smaller than files that throw away some data?

That's when I realized I shouldn't have included the lossless test in the first place. The compression mode comparison was already covered by the two `d=0.19` tests.

After some digging, I found the catch.

If you feed `cjxl.exe` a JPEG or GIF and do not specify `d`, it defaults to `d=0.0`. If you feed it a PNG or certain other formats, it defaults to `d=1.0`. That explains why the "lossless" encodes kept winning. They weren't lossless at all. They were lower quality than my `d=0.19` test files.

I removed the no `d` test from the script and finally turned it loose on the production data. It worked fine.

One thing that did surprise me was that there wasn't a consistent winner between modular and VarDCT. Sometimes modular produced the smaller file, and sometimes VarDCT did. For scanned drawings where the goal is maximum space savings, testing both turned out to be the right approach. I wish I had kept statistics on how often modular beat VarDCT and vice versa, but that wasn't the focus of the testing. I was only interested in keeping whichever output was smaller for each image.

Still, I saved over 2 TB of SAN storage space, and that's not counting backup space saved either.

I ran into an issue with JXL files and wanted to share it in case others see the same behavior.

Background: I’m on Windows with Microsoft’s JXL Image Extensions installed.

Problem: After converting a batch of JPGs to JXL using lossless JPEG transcoding, I noticed that some JXL files appeared rotated incorrectly in Windows Explorer (both thumbnails and the preview pane). The Windows Photos app showed the same problem. The original JPGs displayed correctly.

When I opened the same JXL files in Nomacs, they were oriented correctly. They also appeared correctly when inserted into an Office 365 document. After some digging, I found the cause and a fix.

Cause: JPEG and JXL handle orientation differently. JPEG uses an EXIF orientation tag. The JXL spec ignores EXIF orientation and expects the image data itself to already be correctly oriented. The cjxl encoder reorients the pixel data during conversion, which is correct. However, it appears to leave the EXIF orientation tag in place. That tag doesn’t show up in Nomacs the same way it does for JPGs, but it’s still present apparently. Some viewers (notably Windows Explorer and Windows Photos with the JXL extension) read that leftover tag and apply rotation again, which results in the incorrect orientation. Office 365, despite using the same extensions, does not seem affected.

Fix (Windows): Remove the EXIF orientation tag from the JXL files. You can do this with ExifTool (no install required). Leaving -Orientation= without a value removes the tag:

"<path to tool>\exiftool.exe" -Orientation= *.jxl

This creates backup files by default. To skip creating backups:

"<path to tool>\exiftool.exe" -Orientation= -overwrite_original *.jxl

After running this, thumbnails and previews display correctly.