Computer vision approach for robust artwork segmentation
I'm building an embroidery digitization tool that converts PNG/JPEG/SVG artwork into embroidery-ready files. I'm currently working on the artwork/background segmentation stage.
My current pipeline is roughly:
Image → background/color segmentation → connected components
→ contours/holes → physical filtering → stitch generation
I'm using mostly classical CV right now.
I've encountered two problems:
1. Thin artwork disappears:
One image contains ~11% visible artwork, but my segmentation classifies almost the entire image as background. Thin/anti-aliased features seem to be the main issue.
2. Too many false regions:
Another JPEG produces ~155 regions, but ~99% of the actual artwork is contained in only two major regions. Most of the remaining regions appear to be JPEG/anti-aliasing artifacts.
I can't simply remove small components because some genuinely thin artwork needs to survive. I also know the final physical embroidery size, so pixel dimensions can eventually be converted to mm.
For CV engineers: how would you approach this?
Would you recommend LAB/color clustering, background modeling, edge-based methods, graph segmentation, SAM/SAM2, vectorization, or a hybrid approach?
I'm particularly interested in methods that can preserve thin artwork while rejecting compression/anti-aliasing noise.