Need Help Improving Accuracy - Signature Matching model by training on top of efficientnetb3 base
I'm working on a signature verification project to detect fraudulent signatures on cheques. Here is how my pipeline looks so far:
1. Extraction (Done)
I'm using a fine-tuned YOLOv8 model (from Tech4Humans) to detect and crop out the signatures from the cheque images. It’s working pretty great at isolating the signature area.
2. Preprocessing (The tricky part)
Since cheques come from all kinds of different scanners and phone cameras, the backgrounds and lighting are all over the place. To clean them up before feeding them to the model, I'm doing:
Ink isolation: Removing noise near the boundaries by thresholding pixels based on background brightness.
Line removal: Scanning for long horizontal bars (like the signature lines on the cheque), wiping them out, and filling the gaps based on the surrounding texture.
Contrast stretching: Using percentile contrast stretching to fix dark backgrounds caused by bad phone/scanner scans.
Smoothing: Applying a bilateral filter to smooth out background noise while keeping the signature lines sharp.
CLAHE: Using Contrast Limited Adaptive Histogram Equalization to boost the contrast of the ink against tough backgrounds.
Padding: Adding final padding so the aspect ratio doesn't get warped.
- The Model
Once preprocessed, I’m feeding the images into EfficientNet-B3 to train it to spot the differences between genuine and forged signatures.
The problem is my validation accuracy is not improving beyond 55-57,
what else can I do to improve this?
Is the preprocessing enough is should I improve it more, because I'm still getting some horizontal line like the signature line, and some text in some cases