I trained YOLO on 2,400 Unreal Engine frames. Synthetic validation reached 0.888 mAP50, but real-world recall was 0.350. Here’s what failed.
I trained a single-class YOLOv5n person detector using no real images:
- 2,400 synthetic frames from eight Unreal Engine 5.8 maps
- 9,007 native-scale 640 px tiles
- 63,742 person boxes
- 100 epochs, approximately 30 minutes on one RTX 4090
- int8 deployment on a Coral USB Accelerator
On the float model's synthetic validation split at epoch 65, we measured precision 0.944, recall 0.790, and mAP50 0.888.
Then we evaluated the Edge TPU model on 120 real VisDrone frames. At a 0.15 confidence threshold, recall dropped to 0.350, with precision 0.582. Lowering the threshold to 0.10 only raised recall to 0.374, so this was not just a confidence-calibration problem.
Green = found, orange = missed, red = false positive. On the synthetic scenes, the model was generally reliable when people were isolated, well separated, and standing on open ground.
The eight source captures and dataset downloads: https://huggingface.co/NameFrame
Has anyone here measured a similar contextual domain gap when moving from synthetic scenes to real footage?