Volleyball match analysis from a moving broadcast camera: RF-DETR + RTMPose + court homography + rules
Been working on a volleyball analysis pipeline, this is the current output on a full rally sequence.
The input is just the hall's automatic broadcast camera, so it pans and zooms. No fixed camera, no calibration.
Rough pipeline:
- Detection: RF-DETR Medium at 1280, fine-tuned on of my own labeled frames (ball + person). The ball is tiny (median ~22 px), which is why I stayed at 1280 instead of 640. Test AP50 is 0.92 for the ball, 0.97 for people.
- Camera motion: phase correlation between frames.
- Court: per-frame homography fitted to the white court lines, checked against the floor colour, so everything can be expressed in court metres.
- Ball: my own Kalman tracker with a gravity term, to bridge missed detections and catch bounces and hits.
- Pose: RTMPose on every frame, used for which hand/body part touched the ball and for the first referee's arm signals.
- Players: 6+6 tracks per team, plus referees and bench told apart by position (and shirt colour for the officials).
- Scoreboard: macOS Vision OCR on the broadcast graphic.
- On top of that, a rule layer (FIVB rules) that turns all of this into touches, blocks, rallies and points, and flags it when the scoreboard, the referee and the ball disagree.
It also estimates ball speed per hit (2D, so a lower bound), attack zone and direction, serve position and whistle-to-serve time, landing point in/out, and running distance (experimental).
Hardest parts so far:
- the ball leaving the top of the frame and the tracker jumping to something else
- referee signals: in this hall two staff stand behind the first referee and their boxes are taller than his (his legs are hidden by the stand), so picking "the tallest person on the stand" read the wrong arms
- every new hall = new floor colours, lines, lighting and camera angle
Yes it is all vibe coding with Claude