feat: Add CLEAR MOT tracking evaluation metrics (MOTA, MOTP) with combined TrackingMetrics class#2193
feat: Add CLEAR MOT tracking evaluation metrics (MOTA, MOTP) with combined TrackingMetrics class#2193Preksha2 wants to merge 2 commits intoroboflow:developfrom
Conversation
e50d010 to
424c9a9
Compare
|
@Preksha2 thank you for your proposal. However, I think supervision will deprecate the tracking section since we created it as a separate repo - https://github.com/roboflow/trackers Am I right @SkalskiP" 🦝 |
|
Thanks for the feedback @Borda! That makes sense — I wasn't aware tracking was being separated into its own repo. I'd be happy to move this contribution to https://github.com/roboflow/trackers instead if that's the better home for tracking metrics. The implementation (MOTA, MOTP, TrackingMetrics) follows the same update()/compute() pattern and should integrate cleanly there. Alternatively, if there's another area in supervision where metrics contributions are welcome, I'm happy to adapt. Let me know how you'd like me to proceed! |
|
Thanks for this solid implementation of CLEAR MOT metrics! As we're deprecating the tracker module in |
Before submitting
Description
Adds CLEAR MOT tracking evaluation metrics to supervision's metrics module, implementing the standard defined in Bernardin & Stiefelhagen, "Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics" (JIVP, 2008). Includes MOTA (tracking accuracy), MOTP (tracking precision), and a combined TrackingMetrics class that computes both in a single pass.
Type of Change
Motivation and Context
Supervision has detection metrics (mAP, F1) but no tracking metrics. MOTA and MOTP are the two most widely used multi-object tracking evaluation metrics, standard in benchmarks like MOTChallenge. Tracking metrics were mentioned as welcome contributions in the 0.25.0 release notes. This PR fills that gap.
Changes Made
MultiObjectTrackingAccuracyclass (MOTA) measuring overall tracking quality via false positives, false negatives, and identity switchesMultiObjectTrackingPrecisionclass (MOTP) measuring average localization precision of matched detectionsTrackingMetricsclass computing both MOTA and MOTP in a single pass (single IoU matrix computation and matching per frame)MOTAResult,MOTPResult,MOTMetricsResultdataclasses with full component breakdowntracker_idupdate()/compute()interface consistent with the metrics APITesting
30 tests covering: perfect tracking, false positives, false negatives, identity switches, IoU thresholds, partial overlap localization, empty frames, reset behavior, input validation, and string representations.
MOTA = 1 - (FN + FP + IDSW) / GT
MOTP = sum(IoU of matched pairs) / num_matches
Usage:
Additional Notes
Happy to add documentation entries for autogeneration and adjust the API style if the maintainers have preferences. This is my first contribution to supervision.