Skip to content

Comments

Fix: extended_metrics raises clear ValueError when IoU=0.50 absent from iouThrs#75

Merged
MiXaiLL76 merged 2 commits intomainfrom
copilot/fix-extended-metrics-valueerror
Feb 21, 2026
Merged

Fix: extended_metrics raises clear ValueError when IoU=0.50 absent from iouThrs#75
MiXaiLL76 merged 2 commits intomainfrom
copilot/fix-extended-metrics-valueerror

Conversation

Copy link
Contributor

Copilot AI commented Feb 19, 2026

COCOeval_faster.extended_metrics crashed with a cryptic ValueError: can only convert an array of size 1 to a Python scalar whenever params.iouThrs didn't include 0.50, because np.argwhere(...).item() blows up on an empty array.

Motivation

Any caller customising params.iouThrs to a set that excludes 0.50 (e.g. single-threshold or high-IoU-only evaluation) hits an opaque crash with no indication of what went wrong.

Modification

  • faster_coco_eval/core/faster_eval_api.py: Replace the unconditional .item() call with a guarded np.where(...) lookup. Raises a descriptive ValueError naming the missing threshold when 0.50 is absent.
# Before (crashes opaquely when 0.50 not in iouThrs)
iou50_idx, area_idx, maxdet_idx = (np.argwhere(np.isclose(iou_thrs, 0.50)).item(), 0, -1)

# After (clear error)
_iou50_hits = np.where(np.isclose(iou_thrs, 0.50))[0]
if len(_iou50_hits) == 0:
    raise ValueError(
        "extended_metrics requires IoU threshold 0.50, "
        f"but it is not present in params.iouThrs={iou_thrs.tolist()}."
    )
iou50_idx, area_idx, maxdet_idx = (int(_iou50_hits[0]), 0, -1)
  • tests/test_simple_extra.py: Add _make_coco_eval() helper and two tests — one asserting the new ValueError fires when 0.50 is absent, one asserting normal operation when using default iouThrs.

BC-breaking (Optional)

Previously the crash produced a generic ValueError; it now produces a more descriptive one. Code catching the bare ValueError type is unaffected. Code matching on the old message (unlikely) would need updating.

Checklist

  1. Pre-commit or other linting tools are used to fix the potential lint issues.
  2. The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
  3. If the modification has potential influence on downstream projects, this PR should be tested with downstream projects, like MMDet or MMCls.
  4. The documentation has been modified accordingly, like docstring or example tutorials.
Original prompt

This section details on the original issue you should resolve

<issue_title>Bug: extended_metrics crashes with ValueError when IoU=0.50 is not in iouThrs</issue_title>
<issue_description>## Summary

COCOeval_faster.extended_metrics unconditionally calls .item() on the result of
np.argwhere(...). When params.iouThrs does not contain 0.50, argwhere returns an
empty array and .item() raises ValueError.

Affected property

COCOeval_faster.extended_metrics (faster_eval_api.py, line ~238)

# current (buggy) code
iou50_idx = np.argwhere(np.isclose(iou_thrs, 0.50)).item()
#                                                   ^^^^^^
# raises ValueError when the array is empty (IoU=0.50 not found)

Reproducible example

import numpy as np
from faster_coco_eval.core import COCO
from faster_coco_eval.core import COCOeval_faster as COCOeval

# Minimal single-class setup
image_id = 1
SIZE = 200
bbox = [0.0, 0.0, float(SIZE), float(SIZE)]

coco_gt = COCO()
coco_gt.dataset = {
    "images": [{"id": image_id, "width": 1000, "height": 1000}],
    "annotations": [{"id": 1, "image_id": image_id, "category_id": 1,
                     "bbox": bbox, "area": SIZE*SIZE, "iscrowd": 0}],
    "categories": [{"id": 1, "name": "cat1"}],
}
coco_gt.createIndex()

pred_bbox = [10.0, 10.0, float(SIZE - 20), float(SIZE - 20)]  # IoU ≈ 0.69
coco_dt = coco_gt.loadRes([
    {"image_id": image_id, "category_id": 1, "bbox": pred_bbox, "score": 0.9}
])

coco_eval = COCOeval(coco_gt, coco_dt, iouType="bbox")
coco_eval.params.iouThrs = np.array([0.55, 0.65, 0.75, 0.85, 0.95])  # no 0.50
coco_eval.evaluate()
coco_eval.accumulate()
coco_eval.summarize()

print(coco_eval.extended_metrics)  # raises ValueError

Error

ValueError: can only convert an array of size 1 to a Python scalar

Full traceback:

File ".../faster_coco_eval/core/faster_eval_api.py", line 238, in extended_metrics
    iou50_idx, area_idx, maxdet_idx = (np.argwhere(np.isclose(iou_thrs, 0.50)).item(), 0, -1)
ValueError: can only convert an array of size 1 to a Python scalar

Fix

Guard the lookup and return a sensible fallback (or raise a clear ValueError) when
IoU=0.50 is absent:

hits = np.where(np.isclose(iou_thrs, 0.50))[0]
if len(hits) == 0:
    # Either raise or return nan/zero metrics with a warning
    raise ValueError("iouThrs does not contain 0.50; extended_metrics requires it.")
iou50_idx = int(hits[0])

Notes

  • This is independent of the precision-interpolation bug (separate issue).
  • Any caller that customises params.iouThrs — e.g. to evaluate at a single threshold
    other than 0.50 — is affected.

Discovered in roboflow/rf-detr#231</issue_description>

<agent_instructions>perform test-driven development, validate if the issue is real or just false assumptions, if a real bug, add tests to reproduce the issue, and at last implement the fix</agent_instructions>

Comments on the Issue (you are @copilot in this section)


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

…d tests

Co-authored-by: Borda <6035284+Borda@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix ValueError in extended_metrics when IoU=0.50 is missing Fix: extended_metrics raises clear ValueError when IoU=0.50 absent from iouThrs Feb 19, 2026
Copilot AI requested a review from Borda February 19, 2026 12:48
@Borda Borda marked this pull request as ready for review February 19, 2026 12:51
Copilot AI review requested due to automatic review settings February 19, 2026 12:51
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a bug where COCOeval_faster.extended_metrics crashes with a cryptic error when IoU threshold 0.50 is not present in params.iouThrs. The fix replaces an unconditional .item() call with a guarded lookup that raises a descriptive ValueError when the required threshold is missing.

Changes:

  • Fixed crash in extended_metrics when IoU=0.50 is absent from iouThrs by adding explicit validation
  • Added comprehensive test coverage for both the error case and normal operation

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
faster_coco_eval/core/faster_eval_api.py Replaced unconditional .item() with guarded np.where() lookup and added descriptive error message
tests/test_simple_extra.py Added helper function and two tests to verify error handling and normal operation

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@Borda Borda requested a review from MiXaiLL76 February 19, 2026 15:39
@MiXaiLL76
Copy link
Owner

Thanks for the implementation, everything is great!

@MiXaiLL76 MiXaiLL76 merged commit 8a77cfa into main Feb 21, 2026
18 checks passed
@MiXaiLL76 MiXaiLL76 mentioned this pull request Feb 21, 2026
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: extended_metrics crashes with ValueError when IoU=0.50 is not in iouThrs

3 participants