Skip to content

BOT: Fix #363: Add metric adapter functions for external scoring metrics#1101

Draft
nikosbosse wants to merge 1 commit intomainfrom
fix/363-metric-adapters
Draft

BOT: Fix #363: Add metric adapter functions for external scoring metrics#1101
nikosbosse wants to merge 1 commit intomainfrom
fix/363-metric-adapters

Conversation

@nikosbosse
Copy link
Collaborator

Summary

  • Adds metric_adapt_binary_numeric() — wraps external metrics that expect numeric 0/1 observed values, converting the factor input that scoringutils uses for binary forecasts
  • Adds metric_adapt_swap_args() — wraps external metrics with function(predicted, observed) signature to match scoringutils' function(observed, predicted) convention
  • Both adapters return functions with (observed, predicted, ...) formals, compatible with score(), run_safely(), and purrr::partial()

Fixes #363

Root cause

Users cannot directly use external scoring functions in score() when those functions expect a different argument order or numeric (not factor) binary observed values. purrr::partial() can fix extra parameters but cannot reorder positional arguments or perform type conversion.

What this does

Creates two small adapter/wrapper factory functions that return closures with the correct signature. The adapters:

  • Convert factor observed to 0/1 numeric (metric_adapt_binary_numeric)
  • Swap the first two positional arguments (metric_adapt_swap_args)
  • Pass through ... for additional parameters
  • Are composable with purrr::partial() for further customization

Test coverage

10 test cases covering:

  • Basic factor-to-numeric conversion
  • End-to-end integration with score() for binary, point, and sample forecasts
  • Argument order reversal verification
  • Function object properties (formals, ... pass-through)
  • Composability with purrr::partial()
  • Input validation (non-function inputs error)
  • Extra argument pass-through

Test plan

  • New adapter tests pass (26 expectations)
  • Full test suite passes (704 tests, 0 failures)
  • R CMD check: 0 errors, 0 warnings, 2 pre-existing notes

🤖 Generated with Claude Code

Add metric_adapt_binary_numeric() and metric_adapt_swap_args() to wrap
external scoring functions for use with score(). These handle the two
most common incompatibilities: factor-to-numeric conversion for binary
observed values, and reversed argument order (predicted, observed).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@codecov
Copy link

codecov bot commented Feb 13, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.84%. Comparing base (ac0c01a) to head (889e308).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1101      +/-   ##
==========================================
+ Coverage   97.83%   97.84%   +0.01%     
==========================================
  Files          35       36       +1     
  Lines        1845     1854       +9     
==========================================
+ Hits         1805     1814       +9     
  Misses         40       40              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Collaborator Author

@nikosbosse nikosbosse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CLAUDE: Approve — Clean, minimal implementation of two adapter factory functions (metric_adapt_binary_numeric and metric_adapt_swap_args) that solve a real usability gap for external metric integration. Code follows existing package patterns (identical factor conversion to brier_score), closures have correct formals for run_safely() compatibility, and the 10-test suite covers unit, integration, composability, validation, and pass-through scenarios. No issues found.

@nikosbosse nikosbosse marked this pull request as draft February 13, 2026 08:28
@nikosbosse nikosbosse changed the title Fix #363: Add metric adapter functions for external scoring metrics BOT: Fix #363: Add metric adapter functions for external scoring metrics Feb 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Discussion: design interface for external functions with different input/output formats

1 participant

Comments