Skip to content

BOT: Fix #895: Add get_non_monotonic_forecasts() and monotonicity warning#1081

Draft
nikosbosse wants to merge 1 commit intomainfrom
fix/895-non-monotonic-quantiles
Draft

BOT: Fix #895: Add get_non_monotonic_forecasts() and monotonicity warning#1081
nikosbosse wants to merge 1 commit intomainfrom
fix/895-non-monotonic-quantiles

Conversation

@nikosbosse
Copy link
Collaborator

Summary

  • Adds a new exported get_non_monotonic_forecasts() diagnostic function (modeled on get_duplicate_forecasts()) that identifies forecast units where quantile predictions are not monotonically non-decreasing with increasing quantile levels
  • Adds a warning in assert_forecast.forecast_quantile() (guarded by verbose) so users are alerted at as_forecast_quantile() time rather than only discovering the issue deep inside bias_quantile() during scoring
  • Also addresses Add check to validate_forecast() that predictions must not decrease with quantile #412 (add monotonicity check to forecast validation)

Root cause: Non-monotonic quantile predictions silently pass through as_forecast_quantile() and assert_forecast(). The only check exists deep in bias_quantile_single_vector() as a hard error, which users only encounter at score() time — often after significant pipeline setup.

What changed:

  • New R/get-non-monotonic-forecasts.R with exported get_non_monotonic_forecasts() and internal check_monotonicity()
  • assert_forecast.forecast_quantile() now calls check_monotonicity() and emits a cli_warn() when violations are found
  • Warning is suppressible via verbose = FALSE
  • 15 new tests in test-get-non-monotonic-forecasts.R + 4 new tests in test-class-forecast-quantile.R

Closes #895
Closes #412

Test plan

  • New get_non_monotonic_forecasts() tests: well-formed data returns empty, detects violations, mixed units, counts argument, custom forecast_unit, class checks, data.frame input, equal predictions edge case
  • check_monotonicity() tests: returns TRUE for good data, message string for bad data
  • assert_forecast.forecast_quantile() tests: warns on non-monotonic, no false positive, verbose=FALSE suppresses, end-to-end score works with bias excluded
  • Full test suite passes (699 tests, 0 failures)
  • R CMD check: 0 errors, 0 warnings, 2 notes (pre-existing)

🤖 Generated with Claude Code

Add diagnostic helper and early warning for non-monotonic quantile predictions:
- New exported `get_non_monotonic_forecasts()` function (modeled on
  `get_duplicate_forecasts()`) to identify forecast units where predicted
  values decrease with increasing quantile levels
- New internal `check_monotonicity()` helper
- Warning in `assert_forecast.forecast_quantile()` (guarded by `verbose`)
  so users are alerted during `as_forecast_quantile()` instead of only
  discovering the issue deep inside `bias_quantile()` at scoring time

Also addresses #412.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@codecov
Copy link

codecov bot commented Feb 13, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.86%. Comparing base (ac0c01a) to head (7f00d40).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1081      +/-   ##
==========================================
+ Coverage   97.83%   97.86%   +0.03%     
==========================================
  Files          35       36       +1     
  Lines        1845     1875      +30     
==========================================
+ Hits         1805     1835      +30     
  Misses         40       40              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Collaborator Author

@nikosbosse nikosbosse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CLAUDE: Review verdict — APPROVE. Clean implementation following the established get_duplicate_forecasts()/check_duplicates() pattern. Core detection logic (diff(predicted) < 0) correctly flags strictly decreasing predictions while allowing equal values. Warning (not error) in assert_forecast.forecast_quantile() is the right design choice per issue discussion. All 12 test specifications covered by 15 tests. No issues found.

@nikosbosse nikosbosse marked this pull request as draft February 13, 2026 08:27
@nikosbosse nikosbosse changed the title Fix #895: Add get_non_monotonic_forecasts() and monotonicity warning BOT: Fix #895: Add get_non_monotonic_forecasts() and monotonicity warning Feb 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Check quantile prediction order helper function Add check to validate_forecast() that predictions must not decrease with quantile

1 participant

Comments