Fix two-sided outcome-constraint metrics showing a single direction - #5255
Open
tg9963 wants to merge 1 commit into
Open
Fix two-sided outcome-constraint metrics showing a single direction#5255tg9963 wants to merge 1 commit into
tg9963 wants to merge 1 commit into
Conversation
|
@tg9963 has exported this pull request. If you are a Meta employee, you can view the originating Diff in D113506683. |
Summary: A metric constrained on both sides (a two-sided band with both a `>=` and a `<=` outcome constraint, e.g. `m >= 0.9` and `m <= 1.1`) was rendered on the Ax platform with a single "increase" direction, so the upper bound appeared to be lost. The constraint `op` and `bound` were always stored correctly; the bug was in the metric's derived `lower_is_better`. When building the experiment, Ax derived each metric's `lower_is_better` from only the FIRST outcome constraint referencing it (`if metric_name not in map`). For a two-sided band the first constraint is `>=` (GEQ -> lower_is_better=False -> "Increase"), and the second bound was ignored. A metric bounded on both sides has no single preferred direction. - Extract a shared `InstantiationBase._build_lower_is_better_map` helper. A metric with conflicting constraint ops (both GEQ and LEQ) now maps to `lower_is_better=None` instead of the arbitrary first constraint's direction. - Use the helper in both call sites that previously duplicated the buggy block: `make_experiment` (create path) and `AxClient.set_optimization_config` (update path). Drop the now-unused `ComparisonOp` import in ax_client. Objective directions and single-sided constraints are unchanged. Objective and constraint metric names are always disjoint (an objective metric cannot be constrained), so the two groups never conflict. Differential Revision: D113506683
tg9963
force-pushed
the
export-D113506683
branch
from
July 24, 2026 05:22
055d610 to
fda69e1
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #5255 +/- ##
==========================================
- Coverage 96.58% 96.57% -0.01%
==========================================
Files 621 621
Lines 70663 70682 +19
==========================================
+ Hits 68248 68262 +14
- Misses 2415 2420 +5 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary:
A metric constrained on both sides (a two-sided band with both a
>=and a<=outcome constraint, e.g.m >= 0.9andm <= 1.1) was rendered on the Axplatform with a single "increase" direction, so the upper bound appeared to be
lost. The constraint
opandboundwere always stored correctly; the bug wasin the metric's derived
lower_is_better.When building the experiment, Ax derived each metric's
lower_is_betterfromonly the FIRST outcome constraint referencing it (
if metric_name not in map).For a two-sided band the first constraint is
>=(GEQ -> lower_is_better=False-> "Increase"), and the second bound was ignored. A metric bounded on both sides
has no single preferred direction.
InstantiationBase._build_lower_is_better_maphelper. Ametric with conflicting constraint ops (both GEQ and LEQ) now maps to
lower_is_better=Noneinstead of the arbitrary first constraint's direction.make_experiment(create path) andAxClient.set_optimization_config(update path). Drop the now-unused
ComparisonOpimport in ax_client.Objective directions and single-sided constraints are unchanged. Objective and
constraint metric names are always disjoint (an objective metric cannot be
constrained), so the two groups never conflict.
Differential Revision: D113506683