Skip to content

feat(hitl): decouple bboxes from confidence scores#4639

Merged
mergify[bot] merged 5 commits into
box:masterfrom
dlasecki-box:hitl-bbox-decouple
Jun 18, 2026
Merged

feat(hitl): decouple bboxes from confidence scores#4639
mergify[bot] merged 5 commits into
box:masterfrom
dlasecki-box:hitl-bbox-decouple

Conversation

@dlasecki-box

@dlasecki-box dlasecki-box commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

This PR decouples Bounding Box support from Confidence Score review in the Human-in-the-loop Metadata Extraction flow.

Previously, the entire feature was controlled by a single flag: isConfidenceScoreReviewEnabled. Due to updated business requirements, the work is now split into two independently controlled milestones:

Rendering and saving bounding boxes for extracted metadata values

  • Flag: isBoundingBoxeEnabled

Reviewing extracted metadata fields with LOW confidence

  • Flag: isConfidenceScoreReviewEnabled
bbox only

Summary by CodeRabbit

  • New Features
    • Added AI-extracted metadata field support, including an optional extracted-status indicator and enhanced redesign $details enrichment.
    • Renamed and expanded the review toggle to a combined bounding-box-or-confidence capability that now drives fetching, creation, and unsaved-changes behavior.
  • Tests
    • Added unit tests covering extracted-status behavior, redesign $details payload rules, and bounding-box-enabled fetch/create flows.
  • Chores
    • Updated metadata tooling for template support, including translation bundling and Jest transform configuration, plus related dev dependency updates.

@coderabbitai

coderabbitai Bot commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 2c08e253-95a8-4211-a567-53b211648c37

📥 Commits

Reviewing files that changed from the base of the PR and between f110c55 and 6aeab1f.

⛔ Files ignored due to path filters (1)
  • yarn.lock is excluded by !**/yarn.lock, !**/*.lock
📒 Files selected for processing (1)
  • package.json
🚧 Files skipped from review as they are similar to previous changes (1)
  • package.json

Walkthrough

The PR introduces a combined isBoundingBoxOrConfidenceScoreReviewEnabled flag by merging the existing confidence-score flag with a new metadata.boundingBox.enabled feature flag. A new AI_EXTRACTED_PROCESS constant and checkIsExtractedProcessFieldValue helper are added. The flag and helper are threaded through Metadata.js API methods, sidebar hooks, and UI components. @box/metadata-template-browser is added as a dependency and registered in build/test configs.

Changes

Bounding Box and AI Extracted Process Support

Layer / File(s) Summary
Constants, types, and utility helper
src/constants.js, src/common/types/metadata.js, src/api/utils.js, src/api/__tests__/utils.test.js
Adds AI_EXTRACTED_PROCESS constant, optional isExtracted field on MetadataTemplateInstanceField, checkIsExtractedProcessFieldValue helper that checks details.process === AI_EXTRACTED_PROCESS, replaces hardcoded isAccepted string with AI_ACCEPTED_PROCESS constant, and exports the new helper with full test coverage.
Metadata API: flag rename and $details payload rewrite
src/api/Metadata.js, src/api/__tests__/Metadata.test.js
Renames isConfidenceScoreEnabled to isBoundingBoxOrConfidenceScoreReviewEnabled across all five affected method signatures (getInstances, createTemplateInstance, getTemplateInstances, getMetadata, createMetadataRedesign); marks fields isExtracted using the new helper; rewrites $details payload in createMetadataRedesign to emit AI_EXTRACTED_PROCESS entries and prefer AI_ACCEPTED_PROCESS when accepted; adds 240 lines of new tests covering isExtracted derivation and $details precedence.
useSidebarMetadataFetcher: bounding box parameter and request gating
src/elements/content-sidebar/hooks/useSidebarMetadataFetcher.ts, src/elements/content-sidebar/__tests__/useSidebarMetadataFetcher.test.tsx
Adds isBoundingBoxEnabled parameter, derives isBoundingBoxOrConfidenceScoreReviewEnabled, threads it into fetchMetadata and handleCreateMetadataInstance, separates include_confidence_score and include_reference gating in extractSuggestions, and adds tests for bounding-box-only mode.
useMetadataSidebarUnsavedChangesGuard: flag rename
src/elements/content-sidebar/hooks/useMetadataSidebarUnsavedChangesGuard.ts, src/elements/content-sidebar/__tests__/useMetadataSidebarUnsavedChangesGuard.test.ts
Renames isConfidenceScoreReviewEnabled to isBoundingBoxOrConfidenceScoreReviewEnabled in the Params interface, hook body, modal guard, navigation blocking effect, and all test assertions.
MetadataSidebarRedesign and MetadataInstanceEditor: feature flag wiring
src/elements/content-sidebar/MetadataSidebarRedesign.tsx, src/elements/content-sidebar/MetadataInstanceEditor.tsx, src/elements/content-sidebar/__tests__/MetadataInstanceEditor.test.tsx, src/elements/content-sidebar/__tests__/MetadataSidebarRedesign.test.tsx
MetadataSidebarRedesign reads metadata.boundingBox.enabled, derives the combined flag, and forwards isBoundingBoxEnabled to MetadataInstanceEditor and MetadataInstanceList. MetadataInstanceEditor adds the optional prop and forwards it to MetadataInstanceForm. Tests updated and added to cover the new flag paths.
Package and build config
package.json, scripts/i18n.config.js, scripts/jest/jest.config.js
Adds @box/metadata-template-browser to devDependencies, peerDependencies, translationDependencies, and Jest transformIgnorePatterns allowlist; bumps @box/metadata-editor to ^1.73.4.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~50 minutes

Possibly related PRs

  • box/box-ui-elements#4441: Introduced the original confidence-score feature-flag prop wiring in MetadataSidebarRedesign.tsx and MetadataInstanceEditor.tsx that this PR renames and extends with bounding-box support.
  • box/box-ui-elements#4499: Modified createMetadataRedesign in src/api/Metadata.js to thread the confidence-flag parameter into $details payload generation, which this PR extends to handle extracted fields.
  • box/box-ui-elements#4491: Wired the confidence/reference extraction flow through useSidebarMetadataFetcher using include_confidence_score/include_reference gating, the same area this PR refactors to use the combined bounding-box flag.

Suggested reviewers

  • Lindar90

Poem

🐇 A new flag hops into the field,
Bounding boxes and scores revealed!
AI_EXTRACTED joins the fray,
isExtracted marks the way. 🎯
The rabbit stamps the payload sealed! ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately captures the main objective of the PR—decoupling bounding box functionality from confidence score review in the HITL metadata extraction flow.
Description check ✅ Passed The description clearly explains the purpose of decoupling bounding boxes from confidence scores, the two independent flags introduced, and includes a visual example. However, it lacks structured sections as suggested by the repository's description template.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@socket-security

socket-security Bot commented Jun 18, 2026

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addednpm/​@​box/​metadata-template-browser@​1.24.1377100899950
Updatednpm/​@​box/​metadata-editor@​1.69.6 ⏵ 1.73.1186 +110090 +19950

View full report

feat(hitl): flow fix

feat(hitl): flow fix

feat(hitl): flow fix

feat(hitl): flow fix

feat(hitl): flow fix
@dlasecki-box
dlasecki-box marked this pull request as ready for review June 18, 2026 09:57
@dlasecki-box
dlasecki-box requested review from a team as code owners June 18, 2026 09:57

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (2)
src/elements/content-sidebar/hooks/useSidebarMetadataFetcher.ts (1)

74-74: ⚡ Quick win

Fix typoed flag names before they spread further.

isBoundindBoxEnabled / boundindBoxParams are misspelled. Renaming now will make callsites and future refactors safer.

Proposed rename
-    isBoundindBoxEnabled: boolean = false,
+    isBoundingBoxEnabled: boolean = false,
...
-    const isBoundingBoxOrConfidenceScoreReviewEnabled = isBoundindBoxEnabled || isConfidenceScoreEnabled;
+    const isBoundingBoxOrConfidenceScoreReviewEnabled = isBoundingBoxEnabled || isConfidenceScoreEnabled;
...
-    const boundindBoxParams = isBoundindBoxEnabled ? { include_reference: true } : {};
+    const boundingBoxParams = isBoundingBoxEnabled ? { include_reference: true } : {};
...
-                ...boundindBoxParams,
+                ...boundingBoxParams,

Also applies to: 244-244

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/elements/content-sidebar/hooks/useSidebarMetadataFetcher.ts` at line 74,
The parameter and variable names contain misspelled words where "Bounding" is
written as "Boundind". Rename the parameter `isBoundindBoxEnabled` to
`isBoundingBoxEnabled` at the function signature level, and also rename
`boundindBoxParams` to `boundingBoxParams` throughout the file (including at
line 244 and any other occurrences). Update all references and usages of these
renamed identifiers to maintain consistency across the codebase.
src/elements/content-sidebar/__tests__/useSidebarMetadataFetcher.test.tsx (1)

661-683: ⚡ Quick win

Add a confidence-only regression assertion for include_reference.

This suite now covers bbox-only mode well, but it should also assert that confidence-only mode does not send include_reference, so decoupling stays enforced.

Suggested test addition
+        test('should NOT include include_reference when only isConfidenceScoreEnabled is true', async () => {
+            mockAPI.extractStructured.mockResolvedValue({
+                answer: { field1: 'value1' },
+                created_at: '2026-03-27T08:10:14.106-07:00',
+                completion_reason: 'done',
+            });
+
+            const { result } = setupHook('123', true, false);
+            await result.current.extractSuggestions('templateKey', 'global');
+
+            expect(mockAPI.extractStructured).toHaveBeenCalledWith(
+                expect.not.objectContaining({
+                    include_reference: expect.anything(),
+                }),
+            );
+            expect(mockAPI.extractStructured).toHaveBeenCalledWith(
+                expect.objectContaining({
+                    include_confidence_score: true,
+                }),
+            );
+        });
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/elements/content-sidebar/__tests__/useSidebarMetadataFetcher.test.tsx`
around lines 661 - 683, The current test only verifies that include_reference is
sent when isBoundingBoxEnabled is true, but there is no regression test to
ensure that include_reference is NOT sent when only isConfidenceScoreEnabled is
true. Add a new test case following the same pattern as the existing test, but
configure setupHook to enable confidence score while disabling bounding box,
then assert that the extractStructured API call does not contain the
include_reference parameter to ensure these feature flags remain decoupled.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@package.json`:
- Line 141: The `@box/metadata-editor` package version has been updated to ^1.73.4
in devDependencies but the version in peerDependencies remains at the older
^1.69.6, creating a version mismatch. Update the `@box/metadata-editor` entry in
the peerDependencies section to match the devDependencies version of ^1.73.4 to
ensure consistency and prevent version conflicts for package consumers.

In `@src/api/Metadata.js`:
- Around line 467-471: The getInstances function parameter
isBoundingBoxOrConfidenceScoreReviewEnabled combines two independent feature
controls into a single boolean. Replace this merged parameter with two separate
boolean parameters: one for controlling bounding box serialization and one for
controlling confidence score review. Then update the serialization logic in the
lines around 1258-1287 (and the other referenced locations at lines 640-653) to
independently check these two separate flags instead of checking the single
merged flag, ensuring that confidence details and bounding box details can be
controlled and persisted independently.

In `@src/elements/content-sidebar/hooks/useSidebarMetadataFetcher.ts`:
- Around line 240-252: The boundindBoxParams object is conditioned on
isBoundingBoxOrConfidenceScoreReviewEnabled, which keeps the bounding box
feature coupled to the confidence score feature. Replace the condition
isBoundingBoxOrConfidenceScoreReviewEnabled with isBoundingBoxEnabled so that
bounding box references are fetched based solely on the bounding box flag,
allowing the two features to be independently controlled.

---

Nitpick comments:
In `@src/elements/content-sidebar/__tests__/useSidebarMetadataFetcher.test.tsx`:
- Around line 661-683: The current test only verifies that include_reference is
sent when isBoundingBoxEnabled is true, but there is no regression test to
ensure that include_reference is NOT sent when only isConfidenceScoreEnabled is
true. Add a new test case following the same pattern as the existing test, but
configure setupHook to enable confidence score while disabling bounding box,
then assert that the extractStructured API call does not contain the
include_reference parameter to ensure these feature flags remain decoupled.

In `@src/elements/content-sidebar/hooks/useSidebarMetadataFetcher.ts`:
- Line 74: The parameter and variable names contain misspelled words where
"Bounding" is written as "Boundind". Rename the parameter `isBoundindBoxEnabled`
to `isBoundingBoxEnabled` at the function signature level, and also rename
`boundindBoxParams` to `boundingBoxParams` throughout the file (including at
line 244 and any other occurrences). Update all references and usages of these
renamed identifiers to maintain consistency across the codebase.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 61358c6a-98f4-43e4-a448-511e4cc3d091

📥 Commits

Reviewing files that changed from the base of the PR and between 963422d and 0fb8c69.

⛔ Files ignored due to path filters (1)
  • yarn.lock is excluded by !**/yarn.lock, !**/*.lock
📒 Files selected for processing (17)
  • package.json
  • scripts/i18n.config.js
  • scripts/jest/jest.config.js
  • src/api/Metadata.js
  • src/api/__tests__/Metadata.test.js
  • src/api/__tests__/utils.test.js
  • src/api/utils.js
  • src/common/types/metadata.js
  • src/constants.js
  • src/elements/content-sidebar/MetadataInstanceEditor.tsx
  • src/elements/content-sidebar/MetadataSidebarRedesign.tsx
  • src/elements/content-sidebar/__tests__/MetadataInstanceEditor.test.tsx
  • src/elements/content-sidebar/__tests__/MetadataSidebarRedesign.test.tsx
  • src/elements/content-sidebar/__tests__/useMetadataSidebarUnsavedChangesGuard.test.ts
  • src/elements/content-sidebar/__tests__/useSidebarMetadataFetcher.test.tsx
  • src/elements/content-sidebar/hooks/useMetadataSidebarUnsavedChangesGuard.ts
  • src/elements/content-sidebar/hooks/useSidebarMetadataFetcher.ts

Comment thread package.json
Comment thread src/api/Metadata.js
Comment thread src/elements/content-sidebar/hooks/useSidebarMetadataFetcher.ts
Nefaris
Nefaris previously approved these changes Jun 18, 2026
Comment thread src/elements/content-sidebar/hooks/useSidebarMetadataFetcher.ts Outdated
Comment thread package.json

@jfox-box jfox-box left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@tjuanitas tjuanitas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approving for package.json and yarn.lock

@mergify mergify Bot added the queued label Jun 18, 2026
@mergify

mergify Bot commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Merge Queue Status

  • Entered queue2026-06-18 20:49 UTC · Rule: Automatic strict merge
  • Checks passed · in-place
  • Merged2026-06-18 21:02 UTC · at 6aeab1fbc4d97d651148e94013414c36ba80f1c2 · squash

This pull request spent 12 minutes 31 seconds in the queue, including 12 minutes 13 seconds running CI.

Required conditions to merge
  • #approved-reviews-by >= 1 [🛡 GitHub branch protection]
  • #review-threads-unresolved = 0 [🛡 GitHub branch protection]
  • github-review-decision = APPROVED [🛡 GitHub branch protection]
  • any of [🛡 GitHub branch protection]:
    • check-success = Summary
    • check-neutral = Summary
    • check-skipped = Summary
  • any of [🛡 GitHub branch protection]:
    • check-success = lint_test_build
    • check-neutral = lint_test_build
    • check-skipped = lint_test_build
  • any of [🛡 GitHub branch protection]:
    • check-success = license/cla
    • check-neutral = license/cla
    • check-skipped = license/cla
  • any of [🛡 GitHub branch protection]:
    • check-success = lint_pull_request
    • check-neutral = lint_pull_request
    • check-skipped = lint_pull_request

@mergify
mergify Bot merged commit 78fec65 into box:master Jun 18, 2026
10 of 11 checks passed
@mergify mergify Bot removed the queued label Jun 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants