feat(util-genai): add multimodal upload usage recorder hooks#251
feat(util-genai): add multimodal upload usage recorder hooks#251xiongyunn wants to merge 2 commits into
Conversation
ralf0131
left a comment
There was a problem hiding this comment.
Overall Assessment
Adds an optional MultimodalUsageRecorder hook to FsUploader for recording upload success/error terminal events, with a safe no-op default. Clean, backward-compatible, additive design — no existing behavior changes. Code looks good overall.
Severity Breakdown
🔴 Critical: 0 | 🟠 High: 0 | 🟡 Medium: 1 | 🔵 Low: 0 | ℹ️ Info: 1 | ✅ Praise: 1
Action Required
| Severity | Finding | Location |
|---|---|---|
| 🟡 Medium | Recorder calls are not exception-guarded; a RuntimeError raised by a custom recorder would be caught by the upload retry clause, triggering an unnecessary retry of an already-succeeded upload |
fs_uploader.py:462–464 |
| ℹ️ Info | raise_on_success test-fixture field is defined but never exercised by any test |
test_usage_recorder.py:34 |
Details
[Medium] Guard recorder calls against exceptions — fs_uploader.py:462
In _do_upload, record_success() is called on the success path right before return, inside the while loop whose except (OSError, IOError, RuntimeError) clause (line 464) handles storage retries. If a custom recorder raises RuntimeError from record_upload_success(), the exception is caught by this clause and the upload is retried — re-writing a file that already succeeded. The terminal_recorded flag (set at line 399 before the recorder call) correctly prevents double-counting, so this is a robustness/efficiency concern rather than a correctness bug. The default no-op recorder is safe, but a downstream metrics recorder could misbehave under resource pressure. Consider wrapping the recorder invocations (record_success, record_error, and the direct recorder.record_upload_error(...) calls in upload()) in try/except Exception with a debug-level log, so a faulty recorder cannot disrupt the upload flow — consistent with the repo's "minimal overhead / must not disrupt operation" review focus.
[Info] Untested raise_on_success field — test_usage_recorder.py:34
RecordingUsageRecorder.raise_on_success toggles a RuntimeError in record_upload_success, but no test sets it to True. Either add a test exercising the raise path (e.g. verifying the uploader stays stable when the recorder raises) or drop the field to keep the fixture minimal.
Positive
Excellent work on this contribution! A few things worth calling out:
- The
terminal_recordedclosure is a clean, correct way to guarantee exactly one terminal event per task — no double-counting of success/error. - Test coverage is thorough: success, cache-hit no-op, queue-full, invalid-item, download-failed, storage-error-after-retry, and remote-URI paths are all covered.
provider_label_from_protocolkeeps the provider label low-cardinality (oss/sls/other) — good for metrics hygiene.- The no-op default means zero behavior change for existing users — fully backward compatible.
CI Note
CI workflows were in action_required state (first-time contributor — workflows require maintainer approval to run). I've approved the pending workflow runs so the Lint/Test/Misc checks can execute. I'll re-check CI status on the next patrol and approve once it's green. Reviewing now as COMMENT; will APPROVE after CI passes.
Automated review by github-manager-bot
Description
Add a lightweight
MultimodalUsageRecorderinterface and default no-op implementation toopentelemetry-util-genai, and call it fromFsUploaderon terminal upload success/error.Callers can register a custom recorder via
set_multimodal_usage_recorder()to record upload success (provider + payload bytes) and low-cardinality failure reasons. The default recorder is no-op, so behavior is unchanged when nothing is registered.Fixes # (issue)
Type of change
Please delete options that are not relevant.
How Has This Been Tested?
tox -e py312-test-util-genaiutil/opentelemetry-util-genai/tests/_multimodal_upload/test_usage_recorder.pyutil/opentelemetry-util-genai/tests/_multimodal_upload/test_fs_uploader_usage_metrics.pyDoes This PR Require a Core Repo Change?
Checklist:
See contributing.md for styleguide, changelog guidelines, and more.