[AMORO-4034][Improvement]: MaintainerMetrics Unified interface and Report #4045
[AMORO-4034][Improvement]: MaintainerMetrics Unified interface and Report #4045czy006 wants to merge 2 commits intoapache:masterfrom
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #4045 +/- ##
============================================
+ Coverage 29.93% 30.03% +0.10%
- Complexity 4229 4283 +54
============================================
Files 675 679 +4
Lines 53990 55072 +1082
Branches 6838 6981 +143
============================================
+ Hits 16161 16541 +380
- Misses 36636 37312 +676
- Partials 1193 1219 +26
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
a4468f3 to
0cfa9a4
Compare
majin1102
left a comment
There was a problem hiding this comment.
Thanks for working on this. Left some comments
d69deba to
c63f243
Compare
|
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@amoro.apache.org list. Thank you for your contributions. |
1853680 to
3eb6d57
Compare
0b7ec75 to
d9da077
Compare
|
@j1wonpark Can you help me review code ? This is an improvement to the monitoring abstraction of a multi-lake format. |
| } | ||
|
|
||
| private void expireSnapshots(long olderThan, int minCount, Set<String> exclude) { | ||
| // This method is kept for backward compatibility with potential subclasses |
There was a problem hiding this comment.
This private method is no longer called after the refactor (2-param expireSnapshots now calls expireSnapshotsInternal directly). The "backward compatibility with subclasses" comment doesn't apply since it's private. Can be removed.
| private static class OperationDurationGauge implements Gauge<Long> { | ||
| private volatile long value = 0L; | ||
| private volatile long startTime = 0L; | ||
|
|
There was a problem hiding this comment.
startTime is written in recordStart() but never read — getValue() only returns value. Looks like a leftover from an earlier design. Can be removed along with recordStart().
|
Also left a design suggestion about this in a separate comment below. |
| private void expireSnapshotsInternal( | ||
| long mustOlderThan, int minCount, MaintainerMetrics metrics) { | ||
| long startTime = System.currentTimeMillis(); | ||
| Set<String> exclude = expireSnapshotNeedToExcludeFiles(); |
There was a problem hiding this comment.
Duration is recorded twice for the same operation — once by MaintainerOperationExecutor (table_maintainer_operation_duration_millis) and once inside this method (table_snapshots_expiration_duration_millis). The two values will differ slightly. Consider using a single source of truth for duration.
Same pattern applies to doExpireData and autoCreateTags.
|
Thanks for the contribution! I left a few inline comments on dead code and duration recording. One design suggestion: the tag creation logic from |
5bb09bd to
b642e1d
Compare
[Improvement]: MaintainerMetrics Unified interface and Report (#4034)
Why are the changes needed?
Close #4034 .
Brief change log
How was this patch tested?
Add some test cases that check the changes thoroughly including negative and positive cases if possible
Add screenshots for manual tests if appropriate
Run test locally before making a pull request
Documentation