feat: optimize disk buffering export frequency with configurable delay #1446
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -15,6 +15,7 @@ const val MAX_CACHE_FILE_SIZE: Int = 1024 * 1024 | |
| const val DEFAULT_MAX_FILE_AGE_FOR_WRITE_MS = 30L | ||
| const val DEFAULT_MIN_FILE_AGE_FOR_READ_MS = 33L | ||
| const val DEFAULT_MAX_FILE_AGE_FOR_READ_MS = 18L | ||
| const val DEFAULT_EXPORT_SCHEDULE_DELAY_MS: Long = 10000L | ||
|
|
||
| data class DiskBufferingConfig | ||
| @JvmOverloads | ||
|
|
@@ -31,6 +32,65 @@ data class DiskBufferingConfig | |
| * `null`, a default directory inside the application's cache directory will be used. | ||
| */ | ||
| val signalsBufferDir: File? = null, | ||
| /** | ||
| * The delay in milliseconds between consecutive export attempts. Defaults to 10 seconds (10000 ms). | ||
| * | ||
| * This value controls how frequently the SDK attempts to export buffered signals from disk. | ||
| * The configured value represents the minimum delay between export attempts. | ||
| * | ||
| * When [autoDetectExportSchedule] is true, this value is used as an override: | ||
| * - If explicitly set to a non-default value, it overrides auto-detection | ||
| * - If left at the default value, auto-detection will be used | ||
| * | ||
| * Trade-offs to consider: | ||
| * - Lower values (e.g., 10 seconds): More frequent exports mean fresher data in RUM sessions, | ||
| * but higher resource consumption (CPU, disk I/O, network activity) and potentially higher | ||
| * backend load from more frequent requests. | ||
| * - Higher values (e.g., 60 seconds or more): Reduced resource consumption and backend load, | ||
| * but longer delay before data becomes available in RUM sessions. | ||
| * | ||
| * Configuration recommendations: | ||
| * - 10000 ms (10 seconds): Default value, provides balance between data freshness and | ||
| * resource consumption for typical applications. | ||
| * - 5000 ms (5 seconds) or lower: For applications with critical real-time monitoring needs, | ||
| * where fresher data is worth the additional resource cost. | ||
| * - 30000 ms (30 seconds) or higher: For applications with high telemetry volume where reducing | ||
| * backend load and device resource consumption is prioritized over data freshness. | ||
| * | ||
| * Performance considerations: | ||
| * - Each export cycle may involve reading, serializing, and transmitting buffered signals. | ||
| * Higher export frequency means more frequent resource usage. | ||
| * - Lower export frequency means more data accumulates per export cycle, which could impact | ||
| * memory usage and delay in data availability. | ||
| * | ||
| * Minimum supported value: 1000 ms (1 second). Values less than this will be automatically | ||
| * increased to 1000 ms with a warning. | ||
| * | ||
| * Best practice: Test your configuration with your specific telemetry volume and workload | ||
| * to find the optimal balance for your use case. | ||
| */ | ||
| val exportScheduleDelayMillis: Long = DEFAULT_EXPORT_SCHEDULE_DELAY_MS, | ||
| /** | ||
| * Enables automatic detection of optimal export schedule based on device conditions. | ||
| * | ||
| * When enabled, the SDK will analyze device state and adjust export frequency accordingly: | ||
| * - On low battery or battery saver: Increase interval (less frequent exports) | ||
| * - Under memory pressure: Increase interval (reduce memory consumption) | ||
| * - Normal conditions: Use default or user-configured interval | ||
| * | ||
| * Auto-detection respects user configuration: | ||
| * - If [exportScheduleDelayMillis] is explicitly set to a non-default value, | ||
| * it will be used (auto-detection is overridden) | ||
| * - If left at default value, auto-detection suggestions are applied | ||
| * | ||
| * Default: false (preserves current behavior - no auto-detection) | ||
| * | ||
| * This feature is useful for: | ||
| * - Applications that need to adapt to device conditions | ||
| * - High-volume telemetry scenarios where battery life is critical | ||
| * - Scenarios where users want the best of both worlds: sensible defaults plus explicit control | ||
| */ | ||
| val autoDetectExportSchedule: Boolean = false, | ||
| ) { | ||
| companion object { | ||
| /** | ||
|
|
@@ -49,12 +109,34 @@ data class DiskBufferingConfig | |
| maxCacheFileSize: Int = MAX_CACHE_FILE_SIZE, | ||
| debugEnabled: Boolean = false, | ||
| signalsBufferDir: File? = null, | ||
| exportScheduleDelayMillis: Long = DEFAULT_EXPORT_SCHEDULE_DELAY_MS, | ||
| autoDetectExportSchedule: Boolean = false, | ||
| ): DiskBufferingConfig { | ||
| var minRead = minFileAgeForReadMillis | ||
| if (minFileAgeForReadMillis <= maxFileAgeForWriteMillis) { | ||
| minRead = maxFileAgeForWriteMillis + 5 | ||
| try { | ||
| Log.w(OTEL_RUM_LOG_TAG, "minFileAgeForReadMillis must be greater than maxFileAgeForWriteMillis") | ||
| Log.w( | ||
| OTEL_RUM_LOG_TAG, | ||
| "overriding minFileAgeForReadMillis from $minFileAgeForReadMillis to $minRead", | ||
| ) | ||
| } catch (e: RuntimeException) { | ||
| // Keep going, this is just a warning, and we might be running in a unit test | ||
| } | ||
|
Comment on lines
+118
to
+126
There was a problem hiding this comment. Wrapping logging statements in a try-catch for RuntimeException is unusual and may hide legitimate issues. If this is specifically for unit test environments where logging might fail, consider a more explicit check or documentation explaining why this pattern is necessary. |
||
| } | ||
| var validatedExportDelay = exportScheduleDelayMillis | ||
| if (exportScheduleDelayMillis < 1000L) { | ||
| validatedExportDelay = 1000L | ||
| try { | ||
| Log.w(OTEL_RUM_LOG_TAG, "exportScheduleDelayMillis must be at least 1000 ms (1 second)") | ||
| Log.w( | ||
| OTEL_RUM_LOG_TAG, | ||
| "overriding exportScheduleDelayMillis from $exportScheduleDelayMillis to $validatedExportDelay", | ||
| ) | ||
| } catch (e: RuntimeException) { | ||
| // Keep going, this is just a warning, and we might be running in a unit test | ||
| } | ||
| } | ||
| return DiskBufferingConfig( | ||
| enabled = enabled, | ||
|
|
@@ -65,6 +147,8 @@ data class DiskBufferingConfig | |
| maxCacheFileSize = maxCacheFileSize, | ||
| debugEnabled = debugEnabled, | ||
| signalsBufferDir = signalsBufferDir, | ||
| exportScheduleDelayMillis = validatedExportDelay, | ||
| autoDetectExportSchedule = autoDetectExportSchedule, | ||
| ) | ||
| } | ||
| } | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -15,14 +15,11 @@ import java.util.concurrent.TimeUnit | |
|
|
||
| class DefaultExportScheduler( | ||
| periodicWorkProvider: () -> PeriodicWork, | ||
| private val exportScheduleDelayMillis: Long = TimeUnit.SECONDS.toMillis(10), | ||
|
Contributor
There was a problem hiding this comment. @breedx-splk do we use
Contributor
There was a problem hiding this comment. It's only a "nice-to-have" if we expect java users to be calling it. |
||
| ) : PeriodicRunnable(periodicWorkProvider) { | ||
| @Volatile | ||
| private var isShutDown: Boolean = false | ||
|
|
||
| override fun onRun() { | ||
| val exporter = SignalFromDiskExporter.get() ?: return | ||
|
|
||
|
|
@@ -41,5 +38,5 @@ class DefaultExportScheduler( | |
|
|
||
| override fun shouldStopRunning(): Boolean = isShutDown || (SignalFromDiskExporter.get() == null) | ||
|
|
||
| override fun minimumDelayUntilNextRunInMillis(): Long = exportScheduleDelayMillis | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this comment is too long.