Skip to content

Optimize reboots with Android emulator initialization#5280

Open
jardondiego wants to merge 21 commits into
masterfrom
optimize-android-reboots
Open

Optimize reboots with Android emulator initialization#5280
jardondiego wants to merge 21 commits into
masterfrom
optimize-android-reboots

Conversation

@jardondiego
Copy link
Copy Markdown
Collaborator

@jardondiego jardondiego commented May 16, 2026

Goal

Significantly improve the startup efficiency and fuzzing throughput of Android bots by eliminating redundant, time-consuming device reboots during the initialization and sanitizer setup phases.

Background

Historically, Android bots suffered from excessive hard reboots (~60s penalty each) during task execution:

  1. Unconditional Final Reboots: initialize_device() unconditionally triggered a hard reboot at the end of its sequence to ensure a "good state," even if setup_asan_if_needed() had just performed a clean shell restart seconds prior.
  2. Sanitizer Option Reboots: Setting ASan options (sanitizer.set_options) writes to the /system partition. The helper function adb.write_data_to_file() contained a hardcoded rule forcing a reboot anytime a file in /system was modified.

@jardondiego jardondiego marked this pull request as ready for review May 17, 2026 00:44
@jardondiego jardondiego requested a review from a team as a code owner May 17, 2026 00:44
Comment thread src/clusterfuzz/_internal/platforms/android/device.py
Comment thread src/clusterfuzz/_internal/platforms/android/device.py
Comment thread src/clusterfuzz/_internal/platforms/android/sanitizer.py Outdated
Copy link
Copy Markdown
Collaborator

@IvanBM18 IvanBM18 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, minus some NITs

@jardondiego jardondiego requested a review from letitz May 18, 2026 19:27
Copy link
Copy Markdown
Collaborator

@fernandofloresg fernandofloresg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, would be nice to add those docstrings

Comment thread src/clusterfuzz/_internal/platforms/android/device.py
Copy link
Copy Markdown
Collaborator

@letitz letitz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit confused about the main logic change in initialize_device(), see comment below.

Comment thread src/clusterfuzz/_internal/platforms/android/adb.py Outdated
Comment thread src/clusterfuzz/_internal/platforms/android/device.py Outdated
Comment thread src/clusterfuzz/_internal/platforms/android/device.py Outdated
Comment thread src/clusterfuzz/_internal/platforms/android/device.py Outdated
return

adb.write_data_to_file(sanitizer_options, sanitizer_options_file_path)
# Skip reboot as the app will pick up the options file on restart.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to make sure I understand, this means that the next time the app is started, it will read from the options file?

Copy link
Copy Markdown
Collaborator Author

@jardondiego jardondiego May 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. We write the ASan options to the system file. When the target fuzzer application is launched in a new process shortly after this, the new options file are read from disk during its initialization. A full device reboot isn't required for this step, so we can safely skip it to save time. Forcing a ~60s kernel reboot to update a text string that a process is going to read 2 seconds later is redundant.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, thanks. IIUC the logic in write_data_to_file(), that means we'll leave /system in read-write mode. Is that intentional?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. I don't think it would be a problem for emulated devices since they would be launched and destroyed in every task bit it might be an issue for non-virtual devices since a fuzzing job could break a system file and we would have to address that issue in a physical device, and that could mean having to physically check such device. We could either skip this optimization for virtual devices or manually trigger a read-only re-mounting. What do you suggest?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I applied Option A for now, but I'm all ears for any suggestions from you. Thanks!

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See other comment, I don't think option A was applied in the end.

In any case, I don't think it's a big deal to lose 1m per fuzzing session - they last 3 hours anyway. I would rather go for option B and avoid the risk of weird issues where /system files get corrupted. I agree with you that for emulated devices that should be safe, but I'm not 100% confident :)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think something weird happened in my workspace, I was moving around several branches and got confused. Agreed! I will adjust as suggested!

Comment thread src/clusterfuzz/_internal/tests/core/platforms/android/device_test.py Outdated
Comment thread src/clusterfuzz/_internal/tests/core/platforms/android/device_test.py Outdated
Comment thread src/clusterfuzz/_internal/tests/core/platforms/android/device_test.py Outdated
Comment thread src/clusterfuzz/_internal/tests/core/platforms/android/device_test.py Outdated
Comment thread src/clusterfuzz/_internal/tests/core/platforms/android/device_test.py Outdated
@jardondiego jardondiego changed the title Optimize Android emulator initialization by batching reboots Optimize reboots with Android emulator initialization May 20, 2026
@jardondiego jardondiego force-pushed the optimize-android-reboots branch from bff881e to 2888eab Compare May 21, 2026 02:09
@jardondiego jardondiego requested a review from letitz May 21, 2026 02:28
Copy link
Copy Markdown
Collaborator

@letitz letitz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One substantive comment around skipping reboots after writing ASan options, otherwise just two small nits.

Comment thread src/clusterfuzz/_internal/platforms/android/adb.py Outdated
return

adb.write_data_to_file(sanitizer_options, sanitizer_options_file_path)
# Skip reboot as the app will pick up the options file on restart.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, thanks. IIUC the logic in write_data_to_file(), that means we'll leave /system in read-write mode. Is that intentional?

Comment thread src/clusterfuzz/_internal/tests/core/platforms/android/device_test.py Outdated
Diego Jardon added 12 commits May 21, 2026 18:33
This change reduces the number of reboots during Android device setup by:
- Adding a wait_for_reboot parameter to adb.write_data_to_file.
- Tracking reboot status in initialize_device to skip the final reboot if one already occurred.
- Disabling reboots when setting sanitizer options since the app restart is sufficient.

These optimizations improve bot startup efficiency and overall fuzzing throughput.
Adds `-> bool` return type hints and explanatory docstrings to:
- `configure_system_build_properties`
- `setup_asan_if_needed`

These updates clarify that the returned boolean indicates whether a device reboot occurred during the setup step.
- Revert configure_system_build_properties to not return a bool.
- Simplify initialize_device to only reboot if ASan setup did not reboot the device. This fixes the bug where it would reboot even if no properties changed.
- Refactor device_test.py to use helpers.patch instead of manual mock setups, matching the codebase style.
@jardondiego jardondiego force-pushed the optimize-android-reboots branch from 2888eab to 2c08a43 Compare May 21, 2026 18:34
Diego Jardon and others added 5 commits May 21, 2026 18:36
This restricts the should_reboot=False optimization to emulators only. Physical devices will always fall back to a hard reboot after modifying a system file to guarantee the /system partition is safely locked back to read-only.
Removed redundant 'helpers' import in AddTestAccountsIfNeededTest.setUp and fixed spacing.
Copy link
Copy Markdown
Collaborator

@letitz letitz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM % option b and actually fixing the docstring for write_data_to_file().

Comment thread src/clusterfuzz/_internal/platforms/android/sanitizer.py
return

adb.write_data_to_file(sanitizer_options, sanitizer_options_file_path)
# Skip reboot as the app will pick up the options file on restart.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See other comment, I don't think option A was applied in the end.

In any case, I don't think it's a big deal to lose 1m per fuzzing session - they last 3 hours anyway. I would rather go for option B and avoid the risk of weird issues where /system files get corrupted. I agree with you that for emulated devices that should be safe, but I'm not 100% confident :)

Diego Jardon added 2 commits May 22, 2026 14:59
Previously, the initialization sequence forced physical Android devices to perform a full kernel reboot after modifying the /system partition (e.g., when updating asan.options) to guarantee the partition was safely locked back to read-only.

This commit transitions from that conservative approach (Option A) to a shell-based fallback (Option B). Now, when the 'should_reboot=False' optimization is requested, both emulators AND physical devices skip the 60+ second reboot penalty. To maintain security and prevent accidental filesystem corruption from rogue fuzzers, the script explicitly locks the partition back down using the 'mount -o ro,remount /system' shell command.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants