Improve FFI multithread performance by using atomics#482
Open
oskirby wants to merge 25 commits into
Open
Conversation
I think there's another race hiding in will_accept that goes something like this: A: Receives packet N A: Checks counter is good A: Interrupted B: Receives packet N+N_BITS B: Checks counter is good B: Marks bitmap as received A: Resumes and find bitmap marked by N+N_BITS This is a pretty minor race as the packet would be rejected anyways but we return WireGuardError::DuplicateCounter instead of WireGuardError::InvalidCounter and that causes the unit test to fail.
Collaborator
|
I appreciate the effort here and this sounds like a solid improvement. Unfortunately, I'm unlikely to have the time to give such a large and intrusive change the review attention it needs in order to be merged, anytime in the foreseeable future. This crate is kind of just in bare-minimum maintenance mode at present 😢 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I've been trying to make use of the boringtun FFI interface in a multithreaded environment, and I have been finding that the use of a
Mutexaround the tunnel forces all threads to sequentially access the tunnel structure for every packet encryption and decryption operation.I find this to be suboptimal given that 99% of the encrypt/decrypt operations don't need to modify the session state at all, and this could be much better done using a
RwLockinstead. This allows multithreaded access for most packet operations while still acquiring an exclusive lock whenever we need to handle a handshake packet.To get there, we need to ensure that the odds and ends which do modify counters can still be done race-free, and most of this is simply a matter of using an atomic type instead of a naked counter.
A high level summary of the changes introduced by this PR:
try_encapsulate()andtry_decapsulate()methods that attempt to handle packets without modifying the session state.RwLockinstead of aMutexTimersandRateLimitermodules to useAtomicU64instead ofDurationReceivingKeyCounterValidatorto use atomic types and a spinlock.ReceivingKeyCounterValidatoris correct in multithreaded environments.