Fix BatchProcessor missed wakeup race condition

[pranaysb]

Description

Fixes a severe threading race condition in BatchProcessor that wiped the wakeup signal if a batch filled immediately after a check but before the event was cleared. Moving _worker_awaken.clear() to the start of the wait() loop fixes this and eliminates up to 5-second latency spikes in export pipelines under heavy load.

Fixes #5400

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

• Tested with the local reproducer provided in the issue to verify the race condition vanishes and no spans are stuck in the queue.
• Ran uv run pytest opentelemetry-sdk/tests/shared_internal/test_batch_processor.py (All 17 tests passed).
• Formatted and verified with ruff check and ruff format.

Does This PR Require a Contrib Repo Change?

• Yes. - Link to PR:
• No.

Checklist:

• Followed the style guidelines of this project
• Changelogs have been updated
• Unit tests have been added
• Documentation has been updated

[DylanRussell]

Can you add a test case illustrating the race condition ? I don't follow why this change is necessary..

Take a look at _should_export_batch -- we will continue to call export as long as the batch size is exceeded (this is confusing because we have EXPORT_WHILE_BATCH_EXCEEDS_THRESHOLD strategy in the enum even though we always do that.. we could remove that value from the enum...)

[DylanRussell]

did you actually run into this scenario ? it seems incredibly unlikely to occur, to get a big batch immediately after the export loop exits but before clear() is called ?

If we move the clear call to after export like you are proposing we potentially introduce a new issue -- because emit will set the event if we are over the batch size, and then we will immediately wake up without sleeping, but we could still be in the export loop at that point, still draining the queue at that point... does that make sense ?

I think it is correct where it is placed now...