fix(azure blob sink): Fix Shared Key auth for multipart uploads

[ArunPiduguDD]

Summary

Fix Azure Blob Storage uploads that use an account-key connection string and are large enough to be sent in multiple parts.

When Vector is configured with an account-key connection string, it uses Shared Key authorization. Shared Key authorization constructs a signature from several parts of the request, including its Content-Length.

For uploads of 4 MiB or less, the Azure SDK sends a single request and provides its content-length before Vector constructs the signature. Larger uploads are divided into blocks. After uploading those blocks, the SDK sends a final Put Block List request containing an XML body that identifies the blocks belonging to a single upload.

Previously, this final block-list request was signed before the HTTP transport determined the XML body's content-length. Vector therefore signed an empty content-length field, while Azure validated the signature using the body length it received. Because the two signatures were based on different values, Azure rejected the request with 403 AuthorizationFailure.

This change makes Vector's Shared Key policy add an explicit content-length header from the known request body length when the SDK has not already provided one. Vector then constructs the signature from that header, and the HTTP transport sends the same header to Azure. The policy preserves an explicit nonzero header supplied by the SDK and represents a zero-length body as an empty field in the signature, as required by Azure's Shared Key rules.

Example of the affected request flow before this change:

PUT blob?comp=block&blockid=first   -> 201
PUT blob?comp=block&blockid=second  -> 201
PUT blob?comp=blocklist             -> 403 AuthorizationFailure

After this change:

PUT blob?comp=block&blockid=first   -> 201
PUT blob?comp=block&blockid=second  -> 201
PUT blob?comp=blocklist             -> 201

Vector configuration

No configuration changes are needed. This affects existing Azure Blob sink configurations that use a connection string with an account key, for example:

sinks:
  archive:
    type: azure_blob
    container_name: logs
    connection_string: >-
      DefaultEndpointsProtocol=https;
      AccountName=exampleaccount;
      AccountKey=<account-key>;
      EndpointSuffix=core.windows.net

SAS URL and Microsoft Entra ID authentication are not affected because they do not use this per-request Shared Key signature.

How did you test this PR?

• Added unit tests for Shared Key content-length handling:
  • add and sign the known body length when the header is missing;
  • preserve an existing nonzero header;
  • send Content-Length: 0 while representing it as an empty canonical field.
• Added Azurite integration coverage on both sides of the SDK's 4 MiB multipart boundary:
  • a 3 MiB one-shot upload succeeds;
  • a 5 MiB multipart upload succeeds using Shared Key authentication.
• Confirmed the new 5 MiB test fails before the fix: Azurite accepts both block uploads and rejects ?comp=blocklist with 403 AuthorizationFailure.
• Confirmed the fixed run accepts both block uploads and the final ?comp=blocklist request with 201.
• Ran:

cargo vdev int test --retries 0 azure 3.35.0 \
  azure_blob_uploads_one_shot_with_shared_key \
  azure_blob_uploads_multipart_with_shared_key

cargo test --lib --no-default-features --features sinks-azure_blob \
  shared_key_policy::tests

cargo clippy --lib --no-default-features --features sinks-azure_blob -- -D warnings
cargo vdev check changelog-fragments
cargo fmt

Is this a breaking change?

• Yes
• No

Does this PR include user facing changes?

• Yes. Added changelog.d/azure_blob_shared_key_multipart.fix.md.
• No. A maintainer will apply the no-changelog label to this PR.

References

• Azure Shared Key signing format: https://learn.microsoft.com/rest/api/storageservices/authorize-with-shared-key
• Azure Put Block List request: https://learn.microsoft.com/rest/api/storageservices/put-block-list

[datadog-vectordotdev]

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: e849eb1 | Docs | Datadog PR Page | Give us feedback!}