Skip to content

Implement updated standard retry behavior - #733

Open
Alan4506 wants to merge 12 commits into
smithy-lang:developfrom
Alan4506:retries-2.1-sep-update-v2
Open

Implement updated standard retry behavior#733
Alan4506 wants to merge 12 commits into
smithy-lang:developfrom
Alan4506:retries-2.1-sep-update-v2

Conversation

@Alan4506

@Alan4506 Alan4506 commented Jul 2, 2026

Copy link
Copy Markdown
Collaborator

Description of changes:

Updates the standard retry behavior:

  • Throttling backoff: throttling errors use a larger base backoff (1000ms) than other errors (50ms), based on error.is_throttling_error.
  • Retry quota: RETRY_COST is now 14, throttling retries cost 5, and the timeout-specific cost is removed.
  • DynamoDB defaults: DynamoDB / DynamoDB Streams clients default to 4 max attempts and a 25ms base backoff, applied per-value (a value configured by the customer always wins). The generic runtime stays AWS-agnostic; the DynamoDB knowledge lives in an AWS codegen integration.
  • Long-polling: long-polling operations back off before giving up even when the quota is exhausted. Detection checks the smithy.api#longPoll trait at runtime through APIOperation.long_polling.
  • x-amz-retry-after: honored when returned by the service, capped at 5 seconds above the normal backoff; invalid/missing values fall back to normal backoff and the standard HTTP Retry-After is ignored.

Note on long-polling detection:

Some service models don't apply the smithy.api#longPoll trait yet. We deliberately do not hard-code those operations. Doing this would require codegen interface changes for temporary behavior. The trait lookup is the long-term design regardless. Until those service models apply the trait, those operations won't get long-polling backoff.

Testing:

Added unit tests for each change (throttling vs non-throttling backoff, quota costs, the quota-exhaustion backoff contract, and x-amz-retry-after parsing/clamping/wiring). Also updated affected functional tests. All Python checks pass (make check-py, make test-py) and the Java codegen builds successfully.

Follow-up:

The awsJson protocol's _create_error is not on develop yet (it lives on the json-rpc branch). Once that merges, it needs the same parse_retry_after(response) wiring that awsQuery already has, so x-amz-retry-after is honored for awsJson services (e.g. DynamoDB) too.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@Alan4506
Alan4506 requested a review from a team as a code owner July 2, 2026 16:07
Comment thread packages/smithy-core/src/smithy_core/aio/retries.py Outdated
Comment thread packages/smithy-core/src/smithy_core/aio/retries.py Outdated
Comment thread packages/smithy-http/src/smithy_http/aio/protocols.py Outdated
@Alan4506
Alan4506 requested a review from a team as a code owner July 14, 2026 20:02
@Alan4506
Alan4506 force-pushed the retries-2.1-sep-update-v2 branch from e5e0c7d to 18d19e6 Compare July 14, 2026 21:19
@Alan4506
Alan4506 force-pushed the retries-2.1-sep-update-v2 branch from 18d19e6 to a900e8c Compare July 14, 2026 21:22
@Alan4506
Alan4506 force-pushed the retries-2.1-sep-update-v2 branch from 51e469d to 7b6c346 Compare July 16, 2026 15:32

@arandito arandito left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Alan4506! I left a couple comments. My main concerns related to how we are detecting long polling operations and simplifying the StandardRetryStrategy constructor. Let me know if you have any questions!

@@ -0,0 +1,4 @@
{
"type": "feature",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this need to be a minor bump?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think enhancement is more appropriate. Addressed in my latest commit. Let me know if you think otherwise!

Comment thread packages/smithy-http/src/smithy_http/aio/protocols.py
Comment thread packages/smithy-core/src/smithy_core/aio/retries.py Outdated
Comment thread packages/smithy-core/src/smithy_core/aio/retries.py Outdated
Comment thread packages/smithy-core/tests/unit/aio/test_retries.py
Comment thread packages/smithy-core/tests/unit/aio/test_retries.py
Comment thread packages/smithy-core/tests/unit/aio/test_retries.py
transport=config.transport
)
call = $4T(
${?isLongPolling}operation_context = $4T({"config": config})

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Building on Sam's comment, I still think we can improve how we pass this information into the ClientCall.

The trait should be detectable from APIOperation without needing to code generate any context. We already have precedence for operation trait detection for the idempotency token trait plus streaming traits.

We also already have a reference to an APIOperation object inside of RequestPipeline._retry through call.operation.

We can do something like this:

except RetryError as retry_error:
    if call.operation.long_polling and retry_error.retry_after is not None:
        await sleep(retry_error.retry_after)
    raise output_context.response

I think we may need to unconditionally set retry_error.retry_after for all quota exhaustion errors to avoid passing in the operation into refresh_retry_token_for_retry. Since we are already adding retry_after to the RetryError contract, I think that's a reasonable tradeoff.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call! I agree that this aligns with our existing runtime design, so I've addressed this in my latest commit. And I've removed the temporary hard-coded fallback for services that don't apply the trait, since it would introduce codegen interface changes for temporary behavior.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants