When using streaming response mode, exceptions do not include the message text from the HTTP body

[glennschmidt]

Describe the bug

When making a request which uses ['@http' => ['stream' => true]], if the server returns a HTTP error status such as 400, the SDK is unable to read the response body for inclusion in the exception message, and consequently there's no way to see the reason for the error.

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

The PHP exception should include the error message from the HTTP body, as in non-streaming mode.

Fatal error: Uncaught exception 'Aws\BedrockRuntime\Exception\BedrockRuntimeException' with message 'Error executing "ConverseStream" on "https://bedrock-runtime.us-west-2.amazonaws.com/model/x/converse-stream"; AWS HTTP error: Client error: `POST https://bedrock-runtime.us-west-2.amazonaws.com/model/x/converse-stream` resulted in a `400 Bad Request` response:
{"message":"The provided model identifier is invalid."}

Current Behavior

The PHP exception says Unable to parse error information from response.

Fatal error: Uncaught exception 'Aws\BedrockRuntime\Exception\BedrockRuntimeException' with message 'Error executing "ConverseStream" on "https://bedrock-runtime.us-west-2.amazonaws.com/model/x/converse-stream"; AWS HTTP error: Client error: `POST https://bedrock-runtime.us-west-2.amazonaws.com/model/x/converse-stream` resulted in a `400 Bad Request` response Unable to parse error information from response - Error parsing JSON: Syntax error'

Reproduction Steps

Make a streaming API call, with some invalid parameters so that it would cause a 400 Bad Request response.

$client = new Aws\BedrockRuntime\BedrockRuntimeClient([
    'version' => '2023-09-30',
    'region' => 'us-west-2',
]);

$client->converseStream([
    'modelId' => 'x',
    '@http' => [
        'stream' => true,
    ],
]);

Possible Solution

This could be because the response body stream is non-seekable, and some other part of the SDK has already read it.

Additional Information/Context

No response

SDK version used

3.343.14

Environment details (Version of PHP (php -v)? OS name and version, etc.)

PHP 8.1.31, Linux

[yenfryherrerafeliz]

Hi @glennschmidt, thanks for opening this issue. As you mentioned in your comment, that is exactly the issue. There are different layers where the body is read and the body became not readable or empty after the first pass. We will evaluate a fix soon for this.

Thank you!

[glennschmidt]

I'm pretty sure this also means the SDK retry middleware is unable to detect throttle responses, which are likely to occur when dealing with LLMs.

So this is a pretty significant issue, because it requires us to choose between:

1. Giving chat users a bad experience (no streaming), or
2. Not being able to handle model limits and transient issues properly, and not being able to log meaningful error messages

[stobrien89]

Hi @glennschmidt,

#3214 has been merged, which should address this. Please let us know if you're still having issues after upgrading to the most current SDK version after it's been released today.

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.