HDDS-15958. Return InvalidURI for unreadable S3 object keys on GetObject#10860
HDDS-15958. Return InvalidURI for unreadable S3 object keys on GetObject#10860Gargi-jais11 wants to merge 1 commit into
Conversation
ayushtkn
left a comment
There was a problem hiding this comment.
I see that this logic was likely added based on the AWS S3 docs mentioning 'Non-printable ASCII characters (128–255 decimal characters)' under the 'Characters to avoid' section.
However, there are two issues with enforcing it this way:
AWS S3 doesn't actually block these: The AWS docs list this under 'Characters to avoid' as a recommendation for clients to prevent application-side parsing issues.
AWS S3 itself still accepts valid UTF-8 characters that resolve to this range; it does not throw an InvalidURI error.
This blocks valid international characters: Because I believe keyPath is a Java String (UTF-16), checking c >= 0x80 && c <= 0xFF targets the Unicode Latin-1 Supplement block.
This will hard-reject perfectly valid, printable characters like é (U+00E9 / 233), ñ (U+00F1 / 241), or £ (U+00A3 / 163).
A user uploading a file named café.txt or piñata.jpg will get an unexpected InvalidURI error.
The Ceph test_object_read_unreadable test is likely sending raw, unencoded invalid bytes over the wire to trigger a parsing failure. We should rely on the \uFFFD check to catch malformed UTF-8, and if we want to block actual control characters, we should use Character.isISOControl(c) instead of blocking the entire 128-255 block.
https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-keys.html#object-key-guidelines
Or Am I missing somethign here?
What changes were proposed in this pull request?
S3 Gateway returns
404 NoSuchKeyfor GetObject requests with unreadable/invalid object keys, instead of400 InvalidURIas AWS S3 does.Ozone already defines the correct error in S3ErrorTable.INVALID_URI but never uses it:
INVALID_URI("InvalidURI", "Couldn't parse the specified URI.", HTTP_BAD_REQUEST)The key \xae\x8a- contains non-printable high bytes (\x80–\xff) that Ozone cannot store and that AWS treats as an invalid URI. S3G should reject it before OM lookup.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-15958
How was this patch tested?
Add Unit and Integration Tests.