feat(services/s3): add conditional delete with if-match#7607
Conversation
a6dd63c to
3af96e5
Compare
Seems to be wrong. There is no DeleteObjects v2 and DeleteObjects did support setting etag. For example: POST /?delete HTTP/1.1
Host: Bucket.s3.amazonaws.com
x-amz-mfa: MFA
x-amz-request-payer: RequestPayer
x-amz-bypass-governance-retention: BypassGovernanceRetention
x-amz-expected-bucket-owner: ExpectedBucketOwner
x-amz-sdk-checksum-algorithm: ChecksumAlgorithm
<?xml version="1.0" encoding="UTF-8"?>
<Delete xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<Object>
<ETag>string</ETag>
<Key>string</Key>
<LastModifiedTime>timestamp</LastModifiedTime>
<Size>long</Size>
<VersionId>string</VersionId>
</Object>
...
<Quiet>boolean</Quiet>
</Delete>Please double check. |
|
yes, I confused it with
|
| OPENDAL_S3_SECRET_ACCESS_KEY=minioadmin | ||
| OPENDAL_S3_REGION=us-east-1 | ||
| OPENDAL_TEST_CAPABILITY_OVERRIDES=stat_with_version=false,read_with_version=false,delete_with_version=false,list_with_versions=false,list_with_deleted=false,write_can_append=false,copy_with_if_not_exists=false,copy_with_if_match=false | ||
| OPENDAL_TEST_CAPABILITY_OVERRIDES=stat_with_version=false,read_with_version=false,delete_with_version=false,list_with_versions=false,list_with_deleted=false,write_can_append=false,copy_with_if_not_exists=false,copy_with_if_match=false,delete_with_if_match=false |
There was a problem hiding this comment.
Please double check if minio support:
- copy_with_if_not_exists
- copy_with_if_match
- delete_with_if_match
There was a problem hiding this comment.
Confirmed: MinIO community edition does not support delete_with_if_match — DeleteObject silently ignores If-Match and returns 204 (minio/minio#21677, maintainers: "this is a feature, not a bug, something new AWS added recently"). So delete_with_if_match=false is required here and will stay required regardless of MinIO version.
The other two (copy_with_if_not_exists, copy_with_if_match) are pre-existing overrides from #7600 — this PR doesn't touch them. They may be removable on a newer MinIO image (the wildcard * and 412/404 fixes landed after our pinned RELEASE.2024-09-22)
Which issue does this PR close?
Closes #7090.
Rationale for this change
AWS S3 supports
If-MatchonDeleteObjectsince 2024 (docs). Expose it through OpenDAL so callers can do CAS-style conditional deletesWhat changes are included in this PR?
Core:
Capability::delete_with_if_match.if_match: Option<String>toOpDelete,DeleteOptions, and aFutureDelete::if_matchbuilder.correctness_check.rsS3:
delete_with_if_match: trueon the S3 capability.s3_delete_objectinjectsIf-Matchwhenargs.if_match()is set.s3_delete_objectspropagates per-objectif_matchinto the<ETag>field of theDeleteObjectsXML body, so the batch path stays a single request even with conditional deletes.CI:
delete_with_if_match=falsetoOPENDAL_TEST_CAPABILITY_OVERRIDESin the fourminio_s3*action.yml filesTests:
test_delete_with_if_match_match/_mismatchincore/tests/behavior/async_delete.rs.Bindings:
DeleteOptionsextended withif_match; Pythonpyistubs and async/syncdelete()signatures updated. .NET / Java / Haskell / Ruby / OCaml use..Default::default()and need no changes.Are there any user-facing changes?
Yes, additive only —
op.delete_with(path).if_match(etag)is a new opt-in API. Existingop.delete(path)is unchanged.AI Usage Statement
AI-assisted implementation.