Search before asking
Paimon version
1.4.2 (also reproduced on 1.3.1)
Compute Engine
Flink 1.20 / Spark 3.5 — engine-independent: the failure happens inside
OSSFileIO (paimon-oss) request signing, before any engine-specific code path.
Minimal reproduce step
- Deploy an OSS-compatible endpoint whose hostname doesn't end in
.aliyuncs.com
and is not registered as a CNAME custom domain (e.g. a private-cloud/Apsara
Stack OSS deployment, endpoint like oss-cn-xxx.inter.envNNN.example.com)
- Configure a Paimon catalog with
fs.oss.endpoint pointing to that endpoint
- Attempt any OSS read/write operation (e.g.
INSERT INTO or table scan)
- Observe
SignatureDoesNotMatch errors on every request
What doesn't meet your expectations?
When connecting Paimon's OSSFileIO to a non-public-cloud OSS-compatible endpoint
(e.g. Apsara Stack private-cloud OSS) whose hostname does not match the standard
*.aliyuncs.com suffix, the aliyun-sdk-oss SDK's CNAME auto-detection heuristic
incorrectly treats the endpoint as a CNAME custom domain, and strips the bucket
name from the canonical host used for request signing. This causes every request
to fail with SignatureDoesNotMatch, because the server computes the signature
with the bucket name included while the client's signature omits it.
This is not a bug in the SDK itself — the heuristic is reasonable for genuine
public-cloud CNAME scenarios — but Paimon currently provides no way to disable it
for endpoints that are neither official public-cloud domains nor CNAME domains.
I expect Paimon to expose a configuration key to opt out of this heuristic.
Anything else?
Proposed fix: add a new configuration key fs.oss.cname.enabled (boolean). When
explicitly set to false, OSSFileIO.createFileSystem() disables CNAME
auto-detection by reflectively calling ClientConfiguration.setSupportCname(false)
on the underlying OSSClient — the same reflection pattern already used by
fs.oss.sld.enabled (PR #6413, #6413).
When the key is unset, behavior is unchanged (supportCname defaults to true),
so public-cloud users are unaffected. The change is ~20 lines in
paimon-filesystems/paimon-oss-impl OSSFileIO.
Are you willing to submit a PR?
Search before asking
Paimon version
1.4.2 (also reproduced on 1.3.1)
Compute Engine
Flink 1.20 / Spark 3.5 — engine-independent: the failure happens inside
OSSFileIO (paimon-oss) request signing, before any engine-specific code path.
Minimal reproduce step
.aliyuncs.comand is not registered as a CNAME custom domain (e.g. a private-cloud/Apsara
Stack OSS deployment, endpoint like
oss-cn-xxx.inter.envNNN.example.com)fs.oss.endpointpointing to that endpointINSERT INTOor table scan)SignatureDoesNotMatcherrors on every requestWhat doesn't meet your expectations?
When connecting Paimon's OSSFileIO to a non-public-cloud OSS-compatible endpoint
(e.g. Apsara Stack private-cloud OSS) whose hostname does not match the standard
*.aliyuncs.comsuffix, the aliyun-sdk-oss SDK's CNAME auto-detection heuristicincorrectly treats the endpoint as a CNAME custom domain, and strips the bucket
name from the canonical host used for request signing. This causes every request
to fail with
SignatureDoesNotMatch, because the server computes the signaturewith the bucket name included while the client's signature omits it.
This is not a bug in the SDK itself — the heuristic is reasonable for genuine
public-cloud CNAME scenarios — but Paimon currently provides no way to disable it
for endpoints that are neither official public-cloud domains nor CNAME domains.
I expect Paimon to expose a configuration key to opt out of this heuristic.
Anything else?
Proposed fix: add a new configuration key
fs.oss.cname.enabled(boolean). Whenexplicitly set to
false, OSSFileIO.createFileSystem() disables CNAMEauto-detection by reflectively calling
ClientConfiguration.setSupportCname(false)on the underlying OSSClient — the same reflection pattern already used by
fs.oss.sld.enabled(PR #6413, #6413).When the key is unset, behavior is unchanged (supportCname defaults to true),
so public-cloud users are unaffected. The change is ~20 lines in
paimon-filesystems/paimon-oss-impl OSSFileIO.
Are you willing to submit a PR?