Skip to content

[Bug] OSSFileIO CNAME auto-detection breaks signature for non-public-cloud OSS endpoints #8852

Description

@fwonce

Search before asking

  • I searched in the issues and found nothing similar.

Paimon version

1.4.2 (also reproduced on 1.3.1)

Compute Engine

Flink 1.20 / Spark 3.5 — engine-independent: the failure happens inside
OSSFileIO (paimon-oss) request signing, before any engine-specific code path.

Minimal reproduce step

  1. Deploy an OSS-compatible endpoint whose hostname doesn't end in .aliyuncs.com
    and is not registered as a CNAME custom domain (e.g. a private-cloud/Apsara
    Stack OSS deployment, endpoint like oss-cn-xxx.inter.envNNN.example.com)
  2. Configure a Paimon catalog with fs.oss.endpoint pointing to that endpoint
  3. Attempt any OSS read/write operation (e.g. INSERT INTO or table scan)
  4. Observe SignatureDoesNotMatch errors on every request

What doesn't meet your expectations?

When connecting Paimon's OSSFileIO to a non-public-cloud OSS-compatible endpoint
(e.g. Apsara Stack private-cloud OSS) whose hostname does not match the standard
*.aliyuncs.com suffix, the aliyun-sdk-oss SDK's CNAME auto-detection heuristic
incorrectly treats the endpoint as a CNAME custom domain, and strips the bucket
name from the canonical host used for request signing. This causes every request
to fail with SignatureDoesNotMatch, because the server computes the signature
with the bucket name included while the client's signature omits it.

This is not a bug in the SDK itself — the heuristic is reasonable for genuine
public-cloud CNAME scenarios — but Paimon currently provides no way to disable it
for endpoints that are neither official public-cloud domains nor CNAME domains.

I expect Paimon to expose a configuration key to opt out of this heuristic.

Anything else?

Proposed fix: add a new configuration key fs.oss.cname.enabled (boolean). When
explicitly set to false, OSSFileIO.createFileSystem() disables CNAME
auto-detection by reflectively calling ClientConfiguration.setSupportCname(false)
on the underlying OSSClient — the same reflection pattern already used by
fs.oss.sld.enabled (PR #6413, #6413).
When the key is unset, behavior is unchanged (supportCname defaults to true),
so public-cloud users are unaffected. The change is ~20 lines in
paimon-filesystems/paimon-oss-impl OSSFileIO.

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions