Skip to content

Add checksum and clone template to S3 Anchor#22819

Open
JasMehta08 wants to merge 2 commits into
root-project:masterfrom
JasMehta08:s3-anchor-checksum-clone-template
Open

Add checksum and clone template to S3 Anchor#22819
JasMehta08 wants to merge 2 commits into
root-project:masterfrom
JasMehta08:s3-anchor-checksum-clone-template

Conversation

@JasMehta08

Copy link
Copy Markdown
Contributor

This Pull request:

(Is a part of the GSoC 2026 project S3 Backend for RNTuple.)

Adds checksum and a configurable clone path to the S3 Anchor.

Changes or fixes:

  • Add an XXH3 64-bit checksum to the S3 anchor, computed over a canonical compact JSON representation of all data fields. Verified on read.
  • Add fCloneTemplate field to the anchor with ${baseurl} and ${name} placeholders, replacing the hardcoded /_clone/ path in CloneAsHidden. Defaults to ${baseurl}/_clone/${name}.
  • Add unit tests for checksum round-trip, tamper detection, missing checksum rejection, and wrong checksum type. Add unit tests for clone template round-trip, default value when absent, equality, and a wire test verifying custom template expansion in CloneAsHidden.

Checklist:

  • tested changes locally

Comment thread tree/ntuple/inc/ROOT/RPageStorageS3.hxx Outdated
Comment thread tree/ntuple/inc/ROOT/RPageStorageS3.hxx Outdated
@JasMehta08
JasMehta08 force-pushed the s3-anchor-checksum-clone-template branch from 1373142 to 5f539d7 Compare July 15, 2026 12:56
std::string fUrlTemplate = "${baseurl}/${objid}";
/// Pattern for resolving clone (attribute-set) names to base URLs.
/// ${baseurl} is replaced with the anchor URL, ${name} with the clone name.
std::string fCloneTemplate = "${baseurl}/_clone/${name}";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since you removed SetCloneTemplate, what's the point of this field? Isn't this always going to be set to its default value?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just noticed that these fields are all public...we should probably mirror ROOT::RNTuple and have them private with only the getters (and have the anchor itself should be a class)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once the fields are private, fCloneTemplate should probably be removed and replaced by a cpp-file-local constant

@JasMehta08 JasMehta08 Jul 15, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then having the template in the anchor does not really have a point. but @jblomer had asked for the addition of it into the anchor, in the #22653.

But it does make sense to have it be a file-local constant, as we dont have a use for the fCloneTemplate.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For making the fields private with getters, I will add it as a separate commit, or as a separate PR whichever you would prefer.


TEST(RNTupleAnchorS3, CloneTemplateDefault)
{
EXPECT_EQ("${baseurl}/_clone/${name}", RNTupleAnchorS3().fCloneTemplate);

@silverweed silverweed Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see the point of this test

RNTupleAnchorS3 a, b;
EXPECT_EQ(a, b);

b.fCloneTemplate = "${baseurl}/other/${name}";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see the point of this test

@JasMehta08
JasMehta08 requested a review from silverweed July 15, 2026 13:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants