Skip to content

Encrypting IO as a DelegateFileIO#14876

Open
smaheshwar-pltr wants to merge 1 commit intoapache:mainfrom
smaheshwar-pltr:sm/bulk-encryption-deletion
Open

Encrypting IO as a DelegateFileIO#14876
smaheshwar-pltr wants to merge 1 commit intoapache:mainfrom
smaheshwar-pltr:sm/bulk-encryption-deletion

Conversation

@smaheshwar-pltr
Copy link
Copy Markdown
Contributor

A table having encryption should not discount it from it supporting bulk operations (e.g.

private void deleteBulk(SupportsBulkOperations io, List<String> paths) {
try {
io.deleteFiles(paths);
LOG.info("Deleted {} files using bulk deletes", paths.size());
} catch (BulkDeletionFailureException e) {
int deletedFilesCount = paths.size() - e.numberFailedObjects();
LOG.warn(
"Deleted only {} of {} files using bulk deletes", deletedFilesCount, paths.size(), e);
}
}
private void deleteNonBulk(List<String> paths) {
Tasks.Builder<String> deleteTasks =
Tasks.foreach(paths)
.noRetry()
.executeWith(deleteExecutorService)
.suppressFailureWhenFinished()
.onFailure((file, exc) -> LOG.warn("Failed to delete file: {}", file, exc));
if (deleteFunc == null) {
LOG.info(
"Table IO {} does not support bulk operations. Using non-bulk deletes.",
table.io().getClass().getName());
deleteTasks.run(table.io()::deleteFile);
} else {
LOG.info("Custom delete function provided. Using non-bulk deletes");
deleteTasks.run(deleteFunc::accept);
}
}
) if the underlying IO supports it. But currently EncryptingFileIO#combine will never result in a FIleIO that SupportsBulkOperations.

@github-actions github-actions Bot added the API label Dec 17, 2025
@github-actions
Copy link
Copy Markdown

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.

@github-actions github-actions Bot added the stale label Jan 17, 2026
@smaheshwar-pltr
Copy link
Copy Markdown
Contributor Author

Bump to remove staleness

@github-actions github-actions Bot removed the stale label Jan 18, 2026
@github-actions
Copy link
Copy Markdown

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.

@github-actions github-actions Bot added the stale label Feb 17, 2026
@smaheshwar-pltr
Copy link
Copy Markdown
Contributor Author

not stale

@github-actions github-actions Bot removed the stale label Feb 18, 2026
@github-actions
Copy link
Copy Markdown

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.

@github-actions github-actions Bot added the stale label Mar 21, 2026
@smaheshwar-pltr
Copy link
Copy Markdown
Contributor Author

Bump to remove staleness

@github-actions github-actions Bot removed the stale label Mar 22, 2026
@github-actions
Copy link
Copy Markdown

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.

@github-actions github-actions Bot added the stale label Apr 21, 2026
@smaheshwar-pltr
Copy link
Copy Markdown
Contributor Author

not stale

@github-actions github-actions Bot removed the stale label Apr 22, 2026
@smaheshwar-pltr smaheshwar-pltr force-pushed the sm/bulk-encryption-deletion branch from 52a182a to cd054ad Compare May 5, 2026 23:42
}

if (io instanceof SupportsPrefixOperations) {
if (io instanceof DelegateFileIO) {
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree this isn't the nicest, but just following the existing pattern with SupportsPrefixOperations. DelegateFileIO is a very common case so I think we should support this flow

@smaheshwar-pltr smaheshwar-pltr marked this pull request as ready for review May 5, 2026 23:43
@smaheshwar-pltr
Copy link
Copy Markdown
Contributor Author

cc @huaxingao @ggershinsky @singhpk234 please may you take a look here?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant