Skip to content

HIVE-29563: Iceberg REST Catalog's error logs are too verbose#6425

Open
okumin wants to merge 5 commits intoapache:masterfrom
okumin:HIVE-29563-rest-logging
Open

HIVE-29563: Iceberg REST Catalog's error logs are too verbose#6425
okumin wants to merge 5 commits intoapache:masterfrom
okumin:HIVE-29563-rest-logging

Conversation

@okumin
Copy link
Copy Markdown
Contributor

@okumin okumin commented Apr 12, 2026

What changes were proposed in this pull request?

Stop logging every client-side error, i.e., 40x errors.

https://issues.apache.org/jira/browse/HIVE-29563

Why are the changes needed?

As explained in HIVE-29563, the current REST API prints the entire stack trace on every error.

apache/iceberg has the same problem and sent a patch, but it has not been reviewed: apache/iceberg#14908

Does this PR introduce any user-facing change?

It includes a change for HMS administrators.

How was this patch tested?

standalone-metastore/packaging/src/docker/build.sh -repo local
docker run --rm --name hive-standalone-metastore \
  -p 9083:9083 \
  -p 9001:9001 \
  local/hive:standalone-metastore-4.3.0-SNAPSHOT
curl -i http://localhost:9001/iceberg/v1/namespaces/default/tables/does_not_exist

// The caller is responsible for closing the underlying catalog backing this REST catalog.
response.setStatus(error.code());
RESTObjectMapper.mapper().writeValue(response.getWriter(), error);
return null;
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RESTServerCatalogAdapter on apache/iceberg extends a RESTClient. I don't know the exact reason. The trick is used in some integration tests.
Anyway, as our adapter never behaves as a client, I removed the dependency on RESTClient. As a good side effect, this change ensures only a single path invokes RESTServerCatalogAdapter#execute and allows us to simplify the implementation.

@okumin okumin changed the title [WIP] HIVE-29563: Iceberg REST Catalog's error logs are too verbose HIVE-29563: Iceberg REST Catalog's error logs are too verbose Apr 15, 2026
@okumin okumin marked this pull request as ready for review April 15, 2026 00:31
@ayushtkn
Copy link
Copy Markdown
Member

@henrib can you check once

Copy link
Copy Markdown
Contributor

@henrib henrib left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this does not require any test changes, LGTM 👍

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May be also reduce exception verbosity:

  public static void configureResponseFromException(
      Exception exc, ErrorResponse.Builder errorBuilder) {
    int errorCode = EXCEPTION_ERROR_CODES.getOrDefault(exc.getClass(), 500);
    errorBuilder
        .responseCode(errorCode)
        .withType(exc.getClass().getSimpleName())
        .withMessage(exc.getMessage());
    // avoid exposing stack traces for client errors, but include them for server errors to aid debugging
    if (errorCode == 500) {
      errorBuilder.withStackTrace(exc);
    }
  }

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this aligned with you?
7039752

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes :-)

Copy link
Copy Markdown
Contributor Author

@okumin okumin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your review! It will make upcoming changes easy to review

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this aligned with you?
7039752

Copy link
Copy Markdown
Member

@ayushtkn ayushtkn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@okumin
Copy link
Copy Markdown
Contributor Author

okumin commented May 6, 2026

I merged the master branch and resolved conflicts.

Additionally, I found that we can't see anything when a 5xx error occurs. I updated it so we can see a similar log on server errors, and INFO log for non-server errors for convenience.

2026-05-06T04:41:44,152  INFO [qtp1381730594-59] rest.HMSCatalogAdapter: A client error happened while processing REST request: Table does not exist: default.does_not_exist

@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud Bot commented May 6, 2026

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants