Skip to content

HIVE-29241: Distinguish the default location of the database by catalog #6267

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
zhangbutao wants to merge 3 commits into
base: master
Choose a base branch
from
Open

HIVE-29241: Distinguish the default location of the database by catalog #6267

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f4ccb18
HIVE-29241: Distinguish the default location of the database by catalog
zhangbutao Apr 11, 2026
8926c48
HIVE-29241: Distinguish the default location of the database by catalog
zhangbutao Apr 12, 2026
72df70d
Minor fix
zhangbutao Apr 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestMiniClusters.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@
import org.apache.hadoop.hive.llap.LlapItUtils;
import org.apache.hadoop.hive.llap.daemon.MiniLlapCluster;
import org.apache.hadoop.hive.llap.io.api.LlapProxy;
import org.apache.hadoop.hive.metastore.conf.MetastoreConf;
import org.apache.hadoop.hive.ql.exec.Utilities;
import org.apache.hadoop.hive.ql.exec.tez.TezSession;
import org.apache.hadoop.hive.ql.lockmgr.zookeeper.CuratorFrameworkSingleton;
Expand Down Expand Up @@ -544,6 +545,7 @@ private void setFsRelatedProperties(HiveConf conf, boolean isLocalFs, FileSystem
// Different paths if running locally vs a remote fileSystem. Ideally this difference should not
// exist.
Path warehousePath;
Path warehouseCatPath;
Path jarPath;
Path userInstallPath;
if (isLocalFs) {
Expand All @@ -554,23 +556,27 @@ private void setFsRelatedProperties(HiveConf conf, boolean isLocalFs, FileSystem
// Create a fake fs root for local fs
Path localFsRoot = new Path(path, "localfs");
warehousePath = new Path(localFsRoot, "warehouse");
warehouseCatPath = new Path(localFsRoot, "catalog");
Copy link
Copy Markdown
Member

@deniskuzZ deniskuzZ Apr 11, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't the catalog be under the warehousePath?

Warehouse Root
  └── Catalog (default or named)
        └── Database (.db)
              └── Table

/warehouse/<catalog_name>/<database_name>.db/<table_name>/   # non default catalog
/warehouse/hive/<database_name>.db/<table_name>/             # default catalog
/warehouse/hive/default.db/<table_name>/                     # default catalog & database

Copy link
Copy Markdown
Contributor Author

@zhangbutao zhangbutao Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't the catalog be under the warehousePath?

No. The warehousePath is the path of the default catalog, under which the paths of Hive databases reside. If the catalog path is placed under the warehousePath, it will cause confusion with the path information of Hive databases under the default catalog.

The path information for the default catalog's databases is controlled by the property metastore.warehouse.dir, which defaults to /user/hive/warehouse. The path for non-default catalogs is controlled by the new added propertymetastore.warehouse.catalog.dir, which defaults to /user/hive/catalog/warehouse.

/user/hive/catalog/warehouse/<catalog_name>/<database_name>.db/<table_name>/     # non default catalog
/user/hive/warehouse/<database_name>.db/<table_name>/            				 # default catalog
/user/hive/warehouse/default.db/<table_name>/                     				 # default catalog & database

Copy link
Copy Markdown
Member

@dengzhhu653 dengzhhu653 Apr 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to raise my concern, personally I don't like to add a new metastore.warehouse.catalog.dir as a default location for new catalogs(non-default). this could bring some concerns IMO:

  1. We use catalog to describe the source, we should at least get what/where the catalog is before creating the new catalog, that suggests don't create the catalog at will.
  2. On creating the catalog, we create the default database as well, the default database will stay at the old location even if we change the location of the catalog, this might add some confusions on audit and bill consuming.
  3. It's not a good experience to change the location after the catalog created, as we should consider moving the managed tables, which could be time/cost consuming.

So I would argue the location of catalog is immutable, we should check the location of non-default catalog on creating it, this is not a big change I think.

Copy link
Copy Markdown
Member

@deniskuzZ deniskuzZ Apr 24, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zhangbutao

The warehousePath is the path of the default catalog, under which the paths of Hive databases reside. If the catalog path is placed under the warehousePath, it will cause confusion with the path information of Hive databases under the default catalog.

i don't get that at all. where is confusion here?

The path for non-default catalogs is controlled by the new added property metastore.warehouse.catalog.dir

i don't think we need additional config for that

@dengzhhu653

I don't like to add a new metastore.warehouse.catalog.dir as a default location for new catalogs(non-default). this could bring some concerns IMO

I'm aligned on this as well
+1

On creating the catalog, we create the default database as well, the default database will stay at the old location even if we change the location of the catalog

default DB location should be under it's catalog location IMO

jarPath = new Path(localFsRoot, "jar");
userInstallPath = new Path(localFsRoot, "user_install");
} else {
// TODO Why is this changed from the default in hive-conf?
warehousePath = new Path(fsUriString, "/build/ql/test/data/warehouse/");
warehouseCatPath = new Path(fsUriString, "/build/ql/test/data/catalog/");
jarPath = new Path(new Path(fsUriString, "/user"), "hive");
userInstallPath = new Path(fsUriString, "/user");
}

warehousePath = fs.makeQualified(warehousePath);
warehouseCatPath = fs.makeQualified(warehouseCatPath);
jarPath = fs.makeQualified(jarPath);
userInstallPath = fs.makeQualified(userInstallPath);

conf.set(CommonConfigurationKeysPublic.FS_DEFAULT_NAME_KEY, fsUriString);

// Remote dirs
conf.setVar(ConfVars.METASTORE_WAREHOUSE, warehousePath.toString());
MetastoreConf.setVar(conf, MetastoreConf.ConfVars.WAREHOUSE_CATALOG, warehouseCatPath.toString());
conf.setVar(ConfVars.HIVE_JAR_DIRECTORY, jarPath.toString());
conf.setVar(ConfVars.HIVE_USER_INSTALL_DIR, userInstallPath.toString());
// ConfVars.SCRATCH_DIR - {test.tmp.dir}/scratchdir
Expand Down
4 changes: 2 additions & 2 deletions parser/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1132,10 +1132,10 @@ createCatalogStatement
: KW_CREATE KW_CATALOG
ifNotExists?
name=identifier
catLocation
catLocation?
catalogComment?
(KW_PROPERTIES catprops=properties)?
-> ^(TOK_CREATECATALOG $name catLocation ifNotExists? catalogComment? $catprops?)
-> ^(TOK_CREATECATALOG $name catLocation? ifNotExists? catalogComment? $catprops?)
;
catLocation
Expand Down
26 changes: 22 additions & 4 deletions ql/src/java/org/apache/hadoop/hive/ql/ddl/catalog/create/CreateCatalogAnalyzer.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@

package org.apache.hadoop.hive.ql.ddl.catalog.create;

import org.apache.hadoop.hive.metastore.CatalogUtil;
import org.apache.hadoop.hive.metastore.api.Catalog;
import org.apache.hadoop.hive.ql.QueryState;
import org.apache.hadoop.hive.ql.ddl.DDLSemanticAnalyzerFactory;
Expand All @@ -29,6 +30,7 @@
import org.apache.hadoop.hive.ql.parse.HiveParser;
import org.apache.hadoop.hive.ql.parse.SemanticException;

import java.util.HashMap;
import java.util.Map;

/**
Expand All @@ -43,16 +45,19 @@
@Override
public void analyzeInternal(ASTNode root) throws SemanticException {
String catalogName = unescapeIdentifier(root.getChild(0).getText());
String locationUrl = unescapeSQLString(root.getChild(1).getChild(0).getText());
outputs.add(toWriteEntity(locationUrl));

String locationUrl = null;
boolean ifNotExists = false;
String comment = null;
Map<String, String> props = null;
Map<String, String> props = new HashMap<>();

for (int i = 2; i < root.getChildCount(); i++) {
for (int i = 1; i < root.getChildCount(); i++) {
ASTNode childNode = (ASTNode) root.getChild(i);
switch (childNode.getToken().getType()) {
case HiveParser.TOK_CATALOGLOCATION:
locationUrl = unescapeSQLString(childNode.getChild(0).getText());
outputs.add(toWriteEntity(locationUrl));

Check warning on line 59 in ql/src/java/org/apache/hadoop/hive/ql/ddl/catalog/create/CreateCatalogAnalyzer.java

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

'block' child has incorrect indentation level 10, expected level should be 8. 


See more on https://sonarcloud.io/project/issues?id=apache_hive&issues=AZzh02F3oiSNCgpa0-uC&open=AZzh02F3oiSNCgpa0-uC&pullRequest=6267
break;

Check warning on line 60 in ql/src/java/org/apache/hadoop/hive/ql/ddl/catalog/create/CreateCatalogAnalyzer.java

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

'block' child has incorrect indentation level 10, expected level should be 8. 


See more on https://sonarcloud.io/project/issues?id=apache_hive&issues=AZzh02F3oiSNCgpa0-uD&open=AZzh02F3oiSNCgpa0-uD&pullRequest=6267
case HiveParser.TOK_IFNOTEXISTS:
ifNotExists = true;
break;
Expand All @@ -67,10 +72,23 @@
}
}

// Set default type to hive if not specified
checkCatalogType(props);

CreateCatalogDesc desc = new CreateCatalogDesc(catalogName, comment, locationUrl, ifNotExists, props);
Catalog catalog = new Catalog(catalogName, locationUrl);

rootTasks.add(TaskFactory.get(new DDLWork(getInputs(), getOutputs(), desc)));
outputs.add(new WriteEntity(catalog, WriteEntity.WriteType.DDL_NO_LOCK));
}

private static void checkCatalogType(Map<String, String> props) throws SemanticException {
String catalogType = props.get(CatalogUtil.TYPE);
// Normalize and validate catalog type (fail fast on invalid input)
try {
props.put(CatalogUtil.TYPE, CatalogUtil.normalizeCatalogType(catalogType));
} catch (IllegalArgumentException e) {
throw new SemanticException(String.format("type '%s' is not valid", catalogType));
Comment thread
zhangbutao marked this conversation as resolved.
}
}
}
7 changes: 6 additions & 1 deletion ql/src/java/org/apache/hadoop/hive/ql/ddl/catalog/create/CreateCatalogOperation.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@

import org.apache.hadoop.hive.metastore.api.AlreadyExistsException;
import org.apache.hadoop.hive.metastore.api.Catalog;
import org.apache.hadoop.hive.metastore.conf.MetastoreConf;
import org.apache.hadoop.hive.ql.ErrorMsg;
import org.apache.hadoop.hive.ql.ddl.DDLOperation;
import org.apache.hadoop.hive.ql.ddl.DDLOperationContext;
Expand All @@ -37,7 +38,11 @@ public CreateCatalogOperation(DDLOperationContext context, CreateCatalogDesc des

@Override
public int execute() throws Exception {
Catalog catalog = new Catalog(desc.getName(), desc.getLocationUri());
String catLocationUri = Optional.ofNullable(desc.getLocationUri())
Copy link
Copy Markdown
Member

@deniskuzZ deniskuzZ Apr 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor. i think cat prefix in local var is redundant here since we are inside CreateCatalog class

Copy link
Copy Markdown
Contributor Author

@zhangbutao zhangbutao Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, i didn't get you here. Could you please describe it in detail? Thank you.

Copy link
Copy Markdown
Member

@deniskuzZ deniskuzZ Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can simplify naming to just locationUri since class is about Catalog anyways

.orElse(MetastoreConf.getVar(context.getConf(),
MetastoreConf.ConfVars.WAREHOUSE_CATALOG) + "/" + desc.getName());
Comment thread
zhangbutao marked this conversation as resolved.

Catalog catalog = new Catalog(desc.getName(), catLocationUri);
catalog.setDescription(desc.getComment());
Optional.ofNullable(desc.getCatlogProperties())
.ifPresent(catalog::setParameters);
Expand Down
10 changes: 5 additions & 5 deletions ql/src/java/org/apache/hadoop/hive/ql/ddl/database/create/CreateDatabaseAnalyzer.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,20 +19,21 @@
package org.apache.hadoop.hive.ql.ddl.database.create;

import java.util.Map;
import java.util.Optional;

import org.apache.commons.lang3.tuple.Pair;
import org.apache.hadoop.hive.metastore.api.DataConnector;
import org.apache.hadoop.hive.metastore.api.Database;
import org.apache.hadoop.hive.metastore.api.DatabaseType;
import org.apache.hadoop.hive.metastore.api.PrincipalType;
import org.apache.hadoop.hive.ql.ErrorMsg;
import org.apache.hadoop.hive.ql.QueryState;
import org.apache.hadoop.hive.ql.ddl.DDLUtils;
import org.apache.hadoop.hive.ql.exec.TaskFactory;
import org.apache.hadoop.hive.ql.ddl.DDLSemanticAnalyzerFactory.DDLType;
import org.apache.hadoop.hive.ql.ddl.DDLWork;
import org.apache.hadoop.hive.ql.hooks.ReadEntity;
import org.apache.hadoop.hive.ql.hooks.WriteEntity;
import org.apache.hadoop.hive.ql.metadata.HiveUtils;
import org.apache.hadoop.hive.ql.parse.ASTNode;
import org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer;
import org.apache.hadoop.hive.ql.parse.HiveParser;
Expand All @@ -51,10 +52,9 @@ public CreateDatabaseAnalyzer(QueryState queryState) throws SemanticException {
@Override
public void analyzeInternal(ASTNode root) throws SemanticException {
Pair<String, String> catDbNamePair = DDLUtils.getCatDbNamePair((ASTNode) root.getChild(0));
Copy link
Copy Markdown
Member

@deniskuzZ deniskuzZ Apr 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about record CatalogDb(String catalog, String database) {}
please see #6379 (comment)

Copy link
Copy Markdown
Contributor Author

@zhangbutao zhangbutao Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think using a Pair here is already simple and clear enough, there's no need to use the record CatalogDb.

String catalogName = catDbNamePair.getLeft();
if (catalogName != null && getCatalog(catalogName) == null) {
throw new SemanticException(ErrorMsg.CATALOG_NOT_EXISTS, catalogName);
}
String catalogName = Optional.ofNullable(catDbNamePair.getLeft())
.orElse(HiveUtils.getCurrentCatalogOrDefault(conf));

Comment thread
zhangbutao marked this conversation as resolved.
String databaseName = catDbNamePair.getRight();

boolean ifNotExists = false;
Expand Down
63 changes: 46 additions & 17 deletions ql/src/java/org/apache/hadoop/hive/ql/ddl/database/create/CreateDatabaseOperation.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
import org.apache.hadoop.hive.ql.ddl.DDLOperationContext;
import org.apache.hadoop.hive.ql.exec.Utilities;
import org.apache.hadoop.hive.ql.metadata.HiveException;
import org.apache.hadoop.hive.ql.metadata.HiveUtils;
import org.apache.hadoop.hive.ql.session.SessionState;

/**
Expand All @@ -56,12 +57,12 @@ public int execute() throws HiveException {
if (desc.getManagedLocationUri() != null) {
database.setManagedLocationUri(desc.getManagedLocationUri());
}
makeLocationQualified(database); // TODO catalog. Add catalog prefix for db location. Depend on HIVE-29241.
makeLocationQualified(database);
if (database.getLocationUri().equalsIgnoreCase(database.getManagedLocationUri())) {
throw new HiveException("Managed and external locations for database cannot be the same");
}
} else if (desc.getDatabaseType() == DatabaseType.REMOTE) {
makeLocationQualified(database); // TODO catalog. Add catalog prefix for db location. Depend on HIVE-29241.
makeLocationQualified(database);
database.setConnector_name(desc.getConnectorName());
database.setRemote_dbname(desc.getRemoteDbName());
} else { // should never be here
Expand All @@ -81,34 +82,62 @@ public int execute() throws HiveException {
}

private void makeLocationQualified(Database database) throws HiveException {
String catalogName = database.getCatalogName().toLowerCase();
Comment thread
deniskuzZ marked this conversation as resolved.
String dbName = database.getName().toLowerCase();
boolean isDefaultCatalog = HiveUtils.isDefaultCatalog(catalogName, context.getConf());

// -------- External location --------
if (database.isSetLocationUri()) {
database.setLocationUri(Utilities.getQualifiedPath(context.getConf(), new Path(database.getLocationUri())));
} else {
// Location is not set we utilize WAREHOUSE_EXTERNAL together with database name
String rootDir = MetastoreConf.getVar(context.getConf(), MetastoreConf.ConfVars.WAREHOUSE_EXTERNAL);
if (rootDir == null || rootDir.trim().isEmpty()) {
// Fallback plan
LOG.warn(String.format(
"%s is not set, falling back to %s. This could cause external tables to use to managed tablespace.",
MetastoreConf.ConfVars.WAREHOUSE_EXTERNAL.getVarname(), MetastoreConf.ConfVars.WAREHOUSE.getVarname()));
rootDir = MetastoreConf.getVar(context.getConf(), MetastoreConf.ConfVars.WAREHOUSE);
}
Path path = new Path(rootDir, database.getName().toLowerCase() + DATABASE_PATH_SUFFIX);
String qualifiedPath = Utilities.getQualifiedPath(context.getConf(), path);
database.setLocationUri(qualifiedPath);
String rootDir = getExternalRootDir(isDefaultCatalog);
Path path = buildDbPath(rootDir, catalogName, dbName, isDefaultCatalog);
database.setLocationUri(Utilities.getQualifiedPath(context.getConf(), path));
}

// -------- Managed location --------
if (database.isSetManagedLocationUri()) {
database.setManagedLocationUri(Utilities.getQualifiedPath(context.getConf(),
new Path(database.getManagedLocationUri())));
} else {
// ManagedLocation is not set we utilize WAREHOUSE together with database name
String rootDir = MetastoreConf.getVar(context.getConf(), MetastoreConf.ConfVars.WAREHOUSE);
Path path = new Path(rootDir, database.getName().toLowerCase() + DATABASE_PATH_SUFFIX);
String rootDir = MetastoreConf.getVar(
context.getConf(),
isDefaultCatalog
? MetastoreConf.ConfVars.WAREHOUSE
: MetastoreConf.ConfVars.WAREHOUSE_CATALOG
);

Path path = buildDbPath(rootDir, catalogName, dbName, isDefaultCatalog);
String qualifiedPath = Utilities.getQualifiedPath(context.getConf(), path);
if (!qualifiedPath.equals(database.getLocationUri())) {
database.setManagedLocationUri(qualifiedPath);
}
}
}

private Path buildDbPath(String rootDir, String catalogName, String dbName, boolean isDefaultCatalog) {
return isDefaultCatalog
? new Path(rootDir, dbName + DATABASE_PATH_SUFFIX)
: new Path(rootDir + "/" + catalogName, dbName + DATABASE_PATH_SUFFIX);
Comment thread
zhangbutao marked this conversation as resolved.
}

private String getExternalRootDir(boolean isDefaultCatalog) {
MetastoreConf.ConfVars externalVar = isDefaultCatalog
? MetastoreConf.ConfVars.WAREHOUSE_EXTERNAL
: MetastoreConf.ConfVars.WAREHOUSE_CATALOG_EXTERNAL;

String rootDir = MetastoreConf.getVar(context.getConf(), externalVar);
if (rootDir != null && !rootDir.trim().isEmpty()) {
return rootDir;
}

MetastoreConf.ConfVars fallbackVar = isDefaultCatalog
? MetastoreConf.ConfVars.WAREHOUSE
: MetastoreConf.ConfVars.WAREHOUSE_CATALOG;

LOG.warn("{} is not set, falling back to {}. This could cause external tables to use managed tablespace.",
externalVar.getVarname(), fallbackVar.getVarname());

return MetastoreConf.getVar(context.getConf(), fallbackVar);
}
}
4 changes: 4 additions & 0 deletions ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveUtils.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -573,4 +573,8 @@ public static String getCurrentCatalogOrDefault(Configuration conf) {
.map(SessionState::getCurrentCatalog)
.orElseGet(() -> getDefaultCatalog(conf));
}

public static boolean isDefaultCatalog(String catName, Configuration conf) {
return catName == null || catName.isEmpty() || getDefaultCatalog(conf).equals(catName);
}
}
2 changes: 1 addition & 1 deletion ql/src/java/org/apache/hadoop/hive/ql/metadata/SessionHiveMetaStoreClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@
import static org.apache.hadoop.hive.metastore.client.utils.HiveMetaStoreClientUtils.deepCopyFieldSchemas;
import static org.apache.hadoop.hive.metastore.utils.MetaStoreUtils.compareFieldColumns;
import static org.apache.hadoop.hive.metastore.utils.MetaStoreUtils.getColumnNamesForTable;
import static org.apache.hadoop.hive.metastore.utils.MetaStoreUtils.getDefaultCatalog;

Check warning on line 145 in ql/src/java/org/apache/hadoop/hive/ql/metadata/SessionHiveMetaStoreClient.java

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Remove this unused import 'org.apache.hadoop.hive.metastore.utils.MetaStoreUtils.getDefaultCatalog'. 


See more on https://sonarcloud.io/project/issues?id=apache_hive&issues=AZ2B7HN4s0Fbzs2joUkh&open=AZ2B7HN4s0Fbzs2joUkh&pullRequest=6267

Check warning on line 145 in ql/src/java/org/apache/hadoop/hive/ql/metadata/SessionHiveMetaStoreClient.java

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Unused import - org.apache.hadoop.hive.metastore.utils.MetaStoreUtils.getDefaultCatalog. 


See more on https://sonarcloud.io/project/issues?id=apache_hive&issues=AZ2B7HN4s0Fbzs2joUki&open=AZ2B7HN4s0Fbzs2joUki&pullRequest=6267
import static org.apache.hadoop.hive.metastore.utils.MetaStoreUtils.getPvals;
import static org.apache.hadoop.hive.metastore.utils.MetaStoreUtils.isExternalTable;

Expand Down Expand Up @@ -181,7 +181,7 @@
}

private boolean isDefaultCatalog(String catName) {
return catName == null || catName.isEmpty() || getDefaultCatalog(conf).equals(catName);
return HiveUtils.isDefaultCatalog(catName, conf);
}

/**
Expand Down
32 changes: 27 additions & 5 deletions ql/src/java/org/apache/hadoop/hive/ql/parse/EximUtil.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
import org.apache.hadoop.fs.PathFilter;
import org.apache.hadoop.hive.common.repl.ReplConst;
import org.apache.hadoop.hive.conf.HiveConf;
import org.apache.hadoop.hive.metastore.Warehouse;

Check warning on line 30 in ql/src/java/org/apache/hadoop/hive/ql/parse/EximUtil.java

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Remove this unused import 'org.apache.hadoop.hive.metastore.Warehouse'. 


See more on https://sonarcloud.io/project/issues?id=apache_hive&issues=AZ2B7HMts0Fbzs2joUkf&open=AZ2B7HMts0Fbzs2joUkf&pullRequest=6267
import org.apache.hadoop.hive.metastore.api.Database;
import org.apache.hadoop.hive.metastore.api.FieldSchema;
import org.apache.hadoop.hive.metastore.conf.MetastoreConf;
Expand All @@ -39,6 +40,7 @@
import org.apache.hadoop.hive.ql.hooks.WriteEntity;
import org.apache.hadoop.hive.ql.metadata.Hive;
import org.apache.hadoop.hive.ql.metadata.HiveException;
import org.apache.hadoop.hive.ql.metadata.HiveUtils;
import org.apache.hadoop.hive.ql.metadata.Partition;
import org.apache.hadoop.hive.ql.metadata.Table;
import org.apache.hadoop.hive.ql.parse.repl.DumpType;
Expand Down Expand Up @@ -388,14 +390,34 @@

private static void updateIfCustomDbLocations(Database database, Configuration conf) throws SemanticException {
try {
String whLocatoion = MetastoreConf.getVar(conf, MetastoreConf.ConfVars.WAREHOUSE_EXTERNAL,
MetastoreConf.getVar(conf, MetastoreConf.ConfVars.WAREHOUSE));
Path dbDerivedLoc = new Path(whLocatoion, database.getName().toLowerCase() + DATABASE_PATH_SUFFIX);
String catName = database.getCatalogName();
String dbName = database.getName().toLowerCase();
boolean isDefaultCatalog = HiveUtils.isDefaultCatalog(catName, conf);

// external warehouse root
String whLocation = MetastoreConf.getVar(conf,
isDefaultCatalog ? MetastoreConf.ConfVars.WAREHOUSE_EXTERNAL : MetastoreConf.ConfVars.WAREHOUSE_CATALOG_EXTERNAL,

Check warning on line 399 in ql/src/java/org/apache/hadoop/hive/ql/parse/EximUtil.java

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Line is longer than 120 characters (found 127). 


See more on https://sonarcloud.io/project/issues?id=apache_hive&issues=AZzh02KLoiSNCgpa0-uE&open=AZzh02KLoiSNCgpa0-uE&pullRequest=6267
MetastoreConf.getVar(conf,
isDefaultCatalog ? MetastoreConf.ConfVars.WAREHOUSE : MetastoreConf.ConfVars.WAREHOUSE_CATALOG));

if (!isDefaultCatalog) {
whLocation = new Path(whLocation, catName).toString();
}

Path dbDerivedLoc = new Path(whLocation, dbName + DATABASE_PATH_SUFFIX);
String defaultDbLoc = Utilities.getQualifiedPath((HiveConf) conf, dbDerivedLoc);
database.putToParameters(ReplConst.REPL_IS_CUSTOM_DB_LOC,
Boolean.toString(!defaultDbLoc.equals(database.getLocationUri())));
String whManagedLocatoion = MetastoreConf.getVar(conf, MetastoreConf.ConfVars.WAREHOUSE);
Path dbDerivedManagedLoc = new Path(whManagedLocatoion, database.getName().toLowerCase()

// managed warehouse root
String whManagedLocatoion = MetastoreConf.getVar(conf,
isDefaultCatalog ? MetastoreConf.ConfVars.WAREHOUSE
Copy link
Copy Markdown
Member

@deniskuzZ deniskuzZ Apr 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please see #6267 (comment)

Copy link
Copy Markdown
Contributor Author

@zhangbutao zhangbutao Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replied in #6267 (comment)

: MetastoreConf.ConfVars.WAREHOUSE_CATALOG);

if (!isDefaultCatalog) {
whManagedLocatoion = new Path(whManagedLocatoion, catName).toString();
}
Path dbDerivedManagedLoc = new Path(whManagedLocatoion, dbName
+ DATABASE_PATH_SUFFIX);
String defaultDbManagedLoc = Utilities.getQualifiedPath((HiveConf) conf, dbDerivedManagedLoc);
database.getParameters().put(ReplConst.REPL_IS_CUSTOM_DB_MANAGEDLOC, Boolean.toString(
Expand Down
14 changes: 10 additions & 4 deletions ql/src/java/org/apache/hadoop/hive/ql/queryhistory/repository/AbstractRepository.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
import org.apache.hadoop.hive.ql.QueryState;
import org.apache.hadoop.hive.ql.metadata.Hive;
import org.apache.hadoop.hive.ql.metadata.HiveException;
import org.apache.hadoop.hive.ql.metadata.HiveUtils;

Check warning on line 31 in ql/src/java/org/apache/hadoop/hive/ql/queryhistory/repository/AbstractRepository.java

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Remove this unused import 'org.apache.hadoop.hive.ql.metadata.HiveUtils'. 


See more on https://sonarcloud.io/project/issues?id=apache_hive&issues=AZzh02K6oiSNCgpa0-uJ&open=AZzh02K6oiSNCgpa0-uJ&pullRequest=6267
import org.apache.hadoop.hive.ql.metadata.Table;
import org.apache.hadoop.hive.ql.parse.PartitionTransform;
import org.apache.hadoop.hive.ql.parse.TransformSpec;
Expand All @@ -40,6 +41,8 @@
import java.util.ArrayList;
import java.util.List;

import static org.apache.hadoop.hive.metastore.utils.MetaStoreUtils.getDefaultCatalog;

public abstract class AbstractRepository implements QueryHistoryRepository {
protected Logger LOG = LoggerFactory.getLogger(getClass());
@VisibleForTesting
Expand Down Expand Up @@ -82,9 +85,12 @@
db = hive.getDatabase(QUERY_HISTORY_DB_NAME);
if (db == null) {
LOG.warn("Database ({}) for query history table hasn't been found, auto-creating one", QUERY_HISTORY_DB_NAME);
String location = getDatabaseLocation(QUERY_HISTORY_DB_NAME);
// TODO catalog. The Hive Query History functionality is currently limited to the default catalog. Depend on HIVE-29561

Check warning on line 88 in ql/src/java/org/apache/hadoop/hive/ql/queryhistory/repository/AbstractRepository.java

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Complete the task associated to this TODO comment. 


See more on https://sonarcloud.io/project/issues?id=apache_hive&issues=AZ2B7HOZs0Fbzs2joUkj&open=AZ2B7HOZs0Fbzs2joUkj&pullRequest=6267

Check warning on line 88 in ql/src/java/org/apache/hadoop/hive/ql/queryhistory/repository/AbstractRepository.java

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Line is longer than 120 characters (found 127). 


See more on https://sonarcloud.io/project/issues?id=apache_hive&issues=AZ2B7HOZs0Fbzs2joUkk&open=AZ2B7HOZs0Fbzs2joUkk&pullRequest=6267
db = new Database(QUERY_HISTORY_DB_NAME, QUERY_HISTORY_DB_COMMENT,
location, null);
null, null);
db.setCatalogName(getDefaultCatalog(conf));
String location = getDatabaseLocation(db);
db.setLocationUri(location);
hive.createDatabase(db, false);
}
return db;
Expand All @@ -93,8 +99,8 @@
}
}

private String getDatabaseLocation(String databaseName) throws Exception {
return warehouse.getDefaultExternalDatabasePath(databaseName).toUri().toString();
private String getDatabaseLocation(Database db) throws Exception {
return warehouse.getDefaultExternalDatabasePath(db).toUri().toString();
}

protected Table initTable(Hive hive, Database db) {
Expand Down
Loading
Loading