Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/lakehouse/catalogs/hive-catalog.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ CREATE CATALOG [IF NOT EXISTS] catalog_name PROPERTIES (
'type'='hms', -- required
'hive.metastore.type' = '<hive_metastore_type>', -- optional
'hive.version' = '<hive_version>', -- optional
'fs.defaultFS' = '<fs_defaultfs>', -- optional
'fs.defaultFS' = '<fs_defaultfs>', -- required for write, optional for read-only
{MetaStoreProperties},
{StorageProperties},
{HiveProperties},
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ CREATE CATALOG [IF NOT EXISTS] catalog_name PROPERTIES (
'type'='hms', -- required
'hive.metastore.type' = '<hive_metastore_type>', -- optional
'hive.version' = '<hive_version>', -- optional
'fs.defaultFS' = '<fs_defaultfs>', -- optional
'fs.defaultFS' = '<fs_defaultfs>', -- 写入必填,只读时可选
{MetaStoreProperties},
{StorageProperties},
{HiveProperties},
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
---

Check warning on line 1 in i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/lakehouse/catalogs/hive-catalog.mdx

View workflow job for this annotation

GitHub Actions / Build Check

seo-title-duplicate

Rendered SEO title is duplicated across indexable pages%3A "Hive Catalog - Apache Doris". Add a version%2C locale%2C or page-specific qualifier. Owner%3A @apache/doris-website-maintainers

Check warning on line 1 in i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/lakehouse/catalogs/hive-catalog.mdx

View workflow job for this annotation

GitHub Actions / Build Check

seo-description-length

SEO description should be 80-160 characters; current length is 78. Owner%3A @apache/doris-website-maintainers
{
"title": "Hive Catalog",
"language": "zh-CN",
Expand All @@ -13,6 +13,10 @@

除了 Hive 外,很多其他系统也会使用 Hive Metastore 存储元数据。所以通过 Hive Catalog,我们不仅能访问 Hive 表,也能访问使用 Hive Metastore 作为元数据存储的其他表格式,如 Iceberg、Hudi 等。

:::note
通过 Doris 写入 Hive 的能力自 2.1.3 版本起支持,2.1.3 之前的版本仅支持查询。
:::

## 适用场景

| 场景 | 说明 |
Expand All @@ -30,7 +34,7 @@
'type'='hms', -- required
'hive.metastore.type' = '<hive_metastore_type>', -- optional
'hive.version' = '<hive_version>', -- optional
'fs.defaultFS' = '<fs_defaultfs>', -- optional
'fs.defaultFS' = '<fs_defaultfs>', -- 写入必填,只读时可选
{MetaStoreProperties},
{StorageProperties},
{HiveProperties},
Expand Down Expand Up @@ -542,7 +546,7 @@
<summary>2.1 & 3.0 版本</summary>
<Tabs>
<TabItem value='S3' label='S3' default>
非 EC2 环境下,需要使用 [aws configure ](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html) 配置 Credentials 信息,同时在~/.aws 目录下生成 credentials 文件。

Check notice on line 549 in i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/lakehouse/catalogs/hive-catalog.mdx

View workflow job for this annotation

GitHub Actions / Build Check

link-external-report-only

External link is report-only and was not fetched%3A https%3A//docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html. Owner%3A @apache/doris-website-maintainers

或者显示的在创建 catalog 参数中新增凭据提供方式 <b>"aws.catalog.credentials.provider.factory.class"="com.amazonaws.glue.catalog.credentials.ConfigurationAWSCredentialsProviderFactory"</b>
```sql
Expand Down Expand Up @@ -662,7 +666,7 @@

### 查询 Hive 事务表

Hive Transactional 表是 Hive 中支持 ACID 语义的表。详情可见 [Hive Transactions](https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions)。

Check notice on line 669 in i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/lakehouse/catalogs/hive-catalog.mdx

View workflow job for this annotation

GitHub Actions / Build Check

link-external-report-only

External link is report-only and was not fetched%3A https%3A//cwiki.apache.org/confluence/display/Hive/Hive+Transactions. Owner%3A @apache/doris-website-maintainers

* Hive Transactional 表支持情况

Expand Down Expand Up @@ -859,7 +863,7 @@
);
```

创建后,可以通过 `SHOW CREATE TABLE` 命令查看 Hive 的建表语句。

Check warning on line 866 in i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/lakehouse/catalogs/hive-catalog.mdx

View workflow job for this annotation

GitHub Actions / Build Check

markdown-code-fence-language

Code fence should declare a language. Owner%3A @apache/doris-website-maintainers

注意,不同于 Hive 中的建表语句。在 Doris 中创建 Hive 分区表时,分区列也必须写到 Table 的 Schema 中。同时,分区列必须在所有 Schema 的最后,且顺序保持一致。

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
---

Check warning on line 1 in i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/lakehouse/catalogs/hive-catalog.mdx

View workflow job for this annotation

GitHub Actions / Build Check

seo-description-length

SEO description should be 80-160 characters; current length is 78. Owner%3A @apache/doris-website-maintainers
{
"title": "Hive Catalog",
"language": "zh-CN",
Expand Down Expand Up @@ -30,7 +30,7 @@
'type'='hms', -- required
'hive.metastore.type' = '<hive_metastore_type>', -- optional
'hive.version' = '<hive_version>', -- optional
'fs.defaultFS' = '<fs_defaultfs>', -- optional
'fs.defaultFS' = '<fs_defaultfs>', -- 写入必填,只读时可选
{MetaStoreProperties},
{StorageProperties},
{HiveProperties},
Expand Down Expand Up @@ -636,7 +636,7 @@
<summary>2.1 & 3.0 版本</summary>
<Tabs>
<TabItem value='S3' label='S3' default>
非 EC2 环境下,需要使用 [aws configure ](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html) 配置 Credentials 信息,同时在~/.aws 目录下生成 credentials 文件。

Check notice on line 639 in i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/lakehouse/catalogs/hive-catalog.mdx

View workflow job for this annotation

GitHub Actions / Build Check

link-external-report-only

External link is report-only and was not fetched%3A https%3A//docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html. Owner%3A @apache/doris-website-maintainers

或者显示的在创建 catalog 参数中新增凭据提供方式 <b>"aws.catalog.credentials.provider.factory.class"="com.amazonaws.glue.catalog.credentials.ConfigurationAWSCredentialsProviderFactory"</b>
```sql
Expand Down Expand Up @@ -756,7 +756,7 @@

### 查询 Hive 事务表

Hive Transactional 表是 Hive 中支持 ACID 语义的表。详情可见 [Hive Transactions](https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions)。

Check notice on line 759 in i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/lakehouse/catalogs/hive-catalog.mdx

View workflow job for this annotation

GitHub Actions / Build Check

link-external-report-only

External link is report-only and was not fetched%3A https%3A//cwiki.apache.org/confluence/display/Hive/Hive+Transactions. Owner%3A @apache/doris-website-maintainers

* Hive Transactional 表支持情况

Expand Down Expand Up @@ -955,7 +955,7 @@
);
```

创建后,可以通过 `SHOW CREATE TABLE` 命令查看 Hive 的建表语句。

Check warning on line 958 in i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/lakehouse/catalogs/hive-catalog.mdx

View workflow job for this annotation

GitHub Actions / Build Check

markdown-code-fence-language

Code fence should declare a language. Owner%3A @apache/doris-website-maintainers

注意,不同于 Hive 中的建表语句。在 Doris 中创建 Hive 分区表时,分区列也必须写到 Table 的 Schema 中。同时,分区列必须在所有 Schema 的最后,且顺序保持一致。

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
---

Check warning on line 1 in i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/lakehouse/catalogs/hive-catalog.mdx

View workflow job for this annotation

GitHub Actions / Build Check

seo-description-length

SEO description should be 80-160 characters; current length is 78. Owner%3A @apache/doris-website-maintainers
{
"title": "Hive Catalog",
"language": "zh-CN",
Expand Down Expand Up @@ -30,7 +30,7 @@
'type'='hms', -- required
'hive.metastore.type' = '<hive_metastore_type>', -- optional
'hive.version' = '<hive_version>', -- optional
'fs.defaultFS' = '<fs_defaultfs>', -- optional
'fs.defaultFS' = '<fs_defaultfs>', -- 写入必填,只读时可选
{MetaStoreProperties},
{StorageProperties},
{HiveProperties},
Expand Down Expand Up @@ -638,7 +638,7 @@
<summary>2.1 & 3.0 版本</summary>
<Tabs>
<TabItem value='S3' label='S3' default>
非 EC2 环境下,需要使用 [aws configure ](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html) 配置 Credentials 信息,同时在~/.aws 目录下生成 credentials 文件。

Check notice on line 641 in i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/lakehouse/catalogs/hive-catalog.mdx

View workflow job for this annotation

GitHub Actions / Build Check

link-external-report-only

External link is report-only and was not fetched%3A https%3A//docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html. Owner%3A @apache/doris-website-maintainers

或者显示的在创建 catalog 参数中新增凭据提供方式 <b>"aws.catalog.credentials.provider.factory.class"="com.amazonaws.glue.catalog.credentials.ConfigurationAWSCredentialsProviderFactory"</b>
```sql
Expand Down Expand Up @@ -758,7 +758,7 @@

### 查询 Hive 事务表

Hive Transactional 表是 Hive 中支持 ACID 语义的表。详情可见 [Hive Transactions](https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions)。

Check notice on line 761 in i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/lakehouse/catalogs/hive-catalog.mdx

View workflow job for this annotation

GitHub Actions / Build Check

link-external-report-only

External link is report-only and was not fetched%3A https%3A//cwiki.apache.org/confluence/display/Hive/Hive+Transactions. Owner%3A @apache/doris-website-maintainers

* Hive Transactional 表支持情况

Expand Down Expand Up @@ -957,7 +957,7 @@
);
```

创建后,可以通过 `SHOW CREATE TABLE` 命令查看 Hive 的建表语句。

Check warning on line 960 in i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/lakehouse/catalogs/hive-catalog.mdx

View workflow job for this annotation

GitHub Actions / Build Check

markdown-code-fence-language

Code fence should declare a language. Owner%3A @apache/doris-website-maintainers

注意,不同于 Hive 中的建表语句。在 Doris 中创建 Hive 分区表时,分区列也必须写到 Table 的 Schema 中。同时,分区列必须在所有 Schema 的最后,且顺序保持一致。

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
---

Check warning on line 1 in versioned_docs/version-2.1/lakehouse/catalogs/hive-catalog.mdx

View workflow job for this annotation

GitHub Actions / Build Check

seo-description-length

SEO description should be 80-160 characters; current length is 233. Owner%3A @apache/doris-website-maintainers
{
"title": "Hive Catalog",
"language": "en",
Expand All @@ -13,6 +13,10 @@

In addition to Hive, many other systems use Hive Metastore to store metadata. Therefore, through the Hive Catalog, we can access not only Hive tables but also other table formats that use Hive Metastore for metadata storage, such as Iceberg and Hudi.

:::note
Writing to Hive through Doris is supported from version 2.1.3. Earlier 2.1.x releases only support querying.
:::

## Applicable Scenarios

| Scenario | Description |
Expand All @@ -30,7 +34,7 @@
'type'='hms', -- required
'hive.metastore.type' = '<hive_metastore_type>', -- optional
'hive.version' = '<hive_version>', -- optional
'fs.defaultFS' = '<fs_defaultfs>', -- optional
'fs.defaultFS' = '<fs_defaultfs>', -- required for write, optional for read-only
{MetaStoreProperties},
{StorageProperties},
{HiveProperties},
Expand Down Expand Up @@ -533,7 +537,7 @@
<TabItem value='S3' label='S3' default>
AWS Glue and S3 storage services share the same authentication credentials.

In non-EC2 environments, you need to use [aws configure](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html) to configure Credentials information and generate a credentials file in the ~/.aws directory.

Check notice on line 540 in versioned_docs/version-2.1/lakehouse/catalogs/hive-catalog.mdx

View workflow job for this annotation

GitHub Actions / Build Check

link-external-report-only

External link is report-only and was not fetched%3A https%3A//docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html. Owner%3A @apache/doris-website-maintainers

Alternatively, you can explicitly specify the credentials provider factory class
when creating the catalog:
Expand Down Expand Up @@ -656,7 +660,7 @@

### Querying Hive Transactional Tables

Hive Transactional tables support ACID semantics. For more details, see [Hive Transactions](https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions).

Check notice on line 663 in versioned_docs/version-2.1/lakehouse/catalogs/hive-catalog.mdx

View workflow job for this annotation

GitHub Actions / Build Check

link-external-report-only

External link is report-only and was not fetched%3A https%3A//cwiki.apache.org/confluence/display/Hive/Hive+Transactions. Owner%3A @apache/doris-website-maintainers

* Support for Hive Transactional Tables

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
---

Check warning on line 1 in versioned_docs/version-3.x/lakehouse/catalogs/hive-catalog.mdx

View workflow job for this annotation

GitHub Actions / Build Check

seo-description-length

SEO description should be 80-160 characters; current length is 235. Owner%3A @apache/doris-website-maintainers
{
"title": "Hive Catalog",
"language": "en",
Expand Down Expand Up @@ -30,7 +30,7 @@
'type'='hms', -- required
'hive.metastore.type' = '<hive_metastore_type>', -- optional
'hive.version' = '<hive_version>', -- optional
'fs.defaultFS' = '<fs_defaultfs>', -- optional
'fs.defaultFS' = '<fs_defaultfs>', -- required for write, optional for read-only
{MetaStoreProperties},
{StorageProperties},
{HiveProperties},
Expand Down Expand Up @@ -623,7 +623,7 @@
<TabItem value='S3' label='S3' default>
AWS Glue and S3 storage services share the same authentication credentials.

In non-EC2 environments, you need to use [aws configure](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html) to configure Credentials information and generate a credentials file in the ~/.aws directory.

Check notice on line 626 in versioned_docs/version-3.x/lakehouse/catalogs/hive-catalog.mdx

View workflow job for this annotation

GitHub Actions / Build Check

link-external-report-only

External link is report-only and was not fetched%3A https%3A//docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html. Owner%3A @apache/doris-website-maintainers

Alternatively, you can explicitly specify the credentials provider factory class
when creating the catalog:
Expand Down Expand Up @@ -746,7 +746,7 @@

### Querying Hive Transactional Tables

Hive Transactional tables support ACID semantics. For more details, see [Hive Transactions](https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions).

Check notice on line 749 in versioned_docs/version-3.x/lakehouse/catalogs/hive-catalog.mdx

View workflow job for this annotation

GitHub Actions / Build Check

link-external-report-only

External link is report-only and was not fetched%3A https%3A//cwiki.apache.org/confluence/display/Hive/Hive+Transactions. Owner%3A @apache/doris-website-maintainers

* Support for Hive Transactional Tables

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
---

Check warning on line 1 in versioned_docs/version-4.x/lakehouse/catalogs/hive-catalog.mdx

View workflow job for this annotation

GitHub Actions / Build Check

seo-description-length

SEO description should be 80-160 characters; current length is 235. Owner%3A @apache/doris-website-maintainers
{
"title": "Hive Catalog",
"language": "en",
Expand Down Expand Up @@ -30,7 +30,7 @@
'type'='hms', -- required
'hive.metastore.type' = '<hive_metastore_type>', -- optional
'hive.version' = '<hive_version>', -- optional
'fs.defaultFS' = '<fs_defaultfs>', -- optional
'fs.defaultFS' = '<fs_defaultfs>', -- required for write, optional for read-only
{MetaStoreProperties},
{StorageProperties},
{HiveProperties},
Expand Down
Loading