Skip to content
Merged
19 changes: 19 additions & 0 deletions docs/oss/integrations/athena.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
---
title: "Athena"
---

import AthenaCli from '/snippets/cli/athena-cli.mdx';



<Tabs>
<Tab title="Elementary CLI">
<AthenaCli />
</Tab>
</Tabs>

### Have a question?

We are available
on [Slack](https://elementary-data.com/community), reach out
for any kind of help!
19 changes: 19 additions & 0 deletions docs/oss/integrations/duckdb.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
---
title: "DuckDB"
---

import DuckdbCli from '/snippets/cli/duckdb-cli.mdx';



<Tabs>
<Tab title="Elementary CLI">
<DuckdbCli />
</Tab>
</Tabs>

### Have a question?

We are available
on [Slack](https://elementary-data.com/community), reach out
for any kind of help!
19 changes: 19 additions & 0 deletions docs/oss/integrations/spark.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
---
title: "Spark"
---

import SparkCli from '/snippets/cli/spark-cli.mdx';



<Tabs>
<Tab title="Elementary CLI">
<SparkCli />
</Tab>
</Tabs>

### Have a question?

We are available
on [Slack](https://elementary-data.com/community), reach out
for any kind of help!
19 changes: 19 additions & 0 deletions docs/oss/integrations/trino.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
---
title: "Trino"
---

import TrinoCli from '/snippets/cli/trino-cli.mdx';



<Tabs>
<Tab title="Elementary CLI">
<TrinoCli />
</Tab>
</Tabs>

### Have a question?

We are available
on [Slack](https://elementary-data.com/community), reach out
for any kind of help!
33 changes: 33 additions & 0 deletions docs/snippets/cli/athena-cli.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
### Athena connection profile

After installing Elementary's dbt package upon running `dbt deps`,
you can generate Elementary's profile for usage with `edr` by running the following command within your project:

```shell
dbt run-operation elementary.generate_elementary_cli_profile
```

The command will print to the terminal a partially filled template of the profile that's needed for `edr` to work.

```yml Athena
## ATHENA ##
## By default, edr expects the profile name 'elementary'. ##
## Configure the database and schema of elementary models. ##
## Check where 'elementary_test_results' is to find it. ##

elementary:
outputs:
default:
type: athena
work_group: [athena workgroup]
s3_staging_dir: [s3_staging_dir] # Location to store query results & metadata
s3_data_dir: [s3 data dir] # Location to store table data (if not specified, s3_staging_dir is used)
region_name: [aws region name] # AWS region, e.g. eu-west-1
database: [database name]
schema: [schema name] # elementary schema, usually [schema name]_elementary
threads: [number of threads like 8]
```

We support the same format and connection methods as dbt. Please refer to
dbt's documentation of [Athena profile](https://docs.getdbt.com/reference/warehouse-setups/athena-setup) for
further details.
35 changes: 35 additions & 0 deletions docs/snippets/cli/dremio-cli.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
### Dremio connection profile

After installing Elementary's dbt package upon running `dbt deps`,
you can generate Elementary's profile for usage with `edr` by running the following command within your project:

```shell
dbt run-operation elementary.generate_elementary_cli_profile
```

The command will print to the terminal a partially filled template of the profile that's needed for `edr` to work.

```yml Dremio
## DREMIO ##
## By default, edr expects the profile name 'elementary'. ##
## Configure the database and schema of elementary models. ##
## Check where 'elementary_test_results' is to find it. ##

elementary:
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should show here either the cloud or the software option
https://docs.getdbt.com/docs/core/connect-data-platform/dremio-setup#profiles

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — now shows both Dremio Cloud and Dremio Software profile options using tabs, with the correct fields for each (Cloud: cloud_host, cloud_project_id, pat; Software: software_host, port, password/pat). Also updated the dbt docs link to the current URL.

outputs:
default:
type: dremio
software_host: [hostname]
port: [port]
user: [username]
password: [password]
use_ssl: [true or false]
datalake: [datalake source name]
database: [database name]
schema: [schema name] # elementary schema, usually [schema name]_elementary
threads: [1 or more]
```

We support the same format and connection methods as dbt. Please refer to
dbt's documentation of [Dremio profile](https://docs.getdbt.com/reference/warehouse-setups/dremio-setup) for
further details.
29 changes: 29 additions & 0 deletions docs/snippets/cli/duckdb-cli.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
### DuckDB connection profile

After installing Elementary's dbt package upon running `dbt deps`,
you can generate Elementary's profile for usage with `edr` by running the following command within your project:

```shell
dbt run-operation elementary.generate_elementary_cli_profile
```

The command will print to the terminal a partially filled template of the profile that's needed for `edr` to work.

```yml DuckDB
## DUCKDB ##
## By default, edr expects the profile name 'elementary'. ##
## Configure the database and schema of elementary models. ##
## Check where 'elementary_test_results' is to find it. ##

elementary:
outputs:
default:
type: duckdb
path: [path to your .duckdb file]
schema: [schema name] # elementary schema, usually [schema name]_elementary
threads: [1 or more]
```

We support the same format and connection methods as dbt. Please refer to
dbt's documentation of [DuckDB profile](https://docs.getdbt.com/reference/warehouse-setups/duckdb-setup) for
further details.
33 changes: 33 additions & 0 deletions docs/snippets/cli/spark-cli.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
### Spark connection profile

After installing Elementary's dbt package upon running `dbt deps`,
you can generate Elementary's profile for usage with `edr` by running the following command within your project:

```shell
dbt run-operation elementary.generate_elementary_cli_profile
```

The command will print to the terminal a partially filled template of the profile that's needed for `edr` to work.

```yml Spark
## SPARK ##
## By default, edr expects the profile name 'elementary'. ##
## Configure the database and schema of elementary models. ##
## Check where 'elementary_test_results' is to find it. ##

elementary:
outputs:
default:
type: spark
method: [thrift, http, or odbc]
host: [hostname]
port: [port]
user: [username]
schema: [schema name] # elementary schema, usually [schema name]_elementary
threads: [1 or more]
# token: [optional, used with http method]
```

We support the same format and connection methods as dbt. Please refer to
dbt's documentation of [Spark profile](https://docs.getdbt.com/reference/warehouse-setups/spark-setup) for
further details.
35 changes: 35 additions & 0 deletions docs/snippets/cli/trino-cli.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
### Trino connection profile

After installing Elementary's dbt package upon running `dbt deps`,
you can generate Elementary's profile for usage with `edr` by running the following command within your project:

```shell
dbt run-operation elementary.generate_elementary_cli_profile
```

The command will print to the terminal a partially filled template of the profile that's needed for `edr` to work.

```yml Trino
## TRINO ##
## By default, edr expects the profile name 'elementary'. ##
## Configure the database and schema of elementary models. ##
## Check where 'elementary_test_results' is to find it. ##

elementary:
outputs:
default:
type: trino
host: [hostname]
port: [port]
database: [database name]
schema: [schema name] # elementary schema, usually [schema name]_elementary
threads: [1 or more]
method: [authentication method] # ldap, oauth etc.
user: [username]
# password: [optional, used with ldap authentication ]
# session_properties: [optional, sets Trino session properties used in the connection]
```

We support the same format and connection methods as dbt. Please refer to
dbt's documentation of [Trino profile](https://docs.getdbt.com/reference/warehouse-setups/trino-setup) for
further details.
3 changes: 3 additions & 0 deletions docs/snippets/install-cli.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@ pip install 'elementary-data[databricks]'
pip install 'elementary-data[athena]'
pip install 'elementary-data[trino]'
pip install 'elementary-data[clickhouse]'
pip install 'elementary-data[duckdb]'
pip install 'elementary-data[dremio]'
pip install 'elementary-data[spark]'
## Postgres doesn't require this step
```

Expand Down
44 changes: 44 additions & 0 deletions docs/snippets/oss/adapters-cards.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -342,6 +342,50 @@
</svg>
}
></Card>
<Card
title="dbt-duckdb"
icon={
<svg
height="40"
viewBox="0 0 40 40"
width="40"
xmlns="http://www.w3.org/2000/svg"
>
<circle cx="20" cy="20" r="18" fill="#FFF100" stroke="#333" strokeWidth="1"/>
<ellipse cx="14" cy="16" rx="3" ry="3.5" fill="#333"/>
<ellipse cx="26" cy="16" rx="3" ry="3.5" fill="#333"/>
<path d="M12 26 Q20 34 28 26" fill="#E8760B" stroke="#333" strokeWidth="1"/>
</svg>
}
></Card>
<Card
title="dbt-dremio"
icon={
<svg
height="40"
viewBox="0 0 40 40"
width="40"
xmlns="http://www.w3.org/2000/svg"
>
<rect x="2" y="2" width="36" height="36" rx="6" fill="#3BAE73"/>
<text x="20" y="26" textAnchor="middle" fill="white" fontSize="18" fontWeight="bold" fontFamily="Arial">D</text>
</svg>
}
></Card>
<Card
title="dbt-spark"
icon={
<svg
height="40"
viewBox="0 0 40 40"
width="40"
xmlns="http://www.w3.org/2000/svg"
>
<circle cx="20" cy="20" r="18" fill="#E25A1C"/>
<path d="M20 6 L23 17 L34 17 L25 23 L28 34 L20 27 L12 34 L15 23 L6 17 L17 17 Z" fill="#FFF"/>
</svg>
}
></Card>
<Card
title="Ask us"
href="https://github.com/elementary-data/elementary/issues"
Expand Down
55 changes: 55 additions & 0 deletions docs/snippets/profiles/all-profiles.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -177,4 +177,59 @@ elementary:
# session_properties: [optional, sets Trino session properties used in the connection]
```

```yml DuckDB
## DUCKDB ##
## By default, edr expects the profile name 'elementary'. ##
## Configure the database and schema of elementary models. ##
## Check where 'elementary_test_results' is to find it. ##

elementary:
outputs:
default:
type: duckdb
path: [path to your .duckdb file]
schema: [schema name] # elementary schema, usually [schema name]_elementary
threads: [1 or more]
```

```yml Dremio
## DREMIO ##
## By default, edr expects the profile name 'elementary'. ##
## Configure the database and schema of elementary models. ##
## Check where 'elementary_test_results' is to find it. ##
Comment thread
haritamar marked this conversation as resolved.

elementary:
outputs:
default:
type: dremio
software_host: [hostname]
port: [port]
user: [username]
password: [password]
use_ssl: [true or false]
datalake: [datalake source name]
database: [database name]
schema: [schema name] # elementary schema, usually [schema name]_elementary
threads: [1 or more]
```

```yml Spark
## SPARK ##
## By default, edr expects the profile name 'elementary'. ##
## Configure the database and schema of elementary models. ##
## Check where 'elementary_test_results' is to find it. ##

elementary:
outputs:
default:
type: spark
method: [thrift, http, or odbc]
host: [hostname]
port: [port]
user: [username]
schema: [schema name] # elementary schema, usually [schema name]_elementary
threads: [1 or more]
# token: [optional, used with http method]
```

</CodeGroup>
Loading