From 58693f442fedcc81b83d60ada13c2108920e9911 Mon Sep 17 00:00:00 2001 From: James Broadhead Date: Wed, 27 May 2026 15:46:07 +0000 Subject: [PATCH 1/2] docs(skills): use top-level `databricks aitools` everywhere MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The skills-management and tools commands were promoted to top-level in databricks/cli#4917. The old `databricks experimental aitools …` paths still resolve as deprecated aliases, but new readers should be pointed at the supported command. Updates `.agents/skills/databricks{,-core,-apps,-jobs,-pipelines}/...`. Co-authored-by: Isaac --- .agents/skills/databricks-apps/SKILL.md | 2 +- .agents/skills/databricks-core/SKILL.md | 10 +-- .../databricks-core/data-exploration.md | 62 +++++++++---------- .agents/skills/databricks-jobs/SKILL.md | 2 +- .agents/skills/databricks-pipelines/SKILL.md | 2 +- .agents/skills/databricks/SKILL.md | 10 +-- .agents/skills/databricks/data-exploration.md | 62 +++++++++---------- 7 files changed, 75 insertions(+), 75 deletions(-) diff --git a/.agents/skills/databricks-apps/SKILL.md b/.agents/skills/databricks-apps/SKILL.md index 4a26d97..23da19c 100644 --- a/.agents/skills/databricks-apps/SKILL.md +++ b/.agents/skills/databricks-apps/SKILL.md @@ -129,7 +129,7 @@ npx @databricks/appkit docs ./docs/plugins/analytics.md # example: specific doc Optionally use `--version ` to target a specific AppKit version. - **Required**: `--name`, `--profile`. Name: ≤26 chars, lowercase letters/numbers/hyphens only. Use `--features` only for **optional** plugins the user wants (plugins with `requiredByTemplate: false` or absent); mandatory plugins must not be listed in `--features`. - **Resources**: Pass `--set` for every required resource (each field in `resources.required`) for (1) all plugins with `requiredByTemplate: true`, and (2) any optional plugins you added to `--features`. Add `--set` for `resources.optional` only when the user requests them. - - **Discovery**: Use the parent `databricks-core` skill to resolve IDs (e.g. warehouse: `databricks warehouses list --profile ` or `databricks experimental aitools tools get-default-warehouse --profile `). + - **Discovery**: Use the parent `databricks-core` skill to resolve IDs (e.g. warehouse: `databricks warehouses list --profile ` or `databricks aitools tools get-default-warehouse --profile `). **DO NOT guess** plugin names, resource keys, or property names — always derive them from `databricks apps manifest` output. Example: if the manifest shows plugin `analytics` with a required resource `resourceKey: "sql-warehouse"` and `fields: { "id": ... }`, include `--set analytics.sql-warehouse.id=`. diff --git a/.agents/skills/databricks-core/SKILL.md b/.agents/skills/databricks-core/SKILL.md index 79a2465..a469aa5 100644 --- a/.agents/skills/databricks-core/SKILL.md +++ b/.agents/skills/databricks-core/SKILL.md @@ -60,13 +60,13 @@ databricks apps list # profile not set! ```bash # discover table structure (columns, types, sample data, stats) -databricks experimental aitools tools discover-schema catalog.schema.table --profile +databricks aitools tools discover-schema catalog.schema.table --profile # run ad-hoc SQL queries -databricks experimental aitools tools query "SELECT * FROM table LIMIT 10" --profile +databricks aitools tools query "SELECT * FROM table LIMIT 10" --profile # find the default warehouse -databricks experimental aitools tools get-default-warehouse --profile +databricks aitools tools get-default-warehouse --profile ``` See [Data Exploration](data-exploration.md) for details. @@ -99,8 +99,8 @@ databricks tables get .. --profile # databricks schemas list --catalog-name ← WILL FAIL # databricks tables list --catalog ← WILL FAIL # databricks sql-warehouses list ← doesn't exist, use `warehouses list` -# databricks execute-statement ← doesn't exist, use `experimental aitools tools query` -# databricks sql execute ← doesn't exist, use `experimental aitools tools query` +# databricks execute-statement ← doesn't exist, use `aitools tools query` +# databricks sql execute ← doesn't exist, use `aitools tools query` # When in doubt, check help: # databricks schemas list --help diff --git a/.agents/skills/databricks-core/data-exploration.md b/.agents/skills/databricks-core/data-exploration.md index ad2d3ed..5fdfc63 100644 --- a/.agents/skills/databricks-core/data-exploration.md +++ b/.agents/skills/databricks-core/data-exploration.md @@ -10,17 +10,17 @@ Use `information_schema` to search for tables by keyword — do NOT manually ite ```bash # Find tables matching a keyword -databricks experimental aitools tools query \ +databricks aitools tools query \ "SELECT table_catalog, table_schema, table_name FROM system.information_schema.tables WHERE table_name LIKE '%keyword%'" \ --profile # Then discover schema for the tables you found -databricks experimental aitools tools discover-schema catalog.schema.table1 catalog.schema.table2 --profile +databricks aitools tools discover-schema catalog.schema.table1 catalog.schema.table2 --profile ``` ## Overview -The `databricks experimental aitools tools` command group provides tools for data discovery and exploration: +The `databricks aitools tools` command group provides tools for data discovery and exploration: - **discover-schema**: Batch discover table metadata, columns, types, sample data, and statistics - **query**: Execute SQL queries against Databricks SQL warehouses @@ -45,7 +45,7 @@ Batch discover table metadata including columns, types, sample data, and null co ### Command Syntax ```bash -databricks experimental aitools tools discover-schema TABLE... [flags] +databricks aitools tools discover-schema TABLE... [flags] ``` Tables must be specified in **CATALOG.SCHEMA.TABLE** format. @@ -63,16 +63,16 @@ For each table, returns: ```bash # Discover schema for a single table -databricks experimental aitools tools discover-schema samples.nyctaxi.trips --profile my-workspace +databricks aitools tools discover-schema samples.nyctaxi.trips --profile my-workspace # Discover schema for multiple tables -databricks experimental aitools tools discover-schema \ +databricks aitools tools discover-schema \ catalog.schema.table1 \ catalog.schema.table2 \ --profile my-workspace # Get JSON output -databricks experimental aitools tools discover-schema \ +databricks aitools tools discover-schema \ samples.nyctaxi.trips \ --output json \ --profile my-workspace @@ -83,13 +83,13 @@ databricks experimental aitools tools discover-schema \ 1. **Understanding table structure before querying** ```bash - databricks experimental aitools tools discover-schema catalog.schema.customer_data --profile my-workspace + databricks aitools tools discover-schema catalog.schema.customer_data --profile my-workspace ``` 2. **Comparing schemas across multiple tables** ```bash - databricks experimental aitools tools discover-schema \ + databricks aitools tools discover-schema \ catalog.schema.table_v1 \ catalog.schema.table_v2 \ --profile my-workspace @@ -105,7 +105,7 @@ Execute SQL statements against a Databricks SQL warehouse and return results. ### Command Syntax ```bash -databricks experimental aitools tools query "SQL" [flags] +databricks aitools tools query "SQL" [flags] ``` ### Warehouse Selection @@ -119,7 +119,7 @@ To check which warehouse will be used: ```bash # Get the default warehouse that would be auto-detected -databricks experimental aitools tools get-default-warehouse --profile my-workspace +databricks aitools tools get-default-warehouse --profile my-workspace ``` ### Output @@ -134,23 +134,23 @@ Returns: ```bash # Simple SELECT query -databricks experimental aitools tools query \ +databricks aitools tools query \ "SELECT * FROM samples.nyctaxi.trips LIMIT 5" \ --profile my-workspace # Aggregation query -databricks experimental aitools tools query \ +databricks aitools tools query \ "SELECT vendor_id, COUNT(*) as trip_count FROM samples.nyctaxi.trips GROUP BY vendor_id" \ --profile my-workspace # With JSON output -databricks experimental aitools tools query \ +databricks aitools tools query \ "SELECT * FROM catalog.schema.table WHERE date > '2024-01-01'" \ --output json \ --profile my-workspace # Using specific warehouse -DATABRICKS_WAREHOUSE_ID=abc123 databricks experimental aitools tools query \ +DATABRICKS_WAREHOUSE_ID=abc123 databricks aitools tools query \ "SELECT * FROM samples.nyctaxi.trips LIMIT 10" \ --profile my-workspace ``` @@ -161,17 +161,17 @@ DATABRICKS_WAREHOUSE_ID=abc123 databricks experimental aitools tools query \ ```bash # Check table size - databricks experimental aitools tools query \ + databricks aitools tools query \ "SELECT COUNT(*) FROM catalog.schema.table" \ --profile my-workspace # View sample data - databricks experimental aitools tools query \ + databricks aitools tools query \ "SELECT * FROM catalog.schema.table LIMIT 10" \ --profile my-workspace # Get column statistics - databricks experimental aitools tools query \ + databricks aitools tools query \ "SELECT MIN(column), MAX(column), AVG(column) FROM catalog.schema.table" \ --profile my-workspace ``` @@ -180,12 +180,12 @@ DATABRICKS_WAREHOUSE_ID=abc123 databricks experimental aitools tools query \ ```bash # Check for null values - databricks experimental aitools tools query \ + databricks aitools tools query \ "SELECT COUNT(*) FROM catalog.schema.table WHERE column IS NULL" \ --profile my-workspace # Verify data freshness - databricks experimental aitools tools query \ + databricks aitools tools query \ "SELECT MAX(timestamp_column) FROM catalog.schema.table" \ --profile my-workspace ``` @@ -193,7 +193,7 @@ DATABRICKS_WAREHOUSE_ID=abc123 databricks experimental aitools tools query \ 3. **Quick analytics** ```bash # Group by analysis - databricks experimental aitools tools query \ + databricks aitools tools query \ "SELECT category, COUNT(*), AVG(value) FROM catalog.schema.table GROUP BY category" \ --profile my-workspace ``` @@ -204,12 +204,12 @@ Here's a typical workflow combining both commands: ```bash # 1. Discover the schema first -databricks experimental aitools tools discover-schema \ +databricks aitools tools discover-schema \ samples.nyctaxi.trips \ --profile my-workspace # 2. Based on discovered columns, run targeted queries -databricks experimental aitools tools query \ +databricks aitools tools query \ "SELECT vendor_id, payment_type, COUNT(*) as trips, AVG(fare_amount) as avg_fare FROM samples.nyctaxi.trips GROUP BY vendor_id, payment_type @@ -218,7 +218,7 @@ databricks experimental aitools tools query \ --profile my-workspace # 3. Investigate specific patterns found in the data -databricks experimental aitools tools query \ +databricks aitools tools query \ "SELECT * FROM samples.nyctaxi.trips WHERE fare_amount > 100 LIMIT 20" \ @@ -231,15 +231,15 @@ Remember that each Bash command in Claude Code runs in a separate shell: ```bash # ✅ RECOMMENDED: Use --profile flag -databricks experimental aitools tools discover-schema samples.nyctaxi.trips --profile my-workspace +databricks aitools tools discover-schema samples.nyctaxi.trips --profile my-workspace # ✅ ALTERNATIVE: Chain with && export DATABRICKS_CONFIG_PROFILE=my-workspace && \ - databricks experimental aitools tools query "SELECT * FROM samples.nyctaxi.trips LIMIT 5" + databricks aitools tools query "SELECT * FROM samples.nyctaxi.trips LIMIT 5" # ❌ DOES NOT WORK: Separate export export DATABRICKS_CONFIG_PROFILE=my-workspace -databricks experimental aitools tools query "SELECT * FROM samples.nyctaxi.trips LIMIT 5" +databricks aitools tools query "SELECT * FROM samples.nyctaxi.trips LIMIT 5" ``` ## Flags @@ -276,7 +276,7 @@ Both commands support: 1. Check for default warehouse: ```bash - databricks experimental aitools tools get-default-warehouse --profile my-workspace + databricks aitools tools get-default-warehouse --profile my-workspace ``` 2. List available warehouses: ```bash @@ -284,7 +284,7 @@ Both commands support: ``` 3. Set specific warehouse: ```bash - DATABRICKS_WAREHOUSE_ID= databricks experimental aitools tools query "SELECT 1" --profile my-workspace + DATABRICKS_WAREHOUSE_ID= databricks aitools tools query "SELECT 1" --profile my-workspace ``` 4. Start a stopped warehouse: ```bash @@ -325,13 +325,13 @@ Both commands support: 2. **Use LIMIT for exploration** - When exploring large tables, always use LIMIT to avoid long-running queries: ```bash - databricks experimental aitools tools query "SELECT * FROM large_table LIMIT 100" --profile my-workspace + databricks aitools tools query "SELECT * FROM large_table LIMIT 100" --profile my-workspace ``` 3. **JSON output for parsing** - Use `--output json` when you need to process results programmatically: ```bash - databricks experimental aitools tools query "SELECT * FROM table" --output json --profile my-workspace | jq '.results' + databricks aitools tools query "SELECT * FROM table" --output json --profile my-workspace | jq '.results' ``` 4. **Check table existence** - Before querying, verify the table exists: diff --git a/.agents/skills/databricks-jobs/SKILL.md b/.agents/skills/databricks-jobs/SKILL.md index 2d4b87b..c7565b8 100644 --- a/.agents/skills/databricks-jobs/SKILL.md +++ b/.agents/skills/databricks-jobs/SKILL.md @@ -44,7 +44,7 @@ Verify: `databricks -v` Read the `databricks-core` skill for CLI basics, authentication, and deployment workflow. Read the `databricks-jobs` skill for job-specific guidance. -If skills are not available, install them: `databricks experimental aitools skills install` +If skills are not available, install them: `databricks aitools install` ``` ## Project Structure diff --git a/.agents/skills/databricks-pipelines/SKILL.md b/.agents/skills/databricks-pipelines/SKILL.md index 5c907e0..88bbd3b 100644 --- a/.agents/skills/databricks-pipelines/SKILL.md +++ b/.agents/skills/databricks-pipelines/SKILL.md @@ -201,7 +201,7 @@ Verify: `databricks -v` Read the `databricks-core` skill for CLI basics, authentication, and deployment workflow. Read the `databricks-pipelines` skill for pipeline-specific guidance. -If skills are not available, install them: `databricks experimental aitools skills install` +If skills are not available, install them: `databricks aitools install` ``` ## Pipeline Structure diff --git a/.agents/skills/databricks/SKILL.md b/.agents/skills/databricks/SKILL.md index 5b25bbc..b849379 100644 --- a/.agents/skills/databricks/SKILL.md +++ b/.agents/skills/databricks/SKILL.md @@ -60,13 +60,13 @@ databricks apps list # profile not set! ```bash # discover table structure (columns, types, sample data, stats) -databricks experimental aitools tools discover-schema catalog.schema.table --profile +databricks aitools tools discover-schema catalog.schema.table --profile # run ad-hoc SQL queries -databricks experimental aitools tools query "SELECT * FROM table LIMIT 10" --profile +databricks aitools tools query "SELECT * FROM table LIMIT 10" --profile # find the default warehouse -databricks experimental aitools tools get-default-warehouse --profile +databricks aitools tools get-default-warehouse --profile ``` See [Data Exploration](data-exploration.md) for details. @@ -99,8 +99,8 @@ databricks tables get ..
--profile # databricks schemas list --catalog-name ← WILL FAIL # databricks tables list --catalog ← WILL FAIL # databricks sql-warehouses list ← doesn't exist, use `warehouses list` -# databricks execute-statement ← doesn't exist, use `experimental aitools tools query` -# databricks sql execute ← doesn't exist, use `experimental aitools tools query` +# databricks execute-statement ← doesn't exist, use `aitools tools query` +# databricks sql execute ← doesn't exist, use `aitools tools query` # When in doubt, check help: # databricks schemas list --help diff --git a/.agents/skills/databricks/data-exploration.md b/.agents/skills/databricks/data-exploration.md index 1c4a715..2b05479 100644 --- a/.agents/skills/databricks/data-exploration.md +++ b/.agents/skills/databricks/data-exploration.md @@ -10,17 +10,17 @@ Use `information_schema` to search for tables by keyword — do NOT manually ite ```bash # Find tables matching a keyword -databricks experimental aitools tools query \ +databricks aitools tools query \ "SELECT table_catalog, table_schema, table_name FROM system.information_schema.tables WHERE table_name LIKE '%keyword%'" \ --profile # Then discover schema for the tables you found -databricks experimental aitools tools discover-schema catalog.schema.table1 catalog.schema.table2 --profile +databricks aitools tools discover-schema catalog.schema.table1 catalog.schema.table2 --profile ``` ## Overview -The `databricks experimental aitools tools` command group provides tools for data discovery and exploration: +The `databricks aitools tools` command group provides tools for data discovery and exploration: - **discover-schema**: Batch discover table metadata, columns, types, sample data, and statistics - **query**: Execute SQL queries against Databricks SQL warehouses @@ -45,7 +45,7 @@ Batch discover table metadata including columns, types, sample data, and null co ### Command Syntax ```bash -databricks experimental aitools tools discover-schema TABLE... [flags] +databricks aitools tools discover-schema TABLE... [flags] ``` Tables must be specified in **CATALOG.SCHEMA.TABLE** format. @@ -63,16 +63,16 @@ For each table, returns: ```bash # Discover schema for a single table -databricks experimental aitools tools discover-schema samples.nyctaxi.trips --profile my-workspace +databricks aitools tools discover-schema samples.nyctaxi.trips --profile my-workspace # Discover schema for multiple tables -databricks experimental aitools tools discover-schema \ +databricks aitools tools discover-schema \ catalog.schema.table1 \ catalog.schema.table2 \ --profile my-workspace # Get JSON output -databricks experimental aitools tools discover-schema \ +databricks aitools tools discover-schema \ samples.nyctaxi.trips \ --output json \ --profile my-workspace @@ -83,13 +83,13 @@ databricks experimental aitools tools discover-schema \ 1. **Understanding table structure before querying** ```bash - databricks experimental aitools tools discover-schema catalog.schema.customer_data --profile my-workspace + databricks aitools tools discover-schema catalog.schema.customer_data --profile my-workspace ``` 2. **Comparing schemas across multiple tables** ```bash - databricks experimental aitools tools discover-schema \ + databricks aitools tools discover-schema \ catalog.schema.table_v1 \ catalog.schema.table_v2 \ --profile my-workspace @@ -105,7 +105,7 @@ Execute SQL statements against a Databricks SQL warehouse and return results. ### Command Syntax ```bash -databricks experimental aitools tools query "SQL" [flags] +databricks aitools tools query "SQL" [flags] ``` ### Warehouse Selection @@ -119,7 +119,7 @@ To check which warehouse will be used: ```bash # Get the default warehouse that would be auto-detected -databricks experimental aitools tools get-default-warehouse --profile my-workspace +databricks aitools tools get-default-warehouse --profile my-workspace ``` ### Output @@ -134,23 +134,23 @@ Returns: ```bash # Simple SELECT query -databricks experimental aitools tools query \ +databricks aitools tools query \ "SELECT * FROM samples.nyctaxi.trips LIMIT 5" \ --profile my-workspace # Aggregation query -databricks experimental aitools tools query \ +databricks aitools tools query \ "SELECT vendor_id, COUNT(*) as trip_count FROM samples.nyctaxi.trips GROUP BY vendor_id" \ --profile my-workspace # With JSON output -databricks experimental aitools tools query \ +databricks aitools tools query \ "SELECT * FROM catalog.schema.table WHERE date > '2024-01-01'" \ --output json \ --profile my-workspace # Using specific warehouse -DATABRICKS_WAREHOUSE_ID=abc123 databricks experimental aitools tools query \ +DATABRICKS_WAREHOUSE_ID=abc123 databricks aitools tools query \ "SELECT * FROM samples.nyctaxi.trips LIMIT 10" \ --profile my-workspace ``` @@ -161,17 +161,17 @@ DATABRICKS_WAREHOUSE_ID=abc123 databricks experimental aitools tools query \ ```bash # Check table size - databricks experimental aitools tools query \ + databricks aitools tools query \ "SELECT COUNT(*) FROM catalog.schema.table" \ --profile my-workspace # View sample data - databricks experimental aitools tools query \ + databricks aitools tools query \ "SELECT * FROM catalog.schema.table LIMIT 10" \ --profile my-workspace # Get column statistics - databricks experimental aitools tools query \ + databricks aitools tools query \ "SELECT MIN(column), MAX(column), AVG(column) FROM catalog.schema.table" \ --profile my-workspace ``` @@ -180,12 +180,12 @@ DATABRICKS_WAREHOUSE_ID=abc123 databricks experimental aitools tools query \ ```bash # Check for null values - databricks experimental aitools tools query \ + databricks aitools tools query \ "SELECT COUNT(*) FROM catalog.schema.table WHERE column IS NULL" \ --profile my-workspace # Verify data freshness - databricks experimental aitools tools query \ + databricks aitools tools query \ "SELECT MAX(timestamp_column) FROM catalog.schema.table" \ --profile my-workspace ``` @@ -193,7 +193,7 @@ DATABRICKS_WAREHOUSE_ID=abc123 databricks experimental aitools tools query \ 3. **Quick analytics** ```bash # Group by analysis - databricks experimental aitools tools query \ + databricks aitools tools query \ "SELECT category, COUNT(*), AVG(value) FROM catalog.schema.table GROUP BY category" \ --profile my-workspace ``` @@ -204,12 +204,12 @@ Here's a typical workflow combining both commands: ```bash # 1. Discover the schema first -databricks experimental aitools tools discover-schema \ +databricks aitools tools discover-schema \ samples.nyctaxi.trips \ --profile my-workspace # 2. Based on discovered columns, run targeted queries -databricks experimental aitools tools query \ +databricks aitools tools query \ "SELECT vendor_id, payment_type, COUNT(*) as trips, AVG(fare_amount) as avg_fare FROM samples.nyctaxi.trips GROUP BY vendor_id, payment_type @@ -218,7 +218,7 @@ databricks experimental aitools tools query \ --profile my-workspace # 3. Investigate specific patterns found in the data -databricks experimental aitools tools query \ +databricks aitools tools query \ "SELECT * FROM samples.nyctaxi.trips WHERE fare_amount > 100 LIMIT 20" \ @@ -231,15 +231,15 @@ Remember that each Bash command in Claude Code runs in a separate shell: ```bash # ✅ RECOMMENDED: Use --profile flag -databricks experimental aitools tools discover-schema samples.nyctaxi.trips --profile my-workspace +databricks aitools tools discover-schema samples.nyctaxi.trips --profile my-workspace # ✅ ALTERNATIVE: Chain with && export DATABRICKS_CONFIG_PROFILE=my-workspace && \ - databricks experimental aitools tools query "SELECT * FROM samples.nyctaxi.trips LIMIT 5" + databricks aitools tools query "SELECT * FROM samples.nyctaxi.trips LIMIT 5" # ❌ DOES NOT WORK: Separate export export DATABRICKS_CONFIG_PROFILE=my-workspace -databricks experimental aitools tools query "SELECT * FROM samples.nyctaxi.trips LIMIT 5" +databricks aitools tools query "SELECT * FROM samples.nyctaxi.trips LIMIT 5" ``` ## Flags @@ -276,7 +276,7 @@ Both commands support: 1. Check for default warehouse: ```bash - databricks experimental aitools tools get-default-warehouse --profile my-workspace + databricks aitools tools get-default-warehouse --profile my-workspace ``` 2. List available warehouses: ```bash @@ -284,7 +284,7 @@ Both commands support: ``` 3. Set specific warehouse: ```bash - DATABRICKS_WAREHOUSE_ID= databricks experimental aitools tools query "SELECT 1" --profile my-workspace + DATABRICKS_WAREHOUSE_ID= databricks aitools tools query "SELECT 1" --profile my-workspace ``` 4. Start a stopped warehouse: ```bash @@ -325,13 +325,13 @@ Both commands support: 2. **Use LIMIT for exploration** - When exploring large tables, always use LIMIT to avoid long-running queries: ```bash - databricks experimental aitools tools query "SELECT * FROM large_table LIMIT 100" --profile my-workspace + databricks aitools tools query "SELECT * FROM large_table LIMIT 100" --profile my-workspace ``` 3. **JSON output for parsing** - Use `--output json` when you need to process results programmatically: ```bash - databricks experimental aitools tools query "SELECT * FROM table" --output json --profile my-workspace | jq '.results' + databricks aitools tools query "SELECT * FROM table" --output json --profile my-workspace | jq '.results' ``` 4. **Check table existence** - Before querying, verify the table exists: From 68fdd558afe73fe0ad5d9d944d92b983c926acfa Mon Sep 17 00:00:00 2001 From: James Broadhead Date: Wed, 27 May 2026 20:22:06 +0000 Subject: [PATCH 2/2] skills: keep `experimental aitools tools` refs (only install was promoted) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Only the skills-management commands (`install`, `update`, `uninstall`, `list`, `version`) were promoted to top-level `databricks aitools`. `tools query` / `tools discover-schema` / `tools get-default-warehouse` / `tools statement …` are still only registered under `databricks experimental aitools tools` (see databricks/cli/cmd/aitools/aitools.go). Revert the over-eager renames; keep the `databricks aitools install` rename which is correct. Co-authored-by: Isaac --- .agents/skills/databricks-apps/SKILL.md | 2 +- .agents/skills/databricks-core/SKILL.md | 10 +-- .../databricks-core/data-exploration.md | 62 +++++++++---------- .agents/skills/databricks/SKILL.md | 10 +-- .agents/skills/databricks/data-exploration.md | 62 +++++++++---------- 5 files changed, 73 insertions(+), 73 deletions(-) diff --git a/.agents/skills/databricks-apps/SKILL.md b/.agents/skills/databricks-apps/SKILL.md index 23da19c..4a26d97 100644 --- a/.agents/skills/databricks-apps/SKILL.md +++ b/.agents/skills/databricks-apps/SKILL.md @@ -129,7 +129,7 @@ npx @databricks/appkit docs ./docs/plugins/analytics.md # example: specific doc Optionally use `--version ` to target a specific AppKit version. - **Required**: `--name`, `--profile`. Name: ≤26 chars, lowercase letters/numbers/hyphens only. Use `--features` only for **optional** plugins the user wants (plugins with `requiredByTemplate: false` or absent); mandatory plugins must not be listed in `--features`. - **Resources**: Pass `--set` for every required resource (each field in `resources.required`) for (1) all plugins with `requiredByTemplate: true`, and (2) any optional plugins you added to `--features`. Add `--set` for `resources.optional` only when the user requests them. - - **Discovery**: Use the parent `databricks-core` skill to resolve IDs (e.g. warehouse: `databricks warehouses list --profile ` or `databricks aitools tools get-default-warehouse --profile `). + - **Discovery**: Use the parent `databricks-core` skill to resolve IDs (e.g. warehouse: `databricks warehouses list --profile ` or `databricks experimental aitools tools get-default-warehouse --profile `). **DO NOT guess** plugin names, resource keys, or property names — always derive them from `databricks apps manifest` output. Example: if the manifest shows plugin `analytics` with a required resource `resourceKey: "sql-warehouse"` and `fields: { "id": ... }`, include `--set analytics.sql-warehouse.id=`. diff --git a/.agents/skills/databricks-core/SKILL.md b/.agents/skills/databricks-core/SKILL.md index a469aa5..79a2465 100644 --- a/.agents/skills/databricks-core/SKILL.md +++ b/.agents/skills/databricks-core/SKILL.md @@ -60,13 +60,13 @@ databricks apps list # profile not set! ```bash # discover table structure (columns, types, sample data, stats) -databricks aitools tools discover-schema catalog.schema.table --profile +databricks experimental aitools tools discover-schema catalog.schema.table --profile # run ad-hoc SQL queries -databricks aitools tools query "SELECT * FROM table LIMIT 10" --profile +databricks experimental aitools tools query "SELECT * FROM table LIMIT 10" --profile # find the default warehouse -databricks aitools tools get-default-warehouse --profile +databricks experimental aitools tools get-default-warehouse --profile ``` See [Data Exploration](data-exploration.md) for details. @@ -99,8 +99,8 @@ databricks tables get ..
--profile # databricks schemas list --catalog-name ← WILL FAIL # databricks tables list --catalog ← WILL FAIL # databricks sql-warehouses list ← doesn't exist, use `warehouses list` -# databricks execute-statement ← doesn't exist, use `aitools tools query` -# databricks sql execute ← doesn't exist, use `aitools tools query` +# databricks execute-statement ← doesn't exist, use `experimental aitools tools query` +# databricks sql execute ← doesn't exist, use `experimental aitools tools query` # When in doubt, check help: # databricks schemas list --help diff --git a/.agents/skills/databricks-core/data-exploration.md b/.agents/skills/databricks-core/data-exploration.md index 5fdfc63..ad2d3ed 100644 --- a/.agents/skills/databricks-core/data-exploration.md +++ b/.agents/skills/databricks-core/data-exploration.md @@ -10,17 +10,17 @@ Use `information_schema` to search for tables by keyword — do NOT manually ite ```bash # Find tables matching a keyword -databricks aitools tools query \ +databricks experimental aitools tools query \ "SELECT table_catalog, table_schema, table_name FROM system.information_schema.tables WHERE table_name LIKE '%keyword%'" \ --profile # Then discover schema for the tables you found -databricks aitools tools discover-schema catalog.schema.table1 catalog.schema.table2 --profile +databricks experimental aitools tools discover-schema catalog.schema.table1 catalog.schema.table2 --profile ``` ## Overview -The `databricks aitools tools` command group provides tools for data discovery and exploration: +The `databricks experimental aitools tools` command group provides tools for data discovery and exploration: - **discover-schema**: Batch discover table metadata, columns, types, sample data, and statistics - **query**: Execute SQL queries against Databricks SQL warehouses @@ -45,7 +45,7 @@ Batch discover table metadata including columns, types, sample data, and null co ### Command Syntax ```bash -databricks aitools tools discover-schema TABLE... [flags] +databricks experimental aitools tools discover-schema TABLE... [flags] ``` Tables must be specified in **CATALOG.SCHEMA.TABLE** format. @@ -63,16 +63,16 @@ For each table, returns: ```bash # Discover schema for a single table -databricks aitools tools discover-schema samples.nyctaxi.trips --profile my-workspace +databricks experimental aitools tools discover-schema samples.nyctaxi.trips --profile my-workspace # Discover schema for multiple tables -databricks aitools tools discover-schema \ +databricks experimental aitools tools discover-schema \ catalog.schema.table1 \ catalog.schema.table2 \ --profile my-workspace # Get JSON output -databricks aitools tools discover-schema \ +databricks experimental aitools tools discover-schema \ samples.nyctaxi.trips \ --output json \ --profile my-workspace @@ -83,13 +83,13 @@ databricks aitools tools discover-schema \ 1. **Understanding table structure before querying** ```bash - databricks aitools tools discover-schema catalog.schema.customer_data --profile my-workspace + databricks experimental aitools tools discover-schema catalog.schema.customer_data --profile my-workspace ``` 2. **Comparing schemas across multiple tables** ```bash - databricks aitools tools discover-schema \ + databricks experimental aitools tools discover-schema \ catalog.schema.table_v1 \ catalog.schema.table_v2 \ --profile my-workspace @@ -105,7 +105,7 @@ Execute SQL statements against a Databricks SQL warehouse and return results. ### Command Syntax ```bash -databricks aitools tools query "SQL" [flags] +databricks experimental aitools tools query "SQL" [flags] ``` ### Warehouse Selection @@ -119,7 +119,7 @@ To check which warehouse will be used: ```bash # Get the default warehouse that would be auto-detected -databricks aitools tools get-default-warehouse --profile my-workspace +databricks experimental aitools tools get-default-warehouse --profile my-workspace ``` ### Output @@ -134,23 +134,23 @@ Returns: ```bash # Simple SELECT query -databricks aitools tools query \ +databricks experimental aitools tools query \ "SELECT * FROM samples.nyctaxi.trips LIMIT 5" \ --profile my-workspace # Aggregation query -databricks aitools tools query \ +databricks experimental aitools tools query \ "SELECT vendor_id, COUNT(*) as trip_count FROM samples.nyctaxi.trips GROUP BY vendor_id" \ --profile my-workspace # With JSON output -databricks aitools tools query \ +databricks experimental aitools tools query \ "SELECT * FROM catalog.schema.table WHERE date > '2024-01-01'" \ --output json \ --profile my-workspace # Using specific warehouse -DATABRICKS_WAREHOUSE_ID=abc123 databricks aitools tools query \ +DATABRICKS_WAREHOUSE_ID=abc123 databricks experimental aitools tools query \ "SELECT * FROM samples.nyctaxi.trips LIMIT 10" \ --profile my-workspace ``` @@ -161,17 +161,17 @@ DATABRICKS_WAREHOUSE_ID=abc123 databricks aitools tools query \ ```bash # Check table size - databricks aitools tools query \ + databricks experimental aitools tools query \ "SELECT COUNT(*) FROM catalog.schema.table" \ --profile my-workspace # View sample data - databricks aitools tools query \ + databricks experimental aitools tools query \ "SELECT * FROM catalog.schema.table LIMIT 10" \ --profile my-workspace # Get column statistics - databricks aitools tools query \ + databricks experimental aitools tools query \ "SELECT MIN(column), MAX(column), AVG(column) FROM catalog.schema.table" \ --profile my-workspace ``` @@ -180,12 +180,12 @@ DATABRICKS_WAREHOUSE_ID=abc123 databricks aitools tools query \ ```bash # Check for null values - databricks aitools tools query \ + databricks experimental aitools tools query \ "SELECT COUNT(*) FROM catalog.schema.table WHERE column IS NULL" \ --profile my-workspace # Verify data freshness - databricks aitools tools query \ + databricks experimental aitools tools query \ "SELECT MAX(timestamp_column) FROM catalog.schema.table" \ --profile my-workspace ``` @@ -193,7 +193,7 @@ DATABRICKS_WAREHOUSE_ID=abc123 databricks aitools tools query \ 3. **Quick analytics** ```bash # Group by analysis - databricks aitools tools query \ + databricks experimental aitools tools query \ "SELECT category, COUNT(*), AVG(value) FROM catalog.schema.table GROUP BY category" \ --profile my-workspace ``` @@ -204,12 +204,12 @@ Here's a typical workflow combining both commands: ```bash # 1. Discover the schema first -databricks aitools tools discover-schema \ +databricks experimental aitools tools discover-schema \ samples.nyctaxi.trips \ --profile my-workspace # 2. Based on discovered columns, run targeted queries -databricks aitools tools query \ +databricks experimental aitools tools query \ "SELECT vendor_id, payment_type, COUNT(*) as trips, AVG(fare_amount) as avg_fare FROM samples.nyctaxi.trips GROUP BY vendor_id, payment_type @@ -218,7 +218,7 @@ databricks aitools tools query \ --profile my-workspace # 3. Investigate specific patterns found in the data -databricks aitools tools query \ +databricks experimental aitools tools query \ "SELECT * FROM samples.nyctaxi.trips WHERE fare_amount > 100 LIMIT 20" \ @@ -231,15 +231,15 @@ Remember that each Bash command in Claude Code runs in a separate shell: ```bash # ✅ RECOMMENDED: Use --profile flag -databricks aitools tools discover-schema samples.nyctaxi.trips --profile my-workspace +databricks experimental aitools tools discover-schema samples.nyctaxi.trips --profile my-workspace # ✅ ALTERNATIVE: Chain with && export DATABRICKS_CONFIG_PROFILE=my-workspace && \ - databricks aitools tools query "SELECT * FROM samples.nyctaxi.trips LIMIT 5" + databricks experimental aitools tools query "SELECT * FROM samples.nyctaxi.trips LIMIT 5" # ❌ DOES NOT WORK: Separate export export DATABRICKS_CONFIG_PROFILE=my-workspace -databricks aitools tools query "SELECT * FROM samples.nyctaxi.trips LIMIT 5" +databricks experimental aitools tools query "SELECT * FROM samples.nyctaxi.trips LIMIT 5" ``` ## Flags @@ -276,7 +276,7 @@ Both commands support: 1. Check for default warehouse: ```bash - databricks aitools tools get-default-warehouse --profile my-workspace + databricks experimental aitools tools get-default-warehouse --profile my-workspace ``` 2. List available warehouses: ```bash @@ -284,7 +284,7 @@ Both commands support: ``` 3. Set specific warehouse: ```bash - DATABRICKS_WAREHOUSE_ID= databricks aitools tools query "SELECT 1" --profile my-workspace + DATABRICKS_WAREHOUSE_ID= databricks experimental aitools tools query "SELECT 1" --profile my-workspace ``` 4. Start a stopped warehouse: ```bash @@ -325,13 +325,13 @@ Both commands support: 2. **Use LIMIT for exploration** - When exploring large tables, always use LIMIT to avoid long-running queries: ```bash - databricks aitools tools query "SELECT * FROM large_table LIMIT 100" --profile my-workspace + databricks experimental aitools tools query "SELECT * FROM large_table LIMIT 100" --profile my-workspace ``` 3. **JSON output for parsing** - Use `--output json` when you need to process results programmatically: ```bash - databricks aitools tools query "SELECT * FROM table" --output json --profile my-workspace | jq '.results' + databricks experimental aitools tools query "SELECT * FROM table" --output json --profile my-workspace | jq '.results' ``` 4. **Check table existence** - Before querying, verify the table exists: diff --git a/.agents/skills/databricks/SKILL.md b/.agents/skills/databricks/SKILL.md index b849379..5b25bbc 100644 --- a/.agents/skills/databricks/SKILL.md +++ b/.agents/skills/databricks/SKILL.md @@ -60,13 +60,13 @@ databricks apps list # profile not set! ```bash # discover table structure (columns, types, sample data, stats) -databricks aitools tools discover-schema catalog.schema.table --profile +databricks experimental aitools tools discover-schema catalog.schema.table --profile # run ad-hoc SQL queries -databricks aitools tools query "SELECT * FROM table LIMIT 10" --profile +databricks experimental aitools tools query "SELECT * FROM table LIMIT 10" --profile # find the default warehouse -databricks aitools tools get-default-warehouse --profile +databricks experimental aitools tools get-default-warehouse --profile ``` See [Data Exploration](data-exploration.md) for details. @@ -99,8 +99,8 @@ databricks tables get ..
--profile # databricks schemas list --catalog-name ← WILL FAIL # databricks tables list --catalog ← WILL FAIL # databricks sql-warehouses list ← doesn't exist, use `warehouses list` -# databricks execute-statement ← doesn't exist, use `aitools tools query` -# databricks sql execute ← doesn't exist, use `aitools tools query` +# databricks execute-statement ← doesn't exist, use `experimental aitools tools query` +# databricks sql execute ← doesn't exist, use `experimental aitools tools query` # When in doubt, check help: # databricks schemas list --help diff --git a/.agents/skills/databricks/data-exploration.md b/.agents/skills/databricks/data-exploration.md index 2b05479..1c4a715 100644 --- a/.agents/skills/databricks/data-exploration.md +++ b/.agents/skills/databricks/data-exploration.md @@ -10,17 +10,17 @@ Use `information_schema` to search for tables by keyword — do NOT manually ite ```bash # Find tables matching a keyword -databricks aitools tools query \ +databricks experimental aitools tools query \ "SELECT table_catalog, table_schema, table_name FROM system.information_schema.tables WHERE table_name LIKE '%keyword%'" \ --profile # Then discover schema for the tables you found -databricks aitools tools discover-schema catalog.schema.table1 catalog.schema.table2 --profile +databricks experimental aitools tools discover-schema catalog.schema.table1 catalog.schema.table2 --profile ``` ## Overview -The `databricks aitools tools` command group provides tools for data discovery and exploration: +The `databricks experimental aitools tools` command group provides tools for data discovery and exploration: - **discover-schema**: Batch discover table metadata, columns, types, sample data, and statistics - **query**: Execute SQL queries against Databricks SQL warehouses @@ -45,7 +45,7 @@ Batch discover table metadata including columns, types, sample data, and null co ### Command Syntax ```bash -databricks aitools tools discover-schema TABLE... [flags] +databricks experimental aitools tools discover-schema TABLE... [flags] ``` Tables must be specified in **CATALOG.SCHEMA.TABLE** format. @@ -63,16 +63,16 @@ For each table, returns: ```bash # Discover schema for a single table -databricks aitools tools discover-schema samples.nyctaxi.trips --profile my-workspace +databricks experimental aitools tools discover-schema samples.nyctaxi.trips --profile my-workspace # Discover schema for multiple tables -databricks aitools tools discover-schema \ +databricks experimental aitools tools discover-schema \ catalog.schema.table1 \ catalog.schema.table2 \ --profile my-workspace # Get JSON output -databricks aitools tools discover-schema \ +databricks experimental aitools tools discover-schema \ samples.nyctaxi.trips \ --output json \ --profile my-workspace @@ -83,13 +83,13 @@ databricks aitools tools discover-schema \ 1. **Understanding table structure before querying** ```bash - databricks aitools tools discover-schema catalog.schema.customer_data --profile my-workspace + databricks experimental aitools tools discover-schema catalog.schema.customer_data --profile my-workspace ``` 2. **Comparing schemas across multiple tables** ```bash - databricks aitools tools discover-schema \ + databricks experimental aitools tools discover-schema \ catalog.schema.table_v1 \ catalog.schema.table_v2 \ --profile my-workspace @@ -105,7 +105,7 @@ Execute SQL statements against a Databricks SQL warehouse and return results. ### Command Syntax ```bash -databricks aitools tools query "SQL" [flags] +databricks experimental aitools tools query "SQL" [flags] ``` ### Warehouse Selection @@ -119,7 +119,7 @@ To check which warehouse will be used: ```bash # Get the default warehouse that would be auto-detected -databricks aitools tools get-default-warehouse --profile my-workspace +databricks experimental aitools tools get-default-warehouse --profile my-workspace ``` ### Output @@ -134,23 +134,23 @@ Returns: ```bash # Simple SELECT query -databricks aitools tools query \ +databricks experimental aitools tools query \ "SELECT * FROM samples.nyctaxi.trips LIMIT 5" \ --profile my-workspace # Aggregation query -databricks aitools tools query \ +databricks experimental aitools tools query \ "SELECT vendor_id, COUNT(*) as trip_count FROM samples.nyctaxi.trips GROUP BY vendor_id" \ --profile my-workspace # With JSON output -databricks aitools tools query \ +databricks experimental aitools tools query \ "SELECT * FROM catalog.schema.table WHERE date > '2024-01-01'" \ --output json \ --profile my-workspace # Using specific warehouse -DATABRICKS_WAREHOUSE_ID=abc123 databricks aitools tools query \ +DATABRICKS_WAREHOUSE_ID=abc123 databricks experimental aitools tools query \ "SELECT * FROM samples.nyctaxi.trips LIMIT 10" \ --profile my-workspace ``` @@ -161,17 +161,17 @@ DATABRICKS_WAREHOUSE_ID=abc123 databricks aitools tools query \ ```bash # Check table size - databricks aitools tools query \ + databricks experimental aitools tools query \ "SELECT COUNT(*) FROM catalog.schema.table" \ --profile my-workspace # View sample data - databricks aitools tools query \ + databricks experimental aitools tools query \ "SELECT * FROM catalog.schema.table LIMIT 10" \ --profile my-workspace # Get column statistics - databricks aitools tools query \ + databricks experimental aitools tools query \ "SELECT MIN(column), MAX(column), AVG(column) FROM catalog.schema.table" \ --profile my-workspace ``` @@ -180,12 +180,12 @@ DATABRICKS_WAREHOUSE_ID=abc123 databricks aitools tools query \ ```bash # Check for null values - databricks aitools tools query \ + databricks experimental aitools tools query \ "SELECT COUNT(*) FROM catalog.schema.table WHERE column IS NULL" \ --profile my-workspace # Verify data freshness - databricks aitools tools query \ + databricks experimental aitools tools query \ "SELECT MAX(timestamp_column) FROM catalog.schema.table" \ --profile my-workspace ``` @@ -193,7 +193,7 @@ DATABRICKS_WAREHOUSE_ID=abc123 databricks aitools tools query \ 3. **Quick analytics** ```bash # Group by analysis - databricks aitools tools query \ + databricks experimental aitools tools query \ "SELECT category, COUNT(*), AVG(value) FROM catalog.schema.table GROUP BY category" \ --profile my-workspace ``` @@ -204,12 +204,12 @@ Here's a typical workflow combining both commands: ```bash # 1. Discover the schema first -databricks aitools tools discover-schema \ +databricks experimental aitools tools discover-schema \ samples.nyctaxi.trips \ --profile my-workspace # 2. Based on discovered columns, run targeted queries -databricks aitools tools query \ +databricks experimental aitools tools query \ "SELECT vendor_id, payment_type, COUNT(*) as trips, AVG(fare_amount) as avg_fare FROM samples.nyctaxi.trips GROUP BY vendor_id, payment_type @@ -218,7 +218,7 @@ databricks aitools tools query \ --profile my-workspace # 3. Investigate specific patterns found in the data -databricks aitools tools query \ +databricks experimental aitools tools query \ "SELECT * FROM samples.nyctaxi.trips WHERE fare_amount > 100 LIMIT 20" \ @@ -231,15 +231,15 @@ Remember that each Bash command in Claude Code runs in a separate shell: ```bash # ✅ RECOMMENDED: Use --profile flag -databricks aitools tools discover-schema samples.nyctaxi.trips --profile my-workspace +databricks experimental aitools tools discover-schema samples.nyctaxi.trips --profile my-workspace # ✅ ALTERNATIVE: Chain with && export DATABRICKS_CONFIG_PROFILE=my-workspace && \ - databricks aitools tools query "SELECT * FROM samples.nyctaxi.trips LIMIT 5" + databricks experimental aitools tools query "SELECT * FROM samples.nyctaxi.trips LIMIT 5" # ❌ DOES NOT WORK: Separate export export DATABRICKS_CONFIG_PROFILE=my-workspace -databricks aitools tools query "SELECT * FROM samples.nyctaxi.trips LIMIT 5" +databricks experimental aitools tools query "SELECT * FROM samples.nyctaxi.trips LIMIT 5" ``` ## Flags @@ -276,7 +276,7 @@ Both commands support: 1. Check for default warehouse: ```bash - databricks aitools tools get-default-warehouse --profile my-workspace + databricks experimental aitools tools get-default-warehouse --profile my-workspace ``` 2. List available warehouses: ```bash @@ -284,7 +284,7 @@ Both commands support: ``` 3. Set specific warehouse: ```bash - DATABRICKS_WAREHOUSE_ID= databricks aitools tools query "SELECT 1" --profile my-workspace + DATABRICKS_WAREHOUSE_ID= databricks experimental aitools tools query "SELECT 1" --profile my-workspace ``` 4. Start a stopped warehouse: ```bash @@ -325,13 +325,13 @@ Both commands support: 2. **Use LIMIT for exploration** - When exploring large tables, always use LIMIT to avoid long-running queries: ```bash - databricks aitools tools query "SELECT * FROM large_table LIMIT 100" --profile my-workspace + databricks experimental aitools tools query "SELECT * FROM large_table LIMIT 100" --profile my-workspace ``` 3. **JSON output for parsing** - Use `--output json` when you need to process results programmatically: ```bash - databricks aitools tools query "SELECT * FROM table" --output json --profile my-workspace | jq '.results' + databricks experimental aitools tools query "SELECT * FROM table" --output json --profile my-workspace | jq '.results' ``` 4. **Check table existence** - Before querying, verify the table exists: