feat(datasets)!: remove datasets commands and dataset feature surface#166
Conversation
Removes the datasets feature from the CLI entirely: - Delete the `datasets` command group and src/datasets.rs - Drop the `--dataset-id` index scope (IndexScope::Dataset, list_one_dataset, dataset path arms); indexes now scope to connection tables / managed-database catalogs only - Remove `dataset_refresh` and `create_dataset_index` from `jobs --job-type` - Repoint the SDK X-Workspace-Id header regression test off `datasets().list` onto `jobs().list` - Scrub README and all skill docs of dataset commands, `--dataset-id`, removed job types, and `datasets.main.*` query examples BREAKING CHANGE: `hotdata datasets ...`, `hotdata indexes --dataset-id`, and the `dataset_refresh`/`create_dataset_index` job-type filters are removed. Also fixes pre-existing doc drift: `indexes create --connection-id` (not a valid flag) corrected to `--catalog <connection-name-or-id>`.
There was a problem hiding this comment.
Clean feature removal. Code changes (command/main/indexes/jobs/sdk) are internally consistent — match arms stay exhaustive, error messages preserved, job display unaffected by the parse_job_type narrowing, and no dangling dataset references remain. Docs scrubbed thoroughly and the --catalog connection-scope correction matches the resolve_connection_id code path.
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
# Conflicts: # src/datasets.rs # src/indexes.rs
There was a problem hiding this comment.
Clean feature removal. Verified no leftover dataset references in src/ (module, command enum, handlers, IndexScope::Dataset, job types all removed), docs scrubbed consistently, and the repointed sdk.rs/workspace_env.rs tests preserve their original assertions. The jobs().list repoint matches the 4-arg signature, and indexes create/list retain prior connection-scope behavior.
There was a problem hiding this comment.
Clean, well-scoped removal of the datasets feature. Verified: no leftover dataset references in code/tests; indexes::list match remains exhaustive with the workspace-scan fallback intact; indexes create/delete collapse to connection scope with unchanged resolution behavior and the public schema default preserved; the sdk.rs workspace-header regression test was correctly repointed to another workspace-scoped call. Docs updated consistently.
0.4.0 removes the datasets API/models and adds a usage API. The CLI already dropped its datasets surface in #166, so the removal is a no-op here and the new usage API is simply unused. No source changes needed. Verified: `cargo build`/`clippy` clean, full suite (206) green, and a production smoke test (auth status, databases list, an arrow-decoded query) against workspace AgentRyan succeeds. Co-authored-by: Eddie A Tejeda <669988+eddietejeda@users.noreply.github.com>
Summary
Removes the
datasetsfeature from the CLI entirely (full feature removal across code + docs).Net: 17 files changed, ~126 insertions, ~971 deletions. Clean build, clippy clean, 194 unit + 12 integration tests passing.
What changed
Code
src/datasets.rs(441 lines) and thedatasetscommand group (command.rsenum +main.rshandler + module decl/import).indexes— dropped the--dataset-idscope everywhere: removedIndexScope::Dataset,list_one_dataset, the dataset path arms, and its unit test; simplified thecreate/delete/listhandlers to connection/catalog scope only.jobs— removeddataset_refreshandcreate_dataset_indexfromparse_job_typeand the--job-typevalue list.sdk.rs— thedatasets()client lives in the externalhotdatacrate (not editable here), so its oneX-Workspace-Id-header regression test was repointed offdatasets().listontojobs().list(another generated-client workspace-scoped call) to preserve the assertion.databases.rs— reworded the parquet-only error hint that pointed athotdata datasets create.tests/workspace_env.rs— repointed the workspace-lock test fromdatasets listtoindexes list.Docs
datasetscommand,--dataset-id, the removed job types, anddatasets.main.*query examples (replaced with<catalog>.public.*). "Datasets vs managed databases" sections collapsed to managed-databases-only with consistent anchors.Notes for reviewers
feat(datasets)!:commit carries the breaking-change note.hotdata indexes create --connection-id ..., butcreatehas no--connection-idflag (the binary rejects it). Index-on-a-connection-table is done via--catalog <connection-name-or-id>(resolve_connection_idresolves names/IDs/catalog aliases). Since the dataset removal forced rewriting the "index scopes" prose anyway, these were corrected to--catalograther than shipping broken examples.Test plan
cargo build— clean, no new warningscargo clippy --all-targets— clean (only pre-existing style nits in untouched code)cargo test— 194 unit + 12 integration tests passhotdata datasetsrejected as unrecognized;indexes listhas no--dataset-id;jobs --job-typeshows onlydata_refresh_table,data_refresh_connection,create_index🤖 Generated with Claude Code