Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,6 @@
[submodule "ref/fs"]
path = ref/fs
url = git@github.com:tidbcloud/fs.git
[submodule "integration/tidbcloud-fs-e2b-example"]
path = integration/tidbcloud-fs-e2b-example
url = git@github.com:likidu/tidbcloud-fs-e2b-example.git
13 changes: 13 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,8 @@ Implemented:
`docs/spec/done/0018-fs-token-auth-and-config-free-access.md`
- Server-backed Filesystem token lifecycle management from
`docs/spec/done/0030-file-system-token-lifecycle-management.md`
- Filesystem layer fork, ancestry, deletion, and historical mount workflows
from `docs/spec/done/0032-filesystem-layer-fork-workflows.md`
- install and update distribution from
`docs/spec/done/0012-install-and-update-distribution.md`
- English PingCAP Preview documentation from
Expand Down Expand Up @@ -123,9 +125,12 @@ Implemented:
- `ti fs find-files`
- `ti fs create-layer`
- `ti fs list-layers`
- `ti fs fork-layer`
- `ti fs list-layer-chain`
- `ti fs describe-layer`
- `ti fs diff-layer`
- `ti fs create-layer-checkpoint`
- `ti fs delete-layer`
- `ti fs rollback-layer`
- `ti fs commit-layer`
- `ti fs mount-file-system`
Expand Down Expand Up @@ -584,10 +589,13 @@ Implemented command behavior:
- `ti fs create-layer --layer-id layer-1 --base-root-path /workspace --layer-name task --durability-mode restore-safe --tag task=auth`
- `ti fs list-layers`
- `ti fs list-layers --output text`
- `ti fs fork-layer --parent-layer-ref layer-1 --layer-name child --checkpoint-id cp-1`
- `ti fs list-layer-chain --layer-ref child --output text`
- `ti fs describe-layer --layer-id layer-1`
- `ti fs diff-layer --layer-id layer-1`
- `ti fs copy-file --from-local ./README.md --to-remote /workspace/layered.md --layer-id layer-1`
- `ti fs create-layer-checkpoint --layer-id layer-1 --checkpoint-id cp-1 --label before-commit`
- `ti fs delete-layer --layer-ref child`
- `ti fs rollback-layer --layer-id layer-1`
- `ti fs commit-layer --layer-id layer-1`
- `ti fs pack-file-system --local-root ~/.ti/local/fs/demo --remote-root /workspace --mount-profile portable`
Expand All @@ -600,6 +608,8 @@ Implemented command behavior:
- `ti fs mount-file-system --file-system-id <file-system-id> --mount-path ./workspace --mount-profile portable --pack-path /`
- `ti fs mount-file-system --file-system-id <file-system-id> --mount-path ./workspace --driver fuse --read-cache-size-mb 256 --read-cache-max-file-mb 16`
- `ti fs mount-file-system --file-system-id <file-system-id> --mount-path ./workspace --driver fuse --cache-dir ~/.ti/cache/workspace --write-back-cache=false`
- `ti fs mount-file-system --file-system-id <file-system-id> --mount-path ./workspace --remote-path /workspace --driver fuse --layer-ref layer-1`
- `ti fs mount-file-system --file-system-id <file-system-id> --mount-path ./workspace-v1 --remote-path /workspace --driver fuse --layer-ref layer-1 --checkpoint-id cp-1`
- `ti fs drain-file-system --mount-path ./workspace`
- `ti fs drain-file-system --mount-path ./workspace --timeout 30s`
- `ti fs unmount-file-system --mount-path ./workspace`
Expand Down Expand Up @@ -671,9 +681,12 @@ Registered command surface:
- `ti fs find-files`
- `ti fs create-layer`
- `ti fs list-layers`
- `ti fs fork-layer`
- `ti fs list-layer-chain`
- `ti fs describe-layer`
- `ti fs diff-layer`
- `ti fs create-layer-checkpoint`
- `ti fs delete-layer`
- `ti fs rollback-layer`
- `ti fs commit-layer`
- `ti fs pack-file-system`
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ live-e2e-db: build
$(LIVE_E2E_RUN) -run '^TestLiveDB'

live-e2e-fs: build
$(LIVE_E2E_RUN) -run '^TestLive(FSRemoteInventoryLifecycle|FSCommandSurface|FSFileSystemTokenLifecycle|FSConfigurationFreeAccess|FSDataPlaneLifecycle|FSMountRuntime|FSWebDAVMountRuntime)$$'
$(LIVE_E2E_RUN) -run '^TestLive(FSRemoteInventoryLifecycle|FSCommandSurface|FSFileSystemTokenLifecycle|FSConfigurationFreeAccess|FSDataPlaneLifecycle|FSLayerForkWorkflow|FSMountRuntime|FSWebDAVMountRuntime)$$'

live-e2e-fs-git: build
$(LIVE_E2E_RUN) -run '^TestLiveFSGit'
Expand Down
22 changes: 22 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,25 @@ Automatic mounting uses FUSE on Linux and WebDAV on macOS and Windows. macOS use

Mount commands start the companion runtime in the background, wait until the mount is ready, and then return a structured result. Use `ti fs unmount-file-system` or `ti fs-vault unmount-vault` to end a mount. The public CLI does not expose a foreground mount mode.

Filesystem layers can fork copy-on-write child timelines without copying a workspace. Layer and checkpoint mounts require FUSE; checkpoint mounts are always read-only. Drive9 does not support combining recursive copy with a layer, so seed a directory tree through a writable layer mount, drain it, and then create the checkpoint:

```shell
export TI_FS_FILE_SYSTEM_ID="<file-system-id>"
ti fs create-directory --path /research/q3-market
ti fs create-layer --base-root-path /research/q3-market --layer-name research-base
SEED_MOUNT="$(mktemp -d)"
ti fs mount-file-system --mount-path "$SEED_MOUNT" --remote-path /research/q3-market --driver fuse --layer-ref research-base
tar -C . -cf - . | tar -C "$SEED_MOUNT" -xf -
ti fs drain-file-system --mount-path "$SEED_MOUNT"
ti fs unmount-file-system --mount-path "$SEED_MOUNT"
rmdir "$SEED_MOUNT"
ti fs create-layer-checkpoint --layer-id research-base --checkpoint-id seed
ti fs fork-layer --parent-layer-ref research-base --layer-name experiment-a --checkpoint-id seed
ti fs list-layer-chain --layer-ref experiment-a --output text
```

Use `ti fs delete-layer --layer-ref experiment-a` to abandon a rejected leaf. Deleting a layer with live descendants fails unless `--cascade` is supplied explicitly. Layer names are not durable unique identifiers because abandoned layers remain visible; automation should capture returned layer IDs and use run-unique checkpoint IDs.

`ti fs list-file-systems` reads the region-scoped remote inventory through TiDB Cloud credentials. A profile can access multiple file systems, including resources created on another machine. Data-plane commands never infer a resource from the number of local credentials, so provide `--file-system-id` or set `TI_FS_FILE_SYSTEM_ID`:

```shell
Expand Down Expand Up @@ -308,9 +327,12 @@ ti fs search-file-content
ti fs find-files
ti fs create-layer
ti fs list-layers
ti fs fork-layer
ti fs list-layer-chain
ti fs describe-layer
ti fs diff-layer
ti fs create-layer-checkpoint
ti fs delete-layer
ti fs rollback-layer
ti fs commit-layer
ti fs pack-file-system
Expand Down
4 changes: 2 additions & 2 deletions docs/spec/done/0012-install-and-update-distribution.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Goal

Make `tdc` installable and updatable through deterministic GitHub Releases artifacts. The MVP channel is GoReleaser plus GitHub Releases, with shell and PowerShell installers. Homebrew and Scoop are intentionally deferred to `docs/spec/0032-homebrew-and-scoop-distribution.md`.
Make `tdc` installable and updatable through deterministic GitHub Releases artifacts. The MVP channel is GoReleaser plus GitHub Releases, with shell and PowerShell installers. Homebrew and Scoop are intentionally deferred to `docs/spec/0033-homebrew-and-scoop-distribution.md`.

## User-facing Commands

Expand Down Expand Up @@ -266,7 +266,7 @@ Installer scripts:
- Silent auto-update.
- Updating TiDB Cloud credentials or DB SQL credentials.
- Config migrations that modify user config during update.
- Homebrew tap and Scoop bucket publishing. See `0032-homebrew-and-scoop-distribution.md`.
- Homebrew tap and Scoop bucket publishing. See `0033-homebrew-and-scoop-distribution.md`.
- Linux apt/yum repositories.
- Winget publishing.
- Notarization or binary signing beyond SHA-256 checksums for MVP.
Loading