Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions rust/crates/sift_cli/assets/docs/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
- [ULog](./data/import-ulog.md)
- [Backups](./data/import-backups.md)
- [Exporting Data](./data/exporting.md)
- [Jobs](./data/jobs.md)

# Reference

Expand Down
59 changes: 59 additions & 0 deletions rust/crates/sift_cli/assets/docs/src/data/jobs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# Jobs

Server-side work in Sift — data imports, exports, and rule evaluations — runs
as a job. The `get`, `status`, and `wait` commands inspect and poll those jobs,
which is what makes the CLI usable in CI/CD.

## Why not just pass `--wait`?

`sift-cli import ... --wait` blocks until the job finishes. That is the right
default for interactive use, but it serializes work: firing five imports means
five sequential blocking commands. And it does not compose — you cannot start
several imports in parallel, do other work, then confirm they all landed at
the end.

The verb-first commands split those two concerns:

1. **Kick off work without waiting.** `sift-cli import ...` (no `--wait`)
uploads the file and prints the assigned job ID. It exits `0` even if the
server-side job later fails.
2. **Poll the jobs later.** `sift-cli wait job <ID> [ID ...]` blocks until
every named job reaches a terminal state and exits non-zero if any failed.

## The CI/CD pattern

```sh
# Fire imports in parallel; capture each job id.
JOB_A=$(sift-cli import csv a.csv --asset engine | awk '/Job ID/ {print $NF}')
JOB_B=$(sift-cli import csv b.csv --asset engine | awk '/Job ID/ {print $NF}')
JOB_C=$(sift-cli import csv c.csv --asset engine | awk '/Job ID/ {print $NF}')

# Gate the pipeline on all three finishing successfully.
sift-cli wait job "$JOB_A" "$JOB_B" "$JOB_C"
```

The `import` command prints a `Job ID: <uuid>` line on the no-wait path
specifically so scripts can capture it.

## Inspecting jobs

- `sift-cli get jobs` lists the 50 most recent jobs, newest first. Add
`--job-type` (`data-import`, `data-export`, `rule-evaluation`) or `--status`
(`created`, `running`, `finished`, `failed`, `cancelled`, `cancel-requested`)
to narrow it. `--limit` overrides the page size.
- `sift-cli get job <ID>` prints the full details on one job: type, status,
timestamps, and failure details when the job failed.
- `sift-cli status job <ID>` is the scripting form. It prints one status word
to stdout and exits with a code that reflects the job state.

### Exit codes for `status job`

| Code | Meaning |
| ---- | ------------------------------------------ |
| `0` | Job finished successfully. |
| `1` | Job failed. |
| `2` | Job was cancelled or cancel is requested. |
| `3` | Job is still running or has not started. |

`wait job` uses `0` if every job finished and `1` if any job failed or was
cancelled; per-job status is printed to stderr for the non-success cases.
12 changes: 12 additions & 0 deletions rust/crates/sift_cli/assets/docs/src/reference/commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,18 @@ Export data from Sift. See [Exporting Data](../data/exporting.md).
Verify credentials and connectivity. See
[Verifying Your Setup](../getting-started/verifying.md).

## `get`, `status`, `wait`

Inspect and poll Sift resources. See [Jobs](../data/jobs.md) for the CI/CD
pattern (fire imports without `--wait`, then gate on a single `wait job`).

| Command | Description |
| ------------------------------------------ | ---------------------------------------------------------------------------------- |
| `get jobs [--job-type ...] [--status ...]` | List recent jobs, newest first. |
| `get job <JOB_ID>` | Show full details for one job. |
| `status job <JOB_ID>` | Print a terse status line. Exit `0` finished, `1` failed, `2` cancelled, `3` running. |
| `wait job <JOB_ID> [JOB_ID ...]` | Block until every named job reaches a terminal state. |

## `install`

Install optional tooling.
Expand Down
7 changes: 7 additions & 0 deletions rust/crates/sift_cli/assets/skills/sift/references/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ Key subcommands:
- `import`: `csv`, `parquet flat-dataset`, `parquet cpr`, `tdms`, `hdf5`,
`ulog`, `backups`.
- `export`: `run`, `asset` (to CSV and other formats).
- `get`, `status`, `wait`: inspect and poll server-side jobs (`get jobs`,
`get job <ID>`, `status job <ID>`, `wait job <ID> [ID ...]`).
- `mcp`: start the MCP server.
- `ping`: verify credentials and connectivity.
- `config`: manage profiles and credentials.
Expand Down Expand Up @@ -72,6 +74,11 @@ per session. The rest apply to each subcommand invocation.
it you cannot confirm the data actually landed. Relay the final stdout
line to the user verbatim. `import backups` is the one exception: it
accepts no `--wait`, `--preview`, or `--run`.

Skip `--wait` only when firing multiple imports in parallel. In that
case capture each `Job ID: <uuid>` line from the upload output, then
run `sift-cli wait job <ID> [ID ...]` to block on the whole batch and
report failures. `wait job` exits non-zero if any job failed.
7. **Surface the Explore link from import output.** Each profile must set
`app_uri`. `sift-cli import` prints a `View in Sift: <URL>` tip when this
value is usable. Surface the URL as plain text, in full. Do not wrap it in a
Expand Down
91 changes: 91 additions & 0 deletions rust/crates/sift_cli/src/cli/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,97 @@ pub enum Cmd {
#[cfg(feature = "mcp")]
#[command(hide = true)]
Mcp(McpArgs),

/// Retrieve Sift resources (e.g. jobs)
#[command(subcommand)]
Get(GetCmd),

/// Print a terse status line for a resource; exit code reflects state
#[command(subcommand)]
Status(StatusCmd),

/// Block until one or more resources reach a terminal state
#[command(subcommand)]
Wait(WaitCmd),
}

#[derive(Subcommand)]
pub enum GetCmd {
/// List recent jobs, newest first
Jobs(GetJobsArgs),

/// Show full details for a single job
Job(GetJobArgs),
}

#[derive(clap::Args)]
pub struct GetJobsArgs {
/// Filter by job type
#[arg(long, value_enum)]
pub job_type: Option<JobTypeArg>,

/// Filter by job status
#[arg(long, value_enum)]
pub status: Option<JobStatusArg>,

/// Max jobs to return (clamped to the server's cap)
#[arg(long, default_value_t = 50)]
pub limit: u32,
}

#[derive(clap::Args)]
pub struct GetJobArgs {
/// Job ID
pub job_id: String,
}

#[derive(Subcommand)]
pub enum StatusCmd {
/// Print the current status of a job. Exit code: 0 finished, 1 failed,
/// 2 cancelled or cancel-requested, 3 still running
Job(StatusJobArgs),
}

#[derive(clap::Args)]
pub struct StatusJobArgs {
/// Job ID
pub job_id: String,
}

#[derive(Subcommand)]
pub enum WaitCmd {
/// Block until every named job reaches a terminal state. Exit 0 if all
/// finished; nonzero if any failed or was cancelled
Job(WaitJobArgs),
}

#[derive(clap::Args)]
#[command(
override_usage = "sift-cli wait job <JOB_ID> [JOB_ID ...]",
after_help = "Example:\n \
sift-cli wait job <JOB_ID_1> <JOB_ID_2> <JOB_ID_3>"
)]
pub struct WaitJobArgs {
/// One or more job IDs to wait on, separated by spaces
#[arg(required = true, value_name = "JOB_ID")]
pub job_ids: Vec<String>,
}

#[derive(clap::ValueEnum, Clone, Debug)]
pub enum JobTypeArg {
DataImport,
DataExport,
RuleEvaluation,
}

#[derive(clap::ValueEnum, Clone, Debug)]
pub enum JobStatusArg {
Created,
Running,
Finished,
Failed,
Cancelled,
CancelRequested,
}

#[cfg(feature = "mcp")]
Expand Down
101 changes: 47 additions & 54 deletions rust/crates/sift_cli/src/cmd/import/mod.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
use anyhow::Result;
use crossterm::style::Stylize;
use std::{process::ExitCode, time::Duration};
use tokio::time::sleep;
use std::process::ExitCode;

use sift_rs::{SiftChannel, common::r#type::v1::ChannelConfig, jobs::v1::JobStatus};

Expand Down Expand Up @@ -40,6 +39,7 @@ pub async fn finish_import(
if !wait {
Output::new()
.line(format!("{} file for processing", "Uploaded".green()))
.line(format!("{}: {job_id}", "Job ID".green()))
.tip(pending_import_tip(
&target.location,
target.explore_url.as_deref(),
Expand All @@ -60,72 +60,65 @@ pub async fn wait_for_job_completion(
let spinner = Spinner::new();
spinner.set_message(format!("{} file for processing", "Uploaded".green()));

let mut job_service = JobServiceWrapper::new(grpc_channel.clone());
let mut job_service = JobServiceWrapper::new(grpc_channel);

let Some(mut job) = job_service.get_job(&job_id).await? else {
spinner.finish_and_clear();
let outcome = job_service
.poll_until_terminal(&job_id, |job| match job.job_status() {
JobStatus::Running => {
spinner.set_message(format!("{} imported file", "Processing".green()));
}
JobStatus::CancelRequested => {
spinner.set_message(format!(
"{} was requested but the job may still finish",
"Cancellation".green()
));
}
_ => (),
})
.await?;

spinner.finish_and_clear();

let Some(job) = outcome else {
Output::new()
.line("The file was successfully uploaded but the job was unexpectedly not found")
.tip("Please notify Sift about this bug")
.eprint();
return Ok(ExitCode::FAILURE);
};

loop {
sleep(Duration::from_secs(3)).await;

let Some(updated_job) = job_service.get_job(&job.job_id).await? else {
spinner.finish_and_clear();
match job.job_status() {
JobStatus::Finished => {
let mut tip_text =
format!("The data should be available on the {import_output_location}");
tip_text.push_str(&explore_or_note(explore_url.as_deref()));
Output::new()
.line(format!("{} data import job", "Completed".green()))
.tip(tip_text)
.print();
Ok(ExitCode::SUCCESS)
}
JobStatus::Cancelled => {
Output::new()
.line(format!("{} data import job", "Cancelled".green()))
.print();
Ok(ExitCode::SUCCESS)
}
JobStatus::Failed => {
Output::new()
.line("The file was successfully uploaded but the job was unexpectedly not found")
.line("Processing failed")
.tip("Please check the Sift jobs manage page for further details")
.eprint();
Ok(ExitCode::FAILURE)
}
other => {
Output::new()
.line(format!("unexpected job status `{other:?}`"))
.tip("Please notify Sift about this bug")
.eprint();
return Ok(ExitCode::FAILURE);
};
job = updated_job;

match job.job_status() {
JobStatus::Created => (),
JobStatus::Running => {
spinner.set_message(format!("{} imported file", "Processing".green()));
}
JobStatus::CancelRequested => {
spinner.set_message(format!(
"{} was requested but the job may still finish",
"Cancellation".green()
));
}
JobStatus::Cancelled => {
spinner.finish_and_clear();
Output::new()
.line(format!("{} data import job", "Cancelled".green()))
.print();
break;
}
JobStatus::Failed => {
spinner.finish_and_clear();
Output::new()
.line("Processing failed")
.tip("Please check the Sift jobs manage page for further details")
.eprint();
return Ok(ExitCode::FAILURE);
}
JobStatus::Finished => {
spinner.finish_and_clear();
let mut tip_text =
format!("The data should be available on the {import_output_location}");
tip_text.push_str(&explore_or_note(explore_url.as_deref()));
Output::new()
.line(format!("{} data import job", "Completed".green()))
.tip(tip_text)
.print();
break;
}
_ => (),
Ok(ExitCode::FAILURE)
}
}
Ok(ExitCode::SUCCESS)
}

pub struct TimePreview<'a> {
Expand Down
Loading
Loading