Skip to content

test: execute documented quickstart SPL examples - #77

Open
Chloride233 wants to merge 2 commits into
alibaba:mainfrom
Chloride233:test/docs-query-examples-check
Open

test: execute documented quickstart SPL examples#77
Chloride233 wants to merge 2 commits into
alibaba:mainfrom
Chloride233:test/docs-query-examples-check

Conversation

@Chloride233

Copy link
Copy Markdown
Contributor

Summary

This adds a small CI check for the SPL examples published in the core docs.

The test loads the bundled quickstart sample into an in-memory app, extracts fenced umctl query run demo and umctl query explain demo commands from the English and Chinese docs plus the quickstart README files, and executes the extracted queries through Query Service.

This should catch docs drift when query syntax, quickstart data, or documented examples change. It does not change runtime behavior or any public API.

Validation

  • go test -v -run TestDocsQueryExamplesExecute ./tests/integration/
  • git diff --check

@qiansheng91 qiansheng91 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks — this is a useful guard. I ran it against the PR head: it passes, and it exercises 82 documented query run / query explain demo examples from docs/en, docs/zh, and the quickstart README, so the extract→execute path holds end to end.

One thing worth making explicit: now that docs-query-check is in the ci gate and runs every documented demo query against the default quickstart sample, the rule it enforces is "documented demo queries must be runnable on multi-domain-quickstart." That's true today (82/82), but it's an implicit contract for future docs — an example written for a different sample (incident-investigation, service-localization) or a non-demo workspace would be silently skipped or break CI. A one-line comment in the test (or a note in the docs-contribution guide) stating that would save a future contributor some confusion.

The scoping to docs/en + docs/zh + quickstart-multidomain/README* is the right call — it avoids the example READMEs that load other samples.

LGTM once the Local CI Gate is green.

@Chloride233
Chloride233 force-pushed the test/docs-query-examples-check branch from daea667 to e7c0b57 Compare June 27, 2026 10:11

@qiansheng91 qiansheng91 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Trivial change, looks good.


Automated review by github-manager-bot

@qiansheng91

Copy link
Copy Markdown
Collaborator

Issue Evaluation

Category: enhancement | Status: Evaluated

Feasibility: feasible
Scope: tests/integration/, docs (EN + ZH), quickstart README
Compatibility: no breaking changes — additive CI check only

This is a solid test infrastructure improvement. Adding CI validation for documented SPL examples catches docs drift early and ensures quickstart examples stay functional as query syntax or sample data evolves. The approach (extract fenced commands from docs, execute via Query Service) is clean and non-invasive.

Recommendation: Proceed with implementation. Ensure the test gracefully handles missing quickstart data (skip vs fail) and documents the extraction pattern for future doc authors.


Automated evaluation by github-manager-bot

@qiansheng91 qiansheng91 added the enhancement New feature or request label Jul 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants