-
Notifications
You must be signed in to change notification settings - Fork 9
feat: skills for creating, reviewing, and auditing an AppKit plugin #257
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
789a94f
fd5119b
a4b31e2
e1eb2c6
b69100e
b8df398
65d5805
dd3f954
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. An idea for future: maybe it would be worth to convert those skills to a "Best practices" page in our docs? That would be awesome 👍 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,141 @@ | ||
| --- | ||
| description: Full audit of a core plugin against all best-practices categories with scorecard | ||
| argument-hint: <plugin-name> | ||
| --- | ||
|
|
||
| # Audit Core Plugin | ||
|
|
||
| Perform a full audit of the named plugin against all AppKit plugin best-practices categories and produce a scorecard. | ||
|
|
||
| **Plugin name:** $ARGUMENTS | ||
|
|
||
| ## Step 1: Validate Input | ||
|
|
||
| If `$ARGUMENTS` is empty or missing, stop and output: | ||
|
|
||
| > Usage: /audit-core-plugin <plugin-name> | ||
|
|
||
| Otherwise, set `PLUGIN_NAME` to the value of `$ARGUMENTS` (trimmed, kebab-case). | ||
|
|
||
| Check whether the plugin directory exists at either of these paths: | ||
|
|
||
| - `packages/appkit/src/plugins/{PLUGIN_NAME}/` | ||
| - `packages/appkit/src/connectors/{PLUGIN_NAME}/` | ||
|
|
||
| If **neither** path exists, stop and output: | ||
|
|
||
| > Error: Plugin "{PLUGIN_NAME}" not found. Checked: | ||
| > - `packages/appkit/src/plugins/{PLUGIN_NAME}/` | ||
| > - `packages/appkit/src/connectors/{PLUGIN_NAME}/` | ||
| > | ||
| > Available plugins can be listed with: `ls packages/appkit/src/plugins/` | ||
|
|
||
| If at least one path exists, proceed. | ||
|
|
||
| ## Step 2: Load Best Practices Reference | ||
|
|
||
| Read the full contents of: | ||
|
|
||
| ``` | ||
| .claude/references/plugin-best-practices.md | ||
| ``` | ||
|
|
||
| This defines the 9 audit categories and their NEVER/MUST/SHOULD guidelines. You will evaluate the plugin against every guideline in every category. | ||
|
|
||
| ## Step 3: File Discovery | ||
|
|
||
| Read **all** files under: | ||
|
|
||
| - `packages/appkit/src/plugins/{PLUGIN_NAME}/` (recursively, including `tests/` subdirectory) | ||
| - `packages/appkit/src/connectors/{PLUGIN_NAME}/` (recursively, if this directory exists) | ||
|
|
||
| Collect the full contents of every file. You need the complete source to evaluate all 9 categories. | ||
|
|
||
| ## Step 4: Structural Completeness Check | ||
|
|
||
| If `packages/appkit/src/plugins/{PLUGIN_NAME}/` does not exist (connector-only package), mark Structural Completeness as **N/A** in the scorecard and proceed to Step 5. | ||
|
|
||
| Otherwise, verify the following expected files exist inside `packages/appkit/src/plugins/{PLUGIN_NAME}/`: | ||
|
|
||
| | Expected file | Required? | | ||
| |---|---| | ||
| | `manifest.json` | MUST | | ||
| | Main plugin class file (any `.ts` file containing a class extending `Plugin`) | MUST | | ||
| | `types.ts` | MUST | | ||
| | `defaults.ts` | SHOULD | | ||
| | `index.ts` | MUST | | ||
| | `tests/` directory with at least one `.test.ts` file | MUST | | ||
|
|
||
| Treat each missing `MUST` file as a **MUST**-severity finding under the "Structural Completeness" category. Treat a missing `SHOULD` file as a **SHOULD**-severity finding. | ||
|
|
||
| `defaults.ts` is not universally required for every plugin. It should be present when the plugin exposes execution settings or defines behavior that depends on `execute()` / `executeStream()` defaults, but its absence alone should not be reported as a MUST failure for plugins that do not use those defaults. | ||
|
|
||
| > **Note:** The structural completeness check applies only to the `plugins/{PLUGIN_NAME}/` directory. Connector directories (`connectors/{PLUGIN_NAME}/`) serve a different architectural role and are read as supporting context for the best-practices review, not audited for structural completeness. | ||
|
|
||
| ## Step 5: Full Best-Practices Review | ||
|
|
||
| Before evaluating, read the shared review rules in `.claude/references/plugin-review-guidance.md` and apply them throughout this step (deduplication, cache-key tracing). | ||
|
|
||
| Evaluate the plugin code against **all 9 categories** from the Category Index in `plugin-review-guidance.md`. Check each category's NEVER/MUST/SHOULD rules from the best-practices reference. | ||
|
|
||
| For each guideline in each category, determine whether the plugin **passes**, **violates**, or is **not applicable** (e.g., SSE rules for a non-streaming plugin). Record findings with: | ||
|
|
||
| - **Severity**: NEVER, MUST, or SHOULD (from the guideline prefix) | ||
| - **Category**: Which of the 9 categories | ||
| - **Description**: What the guideline requires and how the plugin violates it | ||
| - **Location**: Specific `file:line` reference(s) | ||
|
|
||
| A category with no findings is a pass. A category with only SHOULD findings is a warn. A category with any MUST or NEVER finding is a fail. | ||
|
|
||
| ## Step 6: Produce Output | ||
|
|
||
| ### Scorecard Table (output first) | ||
|
|
||
| ``` | ||
| ## Scorecard | ||
|
|
||
| | # | Category | Status | Findings | | ||
| |---|----------|--------|----------| | ||
| | 0 | Structural Completeness | {status} | {count} | | ||
| | 1 | Manifest Design | {status} | {count} | | ||
| | 2 | Plugin Class Structure | {status} | {count} | | ||
| | 3 | Route Design | {status} | {count} | | ||
| | 4 | Interceptor Usage | {status} | {count} | | ||
| | 5 | asUser / OBO Patterns | {status} | {count} | | ||
| | 6 | Client Config | {status} | {count} | | ||
| | 7 | SSE Streaming | {status} | {count} | | ||
| | 8 | Testing Expectations | {status} | {count} | | ||
| | 9 | Type Safety | {status} | {count} | | ||
|
Comment on lines
+99
to
+108
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is it correct numbering? Looking at the plugin-review-guidance I can see only 1-9 points 🤔 |
||
| ``` | ||
|
|
||
| Where `{status}` is one of: | ||
| - Pass — no findings | ||
| - Warn — SHOULD-only findings | ||
| - Fail — any NEVER or MUST findings | ||
| - N/A — category does not apply to this plugin (e.g., SSE Streaming for a non-streaming plugin) | ||
|
|
||
| And `{count}` is the number of findings (0 if pass). | ||
|
|
||
| ### Detailed Findings (output second, severity-first) | ||
|
|
||
| Group all findings across all categories and sort by severity per the Severity Ordering rule in `plugin-review-guidance.md`. | ||
|
|
||
| For each finding, output: | ||
|
|
||
| ``` | ||
| ### [{severity}] {category}: {short description} | ||
|
|
||
| **File:** `{file_path}:{line_number}` | ||
|
|
||
| {Explanation of what the guideline requires, what the code does wrong, and how to fix it.} | ||
| ``` | ||
|
|
||
| If there are zero findings across all categories, output: | ||
|
|
||
| > All checks passed. No findings. | ||
|
|
||
| ### Summary (output last) | ||
|
|
||
| End with a one-line summary: | ||
|
|
||
| > **Audit result: {total_findings} findings ({never_count} NEVER, {must_count} MUST, {should_count} SHOULD) across {failing_categories} failing and {warning_categories} warning categories.** | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,125 @@ | ||
| --- | ||
| description: Review plugin changes against AppKit best practices (composes with review-pr) | ||
| argument-hint: [plugin-name or base-branch] | ||
| --- | ||
|
|
||
| # Review Core Plugin Changes | ||
|
|
||
| User input: $ARGUMENTS | ||
|
|
||
| ## Step 0: Parse Input | ||
|
|
||
| Parse `$ARGUMENTS` deterministically: | ||
|
|
||
| - If `$ARGUMENTS` is empty: | ||
| - Use no plugin name filter. | ||
| - Use `origin/main` as the base branch. | ||
| - Otherwise, check whether either of these paths exists: | ||
| - `packages/appkit/src/plugins/$ARGUMENTS` | ||
| - `packages/appkit/src/connectors/$ARGUMENTS` | ||
| - If either path exists: | ||
| - Treat `$ARGUMENTS` as the **plugin name filter**. | ||
| - Use `origin/main` as the base branch. | ||
| - Otherwise: | ||
| - Treat `$ARGUMENTS` as the **base branch**. | ||
| - Use no plugin name filter. | ||
|
|
||
| Do not use a name-pattern heuristic such as "kebab-case with no slashes" to decide whether `$ARGUMENTS` is a plugin name, because common branch names like `feature-x` and `bugfix-foo` would be ambiguous. | ||
|
|
||
| ## Step 1: Core Principles Review | ||
|
|
||
| First, invoke the `review-pr` skill to run the standard Core Principles review. Pass the base branch as the argument (not the plugin name). | ||
|
|
||
| Use the Skill tool: | ||
| - skill: `review-pr` | ||
| - args: the base branch determined in Step 0 (or empty to use default) | ||
|
|
||
| Wait for this review to complete before continuing. | ||
|
|
||
| ## Step 2: Diff Analysis | ||
|
|
||
| Run `git diff <base-branch>...HEAD --name-only` to get all changed files. | ||
|
|
||
| Filter the file list to plugin-relevant paths: | ||
| - `packages/appkit/src/plugins/**` | ||
| - `packages/appkit/src/connectors/**` | ||
|
|
||
| If a specific plugin name was provided in Step 0, further filter to only files matching that plugin name in the path. | ||
|
|
||
| **If no plugin files are found in the diff**, output: | ||
|
|
||
| > No plugin files were changed in this branch. Plugin best-practices review is not applicable. Only the Core Principles review above applies. | ||
|
|
||
| Then stop. Do not continue to subsequent steps. | ||
|
|
||
| ## Step 3: Multi-Plugin Detection | ||
|
|
||
| If no specific plugin name was provided, detect all distinct plugins touched in the diff by extracting the plugin directory name from each changed path: | ||
| - From `packages/appkit/src/plugins/{name}/...` extract `{name}` | ||
| - From `packages/appkit/src/connectors/{name}/...` extract `{name}` | ||
|
|
||
| Deduplicate the list. You will run Steps 4-6 for **each** detected plugin. | ||
|
|
||
| ## Step 4: Category Scoping | ||
|
|
||
| For each plugin being reviewed, use the Category Index from `.claude/references/plugin-review-guidance.md` as the canonical list of categories. Map changed files to relevant categories using the "What to check" column as a guide — match file names and code patterns (e.g., `this.route(` → Route Design, `asUser(` → asUser / OBO Patterns). A single file may trigger multiple categories. | ||
|
|
||
| Read the actual changed file contents with `git diff <base-branch>...HEAD -- <file>` to determine which patterns are present. | ||
|
|
||
| Record which of the 9 categories are **relevant** (at least one changed file maps to them) and which are **skipped** (no changed files map to them). | ||
|
|
||
| ## Step 5: Load Best Practices Reference | ||
|
|
||
| Read the file `.claude/references/plugin-best-practices.md`. | ||
|
|
||
| For each **relevant** category identified in Step 4, extract all NEVER, MUST, and SHOULD guidelines from that category section. | ||
|
|
||
| ## Step 6: Best-Practices Review | ||
|
|
||
| Before evaluating, read the shared review rules in `.claude/references/plugin-review-guidance.md` and apply them throughout this step (deduplication, cache-key tracing). | ||
|
|
||
| For each plugin detected in Step 3, review the changed code against the scoped guidelines from Step 5. | ||
|
|
||
| For each finding: | ||
| - Identify the **severity** (NEVER, MUST, or SHOULD) | ||
| - Identify the **category** (e.g., "Manifest Design", "Route Design") | ||
| - Cite the specific guideline being violated or satisfied | ||
| - Reference the exact file and line(s) involved | ||
| - Provide a concrete fix if it is a violation | ||
|
|
||
| ## Step 7: Output | ||
|
|
||
| ### Format | ||
|
|
||
| For each plugin reviewed, output a section with the plugin name as the heading. | ||
|
|
||
| Order findings by severity per the Severity Ordering rule in `plugin-review-guidance.md`. | ||
|
|
||
| Each finding should follow this format: | ||
|
|
||
| ``` | ||
| ### [SEVERITY] Category Name: Brief description | ||
| - **File:** `path/to/file.ts:L42` | ||
| - **Guideline:** <quote the specific guideline> | ||
| - **Finding:** <what the code does wrong or right> | ||
| - **Fix:** <concrete fix, if a violation> | ||
| ``` | ||
|
|
||
| If a plugin has **no findings** (all scoped guidelines are satisfied), state that explicitly. | ||
|
|
||
| ### Skipped Categories | ||
|
|
||
| At the end of the output (after all plugin reviews), list the categories that were **not relevant** to this diff: | ||
|
|
||
| ``` | ||
| ### Skipped Categories (not relevant to this diff) | ||
| - Category N: <Name> — no changed files matched this category | ||
| - ... | ||
| ``` | ||
|
|
||
| ### Summary | ||
|
|
||
| End with an overall summary: | ||
| - Total findings by severity (e.g., "0 NEVER, 2 MUST, 3 SHOULD") | ||
| - Whether the changes are ready to merge from a plugin best-practices perspective | ||
| - Any categories that deserve attention even though they were skipped (e.g., "No tests were changed — consider adding tests for the new route") |
Uh oh!
There was an error while loading. Please reload this page.