Skip to content

feat(skill): skill search via BM25 index for large skill catalogues #725

@claudianus

Description

@claudianus

Problem

When a user has a large number of skills installed (e.g. 1,000+), the current implementation injects the entire skill listing into the system prompt — including name, description (250 chars), whenToUse, and file path for every skill.

With 1,530 real skills this amounts to ~118,000 tokens in the system prompt alone, which:

  1. Consumes ~46% of a 256K context window
  2. Loads all SKILL.md file bodies into memory at startup (~88 MB JS heap)
  3. Forces the LLM to scan a massive listing to find the right skill — poor accuracy due to lost-in-the-middle effects

Proposed Solution

A 3-tier auto-detection system based on invocable skill count:

Skill count Listing format Strategy
≤ 80 Legacy full listing (name + desc + path) Prompt-cache optimal for small catalogues
81–300 Compact (name + 80-char desc) Search available but not required
> 300 Names only Search required via action: "search"

Key changes:

  • BM25 search index built at startup from skill metadata (name, description, whenToUse). Zero external dependencies, ~24ms for 1,500 skills, <1ms per query.
  • Skill tool extended with action: "search" (new) alongside existing action: "load" (default). Backward-compatible — skill field remains required.
  • Lazy content loading: only frontmatter parsed at startup; full SKILL.md body loaded from disk on-demand when a skill is activated (~0.4ms per activation).
  • Auto-detection: no feature flag needed — tier selected by skill count at session start.

Token savings (measured with real 1,530 skills)

Metric Before After
System prompt 118,120 tokens 8,436 tokens (93% reduction)
Startup memory ~88 MB ~4 MB (95% reduction)
Search latency N/A 0.0–0.2ms

Files changed

  • skill/search.ts (new) — BM25 index with synonym expansion
  • skill/registry.ts — auto-detect tier, lazy load, searchSkills()
  • skill/parser.ts — parseSkillMetaFromFile() for frontmatter-only parsing
  • skill-tool.ts / .md — search action on existing Skill tool
  • system.md — search-first workflow instructions
  • Tests merged into existing registry.test.ts + integration proof in integration-proof.test.ts

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions