feat(indexer): watchfiles-based auto-reindex of VAULT_PATH

[tricamtech]

Problem

vault-mcp's incremental indexing only runs in two situations:

1. At server startup — index_all does an mtime scan and re-embeds anything that changed since last run.
2. When the server's own write tools modify a file — vault_write and vault_edit call indexer.index_file() inline.

External changes are invisible:

• git pull bringing in new content from a synced repo
• Manual edits in Obsidian, VS Code, vim, etc.
• Sync scripts (rsync, syncthing, nextcloud, etc.)
• Another process editing the markdown directly

For any deployment where the vault directory is shared with anything other than vault-mcp itself — which is most of them, since markdown vaults are usually edited directly by humans or version-controlled with git — the index goes stale fast. The only fix today is systemctl restart, which is disruptive (especially for the OAuth-enabled deployment shape, where restart wipes the in-memory token store and forces re-auth on every connected client).

Fix

Add a background asyncio task that uses watchfiles to subscribe to filesystem events under VAULT_PATH and react in real time:

Event	Action
.md file added	indexer.index_file(rel_path)
.md file modified	indexer.index_file(rel_path)
.md file deleted	indexer._delete_files_chunks([rel_path])
Non-.md event	ignored
Watcher exception per-file	logged + skipped, watcher continues
Watcher crashes globally	logged via logger.exception, server keeps running with auto-reindex disabled

The watcher starts immediately after index_all completes (at the end of indexer.start()), so the server is fully ready before the watcher starts emitting events.

Configuration

• VAULT_WATCH env var (default true). Set VAULT_WATCH=false/0/no/off to disable the watcher entirely (revert to the existing restart-to-pick-up behavior).
• VaultIndexer(..., watch_files=True) constructor arg so library users can opt out without env vars.
• watchfiles>=0.21 added as an explicit dep in pyproject.toml. It was already pulled in transitively via uvicorn, but pinning it explicitly means the import won't break if uvicorn drops it later.

Verification

Full add/modify/delete cycle on a live server, no restart between steps:

$ mkdir /tmp/vault && echo '# initial' > /tmp/vault/a.md
$ VAULT_PATH=/tmp/vault SERVER_HOST=127.0.0.1 SERVER_PORT=8794 \
    python -m vault_mcp.server &
... INFO Filesystem watcher started on /tmp/vault

# Add
$ echo '# new file with Shadowfax the horse' > /tmp/vault/b.md
... INFO Watcher: re-indexing b.md (added)
$ vault_search "Shadowfax"
  → b.md, score 0.752

# Modify
$ echo '# revised, mentions Wisteria Lane' > /tmp/vault/a.md
... INFO Watcher: re-indexing a.md (modified)
$ vault_search "Wisteria"
  → a.md with new content, score 0.771
$ vault_search "old content phrase"
  → a.md no longer matches the old content

# Delete
$ rm /tmp/vault/b.md
... INFO Watcher: removing b.md
$ vault_search "Shadowfax"
  → only weak fallback match, b.md gone from the index

Notes

Found while deploying vault-mcp behind a Cloudflare Tunnel for cross-device access from claude.ai web. The vault is also git-synced from a separate machine, so the "restart to pick up new content" UX would have been painful — especially with OAuth enabled, since that flow wipes the in-memory token store on restart and requires re-auth in the connector dialog every time.

Sister to #1 (vault_reindex guard), #2 (MCP_ALLOWED_HOSTS), and #3 (PIN field maxlength).