[BUG] Sync can prepend a SECOND frontmatter block, shadowing `name`/`description` from any parser that reads the first

[dannybyrnes]

Bug Description

Five of 149 notes in a local synced project were found with two frontmatter blocks — a stub containing only permalink, then a blank line, then the note's real frontmatter:

---
permalink: myproject/my-note
---

---
name: my-note
description: The description that actually explains what this note is
metadata:
  node_type: memory
  type: reference
permalink: myproject/my-note
---

# real body starts here

Any consumer that parses the first frontmatter document — the normal behavior for YAML frontmatter — sees only permalink. name and description are invisible.

In my setup description is what an agent reads to decide whether a note is relevant, so those five notes became effectively unfindable despite having perfectly good descriptions. One of them was a 40KB note I rely on constantly. The failure is silent: nothing errors, the file looks fine when you open it, and the description is right there — just in the second block.

Steps To Reproduce

I could not reproduce this on demand. That's the main weakness of this report and I'd rather say so than invent a repro.

Two controlled probes on 0.22.1 both behaved correctly:

1. Note with permalink inside a single block → untouched by sync (md5 stable across 12s of polling).
2. Note without permalink, single block → sync merged permalink into the existing block, correctly. No second block.

What the five affected notes had in common: all carried older-style frontmatter — name / description / type / originSessionId at top level, with no metadata: wrapper and no permalink — i.e. the shape an earlier version produced. Two of the five were files I had just rewritten by hand in roughly that older shape, and the stub reappeared immediately after each write. Once I rewrote them as a single canonical block with permalink inside, it stopped recurring.

Best guess: a path where a note whose on-disk frontmatter doesn't line up with the DB record gets a permalink block emitted rather than merged. I couldn't isolate it further.

Expected Behavior

ensure_frontmatter_on_sync should always merge into the note's existing frontmatter block, and never emit a second YAML document ahead of it.

Actual Behavior

A second block is prepended before the real one, shadowing name and description for any first-block parser.

Environment

• OS: macOS 26.5.2 (Darwin 25.5.0), arm64
• Python version: 3.9 (system python3 used for the audit); basic-memory installed under its own uv-managed interpreter
• Basic Memory version: 0.22.1
• Installation method: uv tool (~/.local/share/uv/tools/basic-memory), running as the Claude Code plugin basicmachines-co/basic-memory 0.22.1
• Claude Desktop version: 1.24012.9 (Claude Code 2.1.219)

Additional Context

Relevant config from ~/.basic-memory/config.json:

{
  "env": "dev",
  "ensure_frontmatter_on_sync": true,
  "sync_changes": true,
  "sync_delay": 1000,
  "permalinks_include_project": true,
  "kebab_filenames": false,
  "disable_permalinks": false
}

Two suggestions that hold regardless of whether the exact trigger is found:

1. Make the writer merge-only. Locate the first --- … --- document and update it in place; never emit a new one ahead of it. That makes the bad state unreachable even if the triggering path stays unknown.
2. Add a detector. A health check — or a warning during sync — for any note with more than one leading frontmatter block would have surfaced this immediately. As it was, it sat undetected for weeks; I only found it because one note's description came back empty during an audit and I went looking for why.

Happy to run diagnostics against a specific code path if that would help narrow it down.

[phernandez]

@claude take a look at this

[claude]

Claude finished @phernandez's task in 5m 13s —— View job • claude/issue-1171-20260730-2344 • Create PR ➔

---

Investigating issue #1171

• Gather context: locate ensure_frontmatter_on_sync code path
• Understand how frontmatter merge/write can emit a duplicate block
• Determine root cause
• Implement fix (merge-only writer)
• Add regression test
• Run lint/typecheck/tests
• Push branch and provide PR link

Root cause

FileService.update_frontmatter_with_result in src/basic_memory/services/file_service.py only stripped the note's existing frontmatter block on the success path of parsing it:

current_fm = {}
if file_utils.has_frontmatter(content):
    try:
        current_fm = file_utils.parse_frontmatter(content)
        content = file_utils.remove_frontmatter(content)   # <- only reached on success
    except (ParseError, yaml.YAMLError) as e:               # pragma: no cover
        current_fm = {}                                     # block NOT stripped

If the existing block has valid --- fences but the YAML inside fails to parse (e.g. an unescaped special character in a description written by an older basic-memory version — plausible for the "older-style" notes described in the issue), the except branch resets the parsed dict to {} but never strips the block from content. The subsequent write then does:

final_content = f"---\n{yaml_fm}---\n\n{content.strip()}"

...which prepends a brand-new block (just the merge updates, e.g. {"permalink": ...}) directly in front of the untouched original block — exactly the two-block shadowing from the issue. This branch was marked # pragma: no cover, consistent with being an untested, hard-to-reproduce path — it explains why the bug was silent and intermittent.

Fix

content = file_utils.remove_frontmatter(content) now always runs once fences are detected, regardless of whether YAML parsing of the block succeeds. A malformed block is replaced in place instead of stacked on top of. This directly satisfies suggestion #1 from the issue ("make the writer merge-only... never emit a new one ahead of it").

Added test_update_frontmatter_replaces_malformed_block_instead_of_stacking in tests/services/test_file_service.py, which reproduces the exact failure (malformed-YAML frontmatter block) and asserts the result never contains a second --- block. just fast-check/lint/format pass on the changed files; the two pre-existing pymilvus typecheck errors are unrelated (missing optional dependency, present before this change). tests/services/test_file_service.py (23 tests) and tests/indexing/test_batch_indexer.py all pass.

Note: I didn't implement suggestion #2 (a health-check detector for multi-block notes) — the merge-only fix makes the bad state unreachable through this writer, which is the higher-leverage change; a detector could be a useful follow-up if maintainers want defense-in-depth against other write paths, but that's a separate scope decision.