Skip to content

Harper's spawn PID-lock contract (mandatory name, numeric version, ExistingProcessWrapper) is load-bearing for components and entirely undocumented; document it or bless an alternative #626

Description

@deliciousmonster

Components that need a once-per-node child process must build on a contract that exists only
in dist/security/jsLoader.js. Grepping the documentation repo at d4cbc1a7:

  • ExistingProcessWrapper: zero hits.
  • The spawn name option, the version option, pids/: zero hits. The only pid docs
    describe Harper's own hdb.pid (reference/cli/overview.md:82-93), a different mechanism.
  • What is documented is allowedSpawnCommands alone
    (reference/configuration/options.md:352-360, release-notes/v5-lincoln/v5-migration.md:108),
    which says nothing about what spawn() returns or requires beyond the allowlist.

The actual contract, read from harper 5.2.1 dist/security/jsLoader.js and proven by tests
(below):

  1. Allowlist before everything. ALLOWED_COMMANDS.has(command.split(' ')[0])
    (jsLoader.js:1025); exact-string match, so a path containing a space can never be
    allowlisted, and the set is captured once at module load.
  2. options.name is mandatory. Spawn without it throws
    (jsLoader.js:1029-1031); the name is also the lock filename.
  3. PID-file lock. openSync('<rootPath>/pids/<name>.pid', 'wx') (jsLoader.js:955);
    whoever creates the file spawns, everyone else gets an ExistingProcessWrapper.
  4. options.version must round-trip parseInt. Line 2 of the PID file is read with
    parseInt (jsLoader.js:948) and compared with !== against the requested value
    (jsLoader.js:967); a mismatch SIGTERMs the running process and re-acquires. Pass a
    string and it never equals its own recorded value, so every thread kills and respawns the
    child forever. Nothing tells the caller the option must be numeric.
  5. Race losers get ExistingProcessWrapper (jsLoader.js:893): an EventEmitter with
    pid, kill(), unref() and an exit event; no stdout/stderr/stdin, no
    spawnargs, and a 1 Hz liveness setInterval that is never unref'd, so a thread that
    does not call unref() has its event loop pinned through shutdown. Any code doing
    child.stdout.on(...) throws a TypeError on exactly the threads that lost the race.

Each of these is proven by execution against a real Harper 5.2.1 boot in the
datadog-agent-binary integration suite (test/integration/harper-spawn.test.ts at bbeb99a):

Behavior Test
spawn without name throws :475 ("NEGATIVE: spawn without a name option throws")
N worker threads collapse to one process and one PID file per name :541
losers hold a wrapper, detected by absent spawnargs :565 (the adopted assertion)
version is one integer shared across threads and recorded verbatim on PID-file line 2 :606
child exit unlinks the PID file :649

The consequence of the doc gap is not hypothetical: the shipped Datadog example
(example/dd-supervisor.js) needs roughly 200 lines of comments reverse-engineering these
five behaviors to run two agent binaries safely, including a startup probe whose only job is
to detect whether the substitution happened at all, because a supervisor imported via a bare
npm specifier (from a package that does not depend on harper) silently receives the real
child_process; no allowlist, no lock, one agent pair per worker thread.

The ask, either way is fine:

  • Document the contract in reference/components/ as supported API: the mandatory
    name, the numeric-version replacement semantics, the pids/<name>.pid format, the
    wrapper's shape (including the unref() requirement and the missing stdio), and which
    import paths receive the substituted module.
  • Or declare it internal and point components at a supported once-per-node primitive.
    The claim-with-heartbeat pattern in nextjs/src/buildLock.ts:54-124 (claim a table row
    with status: 'building', re-stamp on a heartbeat, siblings poll, stale claims expire) is
    a working in-house exemplar that does not depend on how the module was imported and
    survives applications.moduleLoader: native.

Reproduction

  1. In any Harper 5.2.1 component, import { spawn } from 'node:child_process' (the ESM path;
    require bypasses the substitution) and call spawn('node', ['-e', '...']) with no
    name. Observed: synchronous throw naming the missing "name".
  2. Spawn the same allowlisted command with the same name from two worker threads. Observed:
    one real ChildProcess; the other thread's return value has pid, kill, unref, and no
    spawnargs/stdout.
  3. Pass version: '42' (string). Observed after restart cycles: the child is killed and
    respawned on every thread's spawn call, because '42' !== parseInt('42', 10).

Steps 1-2 are automated in test/integration/harper-spawn.test.ts; step 3 is asserted from
the inverse direction at :606 (integer versions round-trip; the string failure mode is
derived from jsLoader.js:948,967 and flagged here as code-read, not separately executed).

Measured on

Component Version
harper 5.2.1 (dist/security/jsLoader.js:893-1071)
Node v24.16.0
OS macOS 26.5.2 (arm64); singleton suite also green on Debian 13 / linux-amd64 and arm64
documentation repo d4cbc1a7 (grep basis)
evidence checkout datadog-agent-binary bbeb99a (test/integration/harper-spawn.test.ts, example/dd-supervisor.js)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions