Skip to content

Commit 489ccf1

Browse files
anvansterclaude
andauthored
feat(release): publish a linux-arm64 engine (#20)
* feat(release): publish a linux-arm64 engine ARM Linux users had no engine at all: platformBinaryName() returned null for linux-arm64, so npm's postinstall printed "unsupported platform" and exited, and the JetBrains resolver refused too. Issue #15 came from one of the few people who worked around that by building from source. Adds the asset to both client mappings and to the release, and pins how it is built: scripts/build-linux-arm64.sh builds in a container because there is no arm64 Linux build machine, and the compatibility floor is easy to get wrong. It does not only build - it refuses to produce a binary that would narrow support: - __libc_single_threaded must land in writable memory. An immutable definition is read-only and, on aarch64, also exported, so glibc's startup write to it faults before main(). That was issue #15, and this is its regression guard. - the glibc floor must stay <= 2.30 and GLIBCXX <= 3.4.29, the floors measured from the shipped x64 asset. Ubuntu 20.04 + gcc-11 is the only combination measured to give both a working link and those floors. The alternatives failed and the reasons are recorded in the script: Debian 11 (gcc 10) cannot resolve ONNX Runtime's __throw_bad_array_new_length or __aarch64_cas8_sync; SLES 15 SP4's own gcc11 ships no outline atomics; Ubuntu 22.04 links but floors at glibc 2.34, which would have given ARM a narrower support range than x64. Verified by running the built binary: it works on SLES 15 SP4, Ubuntu 22.04+, Debian 12+, RHEL 9+ and Amazon Linux 2023, and not on Ubuntu 20.04, Debian 11, RHEL 8 or Amazon Linux 2 - identical to the x64 asset. Note the binding constraint is GLIBCXX_3.4.29, not glibc, which is why it does not run on its own build host. Also collapses three hand-kept copies of the platform list into one. PUBLISHED_BINARIES in bin/fetch-engine.js is now the source of truth, read by publish-release-assets.sh and package-npm.sh, so a sixth platform cannot be published without every client resolving it - the drift that made this change touch five files. A `while read` loop rather than mapfile: these scripts run under macOS's bash 3.2, where mapfile does not exist and would have silently yielded an empty list. The two tests asserting linux-arm64 was unsupported are inverted rather than deleted; both languages now assert it resolves to its own asset and never to the x64 one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017rVbt7rENTwXkdHt3Bpgb5 * docs: state the platform support matrix, including both Linux floors The project published no platform requirements anywhere users could read them, while the internal build notes claimed a Linux range that was wrong. A user on RHEL 8 hitting `GLIBCXX_3.4.29 not found` had nothing to consult. Two floors matter and only one of them is obvious. glibc 2.30 is not the binding constraint: the engine embeds ONNX Runtime, built with GCC 11, so it also needs libstdc++ >= GLIBCXX_3.4.29. Checking only glibc is what produced the earlier claim that Debian 11 and Ubuntu 20.04 were supported; both ship 3.4.28 and have never been able to start the engine. The runs / does-not-run table is measured by executing the 0.20.1 binaries on each distribution, not inferred from symbol versions, and is identical for linux-x64 and linux-arm64. Also documents linux-arm64 as supported, and gives the GLIBCXX error the remedy that does not require a distro upgrade. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017rVbt7rENTwXkdHt3Bpgb5 * no-mistakes(review): gate arm64 build provenance, harden shim check, fix stale docs * no-mistakes(review): record source commit per asset and gate release on agreement * no-mistakes(document): correct stale engine platform counts in client docs * no-mistakes(document): replace dangling build-notes pointers, correct engine download sizes * no-mistakes(lint): restore vscode eslint config, clear resulting lint errors --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
1 parent 71f0f0f commit 489ccf1

23 files changed

Lines changed: 761 additions & 97 deletions

README.md

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -405,6 +405,42 @@ A single Rust binary serves both MCP and LSP protocols.
405405

406406
---
407407

408+
## Supported platforms
409+
410+
The engine is a native binary, downloaded for your platform on first run.
411+
412+
| Platform | Architectures |
413+
|---|---|
414+
| macOS | Apple Silicon (arm64) and Intel (x64) |
415+
| Linux | x64 and arm64 |
416+
| Windows | x64 (Windows on ARM runs the x64 build under emulation) |
417+
418+
**Linux requires glibc 2.30 or newer *and* a libstdc++ from GCC 11 or newer
419+
(`GLIBCXX_3.4.29`).** The second requirement is the binding one, and it is not
420+
implied by the first — the engine embeds ONNX Runtime, which is built with
421+
GCC 11.
422+
423+
| Runs | Does not run |
424+
|---|---|
425+
| SLES 15 SP4 | Ubuntu 20.04 |
426+
| Ubuntu 22.04 and newer | Debian 11 |
427+
| Debian 12 and newer | RHEL / CentOS 8 |
428+
| RHEL 9 and newer | Amazon Linux 2 |
429+
| Amazon Linux 2023 | |
430+
431+
Both Linux architectures have identical requirements. If the engine exits
432+
immediately with a message like
433+
434+
```
435+
version `GLIBCXX_3.4.29' not found (required by codegraph-server)
436+
```
437+
438+
the distribution's C++ runtime is older than the engine needs; installing a
439+
newer `libstdc++` (for example RHEL 8's `gcc-toolset-11`) resolves it without
440+
upgrading the distribution.
441+
442+
---
443+
408444
## Building from Source
409445

410446
```bash

jetbrains/README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -39,8 +39,8 @@ the dependency ever becomes a problem.
3939
## Engine resolution
4040

4141
The plugin does **not** bundle engine binaries, and neither does any other
42-
client any more: bundling all four platforms meant a ~120 MB download for the
43-
one binary a given user can actually run.
42+
client any more: bundling every platform meant a download several times the size
43+
of the one binary a given user can actually run.
4444
The engine is published once as GitHub release assets and each client fetches
4545
what its platform needs, into the shared `~/.codegraph/bin`.
4646

jetbrains/src/main/kotlin/ai/codegraph/jetbrains/indexing/IndexingStartupActivity.kt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ class IndexingStartupActivity : ProjectActivity {
3838

3939
val resolved = CodeGraphServerResolver.resolve(project.basePath, settings.serverPath)
4040
if (resolved == null) {
41-
// Offered rather than done automatically: this is a ~30 MB download
41+
// Offered rather than done automatically: this is a ~120 MB download
4242
// of a native binary that will run with the user's permissions, and
4343
// starting that unasked on project open is not a decision the
4444
// plugin should make for them.

jetbrains/src/main/kotlin/ai/codegraph/jetbrains/server/CodeGraphServerResolver.kt

Lines changed: 22 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -59,13 +59,13 @@ data class ResolverEnvironment(
5959
/**
6060
* Locates the `codegraph-server` engine binary.
6161
*
62-
* Resolution order mirrors `vscode/src/server.ts`, with one deliberate
63-
* difference: the JetBrains plugin does not bundle platform binaries. The VSIX
64-
* carries four of them (100-126 MB each) because VS Code can ship per-platform
65-
* artifacts; the JetBrains Marketplace cannot, so a bundled plugin would be a
66-
* ~120 MB download for every user regardless of platform. Instead the binary is
67-
* resolved from an existing install and, failing that, downloaded once into the
68-
* managed install directory (Phase 1).
62+
* Resolution order mirrors `vscode/src/server.ts`. No client bundles platform
63+
* binaries any more (each engine is 100-126 MB), and the case against it is
64+
* strongest here: VS Code could at least ship one artifact per platform, while
65+
* the JetBrains Marketplace serves a single artifact to everyone, so a bundled
66+
* plugin would carry every published engine to every user. Instead the binary
67+
* is resolved from an existing install and, failing that, downloaded once into
68+
* the managed install directory (Phase 1).
6969
*
7070
* Order:
7171
* 1. Explicit user override (settings)
@@ -83,14 +83,18 @@ object CodeGraphServerResolver {
8383
/**
8484
* Binary name for this platform, or null when no engine is published for it.
8585
*
86-
* Only macOS is built for both architectures. Windows on ARM runs the x64
87-
* build under the OS's own emulation layer, so it is served the x64 asset;
88-
* Linux has no such layer, and falling back to x64 there installs ~30 MB
89-
* that cannot execute, which surfaces as an exec-format error at first use
90-
* instead of as the unsupported platform it is.
86+
* macOS and Linux are both built for x64 and arm64, and are answered by
87+
* exact platform-arch match and nothing else: falling back to x64 on an
88+
* arm64 machine installs ~120 MB that cannot execute, which surfaces as an
89+
* exec-format error at first use instead of as the unsupported platform it
90+
* is. Windows on ARM is the one exception - it runs the x64 build under the
91+
* OS's own emulation layer, so refusing it would leave those users with no
92+
* engine at all.
9193
*
9294
* Mirrors `platformBinaryName()` in `mcp-package/bin/fetch-engine.js`, which
93-
* is the same rule for the JavaScript channels.
95+
* is the same rule for the JavaScript channels. The plugin cannot import
96+
* that list, so this mapping has to be edited in lockstep with it whenever a
97+
* platform is added or dropped.
9498
*/
9599
fun platformBinaryNameOrNull(env: ResolverEnvironment = ResolverEnvironment.fromSystem()): String? {
96100
val os = env.osName.lowercase()
@@ -104,7 +108,11 @@ object CodeGraphServerResolver {
104108
else -> null
105109
}
106110
os.contains("win") -> if (isX64 || isArm64) "codegraph-server-win32-x64.exe" else null
107-
os.contains("linux") -> if (isX64) "codegraph-server-linux-x64" else null
111+
os.contains("linux") -> when {
112+
isArm64 -> "codegraph-server-linux-arm64"
113+
isX64 -> "codegraph-server-linux-x64"
114+
else -> null
115+
}
108116
else -> null
109117
}
110118
}

jetbrains/src/main/kotlin/ai/codegraph/jetbrains/server/EngineDownloader.kt

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -18,11 +18,11 @@ import java.util.Locale
1818
* Fetches the engine for this platform into the managed install directory.
1919
*
2020
* The plugin does not bundle engines: the JetBrains Marketplace serves one
21-
* artifact to every platform, so bundling all four would mean a ~120 MB
22-
* download for every user to obtain the ~30 MB they can run. The alternative
23-
* for users without Node is worse - install a 498 MB npm package for one
24-
* binary - so the engine is fetched directly from the release that
25-
* `scripts/publish-release-assets.sh` produces.
21+
* artifact to every platform, so bundling every published engine would mean a
22+
* download several times the size of the one a given user can run. Sending
23+
* users to the npm package instead is not an answer either: it needs Node, and
24+
* it fetches the same engine from the same release. So the engine is fetched
25+
* directly from the release that `scripts/publish-release-assets.sh` produces.
2626
*
2727
* Downloads are verified against the checksum published beside each asset. An
2828
* engine is a native binary that runs with the user's permissions; TLS says

jetbrains/src/test/kotlin/ai/codegraph/jetbrains/server/CodeGraphServerResolverTest.kt

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -160,18 +160,19 @@ class CodeGraphServerResolverTest : BasePlatformTestCase() {
160160
}
161161
}
162162

163-
fun `test arm64 linux has no published engine but arm64 windows emulates x64`() {
164-
// Handing the x64 asset to an arm64 Linux machine installs something
165-
// that cannot execute, which shows up as an exec-format error rather
166-
// than as the missing build it is. Windows on ARM is the exception: it
167-
// runs x64 binaries under the OS's own emulation, so refusing there
168-
// would leave those users with no engine for no reason.
163+
fun `test arm64 linux gets its own engine and arm64 windows emulates x64`() {
164+
// Linux arm64 has its own published build, so it must resolve to that
165+
// and never to the x64 asset: handing x64 to an arm64 machine installs
166+
// something that cannot execute, which shows up as an exec-format error
167+
// rather than as the wrong build it is. Windows on ARM is the
168+
// exception - it runs x64 under the OS's own emulation, so refusing
169+
// there would leave those users with no engine for no reason.
169170
fun nameFor(os: String, arch: String) = CodeGraphServerResolver.platformBinaryNameOrNull(
170171
ResolverEnvironment(fakeHome, emptyList(), os, arch),
171172
)
172173

173-
assertNull(nameFor("Linux", "aarch64"))
174-
assertNull(nameFor("Linux", "arm64"))
174+
assertEquals("codegraph-server-linux-arm64", nameFor("Linux", "aarch64"))
175+
assertEquals("codegraph-server-linux-arm64", nameFor("Linux", "arm64"))
175176
assertEquals("codegraph-server-win32-x64.exe", nameFor("Windows 11", "aarch64"))
176177
assertEquals("codegraph-server-win32-x64.exe", nameFor("Windows 11", "arm64"))
177178
assertEquals("codegraph-server-linux-x64", nameFor("Linux", "x86_64"))

jetbrains/src/test/kotlin/ai/codegraph/jetbrains/server/EngineDownloaderTest.kt

Lines changed: 21 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -64,9 +64,12 @@ class EngineDownloaderTest : BasePlatformTestCase() {
6464
ResolverEnvironment(homeDir = home, pathEntries = emptyList(), osName = os, osArch = arch)
6565

6666
/**
67-
* Windows and Linux are published for x64 only, so an arm64 environment
68-
* there is an unsupported platform rather than a machine that downloads the
69-
* x64 build.
67+
* Defaults to arm64 because that is the machine these tests describe: macOS
68+
* and Linux each publish their own arm64 engine, and Windows on ARM is
69+
* served the x64 one it emulates. The Windows cases below pass `amd64`
70+
* explicitly so they read as the platform they are testing rather than
71+
* relying on that fallback. Which name each pair resolves to is
72+
* `CodeGraphServerResolverTest`'s subject, not this file's.
7073
*/
7174
private fun downloader(os: String, arch: String = "aarch64") =
7275
EngineDownloader(env(os, arch), baseUrl())
@@ -81,6 +84,21 @@ class EngineDownloaderTest : BasePlatformTestCase() {
8184
assertTrue("the engine must be executable", path.toFile().canExecute())
8285
}
8386

87+
fun `test an arm64 linux ide installs the arm64 engine`() {
88+
// The platform this channel gained. Both Linux assets are published, so
89+
// a mapping that fell back to x64 would install cleanly here and only
90+
// fail when the IDE tried to start the engine - which is why the whole
91+
// download is exercised and not just the name it resolves to.
92+
publish("0.20.1", "codegraph-server-linux-arm64", "arm64 engine".toByteArray())
93+
publish("0.20.1", "codegraph-server-linux-x64", "x64 engine".toByteArray())
94+
95+
val path = downloader("Linux").download("0.20.1")
96+
97+
assertEquals("codegraph-server-linux-arm64", path.fileName.toString())
98+
assertEquals("arm64 engine", Files.readString(path))
99+
assertTrue("the engine must be executable", path.toFile().canExecute())
100+
}
101+
84102
fun `test windows also installs the runtime library the engine loads`() {
85103
publish("0.19.1", "codegraph-server-win32-x64.exe", "engine".toByteArray())
86104
publish("0.19.1", EngineDownloader.WINDOWS_SIDECAR, "onnx".toByteArray())

mcp-package/bin/fetch-engine.js

Lines changed: 34 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -113,6 +113,32 @@ const ARCH_MAP = { arm64: "arm64", x64: "x64", x86_64: "x64" };
113113
*/
114114
const VERSION_MARKER = ".engine-version";
115115

116+
/**
117+
* Every engine binary a release publishes.
118+
*
119+
* This is the single list for everything that can reach it. publish-release-
120+
* assets.sh uploads exactly these and package-npm.sh probes exactly these, both
121+
* by reading this export rather than repeating it - hand-kept copies of the same
122+
* list are how a release ends up publishing a platform no client asks for, or
123+
* asking for one it never published.
124+
*
125+
* One copy cannot be removed: `CodeGraphServerResolver.platformBinaryNameOrNull`
126+
* in the JetBrains plugin is Kotlin and cannot import this file, so it restates
127+
* the same rule and has to be edited alongside this list. Its own test asserts
128+
* the mapping; nothing can cross-check the two automatically.
129+
*
130+
* `onnxruntime.dll` is deliberately absent: it is a sidecar of the Windows
131+
* engine, not an engine, and requiredAssets() is what decides when it is
132+
* needed.
133+
*/
134+
const PUBLISHED_BINARIES = [
135+
"codegraph-server-darwin-arm64",
136+
"codegraph-server-darwin-x64",
137+
"codegraph-server-linux-arm64",
138+
"codegraph-server-linux-x64",
139+
"codegraph-server-win32-x64.exe",
140+
];
141+
116142
/**
117143
* Asset name for the running platform, matching the names
118144
* publish-release-assets.sh uploads. Returns null when unsupported, so callers
@@ -122,16 +148,16 @@ function platformBinaryName(platform = os.platform(), arch = os.arch()) {
122148
const p = PLATFORM_MAP[platform];
123149
const a = ARCH_MAP[arch];
124150
if (!p || !a) return null;
125-
// macOS is the only platform published for both architectures.
126-
if (p === "darwin") return `codegraph-server-darwin-${a}`;
127151
// Windows on ARM runs x64 executables under the OS's own emulation layer, so
128152
// the x64 asset is the correct answer there and refusing it would leave those
129-
// users with no engine at all.
153+
// users with no engine at all. Linux and macOS have no such layer, so they
154+
// are answered by exact platform-arch match and nothing else: handing an x64
155+
// build to an arm64 machine installs ~120 MB that cannot execute, which
156+
// surfaces as an exec-format error at first use rather than as the
157+
// unsupported platform it is.
130158
if (p === "win32") return "codegraph-server-win32-x64.exe";
131-
// Linux has no such layer. Handing the x64 build to an arm64 machine installs
132-
// ~30 MB that cannot execute, which surfaces as an exec-format error at first
133-
// use rather than as the unsupported platform it is.
134-
return a === "x64" ? "codegraph-server-linux-x64" : null;
159+
const name = `codegraph-server-${p}-${a}`;
160+
return PUBLISHED_BINARIES.includes(name) ? name : null;
135161
}
136162

137163
/** Numeric release components, or null when [version] is not one. */
@@ -415,6 +441,7 @@ async function ensureEngine(version, targetDir, options = {}) {
415441
module.exports = {
416442
RELEASE_BASE,
417443
ENGINE_VERSION,
444+
PUBLISHED_BINARIES,
418445
WINDOWS_SIDECAR,
419446
VERSION_MARKER,
420447
EngineInUseError,

mcp-package/test/fetch-engine.test.js

Lines changed: 64 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@ const {
2727
ensureEngine,
2828
requiredAssets,
2929
platformBinaryName,
30+
PUBLISHED_BINARIES,
3031
redirectTarget,
3132
installedVersion,
3233
compareVersions,
@@ -109,6 +110,39 @@ async function run() {
109110
}
110111
}
111112

113+
// --- an arm64 linux machine installs the arm64 engine ----------------
114+
// The whole install, not just the name mapping: before linux-arm64 was
115+
// published this threw, and the failure worth guarding against now is the
116+
// quiet one - the x64 asset is served alongside, so a rule that fell back to
117+
// it would install cleanly here and only fail when the user tried to run it.
118+
{
119+
const dir = scratch();
120+
const release = await startRelease({
121+
"codegraph-server-linux-arm64": { content: "arm64 engine" },
122+
"codegraph-server-linux-x64": { content: "x64 engine" },
123+
});
124+
try {
125+
const { binary, fetched } = await ensureEngine(VERSION, dir, {
126+
platform: "linux",
127+
arch: "arm64",
128+
baseUrl: release.baseUrl,
129+
});
130+
check(
131+
path.basename(binary) === "codegraph-server-linux-arm64",
132+
"an arm64 linux install fetches the arm64 engine"
133+
);
134+
check(fs.readFileSync(binary, "utf8") === "arm64 engine", "and it is the arm64 build");
135+
check(
136+
fetched.length === 1 && !fs.existsSync(path.join(dir, "codegraph-server-linux-x64")),
137+
"and nothing else, least of all the x64 build"
138+
);
139+
check(installedVersion(dir) === VERSION, "and the release it came from is recorded");
140+
} finally {
141+
release.server.close();
142+
fs.rmSync(dir, { recursive: true, force: true });
143+
}
144+
}
145+
112146
// --- a corrupted download installs nothing ---------------------------
113147
{
114148
const dir = scratch();
@@ -412,8 +446,12 @@ async function run() {
412446
// --- only published platform/arch pairs resolve to an asset ----------
413447
// An x64 asset handed to an arm64 Linux machine downloads and chmods cleanly
414448
// and then fails to exec, which is far harder to read than "not published".
415-
// Windows on ARM is the exception: it emulates x64, so the x64 build runs.
416-
check(platformBinaryName("linux", "arm64") === null, "linux-arm64 has no published engine");
449+
// Linux arm64 now has its own build, so it must resolve to that one and never
450+
// to the x64 asset. Windows on ARM stays the exception: it emulates x64.
451+
check(
452+
platformBinaryName("linux", "arm64") === "codegraph-server-linux-arm64",
453+
"linux-arm64 resolves to its own engine, not the x64 one"
454+
);
417455
check(
418456
platformBinaryName("win32", "arm64") === "codegraph-server-win32-x64.exe",
419457
"win32-arm64 uses the x64 engine, which Windows emulates"
@@ -427,7 +465,30 @@ async function run() {
427465
"darwin-arm64 does"
428466
);
429467
check(platformBinaryName("linux", "x64") === "codegraph-server-linux-x64", "linux-x64 does");
430-
check(requiredAssets("linux", "arm64").length === 0, "an unpublished pair needs no assets");
468+
check(
469+
requiredAssets("linux", "arm64").length === 1 &&
470+
requiredAssets("linux", "arm64")[0] === "codegraph-server-linux-arm64",
471+
"linux-arm64 needs the engine and no sidecar"
472+
);
473+
// A platform with no build must still resolve to nothing rather than to
474+
// someone else's binary.
475+
check(platformBinaryName("linux", "riscv64") === null, "an unbuilt arch resolves to nothing");
476+
check(requiredAssets("linux", "riscv64").length === 0, "an unpublished pair needs no assets");
477+
// Every name the mapping can return has to be a name the release publishes,
478+
// or an install fetches a 404.
479+
for (const [p, a] of [
480+
["darwin", "arm64"],
481+
["darwin", "x64"],
482+
["linux", "arm64"],
483+
["linux", "x64"],
484+
["win32", "x64"],
485+
]) {
486+
const name = platformBinaryName(p, a);
487+
check(
488+
PUBLISHED_BINARIES.includes(name),
489+
`${p}-${a} resolves to a published asset (${name})`
490+
);
491+
}
431492

432493
console.log("");
433494
console.log(`${failures} failure(s)`);

0 commit comments

Comments
 (0)