Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -405,6 +405,42 @@ A single Rust binary serves both MCP and LSP protocols.

---

## Supported platforms

The engine is a native binary, downloaded for your platform on first run.

| Platform | Architectures |
|---|---|
| macOS | Apple Silicon (arm64) and Intel (x64) |
| Linux | x64 and arm64 |
| Windows | x64 (Windows on ARM runs the x64 build under emulation) |

**Linux requires glibc 2.30 or newer *and* a libstdc++ from GCC 11 or newer
(`GLIBCXX_3.4.29`).** The second requirement is the binding one, and it is not
implied by the first — the engine embeds ONNX Runtime, which is built with
GCC 11.

| Runs | Does not run |
|---|---|
| SLES 15 SP4 | Ubuntu 20.04 |
| Ubuntu 22.04 and newer | Debian 11 |
| Debian 12 and newer | RHEL / CentOS 8 |
| RHEL 9 and newer | Amazon Linux 2 |
| Amazon Linux 2023 | |

Both Linux architectures have identical requirements. If the engine exits
immediately with a message like

```
version `GLIBCXX_3.4.29' not found (required by codegraph-server)
```

the distribution's C++ runtime is older than the engine needs; installing a
newer `libstdc++` (for example RHEL 8's `gcc-toolset-11`) resolves it without
upgrading the distribution.

---

## Building from Source

```bash
Expand Down
4 changes: 2 additions & 2 deletions jetbrains/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,8 @@ the dependency ever becomes a problem.
## Engine resolution

The plugin does **not** bundle engine binaries, and neither does any other
client any more: bundling all four platforms meant a ~120 MB download for the
one binary a given user can actually run.
client any more: bundling every platform meant a download several times the size
of the one binary a given user can actually run.
The engine is published once as GitHub release assets and each client fetches
what its platform needs, into the shared `~/.codegraph/bin`.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ class IndexingStartupActivity : ProjectActivity {

val resolved = CodeGraphServerResolver.resolve(project.basePath, settings.serverPath)
if (resolved == null) {
// Offered rather than done automatically: this is a ~30 MB download
// Offered rather than done automatically: this is a ~120 MB download
// of a native binary that will run with the user's permissions, and
// starting that unasked on project open is not a decision the
// plugin should make for them.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,13 +59,13 @@ data class ResolverEnvironment(
/**
* Locates the `codegraph-server` engine binary.
*
* Resolution order mirrors `vscode/src/server.ts`, with one deliberate
* difference: the JetBrains plugin does not bundle platform binaries. The VSIX
* carries four of them (100-126 MB each) because VS Code can ship per-platform
* artifacts; the JetBrains Marketplace cannot, so a bundled plugin would be a
* ~120 MB download for every user regardless of platform. Instead the binary is
* resolved from an existing install and, failing that, downloaded once into the
* managed install directory (Phase 1).
* Resolution order mirrors `vscode/src/server.ts`. No client bundles platform
* binaries any more (each engine is 100-126 MB), and the case against it is
* strongest here: VS Code could at least ship one artifact per platform, while
* the JetBrains Marketplace serves a single artifact to everyone, so a bundled
* plugin would carry every published engine to every user. Instead the binary
* is resolved from an existing install and, failing that, downloaded once into
* the managed install directory (Phase 1).
*
* Order:
* 1. Explicit user override (settings)
Expand All @@ -83,14 +83,18 @@ object CodeGraphServerResolver {
/**
* Binary name for this platform, or null when no engine is published for it.
*
* Only macOS is built for both architectures. Windows on ARM runs the x64
* build under the OS's own emulation layer, so it is served the x64 asset;
* Linux has no such layer, and falling back to x64 there installs ~30 MB
* that cannot execute, which surfaces as an exec-format error at first use
* instead of as the unsupported platform it is.
* macOS and Linux are both built for x64 and arm64, and are answered by
* exact platform-arch match and nothing else: falling back to x64 on an
* arm64 machine installs ~120 MB that cannot execute, which surfaces as an
* exec-format error at first use instead of as the unsupported platform it
* is. Windows on ARM is the one exception - it runs the x64 build under the
* OS's own emulation layer, so refusing it would leave those users with no
* engine at all.
*
* Mirrors `platformBinaryName()` in `mcp-package/bin/fetch-engine.js`, which
* is the same rule for the JavaScript channels.
* is the same rule for the JavaScript channels. The plugin cannot import
* that list, so this mapping has to be edited in lockstep with it whenever a
* platform is added or dropped.
*/
fun platformBinaryNameOrNull(env: ResolverEnvironment = ResolverEnvironment.fromSystem()): String? {
val os = env.osName.lowercase()
Expand All @@ -104,7 +108,11 @@ object CodeGraphServerResolver {
else -> null
}
os.contains("win") -> if (isX64 || isArm64) "codegraph-server-win32-x64.exe" else null
os.contains("linux") -> if (isX64) "codegraph-server-linux-x64" else null
os.contains("linux") -> when {
isArm64 -> "codegraph-server-linux-arm64"
isX64 -> "codegraph-server-linux-x64"
else -> null
}
else -> null
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,11 @@ import java.util.Locale
* Fetches the engine for this platform into the managed install directory.
*
* The plugin does not bundle engines: the JetBrains Marketplace serves one
* artifact to every platform, so bundling all four would mean a ~120 MB
* download for every user to obtain the ~30 MB they can run. The alternative
* for users without Node is worse - install a 498 MB npm package for one
* binary - so the engine is fetched directly from the release that
* `scripts/publish-release-assets.sh` produces.
* artifact to every platform, so bundling every published engine would mean a
* download several times the size of the one a given user can run. Sending
* users to the npm package instead is not an answer either: it needs Node, and
* it fetches the same engine from the same release. So the engine is fetched
* directly from the release that `scripts/publish-release-assets.sh` produces.
*
* Downloads are verified against the checksum published beside each asset. An
* engine is a native binary that runs with the user's permissions; TLS says
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -160,18 +160,19 @@ class CodeGraphServerResolverTest : BasePlatformTestCase() {
}
}

fun `test arm64 linux has no published engine but arm64 windows emulates x64`() {
// Handing the x64 asset to an arm64 Linux machine installs something
// that cannot execute, which shows up as an exec-format error rather
// than as the missing build it is. Windows on ARM is the exception: it
// runs x64 binaries under the OS's own emulation, so refusing there
// would leave those users with no engine for no reason.
fun `test arm64 linux gets its own engine and arm64 windows emulates x64`() {
// Linux arm64 has its own published build, so it must resolve to that
// and never to the x64 asset: handing x64 to an arm64 machine installs
// something that cannot execute, which shows up as an exec-format error
// rather than as the wrong build it is. Windows on ARM is the
// exception - it runs x64 under the OS's own emulation, so refusing
// there would leave those users with no engine for no reason.
fun nameFor(os: String, arch: String) = CodeGraphServerResolver.platformBinaryNameOrNull(
ResolverEnvironment(fakeHome, emptyList(), os, arch),
)

assertNull(nameFor("Linux", "aarch64"))
assertNull(nameFor("Linux", "arm64"))
assertEquals("codegraph-server-linux-arm64", nameFor("Linux", "aarch64"))
assertEquals("codegraph-server-linux-arm64", nameFor("Linux", "arm64"))
assertEquals("codegraph-server-win32-x64.exe", nameFor("Windows 11", "aarch64"))
assertEquals("codegraph-server-win32-x64.exe", nameFor("Windows 11", "arm64"))
assertEquals("codegraph-server-linux-x64", nameFor("Linux", "x86_64"))
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -64,9 +64,12 @@ class EngineDownloaderTest : BasePlatformTestCase() {
ResolverEnvironment(homeDir = home, pathEntries = emptyList(), osName = os, osArch = arch)

/**
* Windows and Linux are published for x64 only, so an arm64 environment
* there is an unsupported platform rather than a machine that downloads the
* x64 build.
* Defaults to arm64 because that is the machine these tests describe: macOS
* and Linux each publish their own arm64 engine, and Windows on ARM is
* served the x64 one it emulates. The Windows cases below pass `amd64`
* explicitly so they read as the platform they are testing rather than
* relying on that fallback. Which name each pair resolves to is
* `CodeGraphServerResolverTest`'s subject, not this file's.
*/
private fun downloader(os: String, arch: String = "aarch64") =
EngineDownloader(env(os, arch), baseUrl())
Expand All @@ -81,6 +84,21 @@ class EngineDownloaderTest : BasePlatformTestCase() {
assertTrue("the engine must be executable", path.toFile().canExecute())
}

fun `test an arm64 linux ide installs the arm64 engine`() {
// The platform this channel gained. Both Linux assets are published, so
// a mapping that fell back to x64 would install cleanly here and only
// fail when the IDE tried to start the engine - which is why the whole
// download is exercised and not just the name it resolves to.
publish("0.20.1", "codegraph-server-linux-arm64", "arm64 engine".toByteArray())
publish("0.20.1", "codegraph-server-linux-x64", "x64 engine".toByteArray())

val path = downloader("Linux").download("0.20.1")

assertEquals("codegraph-server-linux-arm64", path.fileName.toString())
assertEquals("arm64 engine", Files.readString(path))
assertTrue("the engine must be executable", path.toFile().canExecute())
}

fun `test windows also installs the runtime library the engine loads`() {
publish("0.19.1", "codegraph-server-win32-x64.exe", "engine".toByteArray())
publish("0.19.1", EngineDownloader.WINDOWS_SIDECAR, "onnx".toByteArray())
Expand Down
41 changes: 34 additions & 7 deletions mcp-package/bin/fetch-engine.js
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,32 @@ const ARCH_MAP = { arm64: "arm64", x64: "x64", x86_64: "x64" };
*/
const VERSION_MARKER = ".engine-version";

/**
* Every engine binary a release publishes.
*
* This is the single list for everything that can reach it. publish-release-
* assets.sh uploads exactly these and package-npm.sh probes exactly these, both
* by reading this export rather than repeating it - hand-kept copies of the same
* list are how a release ends up publishing a platform no client asks for, or
* asking for one it never published.
*
* One copy cannot be removed: `CodeGraphServerResolver.platformBinaryNameOrNull`
* in the JetBrains plugin is Kotlin and cannot import this file, so it restates
* the same rule and has to be edited alongside this list. Its own test asserts
* the mapping; nothing can cross-check the two automatically.
*
* `onnxruntime.dll` is deliberately absent: it is a sidecar of the Windows
* engine, not an engine, and requiredAssets() is what decides when it is
* needed.
*/
const PUBLISHED_BINARIES = [
"codegraph-server-darwin-arm64",
"codegraph-server-darwin-x64",
"codegraph-server-linux-arm64",
"codegraph-server-linux-x64",
"codegraph-server-win32-x64.exe",
];

/**
* Asset name for the running platform, matching the names
* publish-release-assets.sh uploads. Returns null when unsupported, so callers
Expand All @@ -122,16 +148,16 @@ function platformBinaryName(platform = os.platform(), arch = os.arch()) {
const p = PLATFORM_MAP[platform];
const a = ARCH_MAP[arch];
if (!p || !a) return null;
// macOS is the only platform published for both architectures.
if (p === "darwin") return `codegraph-server-darwin-${a}`;
// Windows on ARM runs x64 executables under the OS's own emulation layer, so
// the x64 asset is the correct answer there and refusing it would leave those
// users with no engine at all.
// users with no engine at all. Linux and macOS have no such layer, so they
// are answered by exact platform-arch match and nothing else: handing an x64
// build to an arm64 machine installs ~120 MB that cannot execute, which
// surfaces as an exec-format error at first use rather than as the
// unsupported platform it is.
if (p === "win32") return "codegraph-server-win32-x64.exe";
// Linux has no such layer. Handing the x64 build to an arm64 machine installs
// ~30 MB that cannot execute, which surfaces as an exec-format error at first
// use rather than as the unsupported platform it is.
return a === "x64" ? "codegraph-server-linux-x64" : null;
const name = `codegraph-server-${p}-${a}`;
return PUBLISHED_BINARIES.includes(name) ? name : null;
}

/** Numeric release components, or null when [version] is not one. */
Expand Down Expand Up @@ -415,6 +441,7 @@ async function ensureEngine(version, targetDir, options = {}) {
module.exports = {
RELEASE_BASE,
ENGINE_VERSION,
PUBLISHED_BINARIES,
WINDOWS_SIDECAR,
VERSION_MARKER,
EngineInUseError,
Expand Down
67 changes: 64 additions & 3 deletions mcp-package/test/fetch-engine.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ const {
ensureEngine,
requiredAssets,
platformBinaryName,
PUBLISHED_BINARIES,
redirectTarget,
installedVersion,
compareVersions,
Expand Down Expand Up @@ -109,6 +110,39 @@ async function run() {
}
}

// --- an arm64 linux machine installs the arm64 engine ----------------
// The whole install, not just the name mapping: before linux-arm64 was
// published this threw, and the failure worth guarding against now is the
// quiet one - the x64 asset is served alongside, so a rule that fell back to
// it would install cleanly here and only fail when the user tried to run it.
{
const dir = scratch();
const release = await startRelease({
"codegraph-server-linux-arm64": { content: "arm64 engine" },
"codegraph-server-linux-x64": { content: "x64 engine" },
});
try {
const { binary, fetched } = await ensureEngine(VERSION, dir, {
platform: "linux",
arch: "arm64",
baseUrl: release.baseUrl,
});
check(
path.basename(binary) === "codegraph-server-linux-arm64",
"an arm64 linux install fetches the arm64 engine"
);
check(fs.readFileSync(binary, "utf8") === "arm64 engine", "and it is the arm64 build");
check(
fetched.length === 1 && !fs.existsSync(path.join(dir, "codegraph-server-linux-x64")),
"and nothing else, least of all the x64 build"
);
check(installedVersion(dir) === VERSION, "and the release it came from is recorded");
} finally {
release.server.close();
fs.rmSync(dir, { recursive: true, force: true });
}
}

// --- a corrupted download installs nothing ---------------------------
{
const dir = scratch();
Expand Down Expand Up @@ -412,8 +446,12 @@ async function run() {
// --- only published platform/arch pairs resolve to an asset ----------
// An x64 asset handed to an arm64 Linux machine downloads and chmods cleanly
// and then fails to exec, which is far harder to read than "not published".
// Windows on ARM is the exception: it emulates x64, so the x64 build runs.
check(platformBinaryName("linux", "arm64") === null, "linux-arm64 has no published engine");
// Linux arm64 now has its own build, so it must resolve to that one and never
// to the x64 asset. Windows on ARM stays the exception: it emulates x64.
check(
platformBinaryName("linux", "arm64") === "codegraph-server-linux-arm64",
"linux-arm64 resolves to its own engine, not the x64 one"
);
check(
platformBinaryName("win32", "arm64") === "codegraph-server-win32-x64.exe",
"win32-arm64 uses the x64 engine, which Windows emulates"
Expand All @@ -427,7 +465,30 @@ async function run() {
"darwin-arm64 does"
);
check(platformBinaryName("linux", "x64") === "codegraph-server-linux-x64", "linux-x64 does");
check(requiredAssets("linux", "arm64").length === 0, "an unpublished pair needs no assets");
check(
requiredAssets("linux", "arm64").length === 1 &&
requiredAssets("linux", "arm64")[0] === "codegraph-server-linux-arm64",
"linux-arm64 needs the engine and no sidecar"
);
// A platform with no build must still resolve to nothing rather than to
// someone else's binary.
check(platformBinaryName("linux", "riscv64") === null, "an unbuilt arch resolves to nothing");
check(requiredAssets("linux", "riscv64").length === 0, "an unpublished pair needs no assets");
// Every name the mapping can return has to be a name the release publishes,
// or an install fetches a 404.
for (const [p, a] of [
["darwin", "arm64"],
["darwin", "x64"],
["linux", "arm64"],
["linux", "x64"],
["win32", "x64"],
]) {
const name = platformBinaryName(p, a);
check(
PUBLISHED_BINARIES.includes(name),
`${p}-${a} resolves to a published asset (${name})`
);
}

console.log("");
console.log(`${failures} failure(s)`);
Expand Down
Loading
Loading