feat: supply-chain scanning plugin with chainPhase plugin hooks (RFC / discussion)#1639
Open
dcoric wants to merge 6 commits into
Open
feat: supply-chain scanning plugin with chainPhase plugin hooks (RFC / discussion)#1639dcoric wants to merge 6 commits into
dcoric wants to merge 6 commits into
Conversation
✅ Deploy Preview for endearing-brigadeiros-63f9d0 canceled.
|
| res.set('expires', 'Fri, 01 Jan 1980 00:00:00 GMT'); | ||
| res.set('pragma', 'no-cache'); | ||
| res.set('cache-control', 'no-cache, max-age=0, must-revalidate'); | ||
| res.status(200).send(handleErrPacket(message)); |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1639 +/- ##
==========================================
- Coverage 85.87% 85.86% -0.02%
==========================================
Files 84 84
Lines 8109 8159 +50
Branches 1376 1391 +15
==========================================
+ Hits 6964 7006 +42
- Misses 1117 1124 +7
- Partials 28 29 +1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Add an optional chainPhase option to PushActionPlugin so push plugins can run after the built-in getDiff step, with access to the computed diff and the cloned repo. Default 'start' preserves existing behaviour. Add @finos/git-proxy-plugin-supply-chain, an out-of-tree push plugin that flags common npm supply-chain-attack signatures on changed manifests and lockfiles: install lifecycle scripts, non-registry dependency sources, overrides/resolutions, typosquats, unpinned versions, and off-registry lockfile sources. Non-blocking by default; configurable to hard-block via failOn.
Extend the supply-chain scanner to Python dependency files. Add pattern based manifest classification (requirements*.txt) and a Python analyzer covering setup.py install-time code execution, custom package indexes (index-url and poetry/pipenv source blocks), non-registry sources (vcs/url/editable, inline git/url/path, PEP 508 direct references), unpinned requirements, typosquats against a PyPI reference list, and git/http lockfile sources. Generalise the typosquat helper so it serves both npm and PyPI, and enable the python ecosystem by default.
Add pull/clone protection. Introduce a new afterAuth pull chainPhase and a PullActionPlugin that, once a repo is confirmed authorised, shallow-clones the repository being fetched, enumerates its manifests and scans them with the existing npm/Python analyzers, treating the whole tree as newly introduced. Warn by default (findings logged, clone proceeds); block above pull.failOn, in which case the clone fails with the findings shown in the developer's terminal. HTTPS only; default branch scanned; SSH pulls and inline (non-blocking) terminal warnings are deferred.
A blocked pull/fetch previously reused the receive-pack sideband error
response, which is the wrong content type and framing for git-upload-pack
and a fetching client cannot render it. Send a protocol-correct
PKT-LINE("ERR" SP message) with the application/x-git-upload-pack-result
content type so git clone/fetch aborts and prints the reason. Add
handleErrPacket, route git-upload-pack POSTs to it in sendErrorResponse,
and cover it with unit tests. Also add SUPPLY-CHAIN.md, a manual test and
demo guide.
04105a1 to
2c44057
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is (and what it is not)
This is an RFC / discussion PR: a working proof of concept that demonstrates the idea end to end. We are not asking for this to be merged as-is. What we want is a conversation about two questions:
chainPhase) the right mechanism for plugins that need to run at specific points in the proxy chain?Everything in this branch works and is tested, but names, rule sets, config shape and the core hook design are all up for debate.
One more thing we want to be upfront about: this implementation was produced mostly with LLM assistance (human-directed architecture, design decisions and review; largely AI-generated code). Treat it accordingly. If any part of this proceeds beyond discussion, it should get thorough human review and be reshaped by maintainers first - for a control that can block pushes and clones, humans need full control of and confidence in every line, and we do not consider the AI-assisted authoring and verification below a substitute for that.
Demo
Both recordings are real: a live GitProxy instance with the plugin enabled (
failOn: "high"), against a public, deliberately poisoned testbed repository (dcoric/git-proxy-supplychain-testbed) that you can reproduce with yourself.Pull / clone protection - a developer clones a repository with a poisoned
go.modthrough the proxy and is stopped cold, with the full ranked findings report in their terminal:supply-chain-pull-block.mp4
Push protection (diff-aware) - a commit adding a typosquatted module and a
replaceto a raw-IP host is pushed through the proxy and rejected. Note that only the newly added lines are flagged; the pre-existing content of the repository is not re-reported:supply-chain-push-block.mp4
Why we built it
Supply-chain attacks through package manifests are now routine: install lifecycle scripts that download and execute payloads, typosquatted package names, dependencies redirected to attacker-controlled forks or raw IPs, lockfiles quietly resolving from off-registry hosts. The recent npm worm attacks and classics like event-stream all ride on exactly these signatures.
GitProxy already sits at the perfect enforcement point: every push and every clone in a controlled environment flows through it. Today the chain inspects pushes for policy, but nothing looks at what the code being pushed or pulled will do to the developer's machine at install time. A developer who clones a poisoned repository and runs
npm install/pip install/go buildhas already lost by the time any downstream scanner sees it.So we built protection for both directions:
What it detects
Three ecosystems so far, entirely offline (no network calls, no new runtime dependencies):
replacedirectives redirecting to local paths, forks or raw-IP hosts (the classic Go attack vector), suspicious require hosts (raw IPv4, localhost, plain http), pseudo-versions and +incompatible on new requires,toolchainchanges,excludeentries, typosquats with path-aware matching (host impersonation and tail typos,/vNand gopkg.in normalization), go.sum host checksSeverity is ranked info < low < medium < high < critical. Findings are attached to the action as a step (visible in the review dashboard for pushes) and rendered as a ranked, human-readable report.
Everything is non-blocking by default (
failOn: "off"): findings annotate the push review. SetfailOn/pull.failOnto a severity to hard-block. Config comes from a JSON file viaGIT_PROXY_SUPPLY_CHAIN_CONFIG, with per-ecosystem toggles and allow-lists.How it is built
The core change is deliberately tiny (3 files in
src/), everything else lives in a plugin:chainPhaseplugin option (src/plugin.ts,src/proxy/chain.ts):PushActionPluginandPullActionPluginaccept an optional{ chainPhase }.'afterDiff'inserts the plugin right after the built-ingetDiffstep, so it can read the computed diff and the already-cloned repo.'afterAuth'(pull) runs only for authorized repos. The default'start'preserves the historical index-0 insertion, so existing plugins are completely unaffected. This is the piece we would most like design feedback on: it is generic infrastructure, not supply-chain-specific, and other plugin authors likely have the same "I need the diff" problem.The scanner plugin (
plugins/git-proxy-plugin-supply-chain/, mirrorsgit-proxy-plugin-samples): purelib/analyzer modules with zero@finos/git-proxyimports (unit-testable without a build), a thinindex.jsthat wires them into push and pull plugin classes. Adding an ecosystem is mechanical: onelib/ecosystems/<eco>.js, a filename rule, a registry entry, a config flag. Go was added in exactly one commit on top of the npm/Python base, which we take as evidence the extension seams are right.Protocol-correct pull blocking (
src/proxy/routes/index.ts): blocked pulls previously reused the receive-pack sideband error path, which git clients cannot render for upload-pack responses. We added anERRPKT-LINE response with the correct content type, so a blocked clone shows the full findings report in the developer's terminal viaremote:lines and the clone fails cleanly.Where sideband streaming fits (#1637)
Blocking works today, and blocked clients see the full report. The gap is non-blocking warnings on pull: when findings are below the block threshold, the clone proceeds and the warnings only exist in server logs. There is currently no channel to whisper "this repo you just cloned has a postinstall script that curls a raw IP" into the developer's terminal while still letting the clone succeed.
@jescalada's sideband streaming work in #1637 is exactly the missing delivery mechanism. With progress/sideband streaming wired into the chain, warn-level supply-chain findings could be streamed live to the client during clone/fetch (and richer live feedback during push) without blocking anything. We built this expecting that synergy: the findings report is already rendered as terminal-friendly text and attached to the action step, waiting for a transport. If #1637 lands, this plugin becomes dramatically more useful in warn-only mode, which is how we expect most organizations would run it initially.
What we verified
test/plugins/supply-chain/), full suite at 1244 passing. Every rule has positive and negative cases, including malformed-input tests (analyzers never throw; the scanner never breaks a push or clone on its own error).pullExec/execplugin code against a live, deliberately poisoned GitHub testbed repo. Clone: 11 findings, blocked at critical, correct client-facing message. Push (clean manifest to poisoned manifest across two commits): 9 findings, blocked, and diff-awareness confirmed (only newly added modules flagged, pre-existing ones suppressed).npm run server, plugin loaded through the actual PluginLoader), a realgit clonethroughhttp://localhost:8000aborted withfatal: remote error:and the full report, and a realgit pushrejected over the receive-pack sideband withremote:lines. This exercised the entire stack including the Express routes, not just the chain.ERRpacket is additionally byte-verified intest/testProxyRoute.test.ts.load-plugin'skeydefaults to'default'), so the pull plugin's named export was silently dropped. Fixed by shipping the pull scanner as its own entry file (pull.js) registered as a secondpluginsentry. Worth noting for the plugin-system discussion: a module cannot currently export both a push and a pull plugin.SUPPLY-CHAIN.mdin this branch contains a 60-second no-proxy demo and a full proxy walkthrough for both push and pull protection.Honest limitations (we would rather over-disclose)
audit.tsskips PULL), so a blocked clone has no dashboard record.Where this could go next
lib/ecosystemsmodule plus registry entries).If the direction resonates we are happy to split this into reviewable pieces (core
chainPhasehook first, plugin second), rewrite whatever the maintainers want reshaped, or fold parts into existing efforts. Tear it apart.