Support for vendoring on a single platform

[nipunn1313]

View all comments

See alexcrichton/cargo-vendor#70

Would be nice to be able to vendor on linux without pulling in windows-specific dependencies.
This would still be something we would use.

Related issues:

• Don't require existence of dependencies not needed for the current --target #5896
• Allow specifying a set of supported target platforms in Cargo.toml #6179
• Resolve dependencies based on feasible target/target_os platform combinations #15563

[kvark]

@alexcrichton this is actually hurting us a bit with Gecko as well. I'm trying to vendor https://github.com/grovesNL/spirv_cross which has a wasm-specific dependency:

[target.wasm32-unknown-unknown.dependencies]
wasm-bindgen = "0.2.33"
js-sys = "0.3.10"

The difference with the original issue is that we'd need a set of targets to filter on - the set Gecko needs to be building for.

[alexcrichton]

It's worth copying over the context of alexcrichton/cargo-vendor#70 (comment) here I think. tl;dr; this is, I believe, blocked on #5133

Certainly a feature I'd like to implement! Supporting this will be pretty difficult though, unfortunately. Cargo currently requires that the lock file is platform-independent, meaning that the lock file (and generating the lock file) always has to locate all crates for all possible platforms. This means that if you were to remove the winapi folder, for example, then a lock file would no longer be correctly generated.

There's a tracking issue for this at #5133 which is a possible way to pass a flag to Cargo and say "please generate a minimal lock file", and with that we can safely delete all the dependencies for other platforms.

[thefallentree]

+1 this would be very useful.

[est31]

An alternative to solve this issue with platform specific lockfiles would be to generate empty stubs for crates left out from the build, just with enough information for cargo to make sense of the passed lockfile.

To give a different way of looking at this proposal: this method is already implemented for unvendored crates.io crates in the form of the index. Stub Cargo.tomls would be nothing else than extracting those parts of the index that are relevant so that the vendor setting doesn't need to use the index. Only the format would be different from the index's json format and use Cargo.toml instead.

The advantage: it would only require functional changes in cargo-vendor but not cargo itself, and especially not require complicated resolver changes.

I think from a practical standpoint, this approach could solve the main objections of people, weird files* included by dependencies and the size bloat. It might seem like a hack at first sight, but as I've outlined above, the same thing is done with the index already.

*: Note that I wouldn't be unhappy if raw dylibs finally got implemented so that winapi can drop those files

[ehuss]

@Eh2406 wrote some details about this at #8723 (comment).

[avindra]

This affects https://github.com/denoland/deno too. Here is the layout of Deno's vendor tarball created on Linux. >20% of the size of the vendor directory comes from Winapi, which is amazing considering it also contains the v8 engine and portions of Chrome's bespoke build system.

[avindra]

Sharing a workaround to remove the bulk of the bulky .a files from winapi*gnu. I am still able to compile Deno on linux after removing these files:

rm -fr vendor/winapi*gnu*/lib/*.a

Those files account for the bulk of the contribution from the winapi folders (they are the orange boxes in the two largest yellow regions of the previously attached screenshot).