Support json mode for build context transfer#1739
Open
JaewonHur wants to merge 2 commits into
Open
Conversation
* Add "transfer-mode" key to build metadata, and plumb CLI to support "tar", and "json" modes. * Handle json mode in BuildFSSync proxy. Json mode doesn't archive build context, but sends only the file metadata to the builder shim. * Fix constructing file metadata for symlink. i) all ancestor of a symlink target should be included in the file metadata to shim, ii) all child entries under directory target should be included, iii) isDir must be false for symlink.
Test both tar and json mode build context transfer. This doubles the test time, but total time for build is not that large.
JaewonHur
force-pushed
the
context-mode-json
branch
from
183c722 to
820892a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Type of Change
Motivation and Context
This resolves #1515.
This relies on apple/container-builder-shim#84.
Json mode avoids computing overhead on host side due to the archiving of build context. Instead, it feeds file on demand as the buildkitd ask for the file contents. Performance tradeoff depends on each users' workloads. If build context contains lots of files which cannot be ignored, tar mode would be better, if most of the files can be ignored, then json mode is better.
Testing