merge: Currency merge with upstream 2.18 branch#53
Draft
erickshepherdNI wants to merge 86 commits into
Draft
Conversation
In the past, broken SSL certificates were common on subversion servers. As such, the subversion fetcher used to ignore these issues. Cert infrastructure has massively improved since that decision was made and things like self signed certificates should no longer be common place. We should follow good security practises and not have this as a default anymore, remove the --trust-server-cert commandline option by default. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When the user provide a path that does not exist it fails with: | stat: cannot read file system information for '/path/to/my/hashserve': No such file or directory We can strip the non exising part in the path and pass just that to the stat. Signed-off-by: Jose Quaresma <jose.quaresma@oss.qualcomm.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
…hing BBMASK matches files in the directories matching the regex, and also in their respective subdirectories, so make that clear in the wording leading to the example. Signed-off-by: Quentin Schulz <quentin.schulz@cherry.de> Reviewed-by: Antonin Godard <antonin.godard@bootlin.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
…s be consistent It's a bit confusing to have a mix of paths with and without a leading slash. The behavior is actually different. The next commit will explain when/why to add a leading slash. Reported-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Quentin Schulz <quentin.schulz@cherry.de> Reviewed-by: Antonin Godard <antonin.godard@bootlin.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
…explanations The documentation doesn't explain the side-effect of putting a leading slash, only the trailing slash. The leading slash is not for making the regular expression match an absolute path, but to force the match on the directory or file that exactly starts (and ends if there is a trailing slash) with the specified string. So let's explain that. This also explains that this doesn't prevent more than the intended path to be caught, specifically because it is NOT a regular expression matching an absolute path. Because any pattern not starting with a ^ character can match multiple directories from multiple layers, let's make the usage of BBFILE_PATTERN_my-layer the recommended one. Keep the rest as hints at what can happen when not using the variable but reiterate that users should be really be using that variable. Reported-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Quentin Schulz <quentin.schulz@cherry.de> Reviewed-by: Antonin Godard <antonin.godard@bootlin.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
…rrent meta-ti meta-ti git repo now has multiple layers in it since kirkstone, so let's update the examples to match the current state of meta-ti by using meta-ti-bsp. Signed-off-by: Quentin Schulz <quentin.schulz@cherry.de> Reviewed-by: Antonin Godard <antonin.godard@bootlin.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
…HTTPS It's 2026, use HTTPS for the link. Signed-off-by: Quentin Schulz <quentin.schulz@cherry.de> Reviewed-by: Antonin Godard <antonin.godard@bootlin.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will be re-used in the next commits to force subprocess commands to color their outputs, as passing --color=auto in them always render without coloring. Signed-off-by: Antonin Godard <antonin.godard@bootlin.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bold the configuration names and align the descriptions for improved readability. For this, define a print_configs() function which should also help factorizing the code as this is done in multiple places. If color_enabled() is false, keep the text plain. Signed-off-by: Antonin Godard <antonin.godard@bootlin.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix the switchers.js script of Bitbake to show what are the supported versions and be able to switch between them. The default landing page is the stable branch and shows the BitBake version along with the corresponding Yocto Project codename. This hopefully makes it easier to remember the correspondance between the BitBake version and the Yocto Project version. This works thanks to a setversions.py script, imported and executed in conf.py, which is largely inspired from the one in yocto-docs. It reads the tags from the repository and tries to guess the currently checked out version of BitBake on which we are. The "obsolete" warning is now also shown when browsing outdated manuals, meaning any version not part of activereleases in setversions.py and "dev"/"next". Signed-off-by: Antonin Godard <antonin.godard@bootlin.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Define a get_diff_color_param() function that returns the --color value for diff commands. It function also uses color_enabled() to force showing color or not in the spawned subprocess (--color=auto would never show color). Signed-off-by: Antonin Godard <antonin.godard@bootlin.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When a bbappend file is parsed, the FILE variable is set to the name of the actual file being parsed (e.g. the name of the bbappend). Since PN/PV etc. are derived from FILE, this means that they can be misleading in the event that bbappend is using wildcards (e.g. PV might have the value of "%" instead of the actual version number). In order to allow bbappends to derived the actual information of the recipe, capture the name of the original recipe being parsed as __BB_RECIPE_FILE when parsing a new recipe. The value of this variable doesn't change when parsing .bbappend or .inc file associated with the recipe Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When calling 'bitbake-getvar -r somerecipe FOO', show close matches when 'somerecipe' doesn't exist but close matches are found. This replicates the behavior of 'bitbake -e'. Example output: $ bitbake-getvar -r binutils-cross FOO Unable to find any recipe file matching "binutils-cross". Close matches: binutils-cross-aarch64 binutils util-macros Signed-off-by: Antonin Godard <antonin.godard@bootlin.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
…default BB_ENV_PASSTHROUGH_ADDITIONS (anymore) BB_ENV_PASSTHROUGH_ADDITIONS variable doesn't contain BB_ENV_PASSTHROUGH anymore since commit 0a33b56 ("utils: remove BB_ENV_PASSTHROUGH from preserved_envvars()") so let's fix this oversight. Reported-by: Robert P. J. Day <rpjday@crashcourse.ca> Suggested-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Quentin Schulz <quentin.schulz@cherry.de> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use correct spelling of "BitBake" in a few places. Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
… subsection Out of all the directives listed in this document, only inherit_defer is documented in a subsubsection rather than in a subsection. Therefore, let's be consistent and make inherit_defer documentation into a subsection. Fixes: eb10df5 ("doc: bitbake-user-manual: document inherit_defer") Signed-off-by: Quentin Schulz <quentin.schulz@cherry.de> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
…asks Javier Tia reported[1] they keep forgetting `[[` conditional isn't supported and scouring our BugZilla, I found[2] another "bug" where arithmetic expansion isn't supported either. Let's add a subsection of the shell tasks with the known limitations. [1] https://lore.kernel.org/bitbake-devel/20250320003358.977774-1-javier.tia@linaro.org/ [2] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11314 Reported-by: Javier Tia <javier.tia@linaro.org> Suggested-by: Javier Tia <javier.tia@linaro.org> Signed-off-by: Quentin Schulz <quentin.schulz@cherry.de> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Many CI setups rely on the GIT_CONFIG_GLOBAL environment variable to define project-specific gitconfigs, often to inject credentials using git's url.<base>.insteadOf configuration. However, BitBake restricts the environment variables passed to the fetch task to a fixed list (FETCH_EXPORT_VARS). Because GIT_CONFIG_GLOBAL is missing from this default list, the variable is stripped from the environment. Add GIT_CONFIG_GLOBAL to FETCH_EXPORT_VARS to natively support this credential injection method (and GIT_CONFIG_GLOBAL in general). This also eliminates confusion for users who expect the variable to propagate after adding it to `BB_ENV_PASSTHROUGH_ADDITIONS`, only to find it has no effect during the fetch task. Signed-off-by: Florian Schmaus <florian.schmaus@codasip.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We cannot assume ud has the same fetcher with origud. For example, if we set map git:// to file:// in PREMIRRORS, ud is using local fetcher and origud is using git fetcher. In such case, the ud does have the 'shallow' attribute. And we'll see the following error: Exception: AttributeError: 'FetchData' object has no attribute 'shallow' Looking at the logic of this function, I think it's the origud's shallow that should be checked. So the logic becomes: if origud is using shallow and its full shallow tarball does not exist yet, symlink to ensure it exists. This should make more sense. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
… error When using git:// -> file:// in PREMIRRORS and we have the same url in SRC_URI twice, we reveal the following error: AttributeError: 'FetchData' object has no attribute 'shallow' Add a test case to ensure future changes don't introduce the error back. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Otherwise, we see errors like: |ERROR: An uncaught exception occurred in runqueue |Traceback (most recent call last): | File "/srv/pokybuild/yocto-worker/check-layer/build/layers/bitbake/lib/bb/runqueue.py", line 1663, in execute_runqueue | return self._execute_runqueue() | ~~~~~~~~~~~~~~~~~~~~~~^^ | File "/srv/pokybuild/yocto-worker/check-layer/build/layers/bitbake/lib/bb/runqueue.py", line 1607, in _execute_runqueue | retval = self.dump_signatures(dumpsigs) | File "/srv/pokybuild/yocto-worker/check-layer/build/layers/bitbake/lib/bb/runqueue.py", line 1718, in dump_signatures | p.start() | ~~~~~~~^^ | File "/usr/lib64/python3.14/multiprocessing/process.py", line 121, in start | self._popen = self._Popen(self) | ~~~~~~~~~~~^^^^^^ | File "/usr/lib64/python3.14/multiprocessing/context.py", line 224, in _Popen | return _default_context.get_context().Process._Popen(process_obj) | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^ | File "/usr/lib64/python3.14/multiprocessing/context.py", line 300, in _Popen | return Popen(process_obj) | File "/usr/lib64/python3.14/multiprocessing/popen_forkserver.py", line 35, in __init__ | super().__init__(process_obj) | ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^ | File "/usr/lib64/python3.14/multiprocessing/popen_fork.py", line 20, in __init__ | self._launch(process_obj) | ~~~~~~~~~~~~^^^^^^^^^^^^^ | File "/usr/lib64/python3.14/multiprocessing/popen_forkserver.py", line 47, in _launch | reduction.dump(process_obj, buf) | ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^ | File "/usr/lib64/python3.14/multiprocessing/reduction.py", line 60, in dump | ForkingPickler(file, protocol).dump(obj) | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^ |_pickle.PicklingError: Can't pickle local object <function CoreRecipeInfo.init_cacheData.<locals>.<lambda> at 0x7f7925667d70> |when serializing tuple item 0 |when serializing collections.defaultdict reconstructor arguments |when serializing collections.defaultdict object |when serializing dict item 'rundeps' |when serializing bb.cache.CacheData state |when serializing bb.cache.CacheData object |when serializing dict item '' |when serializing dict item 'recipecaches' |when serializing bb.cooker.BBCooker state |when serializing bb.cooker.BBCooker object |when serializing dict item 'cooker' |when serializing bb.runqueue.RunQueue state |when serializing bb.runqueue.RunQueue object |when serializing tuple item 0 |when serializing method reconstructor arguments |when serializing method object |when serializing dict item '_target' |when serializing multiprocessing.context.Process state |when serializing multiprocessing.context.Process object Fixes: [YOCTO #16184] Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake-worker relies on multiprocessing using the "fork" method to pass data to the child process. Starting in Python 3.13 the default changed to "forkserver", so the bb.multiprocessing context needs to be used in order to preserve the "fork" method. Note that bb.multiprocessing is a context, so Lock can't be imported from it; instead a local alias for Lock must be used Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
If config_id (given on command line in 'init' operation) is provided and interpreted as a relative path to a local file (by checking for the file existence), it should be recorded into configuration instance as an absolute path. Otherwise status/update operations will fail if executed from a different directory that the template file was in, as they won't be able to find the file. Note that the patch context doesn't show where having the absolute path actually matters: further down in the function, config_id is used to set a 'non-interactive-cmdline-options' property in configuration instance dictionary. This property is used by status/update operations. Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There's no need to repeatedly do it in the loop, as it doesn't change. Also reposition 'remotes' and 'changed' variables assignments, as this will aid the code readability after the following commit. Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
…hen it is fixed to a commit id The code was relying on git fetcher's latest_revision() method, which works properly if the revision in the json config template is set to a branch or a tag, but does not work if it is set to a fixed revision (the method will return the latest revision on the respective branch, and bitbake-setup will incorrectly determine that the layer needs to be updated to that revision). Add a guard to check if the revision is a fixed commit id, and do a direct revision comparison if so, skipping the fetcher query. [YOCTO #16190] Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was introduced as a test for the fix in 6ec70d5 however the way github works has changed in the decade since and this test is no longer testing what was intended. The test is failing intermittently on our infrastructure thanks to spam and IP block throttling restrictions which lead to occasional failures. At this point it is probably simplest just to drop the url from testing. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We see occasional connection errors in wget testing of sstate mirrors.
It appears there is a case where http.client.RemoteDisconnected is
returned against getrepsonse() which the current code doesn't handle
well.
Rather than trying to handle very specific error cases, catch any errors
and drop the cached connection in those cases in the do_open() code.
Similarly, try again in a more complete set of error cases (all of
OSError instead of special cases and RemoteDisconnected).
[YOCTO #15945]
The traceback from the logs is included below for reference.
Traceback (most recent call last):
File "/srv/pokybuild/yocto-worker/a-full/build/layers/bitbake/lib/bb/fetch2/wget.py", line 415, in checkstatus
with opener.open(r, timeout=100) as response:
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/urllib/request.py", line 519, in open
response = self._open(req, data)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/urllib/request.py", line 536, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/urllib/request.py", line 496, in _call_chain
result = func(*args)
^^^^^^^^^^^
File "/srv/pokybuild/yocto-worker/a-full/build/layers/bitbake/lib/bb/fetch2/wget.py", line 178, in http_open
return self.do_open(HTTPConnectionCache, req)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/srv/pokybuild/yocto-worker/a-full/build/layers/bitbake/lib/bb/fetch2/wget.py", line 249, in do_open
r = h.getresponse()
^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/http/client.py", line 1374, in getresponse
response.begin()
File "/usr/lib/python3.11/http/client.py", line 318, in begin
version, status, reason = self._read_status()
^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/http/client.py", line 287, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/srv/pokybuild/yocto-worker/a-full/build/layers/bitbake/lib/bb/fetch2/wget.py", line 228, in do_open
h.request(req.get_method(), req.selector, req.data, headers)
File "/usr/lib/python3.11/http/client.py", line 1282, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.11/http/client.py", line 1328, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.11/http/client.py", line 1277, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.11/http/client.py", line 1037, in _send_output
self.send(msg)
File "/usr/lib/python3.11/http/client.py", line 998, in send
self.sock.sendall(data)
OSError: [Errno 9] Bad file descriptor
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/srv/pokybuild/yocto-worker/a-full/build/layers/bitbake/lib/bb/fetch2/wget.py", line 415, in checkstatus
with opener.open(r, timeout=100) as response:
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/urllib/request.py", line 519, in open
response = self._open(req, data)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/urllib/request.py", line 536, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/urllib/request.py", line 496, in _call_chain
result = func(*args)
^^^^^^^^^^^
File "/srv/pokybuild/yocto-worker/a-full/build/layers/bitbake/lib/bb/fetch2/wget.py", line 178, in http_open
return self.do_open(HTTPConnectionCache, req)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/srv/pokybuild/yocto-worker/a-full/build/layers/bitbake/lib/bb/fetch2/wget.py", line 246, in do_open
raise urllib.error.URLError(err)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This check can be done earlier when building the documentation, and does not need any version information. Move it to the beginning of conf.py. Signed-off-by: Antonin Godard <antonin.godard@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
While this was inspired with yocto-docs' set_versions.py file[1], some of the logic here can be simplified with early returns. As a consequence of these early returns, move the print() call showing the computed version to the conf.py file. Signed-off-by: Antonin Godard <antonin.godard@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Like what was done for yocto-docs[1], build using the remote releases.json file we can fetch from [2]. The file is downloaded if not present, and not re-downloaded for subsequent builds. In case of fetch failure, fallback to default values. [1]: https://git.yoctoproject.org/yocto-docs/commit/?id=5bebe38a808a33fea3deefc21dda39a35d90a7dd [2]: https://dashboard.yoctoproject.org/releases.json Signed-off-by: Antonin Godard <antonin.godard@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This change introduces a function to generate a VSCode workspace file
(bitbake.code-workspace). The --init-vscode flag added to bitbake-setup
init defaults to True when the code binary is found on PATH, and can be
passed explicitly to exercise the feature on machines without code (e.g.
when running tests on an autobuilder). Once the workspace file exists,
it is updated automatically on every subsequent run.
This workspace file is preferred over a project-specific
.vscode/settings.json for several reasons:
- It allows for a multi-root workspace, which is ideal for a bitbake
project structure setup with bitbake-setup. This enables including all
layer repositories and the build configuration directory as top-level
folders in the explorer.
- The workspace file can be located at the top level of the setup,
outside of any version-controlled source directory. This avoids
cluttering the git repositories with editor-specific configuration.
- It provides a centralized place for all VSCode settings related to the
project, including those for the bitbake extension, Python language
server, and file associations, ensuring a consistent development
environment for all users of the project.
The Python analysis paths (python.analysis.extraPaths) are configured
with absolute paths. This is a workaround for a limitation in the Pylance
extension, which does not correctly resolve ${workspaceFolder:...}
variables in a multi-root workspace context for import resolution. Using
absolute paths ensures that Pylance can find all necessary modules from
the various layers.
There is room for improvement. Starting a terminal (bitbake or any other)
is cumbersome, as VSCode wants to start it for one of the layers rather
than the build directory.
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add test_vscode to BitbakeSetupTest covering the --init-vscode option introduced in 'bitbake-setup: generate config files for VSCode': - init --init-vscode creates bitbake.code-workspace with the expected top-level structure (folders, settings, extensions). - Folders list conf and each non-symlink git repo in layers/; all paths are relative. - Bitbake extension settings (pathToBuildFolder, pathToEnvScript, disableConfigModification) are set correctly. - File associations (*.conf, *.inc) and python.analysis.extraPaths / python.autoComplete.extraPaths are populated. - init --no-init-vscode does not create a workspace file. - update after a layer change preserves user-added folders and settings while updating managed ones. - update with a corrupt workspace file logs an error and leaves the file unchanged. Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When unpack_update() raises LocalModificationsError (uncommitted changes) or RebaseError (local commits that could not be rebased), the caller now has a choice of how to handle it, controlled by the new --rebase-conflicts-strategy option on the 'update' subcommand: - abort (default): re-raise the error so the update stops with a clear message that names the affected source and its path. The fetcher has already aborted the rebase and restored the checkout to its previous state. - backup: rename the directory to a timestamped <name>-backup path to preserve local work, then re-clone from upstream via fetcher.unpack(). Both exception types share a single except clause; the exception message already describes the specific failure (uncommitted changes vs. failed rebase with git output), so it is forwarded directly to the warning log. The strategy is threaded from the CLI argument through build_status() and update_build() down to checkout_layers(). Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a dedicated test for the --rebase-conflicts-strategy option introduced in
'bitbake-setup: add --rebase-conflicts-strategy to the update command'.
Three scenarios are covered that are not exercised by the existing
test_setup:
1. Uncommitted tracked-file change (LocalModificationsError), default
'abort' strategy: update must fail with an error that contains
'has uncommitted changes' and the '--rebase-conflicts-strategy=backup'
hint; no backup directory is created.
2. Same uncommitted change, 'backup' strategy: the layer directory is
renamed to a timestamped backup, the layer is re-cloned from
upstream, and the result is clean (upstream content, no local
modifications).
3. Committed local change that conflicts with the next upstream commit
(RebaseError):
a. Default 'abort' strategy: update must fail with an error that
contains 'Merge conflict' and the '--rebase-conflicts-strategy=backup'
hint; no backup directory is created.
b. 'backup' strategy: instead of the hard rebase-conflict failure,
the conflicted directory is backed up and re-cloned successfully.
A small _count_layer_backups() helper is added to the class and reused
across scenarios.
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
…sage Without a top-level exception handler, any unexpected Python exception reaches the user as a raw traceback, which is confusing and exposes implementation details. Add a try/except block around the main dispatch in main() that: - catches all unexpected exceptions and logs them as a single ERROR line, consistent with how other errors are already presented - re-raises when --debug is active, so the full traceback is still available for diagnosis - explicitly re-raises SystemExit and KeyboardInterrupt so sys.exit() calls and Ctrl-C continue to work normally Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Document the two new options introduced in the setup command: - --init-vscode / --no-init-vscode (66c26e5f0): generates a bitbake.code-workspace file for VS Code in the Setup directory, with layer folders, BitBake extension settings, and file associations. Default is enabled when the 'code' binary is found on PATH. - --rebase-conflicts-strategy (60ce61cca): controls the fallback behaviour when a layer repository cannot be updated in place due to local modifications or conflicting local commits. Documents both the 'abort' (default) and 'backup' values. Also documents the in-place update behaviour of the update command (fetch via dldir remote + rebase), with a cross-reference to the unpack_update() section in the fetching chapter. Also document that the update command is intended to be run from a shell with the BitBake environment sourced, and why --autostash is not used (transparency: uncommitted changes are surfaced rather than silently stashed and re-applied). Add examples for the four distinct update scenarios, using real output from an actual update session: - normal update with local commits rebased and preserved - update blocked by uncommitted changes, with stash + retry steps - rebase conflict with manual resolution via the dldir git remote, including the rebase --continue + re-run sequence - rebase conflict resolved via --rebase-conflicts-strategy=backup, showing the real WARNING output and timestamped backup directory name, with a step-by-step git workflow (cd into the fresh clone, fetch the backup branch by local path, cherry-pick oldest-first) to recover commits from the backup, and cleanup instructions (rm -rf the backup, or remove the VS Code workspace folder via the UI or by re-running bitbake-setup update after deletion) Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When I was using `bitbake-layers layerindex-fetch ...` I noticed it being quite slow. Upon further investigation, I found that the call to `load_layerindex()` inside layerindexlib was called with the deafult `load` argument. The deafault was set to ['layerDependencies', 'recipes', 'machines', 'distros'] where the `layerindex-fetch` effectively requires only ['layerDependencies']. So whenever a `layerindex-fetch` is used, we pull for example 'recipes', which is about 18-20MB in size - it takes time to produce that response by the layerindex-web and ship over the internet, just to be discarded. This change addresses the issue by setting a "sane" default (empty `load` list) and sets a correct, explicit `load` list whenever it's needed (e.g. toaster) by the caller. Lastly, there seems to be an error when the cooker index is loaded in the process with the `load='layerDependencies'`, that's incorrect, as the implementation expects also a list, so that's corrected as well. Signed-off-by: Piotr Buliński <piotr@qbee.io> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use '?=' instead of '=' when writing DL_DIR to site.conf, so an exported DL_DIR can be reused across environments. This keeps a default value while allowing environment-based cache configuration in mixed workflows (e.g. container/host). Signed-off-by: Zhangfei Gao <zhangfei.gao@linaro.org> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This fixes another corner case: 'bitbake-setup update --setup-dir ./relative/path' Normally setup-dir is deduced from bitbake environment, but when it is passed in on the command line as a relative path, and there is something to update, new bblayer.conf is created with relative paths in it, causing breakage. Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
…meter [YOCTO #15558] Add a warning on the subpath option which has an unexpected behavior with regards to how the HEAD is updated in the checked out repository. Signed-off-by: Antonin Godard <antonin.godard@bootlin.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
…agments OE_FRAGMENTS was meant as a code-block and thus needs two colons, a newline and an increased indent for it to work, however we were missing the second colon. Let's fix that oversight. Fixes: 3b9d7be ("parse/ast: add support for 'built-in' fragments") Signed-off-by: Quentin Schulz <quentin.schulz@cherry.de> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We've decided code-blocks should be indented by three whitespaces compared to the parent block and this only had two, so let's add the missing whitespace. This is cosmetic in the sources only, it doesn't change the output. Fixes: 3b9d7be ("parse/ast: add support for 'built-in' fragments") Signed-off-by: Quentin Schulz <quentin.schulz@cherry.de> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We've decided bullet lists should be started by a dash followed by two whitespaces and the other lines in the bullet list should be indented at that level, that is with three whitespaces compared to the parent block. This only has two while the rest of the file has three, so let's add the missing whitespace. This is cosmetic in the sources only, it doesn't change the output. Fixes: 6bc65e6 ("documentation: bitbake: add file-checksums to varflags section") Signed-off-by: Quentin Schulz <quentin.schulz@cherry.de> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Two of the tests were still using git protocol to access git services. For the submodule test, the upstream repo has been updated. In the other case, we need to pass the correct command to the manual git commandline, we can't use a recipe url that previously just happened to work. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A problem can happen with the current code: if the current stable release is made EOL before the development branch is made the current stable, then we only have LTS releases displayed. Change the logic to _always_ display the latest non-dev branch as the default page, even if EOL. Signed-off-by: Antonin Godard <antonin.godard@bootlin.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Fix comment typos in RunQueueData.prepare Signed-off-by: Yann Dirson <yann.dirson@vates.tech> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
When subpath is set the checkout code uses read-tree + checkout-index, but never updates HEAD. It stays on whatever branch the clone defaulted to instead of pointing to the requested revision. Fix by pointing HEAD at the correct revision after checkout-index, and creating the branch ref when branch= is specified. Also remove the warning note in the documentation that described this limitation. Signed-off-by: Mark-Pk Tsai <mark-pk.tsai@mediatek.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
This function is known python/cpython#72680 for false-positives. With python 3.13.5 there is one with https://vault.almalinux.org/10.0/CRB/x86_64_v2/os/Packages/jdom2-2.0.6.1-8.el10.noarch.rpm The double "is_zipfile = False" is redundant but likely more clear. Signed-off-by: Yann Dirson <yann.dirson@vates.tech> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
When packages appear multiple times it gets hard to make any sense of the list display. Using sets does the job for us. Signed-off-by: Yann Dirson <yann.dirson@vates.tech> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
This ensures that the main function is only executed if the module is executed in the top-level environment. Signed-off-by: Rob Woolley <rob.woolley@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the new release Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are cases where we would like the filter functionality but don't want to read the data from a variable. Add a new function, filter_string to allow this, basically separating filter() into two functions. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This avoids the 1 req/sec that the API has. Reference: rust-lang/crates.io#13482 Reference: https://blog.rust-lang.org/2024/03/11/crates-io-download-changes/ Signed-off-by: Johan Anderholm <johanam@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the crate host is crates.io then we can fetch the index for the crate from index.crates.io instead of hitting the API. This is the recommended way to do automated checks and means we don't break the data access policy[1] by not setting an explicit User-Agent. [1] https://crates.io/data-access Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We create a temporary directory for holding a clone but we never clean it up. Fix this by using a context manager areound the temporary directory. This resolves a buildup of tmp directories in DL_DIR in builds. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Whilst this code doesn't leak a temporary directory, it is a little unsafe around directory creation/reuse due to the cleanup and retry logic. Restructure the function so it can use tempfile.TemporaryDirectory as a context manager with a context per loop iteration. Behavour shouldn't be changed, the deletion/recreation is just removed. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
…xactly This change is in code that iterates over a html page, picking out links to versioned subdirectories to check further for new versions. Without the ^ and $ guards, there would be sub-string matches. This results in querying the server multiple times for the same directory (for example, perl check where the page contains multiple references to 5.0/something-unrelated) or querying the server for directory that doesn't exist (nasm check, where existence of 3.02rcX directories would result in querying the server for non-existing 3.02). I have confirmed that the overall oe-core version check returns same results as before, with less network churn and nasm version check working properly again. Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 4af5f6a) Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
That might help new users send correct first stable patches. Cc: Quentin Schulz <quentin.schulz@cherry.de> Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
That might help new users send correct first stable patches. Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
merge: Currency merge with upstream 2.18 branch
No conflicts
AB#3738531
Testing