Skip to content

ci: cache gnu32 nix store#256

Merged
ryanofsky merged 1 commit intobitcoin-core:masterfrom
Sjors:2026/03/nix-cache
Mar 26, 2026
Merged

ci: cache gnu32 nix store#256
ryanofsky merged 1 commit intobitcoin-core:masterfrom
Sjors:2026/03/nix-cache

Conversation

@Sjors
Copy link
Copy Markdown
Member

@Sjors Sjors commented Mar 12, 2026

gnu32 is by far the slowest ci job and it spends most of it's time building Nix stuff.

Caching the Nix store drops subsequent runs to just 3 minutes.

Not caching the other ones, because the cache is quite large and Github limits us to 10GB total.

Using https://github.com/nix-community/cache-nix-action

Closes #254

@DrahtBot
Copy link
Copy Markdown

DrahtBot commented Mar 12, 2026

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.

Type Reviewers
ACK ryanofsky

If your review is incorrectly listed, please copy-paste <!--meta-tag:bot-skip--> into the comment that the bot should ignore.

@Sjors Sjors force-pushed the 2026/03/nix-cache branch from 915a7f8 to 7aa3271 Compare March 12, 2026 13:42
@Sjors
Copy link
Copy Markdown
Member Author

Sjors commented Mar 12, 2026

@ryanofsky can you whitelist nix-community/cache-nix-action@v7?

@ryanofsky
Copy link
Copy Markdown
Collaborator

@ryanofsky can you whitelist nix-community/cache-nix-action@v7?

Thanks, added. This sorry for the delay. If there is a way we could give you github permissions to change these settings that would seem nice too

@Sjors
Copy link
Copy Markdown
Member Author

Sjors commented Mar 12, 2026

There's no rush and I don't expect to have to make many such changes. Basic ccache and more tailored nix cache should do the trick.

@Sjors
Copy link
Copy Markdown
Member Author

Sjors commented Mar 12, 2026

About 800 MB, not too bad:
Scherm­afbeelding 2026-03-12 om 20 20 26

Runtime was 60 minutes. I'm going to amend the commit (date) to trigger a re-run which should use this cache...

@Sjors Sjors force-pushed the 2026/03/nix-cache branch 2 times, most recently from 3fa8a2c to 34da50f Compare March 12, 2026 19:54
@Sjors
Copy link
Copy Markdown
Member Author

Sjors commented Mar 12, 2026

Looks like the cache got trimmed the first write. Trying again.

This push to 34da50f is not expected to be substantially faster, the next push should be.

@Sjors
Copy link
Copy Markdown
Member Author

Sjors commented Mar 12, 2026

Depsite the incomplete cache it ran in only 30 minutes. The new cache entry is 1.7 GB. Let's run it again...

@Sjors Sjors force-pushed the 2026/03/nix-cache branch from 34da50f to 6dc9700 Compare March 12, 2026 20:35
if: matrix.config == 'gnu32'
uses: nix-community/cache-nix-action@v7
with:
primary-key: nix-${{ runner.os }}-${{ matrix.config }}-${{ env.NIXPKGS_CHANNEL }}-${{ hashFiles('shell.nix', 'ci/patches/*.patch', 'ci/configs/gnu32.bash') }}-${{ github.sha }}
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be better to drop -${{ github.sha }} so our cache won't get flushed as often.

@Sjors
Copy link
Copy Markdown
Member Author

Sjors commented Mar 12, 2026

Down to 3 minutes, nice!

@Sjors Sjors marked this pull request as ready for review March 12, 2026 20:41
@Sjors Sjors force-pushed the 2026/03/nix-cache branch from 6dc9700 to e734375 Compare March 12, 2026 21:14
@Sjors
Copy link
Copy Markdown
Member Author

Sjors commented Mar 12, 2026

Trying a different caching approach, using the latest nix channel commit - which doesn't change that often for nixos-25.05. It should now save the cache only once.

If it can't get an exact match it will try a broader match, so IIUC even if something in nix changes from under us, we might still get some benefit out of earlier cache entries. Once the build succeeds, it produces a fresh one that's then used in subsequent runs.

@Sjors Sjors force-pushed the 2026/03/nix-cache branch 2 times, most recently from 7664d7c to 76d3e51 Compare March 12, 2026 21:27
@Sjors Sjors marked this pull request as draft March 12, 2026 21:28
@Sjors Sjors force-pushed the 2026/03/nix-cache branch from 76d3e51 to 6562d22 Compare March 12, 2026 21:35
@Sjors
Copy link
Copy Markdown
Member Author

Sjors commented Mar 12, 2026

Alright, 76d3e51 created a new entry nix-Linux-gnu32-ac62194c3917d5f474c1a844b6fd6da2db95077d-67a1dfa5824ce68902b067211c47c342ba705ad71d773f96e6885ba6d3defc3e which the next run should reuse and not resave.

Pushed an amended date commit 6562d22 to test that.

@Sjors
Copy link
Copy Markdown
Member Author

Sjors commented Mar 12, 2026

As expected, the last gnu32 CI run found a match for the primary key and did not re-save it.

https://github.com/bitcoin-core/libmultiprocess/actions/runs/23025035895/job/66870747483?pr=256

Runtime slightly under 3 minutes.

So this should be good to go now.

@Sjors Sjors marked this pull request as ready for review March 12, 2026 21:39

# Use an explicit save step instead of the action post-step so we only
# archive the store after the build succeeded and the shell closure is
# rooted against the save-time garbage collection pass.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no idea what "shell closure is rooted against the save-time garbage collection pass" means, so it would be good for someone who knows Nix to sanity check this.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no idea what "shell closure is rooted against the save-time garbage collection pass" means, so it would be good for someone who knows Nix to sanity check this.

It makes sense, but is a confusing comment. Would suggest splitting it up like "Use an explicit save step instead of the action post-step so we only save after a successful build. Create a GC root for the gnu32 shell closure here, so the cache-nix-action step below does not garbage-collect it. "

Copy link
Copy Markdown
Collaborator

@ryanofsky ryanofsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review ACK 6562d22. Nice to speed up this very slow job.

Am happy to merge as-is but it would be nice not to hardcode this logic for the gnu32 job, also to not have such complicated bash commands in the yaml. More ideally, i think this would do something like:

  • Add CI_CACHE_STORE=true in ci/configs/gnu32.bash
  • Add a ci/scripts/config.sh script and build step that sources the config and outputs whatever variables the yml file needs to do caching
  • Add command to ci/scripts/run.sh to create the .nix-gc-roots link needed to prevent garbage collection.

Also from https://github.com/nix-community/cache-nix-action/#a-typical-job it seems we don't need two separate cache-nix-action steps before and after the build. Just a single step before the build should be sufficient if the goal is to only save the cache on success.


# Use an explicit save step instead of the action post-step so we only
# archive the store after the build succeeded and the shell closure is
# rooted against the save-time garbage collection pass.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no idea what "shell closure is rooted against the save-time garbage collection pass" means, so it would be good for someone who knows Nix to sanity check this.

It makes sense, but is a confusing comment. Would suggest splitting it up like "Use an explicit save step instead of the action post-step so we only save after a successful build. Create a GC root for the gnu32 shell closure here, so the cache-nix-action step below does not garbage-collect it. "

@Sjors
Copy link
Copy Markdown
Member Author

Sjors commented Mar 24, 2026

not have such complicated bash commands in the yaml

I you merge #253 first I'll rebase this to take advantage of the bash scripts introduced there.

@Sjors Sjors marked this pull request as draft March 24, 2026 11:13
@Sjors Sjors force-pushed the 2026/03/nix-cache branch from 6562d22 to 0181f8b Compare March 25, 2026 17:54
@Sjors
Copy link
Copy Markdown
Member Author

Sjors commented Mar 25, 2026

TSan tripped over: AssertionError: [node 0] Expected message(s) ['canceled while executing'] not found in log while "Running disconnect during BlockTemplate.waitNext": https://github.com/bitcoin-core/libmultiprocess/actions/runs/23555980083/job/68582763611?pr=256#step:21:2025

Probably need to bump the TEST_RUNNER_TIMEOUT_FACTOR.

If the gnu32 job works (so far so good), I'll push a comment improvement to see if it picks up the cache.

@Sjors Sjors force-pushed the 2026/03/nix-cache branch from 0181f8b to 293b0e0 Compare March 25, 2026 19:04
@Sjors Sjors marked this pull request as ready for review March 25, 2026 19:11
@Sjors
Copy link
Copy Markdown
Member Author

Sjors commented Mar 25, 2026

From one hour to two minutes, nice.


Asan failure looks like another timeout again, see #263.

Expected message(s) ['canceled while executing'] not found in log

Though the tail of the error looks more worrying:

 node0 stderr /usr/include/capnp/capability.h:1129:16: runtime error: member call on address 0x511000092940 which does not point to an object of type 'capnp::CallContextHook'
0x511000092940: note: object has invalid vptr
 02 00 00 00  6d 7f 00 00 02 00 00 00  20 35 1f 06 6b 7f 00 00  00 00 00 00 be be be be  88 18 06 00
              ^~~~~~~~~~~~~~~~~~~~~~~
              invalid vptr 

Or is that in the Python framework?

https://github.com/bitcoin-core/libmultiprocess/actions/runs/23559019643/job/68593574084?pr=256

Copy link
Copy Markdown
Collaborator

@ryanofsky ryanofsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review ACK 293b0e0. This seems more understandable now with one cache step instead of two and more code moved to bash scripts. Left some comments and suggestions, but nothing blocking.

re: #256 (comment)

The capnp::CallContextHook "object has invalid vptr" error is pretty bad looking, but it looks like that happens after a test timeout so #263 should prevent it from happening. The actual bug causing that error might also be fixed by one of the fixes in #249, but that's speculative.

Comment on lines +169 to +170
# Cache the heaviest Nix job to stay within GitHub's cache budget while
# still avoiding repeated gnu32 cross-toolchain downloads and builds.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In commit "ci: cache gnu32 nix store" (293b0e0)

Probably should just drop this comment since there is no longer any code here referencing gnu32.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll move it to the gnu32 script to explain why the other scripts don't.

nix-build shell.nix \
-o .nix-gc-roots/shell \
"${NIX_ARGS[@]+"${NIX_ARGS[@]}"}"
nix-store --query --requisites .nix-gc-roots/shell >/dev/null
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In commit "ci: cache gnu32 nix store" (293b0e0)

Maybe add a comment about why this is here. It seems like it doesn't do anything by default but could print helpful errors if the closure is not complete? Not sure. Would be good to explain or remove.

Copy link
Copy Markdown
Member Author

@Sjors Sjors Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a comment, it's a guard against creating a bad cache.

Good results are ignored, and IIUC if something goes wrong it will go to stderr, but I didn't test that.

ryanofsky added a commit that referenced this pull request Mar 25, 2026
09f10e5 ci: bump timeout factor to 40 (Sjors Provoost)

Pull request description:

  Matches the default `TEST_RUNNER_TIMEOUT_FACTOR` on Bitcoin Core CI.

  Should prevent false positives like #256 (comment)

  That test relies on a timeout:

  ```python
  disconnected_log_check.enter_context(node.assert_debug_log(expected_msgs=["IPC server: socket disconnected", "canceled while executing"], timeout=2))
  ```

ACKs for top commit:
  ryanofsky:
    Code review ACK 09f10e5. Obviously good to match bitcoin core timeouts when running bitcoin core tests.

Tree-SHA512: 48bbbc74813f7ec557a171fef43a381912dce5381c3e568a6dfe1e27fc1d7bdd66376fe9a48c59de9fdb2f1262fba446d9d47b5de7091632d51400f9aae19ae0
@Sjors Sjors force-pushed the 2026/03/nix-cache branch from 293b0e0 to be86228 Compare March 26, 2026 07:44
@Sjors
Copy link
Copy Markdown
Member Author

Sjors commented Mar 26, 2026

Rebased and addressed comments.

The script change invalidated the cache primary key, but the fallback worked, so it's still fast. This fallback mechanism is also why the sanity check before storing cache is important: #256 (comment)

Searching for a cache with the key "nix-Linux-gnu32-ac62194c3917d5f474c1a844b6fd6da2db95077d-129eacb3982a51870d08f654586be1b8b91c598a2ff3b8b1384a2bcea6fac601".
Could not find a cache with the given "primary-key" and "paths".
Searching for a cache using the "restore-prefixes-first-match":
["nix-Linux-gnu32-ac62194c3917d5f474c1a844b6fd6da2db95077d-","nix-Linux-gnu32-","nix-Linux-"]
Cache hit for: nix-Linux-gnu32-ac62194c3917d5f474c1a844b6fd6da2db95077d-bc4e3ad07d6fb99b43a0ff79cb58a5af0d4496c460a796783fa938faefe49cef
...

@maflcko
Copy link
Copy Markdown
Contributor

maflcko commented Mar 26, 2026

CI failure:

2026-03-26T08:10:29.5265259Z Temporary test directory at /tmp/test_runner_₿_🏃_20260326_081029
2026-03-26T08:10:57.0704247Z 1/600 - interface_ipc.py passed, Duration: 22 s
2026-03-26T08:10:57.0711459Z 2/600 - interface_ipc.py passed, Duration: 22 s
2026-03-26T08:10:57.3870521Z 3/600 - interface_ipc.py passed, Duration: 22 s
2026-03-26T08:10:57.4656979Z 4/600 - interface_ipc.py passed, Duration: 22 s
2026-03-26T08:10:57.4887463Z 5/600 - interface_ipc.py passed, Duration: 22 s
2026-03-26T08:10:57.5960759Z 6/600 - interface_ipc.py passed, Duration: 22 s
2026-03-26T08:10:57.6598446Z 7/600 - interface_ipc.py passed, Duration: 22 s
2026-03-26T08:10:57.9096218Z 8/600 - interface_ipc.py passed, Duration: 23 s
2026-03-26T08:10:58.1457589Z 9/600 - interface_ipc.py passed, Duration: 23 s
2026-03-26T08:10:58.1540397Z 10/600 - interface_ipc.py passed, Duration: 23 s
2026-03-26T08:10:58.2227770Z 11/600 - interface_ipc.py passed, Duration: 23 s
2026-03-26T08:10:58.2308253Z 12/600 - interface_ipc.py passed, Duration: 23 s
2026-03-26T08:10:58.2974238Z 13/600 - interface_ipc.py passed, Duration: 23 s
2026-03-26T08:10:58.3686416Z 14/600 - interface_ipc.py passed, Duration: 23 s
2026-03-26T08:10:58.3736080Z 15/600 - interface_ipc.py passed, Duration: 23 s
2026-03-26T08:10:58.4542169Z 16/600 - interface_ipc.py passed, Duration: 23 s
2026-03-26T08:10:58.4688903Z 17/600 - interface_ipc.py passed, Duration: 23 s
2026-03-26T08:10:58.4693534Z 18/600 - interface_ipc.py passed, Duration: 23 s
2026-03-26T08:10:58.4842191Z 19/600 - interface_ipc.py passed, Duration: 23 s
2026-03-26T08:10:58.6343568Z 20/600 - interface_ipc.py passed, Duration: 23 s
2026-03-26T08:11:12.9192987Z 21/600 - interface_ipc.py passed, Duration: 15 s
2026-03-26T08:11:13.0433624Z 22/600 - interface_ipc.py passed, Duration: 15 s
2026-03-26T08:11:13.4416511Z 23/600 - interface_ipc.py passed, Duration: 16 s
2026-03-26T08:11:13.5990535Z 24/600 - interface_ipc.py passed, Duration: 16 s
2026-03-26T08:11:14.2660376Z 25/600 - interface_ipc.py passed, Duration: 15 s
2026-03-26T08:11:14.4723777Z 26/600 - interface_ipc.py passed, Duration: 16 s
2026-03-26T08:11:14.5929731Z 27/600 - interface_ipc.py passed, Duration: 16 s
2026-03-26T08:11:14.7083757Z 28/600 - interface_ipc.py passed, Duration: 16 s
2026-03-26T08:11:14.8849051Z 29/600 - interface_ipc.py passed, Duration: 16 s
2026-03-26T08:11:14.9215983Z 30/600 - interface_ipc.py passed, Duration: 16 s
2026-03-26T08:11:22.4715850Z 31/600 - interface_ipc.py passed, Duration: 9 s
2026-03-26T08:11:22.8364245Z 32/600 - interface_ipc.py passed, Duration: 9 s
2026-03-26T08:11:22.8723167Z 33/600 - interface_ipc.py passed, Duration: 8 s
2026-03-26T08:11:23.8324036Z 34/600 - interface_ipc.py passed, Duration: 8 s
2026-03-26T08:11:24.5614193Z 35/600 - interface_ipc.py passed, Duration: 9 s
2026-03-26T08:12:01.8098556Z ##[error]The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.
2026-03-26T08:12:01.8165252Z ##[error]Process completed with exit code 143.
2026-03-26T08:12:02.7667771Z Cleaning up orphan processes
2026-03-26T08:12:05.7207024Z Terminate orphan process: pid (14932) (b-shutoff)
2026-03-26T08:12:05.8932171Z Terminate orphan process: pid (14934) (b-shutoff)
2026-03-26T08:12:05.9839329Z Terminate orphan process: pid (14936) (b-shutoff)
2026-03-26T08:12:06.0753851Z Terminate orphan process: pid (14945) (b-shutoff)
2026-03-26T08:12:06.1362572Z Terminate orphan process: pid (14951) (b-shutoff)
2026-03-26T08:12:06.2211871Z Terminate orphan process: pid (14963) (bitcoin-node)
2026-03-26T08:12:06.2538740Z Terminate orphan process: pid (14978) (b-shutoff)
2026-03-26T08:12:06.2638800Z Terminate orphan process: pid (14981) (b-shutoff)
2026-03-26T08:12:06.2910155Z Terminate orphan process: pid (14982) (b-shutoff)
2026-03-26T08:12:06.3320706Z Terminate orphan process: pid (16439) (b-shutoff)
2026-03-26T08:12:06.3381871Z Terminate orphan process: pid (16440) (b-shutoff)
2026-03-26T08:12:06.3418611Z ##[warning]Node.js 20 actions are deprecated. The following actions are running on Node.js 20 and may not work as expected: actions/cache/restore@v4, actions/cache@v4, actions/checkout@v4. Actions will be forced to run with Node.js 24 by default starting June 2nd, 2026. Please check if updated versions of these actions are available that support Node.js 24. To opt into Node.js 24 now, set the FORCE_JAVASCRIPT_ACTIONS_TO_NODE24=true environment variable on the runner or in your workflow file. Once Node.js 24 becomes the default, you can temporarily opt out by setting ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION=true. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/

@Sjors
Copy link
Copy Markdown
Member Author

Sjors commented Mar 26, 2026

Another 143 exit, maybe they're running out of RAM with the longer timeouts?

@ryanofsky
Copy link
Copy Markdown
Collaborator

ryanofsky commented Mar 26, 2026

Another 143 exit, maybe they're running out of RAM with the longer timeouts?

Yes these seem to happen reliably on master since #263 was merged. I suspect what is happening is that with the longer timeouts, the test waits forever for node.assert_debug_log(expected_msgs=["IPC server: socket disconnected", "canceled while executing"], timeout=2) log messages that never arrive. Then the jobs are eventually killed with SIGTERM (exit code 143). If this is the case, it's possible #249 could fix this.

EDIT: This explanation is not right. After rebasing #249, we see SIGTERM/143 errors after the test has only run for 6 minutes (https://github.com/bitcoin-core/libmultiprocess/actions/runs/23587830246/job/68685257218?pr=249). So it seems like SIGTERM is not sent because the test is hanging for a very long time. More discussion about this is in #263 (comment)

Copy link
Copy Markdown
Collaborator

@ryanofsky ryanofsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review ACK be86228. Thanks for the updates! Just suggested changes updating comments and tweaking CI_CACHE_NIX_STORE evaluation since last review.

I think I will go ahead and merge this since the Bitcoin Core CI failures should be unrelated, and are also happening in master

@ryanofsky ryanofsky merged commit b090beb into bitcoin-core:master Mar 26, 2026
11 of 13 checks passed
ryanofsky added a commit to ryanofsky/bitcoin that referenced this pull request Mar 26, 2026
…bf18

ff0eed1bf18 refactor: Use loop variable in type-context.h
ff1d8ba172a refactor: Move type-context.h getParams() call closer to use
1dbc59a4aa3 race fix: m_on_cancel called after request finishes
1643d05ba07 test: m_on_cancel called after request finishes
f5509a31fcc race fix: getParams() called after request cancel
4a60c39f24a test: getParams() called after request cancel
f11ec29ed20 race fix: worker thread destroyed before it is initialized
a1d643348f4 test: worker thread destroyed before it is initialized
e606fd84a8c Merge bitcoin-core/libmultiprocess#264: ci: reduce nproc multipliers
336023382c4 ci: reduce nproc multipliers
b090beb9651 Merge bitcoin-core/libmultiprocess#256: ci: cache gnu32 nix store
be8622816da ci: cache gnu32 nix store
975270b619c Merge bitcoin-core/libmultiprocess#263: ci: bump timeout factor to 40
09f10e5a598 ci: bump timeout factor to 40
db8f76ad290 Merge bitcoin-core/libmultiprocess#253: ci: run some Bitcoin Core CI jobs
55a9b557b19 ci: set Bitcoin Core CI test repetition
fb0fc84d556 ci: add TSan job with instrumented libc++
0f29c38725b ci: add Bitcoin Core IPC tests (ASan + macOS)
3f64320315d Merge bitcoin-core/libmultiprocess#262: ci: enable clang-tidy in macOS job, use nullptr
cd9f8bdc9f0 Merge bitcoin-core/libmultiprocess#258: log: add socket connected info message and demote destroy logs to debug
b5d6258a42f Merge bitcoin-core/libmultiprocess#255: fix: use unsigned char cast and sizeof in LogEscape escape sequence
d94688e2c32 Merge bitcoin-core/libmultiprocess#251: Improved CustomBuildField for std::optional in IPC/libmultiprocess
a9499fad755 mp: use nullptr with pthread_threadid_np
f499e37850f ci: enable clang-tidy in macOS job
98f1352159d log: add socket connected info message and demote destroy logs to debug
554a481ea73 fix: use unsigned char cast and sizeof in LogEscape escape sequence
1977b9f3f65 Use std::forward in CustomBuildField for std::optional to allow move semantics, resolves FIXME
22bec918c97 Merge bitcoin-core/libmultiprocess#247: type-map: Work around LLVM 22 "out of bounds index" error
8a5e3ae6ed2 Merge bitcoin-core/libmultiprocess#242: proxy-types: add CustomHasField hook to map Cap'n Proto values to null C++ values
e8d35246918 Merge bitcoin-core/libmultiprocess#246: doc: Bump version 8 > 9
97d877053b6 proxy-types: add CustomHasField hook for nullable decode paths
8c2f10252c9 refactor: add missing includes to mp/type-data.h
b1638aceb40 doc: Bump version 8 > 9
f61af487217 type-map: Work around LLVM 22 "out of bounds index" error

git-subtree-dir: src/ipc/libmultiprocess
git-subtree-split: ff0eed1bf183627c89007e71d631f039bd61bfb5
ryanofsky added a commit to ryanofsky/bitcoin that referenced this pull request Mar 27, 2026
…bf18

ff0eed1bf18 refactor: Use loop variable in type-context.h
ff1d8ba172a refactor: Move type-context.h getParams() call closer to use
1dbc59a4aa3 race fix: m_on_cancel called after request finishes
1643d05ba07 test: m_on_cancel called after request finishes
f5509a31fcc race fix: getParams() called after request cancel
4a60c39f24a test: getParams() called after request cancel
f11ec29ed20 race fix: worker thread destroyed before it is initialized
a1d643348f4 test: worker thread destroyed before it is initialized
e606fd84a8c Merge bitcoin-core/libmultiprocess#264: ci: reduce nproc multipliers
336023382c4 ci: reduce nproc multipliers
b090beb9651 Merge bitcoin-core/libmultiprocess#256: ci: cache gnu32 nix store
be8622816da ci: cache gnu32 nix store
975270b619c Merge bitcoin-core/libmultiprocess#263: ci: bump timeout factor to 40
09f10e5a598 ci: bump timeout factor to 40
db8f76ad290 Merge bitcoin-core/libmultiprocess#253: ci: run some Bitcoin Core CI jobs
55a9b557b19 ci: set Bitcoin Core CI test repetition
fb0fc84d556 ci: add TSan job with instrumented libc++
0f29c38725b ci: add Bitcoin Core IPC tests (ASan + macOS)
3f64320315d Merge bitcoin-core/libmultiprocess#262: ci: enable clang-tidy in macOS job, use nullptr
cd9f8bdc9f0 Merge bitcoin-core/libmultiprocess#258: log: add socket connected info message and demote destroy logs to debug
b5d6258a42f Merge bitcoin-core/libmultiprocess#255: fix: use unsigned char cast and sizeof in LogEscape escape sequence
d94688e2c32 Merge bitcoin-core/libmultiprocess#251: Improved CustomBuildField for std::optional in IPC/libmultiprocess
a9499fad755 mp: use nullptr with pthread_threadid_np
f499e37850f ci: enable clang-tidy in macOS job
98f1352159d log: add socket connected info message and demote destroy logs to debug
554a481ea73 fix: use unsigned char cast and sizeof in LogEscape escape sequence
1977b9f3f65 Use std::forward in CustomBuildField for std::optional to allow move semantics, resolves FIXME
22bec918c97 Merge bitcoin-core/libmultiprocess#247: type-map: Work around LLVM 22 "out of bounds index" error
8a5e3ae6ed2 Merge bitcoin-core/libmultiprocess#242: proxy-types: add CustomHasField hook to map Cap'n Proto values to null C++ values
e8d35246918 Merge bitcoin-core/libmultiprocess#246: doc: Bump version 8 > 9
97d877053b6 proxy-types: add CustomHasField hook for nullable decode paths
8c2f10252c9 refactor: add missing includes to mp/type-data.h
b1638aceb40 doc: Bump version 8 > 9
f61af487217 type-map: Work around LLVM 22 "out of bounds index" error

git-subtree-dir: src/ipc/libmultiprocess
git-subtree-split: ff0eed1bf183627c89007e71d631f039bd61bfb5
ryanofsky added a commit to ryanofsky/bitcoin that referenced this pull request Mar 27, 2026
…da8f

70f632bda8f Merge bitcoin-core/libmultiprocess#265: ci: set LC_ALL in shell scripts
8e8e564259a Merge bitcoin-core/libmultiprocess#249: fixes for race conditions on disconnects
05d34cc2ec3 ci: set LC_ALL in shell scripts
e606fd84a8c Merge bitcoin-core/libmultiprocess#264: ci: reduce nproc multipliers
ff0eed1bf18 refactor: Use loop variable in type-context.h
ff1d8ba172a refactor: Move type-context.h getParams() call closer to use
1dbc59a4aa3 race fix: m_on_cancel called after request finishes
1643d05ba07 test: m_on_cancel called after request finishes
f5509a31fcc race fix: getParams() called after request cancel
4a60c39f24a test: getParams() called after request cancel
f11ec29ed20 race fix: worker thread destroyed before it is initialized
a1d643348f4 test: worker thread destroyed before it is initialized
336023382c4 ci: reduce nproc multipliers
b090beb9651 Merge bitcoin-core/libmultiprocess#256: ci: cache gnu32 nix store
be8622816da ci: cache gnu32 nix store
975270b619c Merge bitcoin-core/libmultiprocess#263: ci: bump timeout factor to 40
09f10e5a598 ci: bump timeout factor to 40
db8f76ad290 Merge bitcoin-core/libmultiprocess#253: ci: run some Bitcoin Core CI jobs
55a9b557b19 ci: set Bitcoin Core CI test repetition
fb0fc84d556 ci: add TSan job with instrumented libc++
0f29c38725b ci: add Bitcoin Core IPC tests (ASan + macOS)
3f64320315d Merge bitcoin-core/libmultiprocess#262: ci: enable clang-tidy in macOS job, use nullptr
cd9f8bdc9f0 Merge bitcoin-core/libmultiprocess#258: log: add socket connected info message and demote destroy logs to debug
b5d6258a42f Merge bitcoin-core/libmultiprocess#255: fix: use unsigned char cast and sizeof in LogEscape escape sequence
d94688e2c32 Merge bitcoin-core/libmultiprocess#251: Improved CustomBuildField for std::optional in IPC/libmultiprocess
a9499fad755 mp: use nullptr with pthread_threadid_np
f499e37850f ci: enable clang-tidy in macOS job
98f1352159d log: add socket connected info message and demote destroy logs to debug
554a481ea73 fix: use unsigned char cast and sizeof in LogEscape escape sequence
1977b9f3f65 Use std::forward in CustomBuildField for std::optional to allow move semantics, resolves FIXME
22bec918c97 Merge bitcoin-core/libmultiprocess#247: type-map: Work around LLVM 22 "out of bounds index" error
8a5e3ae6ed2 Merge bitcoin-core/libmultiprocess#242: proxy-types: add CustomHasField hook to map Cap'n Proto values to null C++ values
e8d35246918 Merge bitcoin-core/libmultiprocess#246: doc: Bump version 8 > 9
97d877053b6 proxy-types: add CustomHasField hook for nullable decode paths
8c2f10252c9 refactor: add missing includes to mp/type-data.h
b1638aceb40 doc: Bump version 8 > 9
f61af487217 type-map: Work around LLVM 22 "out of bounds index" error

git-subtree-dir: src/ipc/libmultiprocess
git-subtree-split: 70f632bda8f80449b6240f98da768206a535a04e
ryanofsky added a commit to ryanofsky/bitcoin that referenced this pull request Mar 30, 2026
…da8f

70f632bda8f Merge bitcoin-core/libmultiprocess#265: ci: set LC_ALL in shell scripts
8e8e564259a Merge bitcoin-core/libmultiprocess#249: fixes for race conditions on disconnects
05d34cc2ec3 ci: set LC_ALL in shell scripts
e606fd84a8c Merge bitcoin-core/libmultiprocess#264: ci: reduce nproc multipliers
ff0eed1bf18 refactor: Use loop variable in type-context.h
ff1d8ba172a refactor: Move type-context.h getParams() call closer to use
1dbc59a4aa3 race fix: m_on_cancel called after request finishes
1643d05ba07 test: m_on_cancel called after request finishes
f5509a31fcc race fix: getParams() called after request cancel
4a60c39f24a test: getParams() called after request cancel
f11ec29ed20 race fix: worker thread destroyed before it is initialized
a1d643348f4 test: worker thread destroyed before it is initialized
336023382c4 ci: reduce nproc multipliers
b090beb9651 Merge bitcoin-core/libmultiprocess#256: ci: cache gnu32 nix store
be8622816da ci: cache gnu32 nix store
975270b619c Merge bitcoin-core/libmultiprocess#263: ci: bump timeout factor to 40
09f10e5a598 ci: bump timeout factor to 40
db8f76ad290 Merge bitcoin-core/libmultiprocess#253: ci: run some Bitcoin Core CI jobs
55a9b557b19 ci: set Bitcoin Core CI test repetition
fb0fc84d556 ci: add TSan job with instrumented libc++
0f29c38725b ci: add Bitcoin Core IPC tests (ASan + macOS)
3f64320315d Merge bitcoin-core/libmultiprocess#262: ci: enable clang-tidy in macOS job, use nullptr
cd9f8bdc9f0 Merge bitcoin-core/libmultiprocess#258: log: add socket connected info message and demote destroy logs to debug
b5d6258a42f Merge bitcoin-core/libmultiprocess#255: fix: use unsigned char cast and sizeof in LogEscape escape sequence
d94688e2c32 Merge bitcoin-core/libmultiprocess#251: Improved CustomBuildField for std::optional in IPC/libmultiprocess
a9499fad755 mp: use nullptr with pthread_threadid_np
f499e37850f ci: enable clang-tidy in macOS job
98f1352159d log: add socket connected info message and demote destroy logs to debug
554a481ea73 fix: use unsigned char cast and sizeof in LogEscape escape sequence
1977b9f3f65 Use std::forward in CustomBuildField for std::optional to allow move semantics, resolves FIXME
22bec918c97 Merge bitcoin-core/libmultiprocess#247: type-map: Work around LLVM 22 "out of bounds index" error
8a5e3ae6ed2 Merge bitcoin-core/libmultiprocess#242: proxy-types: add CustomHasField hook to map Cap'n Proto values to null C++ values
e8d35246918 Merge bitcoin-core/libmultiprocess#246: doc: Bump version 8 > 9
97d877053b6 proxy-types: add CustomHasField hook for nullable decode paths
8c2f10252c9 refactor: add missing includes to mp/type-data.h
b1638aceb40 doc: Bump version 8 > 9
f61af487217 type-map: Work around LLVM 22 "out of bounds index" error

git-subtree-dir: src/ipc/libmultiprocess
git-subtree-split: 70f632bda8f80449b6240f98da768206a535a04e
fanquake added a commit to bitcoin/bitcoin that referenced this pull request Mar 30, 2026
…n disconnects

2478a15 Squashed 'src/ipc/libmultiprocess/' changes from 1868a84451f..70f632bda8f (Ryan Ofsky)

Pull request description:

  Includes:

  - bitcoin-core/libmultiprocess#246
  - bitcoin-core/libmultiprocess#242
  - bitcoin-core/libmultiprocess#247
  - bitcoin-core/libmultiprocess#251
  - bitcoin-core/libmultiprocess#255
  - bitcoin-core/libmultiprocess#258
  - bitcoin-core/libmultiprocess#262
  - bitcoin-core/libmultiprocess#253
  - bitcoin-core/libmultiprocess#263
  - bitcoin-core/libmultiprocess#256
  - bitcoin-core/libmultiprocess#264
  - bitcoin-core/libmultiprocess#249
  - bitcoin-core/libmultiprocess#265

  The main change is bitcoin-core/libmultiprocess#249 which fixes 3 intermittent race conditions detected in bitcoin core CI and antithesis: #34711/#34756, #34777, and #34782.

  The changes can be verified by running `test/lint/git-subtree-check.sh src/ipc/libmultiprocess` as described in [developer notes](https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#subtrees) and [lint instructions](https://github.com/bitcoin/bitcoin/tree/master/test/lint#git-subtree-checksh)

ACKs for top commit:
  Sjors:
    ACK 613a548
  ismaelsadeeq:
    ACK 613a548

Tree-SHA512: d99eebc8b4f45b3c3099298167362cf5e7f3e9e622eef9f17af56388ee5207d77a04b915b2a5a894493e0395aeda70111216f2da0d2a6553f4f6396b3d31a744
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci: cache nix store

4 participants