Merge 3008.1-1 into 3008.x#69865
Open
dwoz wants to merge 16 commits into
Open
Conversation
Introduce first-class support for patch releases (e.g. 3008.1-1) across the full packaging stack: - salt/version.py: add 'patch' field to SaltStackVersion with word-boundary lookahead to disambiguate patch suffix from git-describe commit count; parse() uses groupdict() to keep patch keyword-only and avoid positional arg breakage - tools/changelog.py: _get_salt_version() reads _version.txt directly when present; update_rpm() splits post-release into Version/Release fields - tools/pkg/__init__.py: guard --release stamp against post-releases - tools/pkg/build.py: pass -b to debuild for patch releases to avoid dpkg-source native+hyphen incompatibility - pkg/windows/msi/build_pkg.ps1: extend version regex and produce 4-part InternalVersion (major1.major2.minor.patch) - .github/workflows/templates/layout.yml.jinja: add patch-branch glob '[0-9][0-9][0-9][0-9].[0-9]*-[0-9]*' to on-push triggers; regenerate ci.yml - tests/pytests/unit/test_version.py: add parsing and ordering tests - tests/pytests/pkg/integration/test_version.py: add RPM NVR ordering test
…ide RCs - tools/ci.py: get_release_changelog_target() now matches patch branches (e.g. refs/heads/3008.1-patch) via regex fallback when the standard release_branches substring check doesn't match - staging.yml.jinja + staging.yml: document 3008.1-1 as valid input form - release.yml: document patch release version format and RPM Release: N sort ordering in the salt-version input description
setuptools normalizes "3008.1-1" to "3008.1.post1" per PEP 440, producing salt-3008.1.post1.tar.gz. After recompression, rename it to salt-3008.1-1.tar.gz so that CI artifact names and the release workflow's download paths stay consistent with the version string from salt/_version.txt.
The old code called .split("-")[0] which stripped the patch release
suffix (-1) from every artifact type, so pep440_public_equal("3008.1-1",
"3008.1") always returned False and all 63 package tests failed.
For RPM-family artifacts the release digit lives in regex group 2
(-N.el, -N.fc, -N.am); reconstruct "version-release" when release > 0.
For non-RPM artifacts (DEB, macOS pkg, MSI, exe) the patch digit may
already be in group 1 ("3008.1-1" or "3008.1-1-macos"); preserve a
purely-numeric first hyphen segment and strip platform suffixes.
Newer salt.repo files downloaded from the install guide enable both salt-repo-3006-lts and salt-repo-3008-lts by default. The 3006.x branch in install_previous() disabled salt-repo-3007-sts but not the 3008 LTS repo, so an unversioned `yum install salt` would pull 3008.2 (the latest in the 3008 LTS channel) instead of staying on 3006.x. For patch releases (e.g. 3008.1-1), this caused test_salt_upgrade to fail: start_version=3008.2 > artifact_version=3008.1.post1. Don't call _check_retcode here because older salt.repo files without the [salt-repo-3008-lts] stanza would return non-zero from config-manager.
The rpm_re pattern hardcoded release=0 (-0.) which never matches a patch release RPM like salt-3008.1-1.x86_64.rpm (release=1). The fallback filename builder also always appended -0. Both paths now handle any release number; the fallback derives the correct NVR from packaging.version for post-releases.
The downloaded salt.repo now enables salt-repo-3008-lts by default (3008 is current LTS). When install_previous() enabled salt-repo-3007-sts to install a 3007.x base version, the 3008-lts repo remained active and yum resolved the unversioned `salt` install to 3008.2 instead of 3007.x, causing upgrade tests to assert 3008.2 <= 3008.1.post1. Mirrors the same fix already applied in the 3006.x else-branch.
Both modules unconditionally imported the OTel SDK at module load, even though tracing.enabled and metrics.enabled default to false. Every salt daemon entry point transitively imports both modules, so a ~15-process salt-master container was paying ~15 MB per subsystem per process (~450 MB total) for functionality nobody was using. Move the OTel imports into _load_otel() helpers invoked only after is_enabled() returns True. Public API is preserved. Backport of 73bca44 from PR saltstack#69856.
When HighState.__init__ or State.__init__ raised after allocating their fileclient (e.g. BaseHighState.__init__ failing during master_opts(), or _gather_pillar() failing during pillar compilation), the caller never received the instance and therefore never called .destroy(). The fileclient's ZeroMQ RequestClient was finalized during garbage collection with _closing = False, tripping the ``TransportWarning: Unclosed transport!`` warning that PR saltstack#65559 added. Wrap both constructors' post-allocation bodies in try/except that destroys the freshly-allocated fileclient before re-raising. Also close the temporary Pillar object built by State._gather_pillar() in a try/finally so its channel doesn't rely on __del__ ordering at shutdown. Backport of cb09894 from PR saltstack#69675.
…altstack#69857) Three orthogonal leaks in the salt-master EventPublisher / salt-api TCP transport that together drove multi-GB steady-state growth under sustained load: 1. MessageClient.close() polled ``send_future_map`` at 1s intervals via ``check_close`` and only actually tore down the transport when that map drained. One orphaned in-flight future (e.g. a coroutine cancelled by cherrypy mid-request) kept the map non-empty forever, so the whole MessageClient graph -- Unpacker, IOStream, LazyLoaders reachable via self -- stayed alive. Under salt-api load memray showed ~18 MessageClient objects/s leaking. Close synchronously instead: cancel pending futures with SaltReqTimeoutError, tear the tcp client and stream down before returning, and guard connect() against clobbering _closing/_closed after close() has run. Backport of fb6f95a. 2. PubServer had no way to notice a subscriber disconnect promptly -- ``_stream_read`` only observes the close on the next ``read_bytes`` return, but event-bus subscribers never write, so the read is a permanently-pending await. Result: subscribers accumulate in ``self.clients`` from the moment the peer half-closes until ``publish_payload`` throws on the next write to that stream, and the Tornado IOStream's ``_read_buffer`` / ``_write_buffer`` bytearrays stay pinned. Observed on a live 3008.2 master: 7500+ leaked subscriber sockets / 150 GB RSS in 24 h. Register a ``stream.set_close_callback`` alongside ``clients.add(client)`` so ``clients.discard(client)`` fires the instant the peer goes away (mirrors 3006.x ``IPCMessagePublisher``'s ``discard_after_closed``). 3. TCPPuller.handle_stream fired each payload via ``self.io_loop.create_task(self.payload_handler(body))`` and immediately looped back to read the next framed message. Under sustained publish load (~5000 events/sec) tasks accumulated in the io_loop faster than they could complete: 909,120 pending tasks on the EventPublisher after ~5 min drove RSS to 10 GB (each task frame plus retained event payload is ~11 kB). The 3006.x equivalent (``IPCMessagePublisher._write``, commit d4e2e07) solved the same accumulation by switching from ``@gen.coroutine`` to a plain function with ``future.add_done_callback``. On 3008.1's asyncio-native transport the natural equivalent is to apply backpressure at the reader: await the handler inline. A misbehaving handler is caught and logged so a single bad event can't kill the reader loop. Combined, these keep EP RSS flat and pending-task count in the low tens under identical stress load.
Copies tests/monitoring/ and .github/workflows/nightly-stress-test.yml verbatim from origin/3008.x head so 3008.1-patch can dispatch and schedule the same stress harness we use to validate memory profile changes on 3008.x. Contents (all newly added on this branch, no existing 3008.1-patch files touched): - .github/workflows/nightly-stress-test.yml -- runs at 02:00 UTC and on workflow_dispatch (with configurable ``duration``) - tests/monitoring/Dockerfile.salt + docker-compose.yml + master.conf + minion.conf -- 3-minion salt-master + salt-api + prometheus + grafana + cadvisor test rig - tests/monitoring/stress_test.sh + stress_api.sh -- driver scripts - tests/monitoring/analyze_stats.py + render_panels.py -- post-run Prometheus queries, slope analysis, and PNG dashboard rendering - tests/monitoring/srv/salt/ -- test states, event flooders, and observability sidecar for the master container - tests/monitoring/grafana/provisioning/ -- dashboards + datasource All copied at origin/3008.x tip; no local modifications.
The tests/monitoring directory contains helper scripts (analyze_stats.py, render_panels.py, srv/salt/flood_events.py, etc.) that aren't pytest modules -- they're driver / observer scripts for the stress-test harness -- and don't follow the ``test_*.py`` naming convention. test_module_names.py::BadTestModuleNamesTestCase flags them and the ``unit / zeromq`` job under the Stage Release workflow fails. Mirror the same EXCLUDED_DIRS entry origin/3008.x has for the same directory.
Resolved conflicts in: - salt/transport/tcp.py: Accepted more detailed comments from upstream explaining the TCPPuller backpressure mechanism and memory impact - tests/pytests/unit/transport/test_tcp.py: Updated tests to use proper 4-byte length-prefixed framing that matches the current TCPPuller implementation; removed obsolete _PullerFakeStream class
dwoz
force-pushed
the
merge/3008.1-patch/3008.x
branch
from
July 24, 2026 21:10
0c55c59 to
eb4266a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.