build: harden 1.0 release validation - #295
Merged
Merged
Conversation
Python 3.14 formats typing.Union[a, b] as "a | b" while Python 3.10-3.13 format "Union[a, b]", so the get_persons signature manifest entry failed only on 3.14. The annotation object is unchanged; normalize the legacy rendering so one manifest stays valid on every supported interpreter. Co-authored-by: Matthew Spah <spahmatthew@gmail.com>
The release validator only clean-installed the wheel and only proved the strict HTTP default through constructor signatures, so a broken source distribution or a behavioral regression in strict handling could pass. Offline CI also still watched the stale release/0.9.0 branch and tested only Python 3.10-3.12. Validator: - clean-install the wheel and the source distribution in separate throwaway virtual environments and run the same smoke test against both - generalize wheel-only terminology to cover both distribution artifacts - verify a final fake 403 raises MlbHttpError under the default and under explicit strict_http=True, for Mlb and for MlbDataAdapter on v1 and v1.1 - verify strict_http=False returns the historical empty result and emits exactly one MlbHttpCompatibilityWarning mentioning strict_http=False - assert MlbHttpError status_code, reason, method, url, and response_data without freezing the exception or warning strings - verify an injected requests.Session keeps its headers and its exact adapter objects, never receives the library retry policy, and is never closed by the library - verify a library-created Session carries the installed-metadata User-Agent and the documented retry policy - label reverted strict defaults explicitly instead of raising a bare AssertionError - name the failing artifact, field, or path with expected and actual values in every validation error - expand the required source-distribution paths to the files the archive intentionally carries Tests: - unit-cover the validator helpers with synthetic wheel ZIPs and sdist tarballs, including every required failure mode - prove validate() clean-installs both artifacts in separate environments - lock the documentation Python-example checks to every release-notes file while keeping current-version checks off historical notes - require the release/1.0.0 CI trigger literally instead of deriving it from the still-unbumped package version - cover the Python matrix, twine check, and publishing-safety contract CI and docs: - watch main and release/1.0.0; drop release/0.9.0 - test Python 3.10 through 3.14 and build on 3.14 - add twine as a development dependency and run twine check on both artifacts; nothing is uploaded - state the 3.10 minimum and 3.10-3.14 CI coverage in the current docs The package version stays 0.9.0; the 1.0.0 bump belongs to a separate issue and the validator keeps reading the expected version from pyproject.toml. Co-authored-by: Matthew Spah <spahmatthew@gmail.com>
Mattsface
marked this pull request as ready for review
August 6, 2026 04:44
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #288
Why
The release validator proved less than the 1.0 contract needs.
It clean-installed only the wheel, so a source distribution that cannot build, omits a package file, or loses a runtime dependency would have shipped unnoticed. It proved the strict HTTP default only through constructor signatures, so a behavioral regression in strict handling could pass while the signature still read
strict_http: bool = True. Its error messages did not always name the failing artifact, and its terminology and usage example implied a wheel-only,0.9.0-only tool.Deterministic CI had two matching gaps: it still watched the stale
release/0.9.0branch, and it tested only Python 3.10 through 3.12 with notwine checkat all. The existing branch-trigger test derived the release branch from the still-unbumped0.9.0package version, sorelease/0.9.0satisfied it.Nothing in this pull request publishes anything.
What
Validator (
scripts/validate_release.py)Clean-install coverage:
mlbstatsapi.__file__resolves under the environment'spureliband that it is running inside a virtual environment, so the checkout cannot shadow the installed artifactinstalling wheel,running wheel smoke test,installing source distribution,running source distribution smoke testStrict behavior, not only signatures. The smoke test keeps the
strict_httpsignature-default assertions and adds deterministic offline behavior driven by a fake injected Session. The fake builds a realisticrequests.Response(status_code=403,reason="Forbidden", a Stats API-shaped URL derived from the actual request, and a small JSON body) and performs no network I/O:Mlb(session=fake)(nostrict_http, so the real default runs)MlbHttpErrorMlb(session=fake, strict_http=True)MlbHttpErrorMlb(session=fake, strict_http=False)get_sports()returns[]and warns exactly onceFor each raised error the smoke test verifies
status_code == 403,reason == "Forbidden",method == "GET", the expected URL, and thatresponse_datacontains the JSON payload. For compatibility mode it verifies exactly one captured warning, that its category isMlbHttpCompatibilityWarning, and that the message mentionsstrict_http=False. Neither the exception string nor the warning string is frozen.Direct adapter construction keeps its signature checks and now also confirms, for both
v1andv1.1, that the default and explicit strict adapters raiseMlbHttpErrorfor the fake final 403, and that explicit compatibility mode returns an emptyMlbResult(status_code == 403,message == "Forbidden",data == {}) with the public warning.Injected Session preservation now uses a real
requests.Sessionsubclass that recordsclose(). It sets a customUser-Agent, a custom application header, and custom HTTPS and HTTP adapters mounted withmax_retries=0, captures the exact adapter object identities, and afterwith Mlb(session=session): ...verifies the Session was never closed by the library, both headers are unchanged, both adapters are the exact same objects, and neither adapter carries the library retry policy. The smoke test closes the Session itself in afinallyblock.Library-created Session validation is retained and extended: the
User-Agentmust equalpython-mlb-statsapi/<installed-version>,create_retry_policy()must return a freshRetryper call, and both mounted adapters must carry the documented retry values without freezing unrelated Requests internals.The public API contract checks are unchanged in substance: the same package-root imports, constructor signatures, exception and warning inheritance,
MlbResultbehavior, legacy helpers, and no__all__in 1.0. No accidentally exposed submodule was promoted and no legacy helper was removed.Diagnostics and terminology:
Mlb.strict_http must default to True for the 1.0 contractandMlbDataAdapter.strict_http must default to True for the 1.0 contractrather than a bareAssertionError--expected-version 1.0.0instead of presenting0.9.0as the only expected versionREQUIRED_SDIST_PATHSgrew from three entries to the eleven paths the generated archive actually carries (PKG-INFO,LICENSE,README.md,pyproject.toml, and the sixmlbstatsapimodules plusmlbstatsapi/models/__init__.py), verified againsttar -tzf. Tests, docs, and scripts are intentionally absent from the sdist and are not required.Validator unit tests (
tests/test_release_validation.py)scripts/validate_release.pyis imported withimportlibsincescripts/is not a package. Synthetic wheel ZIPs and sdist tarballs stand in for real artifacts, clean installs are stubbed, and no test creates a virtual environment or makes a network request.Failure coverage added: missing
distdirectory, no matching wheel, no matching source distribution, multiple stale wheels, multiple stale source distributions, incorrect wheelName, incorrect wheelVersion, incorrectRequires-Python, ambiguous wheelMETADATA, each required sdist path missing in turn, failed wheel installation, failed sdist installation, and failed smoke test for both artifacts. Each assertion checks that the message names the artifact and the incorrect field or path with expected and actual values.Other coverage:
validate()clean-installs both artifacts in that order, each artifact gets a distinct virtual environment, the smoke script is written outside the checkout, the smoke-test source compiles, and the strict-default failure messages remain present in the smoke-test contract.The documentation Python-example syntax test now covers
README.md,docs/http-transport.md,docs/public-api.md, and every file indocs/releases/instead of hard-coding0.9.0.md. Current-version checks are scoped toREADME.md,docs/http-transport.md,docs/public-api.md, anddocs/releases/1.0.0.md; historical notes are asserted to keep their own version-specificUser-Agentstatements, and no historical 0.7.1, 0.8.0, or 0.9.0 fact was rewritten.The CI branch-trigger test now requires
release/1.0.0literally and rejectsrelease/0.9.0, so it no longer depends on the version bump owned by #289.CI (
.github/workflows/build-and-test.yml)mainandrelease/1.0.0;release/0.9.0is removed;workflow_dispatchis keptfail-fast: falseso a single-version failure is easy to isolate. Python 3.15 is deliberately excluded while it is a prereleaserm -rf dist,poetry build,python scripts/validate_release.py, thenpoetry run twine check dist/*. Both the validator andtwine checkare required, in that orderpyproject.tomlandpoetry.lock), never a runtime dependencyDocumentation
docs/public-api.md,docs/releases/1.0.0.md, andREADME.mdnow state a minimum declared Python version of 3.10 and CI-validated versions 3.10 through 3.14, replacing the "3.10, 3.11, 3.12" wording and the note that 3.13 coverage was still outstanding. No upper Python bound was added, the runtime requirement stays>=3.10, and no document claims Python 3.15.docs/releases/1.0.0.mdand the README development section also describe the two-artifact validator andtwine check.Python 3.14 incompatibility found and fixed
Adding 3.14 to the matrix surfaced one real failure, committed separately as
test: normalize Union rendering across the Python matrix:Python 3.14 renders
typing.Union[a, b]throughinspect.formatannotationasstr | List[int], while 3.10 through 3.13 renderUnion[str, List[int]]. The annotation object and the public API are unchanged; only the rendering differs. The signature manifest now normalizes the legacy spelling so one manifest is valid on every supported interpreter. This was not a dependency problem: every locked dependency, includingpydantic-core 2.41.5andcryptography 50.0.0, installs and runs on 3.14, so no dependency constraint needed relaxing and Python 3.14 was not removed from the matrix.Tests
Focused suite (Python 3.12, Poetry environment):
Full deterministic suite:
The baseline on
release/1.0.0was 521 passing tests, so this adds 89.Package checks:
Validator output shows separate installation and smoke testing per artifact:
Local multi-version runs, each with the locked dependency versions installed into a fresh environment. GitHub Actions remains the authoritative matrix:
scripts/validate_release.pywas also run end to end under Python 3.14 (the build-job interpreter) and passed both artifact installs, andpoetry installresolved the lock cleanly on 3.14.Negative testing outside the automated suite. The smoke test was extracted and run against a deliberately broken installed copy to confirm the failure messages are understandable:
strict_httpreverted toFalsein both constructors:AssertionError: Mlb.strict_http must default to True for the 1.0 contractTrue:AssertionError: Mlb.strict_http must default to True for the 1.0 contract: a final 403 did not raise MlbHttpErrorThe second case is the one signature checks alone could not catch.
CI configuration audit:
release/1.0.0appears in both the pull-request and push trigger lists,release/0.9.0is gone, 3.10 through 3.14 are present, 3.15 appears only in the comment explaining its exclusion, andpoetry run twine check dist/*is present.Live MLB API tests were not run and were not merged into deterministic CI.
Version-bump boundary with #289
pyproject.tomlstill declaresversion = "0.9.0". The 1.0.0 bump belongs to #289.The validator continues to read the expected version from
pyproject.tomlby default, so today it accepts correctly built0.9.0-named artifacts that carry the already-implemented 1.0 behavior, and after #289 the same unmodified validator will require1.0.0artifacts.1.0.0appears only as a usage example in the docstring and as a synthetic expected-version fixture in tests; it is never hard-coded as the only valid artifact version. A unit test asserts the validator stays version-driven.Publishing safety
No workflow contains
poetry publish,twine upload, a PyPI or TestPyPI token,pypa/gh-action-pypi-publish,softprops/action-gh-release,gh release create, orgit tag. A parametrized test enforces this across both workflows. Twine is used only fortwine check, which reads metadata and uploads nothing. No release, tag, or publish action was performed.Risk and impact
Minimal.
No library code changed.
mlbstatsapiHTTP decision logic, constructor defaults, the exception hierarchy, the warning implementation, retry values, Session ownership behavior, endpoint models, endpoint signatures, and the package version are all untouched. The diff is limited toscripts/validate_release.py, two test modules, the offline CI workflow, the Twine development dependency, and current documentation. Historical release notes and the live-test workflow were not modified.The realistic failure modes are CI-side. Validation now runs two clean installs instead of one, so the build job takes roughly twice as long in that step (about 11 seconds locally) and the sdist install needs network access for build isolation, which CI already has. Python 3.13 and 3.14 are new required jobs, so a future dependency that drops support for one of them will fail CI rather than pass silently, which is the intent. The smoke test reads
Mlb._sessionandMlb._strict_httpand buildsrequests.Response._content; these are private, so a future internal refactor could require a validator update. That trade-off is already accepted for release validation and documentation is explicitly held to the public API instead.If something does go wrong, the blast radius is a failing CI job or a failing release validation run. No published artifact and no runtime behavior can be affected by this change.
Intentionally left out
docs/public-api.mdalready tracks as a separate focused issuepoetry checkwarnings about migrating[tool.poetry]metadata to[project], which would change package metadata layout and belongs in its own change