Skip to content

fix(webapp): survive asset hash rotation across rolling deploys#4260

Merged
kathiekiwi merged 12 commits into
mainfrom
fix/css-loading
Jul 14, 2026
Merged

fix(webapp): survive asset hash rotation across rolling deploys#4260
kathiekiwi merged 12 commits into
mainfrom
fix/css-loading

Conversation

@kathiekiwi

@kathiekiwi kathiekiwi commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator

Problem

Webapp HTML references content-hashed /build assets, and each Docker image contains exactly one build with a hard 404 for unknown hashes. During a rolling deploy, a client holding HTML from the old build may request old asset hashes from a replica running the new image, causing missing styles or failed chunk loads.

The page should recover automatically once a compatible build becomes available, without reload loops or unnecessary interruptions during normal deployments.

What changed

  • Build changes alone do nothing — no polling, no automatic reloads.
  • If a CSS or JavaScript asset fails to load, a recovery overlay is shown immediately.
  • While the server still reports the same build, the client polls for a newer build using exponential backoff (up to ~60s). As soon as a newer build is detected, the page reloads automatically.
  • If no newer build appears within the timeout, recovery falls back to a manual Reload action.
  • If recovery still fails after the automatic reload, the client stops retrying and displays a final recovery screen instead of entering a reload loop.
  • Recovery preserves form values and scroll position across the automatic reload.

@changeset-bot

changeset-bot Bot commented Jul 14, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 1726d57

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@coderabbitai

coderabbitai Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

The deployment workflow resolves a prior image and passes it to Docker. Docker preserves previous hashed web assets, overlays current assets, prunes files older than 14 days, and enforces a size limit. Document responses default to Cache-Control: no-cache. A production-only head script detects failed build assets and dynamic imports, then performs limited delayed reloads in normal and error document output.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check ⚠️ Warning The description omits the required template sections such as Closes #, checklist, Testing, Changelog, and Screenshots. Rewrite the PR description to match the template and add the missing issue link, checklist items, testing steps, changelog, and screenshots section.
✅ Passed checks (3 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title is concise and clearly summarizes the main change: surviving asset hash rotation during rolling deploys.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/css-loading

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Documents are content-hash-coupled to the build baked into the image
that rendered them, so during a rolling deploy a stale document requests
/build asset URLs that 404 on the new image: unstyled pages and dead
buttons from partial hydration.

Three layers:
- Carry the previous published image's /build assets into each new image
  (PREV_IMAGE build arg, resolved in the publish workflow, no-op default
  for forks/local builds), pruned after 14 days.
- Default document responses to Cache-Control: no-cache so browsers
  always revalidate HTML.
- Inline recovery script that force-reloads (at most twice, 30s apart)
  when a /build stylesheet/script fails to load or a chunk import
  rejects - covers rollbacks and tabs older than the retention window.
coderabbitai[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

Pruning before the current build lands makes the current build's assets
unreachable by the prune, whatever mtimes the builder produces - a
broken prune can only mean slow image growth, never a broken image.
@kathiekiwi kathiekiwi marked this pull request as ready for review July 14, 2026 14:28
Comment thread .server-changes/stale-deploy-asset-recovery.md
… semver fallback

The 14-day prune depends on the builder preserving mtimes through COPY.
Assert the outcome instead of the mechanism: fail the image build if
public/build exceeds 1GB, so silent retention failure becomes a loud CI
error. Also try :latest before :main when resolving the previous image,
so semver releases carry forward their true predecessor's assets.
devin-ai-integration[bot]

This comment was marked as resolved.

@nicktrn nicktrn left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to find a different solution for this that doesn't involve creating Docker images with assets from two different builds. Lots of additional complexity to debug when things go wrong.

Mixing assets from two builds in one image adds too much debugging
complexity. Stale clients during a deploy window are covered by the
client-side recovery reload; a longer-term fix (serving /build from
object storage with retention) can be designed separately.
Instead of blindly reloading on a /build asset failure, the recovery
script now polls a new /build-version endpoint (no-store) with backoff
and reloads only once the server reports a different build than the one
the page was rendered with (window.__remixManifest.version). If the
versions never diverge - the asset failed for some other reason - or the
reload budget is spent, it shows a manual-reload banner instead of
leaving a dead page. Turns a speculative reload into a deterministic
wait for a compatible deployment state.
devin-ai-integration[bot]

This comment was marked as resolved.

…ate restore

Final recovery policy:
- All responses carry X-Build-Id. A working page is left alone when a
  new build exists; when a Remix loader fetch during navigation reveals
  a different build, the navigation becomes a full document load - the
  user lands on the new build without seeing an incompatible state.
- Only real breakage (a /build asset 404 or failed chunk import) starts
  recovery: an overlay goes up immediately, the client polls
  /build-version and reloads once the server reports a different build.
  If versions never diverge or the reload budget is spent, the overlay
  offers a manual reload instead of a dead page.
- Form fields and scroll position are snapshotted to sessionStorage
  before a recovery reload and restored after (native value setters +
  input events so React state stays in sync). history.state is
  deliberately not restored - the Remix router owns it.
devin-ai-integration[bot]

This comment was marked as resolved.

@kathiekiwi kathiekiwi requested a review from nicktrn July 14, 2026 15:50

@nicktrn nicktrn left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, small comment

Comment thread .server-changes/stale-deploy-asset-recovery.md Outdated
@triggerdotdev triggerdotdev deleted a comment Jul 14, 2026
@triggerdotdev triggerdotdev deleted a comment Jul 14, 2026
@kathiekiwi kathiekiwi merged commit 2aa6420 into main Jul 14, 2026
30 checks passed
@kathiekiwi kathiekiwi deleted the fix/css-loading branch July 14, 2026 22:48
@github-actions github-actions Bot mentioned this pull request Jul 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants