Skip to content

[WIP][SPARK-56015][INFRA][DOCS] Cleanup docs container, remove unused R deps, and fix x86 build.#54974

Draft
holdenk wants to merge 10 commits intoapache:masterfrom
holdenk:SPARK-56015-remove-r-deps-from-docs-container-r2
Draft

[WIP][SPARK-56015][INFRA][DOCS] Cleanup docs container, remove unused R deps, and fix x86 build.#54974
holdenk wants to merge 10 commits intoapache:masterfrom
holdenk:SPARK-56015-remove-r-deps-from-docs-container-r2

Conversation

@holdenk
Copy link
Contributor

@holdenk holdenk commented Mar 24, 2026

What changes were proposed in this pull request?

Update the documentation build README, container, and steps. Takes over from #54838

Why are the changes needed?

  • SparkR is deprecated and docs are built outside the container anyways
  • Current docs build is hard coded to assume ARM
  • For building the deprecated R docs people need to install an extra package
  • We shouldn't be creating a bunch of files as root in the users spark directtory if we can avoid it.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Built docs docker container locally, built docs with the doc build shell script locally, built website with docs container locally.

Was this patch authored or co-authored using generative AI tooling?

  • Asked chat gpt about the different CRAN install methods.
  • copilot in autocomplete mode for some of it
  • Various asking claude about R & Jekyll stuff I'm not super up to date on
  • case/switch in shell script generated by claude

sfc-gh-hkarau and others added 10 commits March 17, 2026 15:51
Co-authored-by: Holden Karau <holden@pigscanfly.ca>
…th root permissions, fix we use Python 3.12 not 3.11 anymore, handle amd64/arm64 builds

Co-authored-by: Holden Karau <holden@pigscanfly.ca>
… bind mount permissions

Co-authored-by: Holden Karau <holden@pigscanfly.ca>
… out of scope for fixing that here today lets just get the current system working again on x86.

Co-authored-by: Holden Karau <holden@pigscanfly.ca>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants