[SPARK-55330][K8S] Make spark.kubernetes.legacy.useReadWriteOnceAccessMode a public config

[nitrajen]

What changes were proposed in this pull request?

spark.kubernetes.legacy.useReadWriteOnceAccessMode was introduced in SPARK-46945 as an .internal() config. This PR removes the .internal() marker and improves the doc string so the config appears in the public documentation and can be used by operators.

Why are the changes needed?

SPARK-46786 (Spark 4.0) changed the default PVC access mode from ReadWriteOnce to ReadWriteOncePod. The Kubernetes documentation states that whether fsGroup is applied to a mounted volume depends on the CSI driver's fsGroupPolicy setting, and certain policies do not apply fsGroup for ReadWriteOncePod volumes. This can cause non-root executor processes to fail at job startup when trying to create the blockmgr-xxx directory under a PVC-backed local dir:

ERROR JavaUtils: Failed to create directory /apps/application/data/blockmgr-2469dd51-bfd2-478d-9b3c-d5593bf21c26
java.nio.file.AccessDeniedException: /apps/application/data/blockmgr-2469dd51-bfd2-478d-9b3c-d5593bf21c26

The escape hatch config (spark.kubernetes.legacy.useReadWriteOnceAccessMode=true) already exists in the codebase to restore ReadWriteOnce behavior, but because it is marked internal it does not appear in the docs and users cannot find it.

Does this PR introduce any user-facing change?

Yes. spark.kubernetes.legacy.useReadWriteOnceAccessMode becomes a public, documented configuration option (versioned as 4.2.0). Previously it was internal and invisible to users.

How was this patch tested?

Added two unit tests to MountVolumesFeatureStepSuite:

• SPARK-55330: OnDemand PVC uses ReadWriteOncePod access mode by default — verifies the default PVC access mode is ReadWriteOncePod
• SPARK-55330: OnDemand PVC uses ReadWriteOnce when legacy access mode is enabled — verifies that setting spark.kubernetes.legacy.useReadWriteOnceAccessMode=true results in PVCs being created with ReadWriteOnce

Both tests use getAdditionalKubernetesResources() on MountVolumesFeatureStep and assert the accessModes field on the returned PersistentVolumeClaim.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Sonnet 4.6

[dongjoon-hyun]

Do you know ReadWriteOnce means that multiple executors on the same node share a single PVC in the worst case? Apache Spark community wants to avoid those situations completely. For those cases, please you can use hostPath.

Thus, -1 for this proposal, @nitrajen .