docs: document the v1 Materialize CRD and installV1CRD opt-in#35508
docs: document the v1 Materialize CRD and installV1CRD opt-in#35508alex-hunt-materialize wants to merge 11 commits into
Conversation
|
Thanks for opening this PR! Here are a few tips to help make the review process smooth for everyone. PR title guidelines
Pre-merge checklist
|
|
@kay-kim I'm keeping this as a draft so we don't accidentally merge it, but it is ready for review. I may need to rebase if something comes up in the original PR, but I think the docs commit shouldn't change. |
|
Thank you!!! |
d7820a1 to
731208e
Compare
|
Note to self, we need to document how to run the new orchestratord with webhooks if they aren't using our terraform. |
731208e to
5066939
Compare
5066939 to
e7c2da3
Compare
| @@ -0,0 +1,70 @@ | |||
| - v26.19.0 introduces a new `v1alpha2` version of the Materialize CRD | |||
There was a problem hiding this comment.
I'd recommend phrasing this as something like this:
--
v26.19.0 introduces a new version of the Materialize CRD,v1alpha2. The new CRD simplifies rollouts by automatically detecting changes. This means you will no longer need to manually rotate a UUID to trigger a rollout.
Before, on v1alpha1:
yamlapiVersion: materialize.cloud/v1alpha1
kind: Materialize
metadata:
name: 12345678-1234-1234-1234-123456789012
namespace: materialize-environment
spec:
environmentdImageRef: materialize/environmentd:v26.16.0
requestRollout: 22222222-2222-2222-2222-222222222222 # ← MUST set a new UUID every upgrade
# forceRollout: 33333333-3333-3333-3333-333333333333 # ← for forced rollouts
rolloutStrategy: WaitUntilReady
backendSecretName: materialize-backendAfter, on v1alpha2:
apiVersion: materialize.cloud/v1alpha2
kind: Materialize
metadata:
name: 12345678-1234-1234-1234-123456789012
namespace: materialize-environment
spec:
environmentdImageRef: materialize/environmentd:v26.16.0 # ← just change this
# forceRollout: 33333333-3333-3333-3333-333333333333 # ← only for forced rollouts
rolloutStrategy: WaitUntilReady
backendSecretName: materialize-backendWith the new change, the requestRollout field will be removed, along with all previously deprecated fields.
Switching to v1alpha2 is opt-in. You may continue to apply v1alpha1 CRs and your existing instances will behave exactly as before. We recommend you opt-in to v1alpha2 at your convenience, as we will migrate to the v1alpha2 behavior in the next major release. For step-by-step migration instructions, see Switching from v1alpha1 to v1alpha2.
|
|
||
| {{< important >}} | ||
|
|
||
| v26.19 requires infrastructure changes to support migration to v1alpha2 resources. |
There was a problem hiding this comment.
Do these infrastructure changes need to be made for v26.19 @alex-hunt-materialize ? Or is it only for migrating to v1alpha2?
If it is only to migrate to v1alpha2, I'd actually recommend that we move this into a separate section. At a high level, I'd imagine that the structure can be like this:
- Self-Managed deployments
- Upgrading page: Give the user basic information about what has changed in v26.19
- Materialize CRDs Page: Give the user information about v1alpha1 and v1alpha2. Include information about how to migrate from alpha1 to alpha2.
Does that make sense?
There was a problem hiding this comment.
This applies regardless of which CRD version you use. I'll rephrase this line to say:
v26.19 requires infrastructure changes to support converting between `v1alpha1` and `v1alpha2` resources.
|
|
||
| {{< /important >}} | ||
|
|
||
| The Materialize Operator now hosts conversion webhooks to convert between |
There was a problem hiding this comment.
I'd rephrase this a little:
Materialize uses webhooks to allow you to gracefully migrate from alphav1 to alphav2. To enable these webhooks to be delivered, you need to install cert-manager and allow internal network ingress on port 8001. If you're using our terraform modules, this will be handled for you automatically. If you're not using our terraform, you will need to complete these steps manually before upgrading to v26.19.
- Tab for how to upgrade using our terraform
- Tab for how to upgrade if customers are doing this manually
There was a problem hiding this comment.
- Tab for how to upgrade using our terraform
They should already know how to upgrade the terraform code. They should have been doing that for literally every release. Do we really need to tell them how to bump a version number? If they can't handle that, they should not be running self-managed.
- Tab for how to upgrade if customers are doing this manually
That's the stuff directly below this.
There was a problem hiding this comment.
They should already know how to upgrade the terraform code. They should have been doing that for literally every release. Do we really need to tell them how to bump a version number? If they can't handle that, they should not be running self-managed.
Yes. Please be explicit here - customers are going to get tripped up otherwise. If I skim this document, I am going to miss the instructions for how to do the upgrade using the MZ terraform.
There was a problem hiding this comment.
What I'd recommend doing here is:
- 1 tab for users on our supported terraform
- 1 tab for folks on our legacy terraform. Give them instructions / details for how to migrate to the new terraform.
- 1 tab for manual instructions
There was a problem hiding this comment.
Done, please take another look.
| `8001` from the API server to pods with the label | ||
| `app.kubernetes.io/name: materialize-operator`. | ||
|
|
||
| **Kubernetes NetworkPolicy:** Add a policy that allows ingress on port |
There was a problem hiding this comment.
Could we explicitly specify that this policy will only allow ingress from within the cluster? And not ingress from public internet?
There was a problem hiding this comment.
This example policy allows ingress from everywhere, but they only need to allow it from the Kubernetes API servers. Kubernetes network policies only allow specifying rules based on CIDR blocks or pod labels. The only way to restrict this for the API servers is by CIDR block, which we don't have available here.
Ingress to this port on this pod is safe, there is nothing bad that anyone can do with these endpoints, beyond making orchestratord generate a response.
Additionally, this policy only affects traffic that has reached Kubernetes. They should be running their clusters in an isolated VPC, and the normal cloud firewall rules still apply (ie: security groups for nodes).
The new CRD version is now called v1 instead of v1alpha2, and the operator only installs the v1 CRD and its conversion webhook when the operator.args.installV1CRD helm value is set. Update the version docs to v26.29, regenerate the CRD field descriptions from the v1 schema, and pick up the newer rollout-cancellation instructions on the v1alpha1 page. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
02dcdd7 to
ef7d339
Compare
Docs follow up for #35418
Motivation
Separate PR so we don't update the docs until we're ready to release.
Description
This branch is based on #35418; only the commits on top of it are the docs changes.
Documents the new
v1CRD version (hash-based rollouts, norequestRollout), which is opt-in twice over:v1CRD and its conversion webhook when theoperator.args.installV1CRDhelm value is set. cert-manager (or a user-provided certificate) and API-server ingress to port 8001 are only required when opting in.v1alpha1CRs keep working unchanged; instances are switched individually by applying av1CR.Also regenerates the CRD field description data from the
v1schema and updates thecrd-writer/bump-versiontooling for the version rename.