-
Notifications
You must be signed in to change notification settings - Fork 1.9k
OCPBUGS-99480: Harden Gateway API docs against route hijacking #117235
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -13,11 +13,11 @@ When you create a gateway resource, you must configure it for automatic address | |
|
|
||
| * You have access to the cluster as a user with the `cluster-admin` role. | ||
| * You have installed the {oc-first}. | ||
| * You have an existing `GatewayClass` resource, such as `openshift-default`. | ||
| * You have an existing `GatewayClass` custom resource, such as `openshift-default`. | ||
|
|
||
| .Procedure | ||
|
|
||
| . Create a YAML file, such as `hello-gateway.yaml`, that defines your `Gateway` object without the addresses field: | ||
| . Create a YAML file, such as `hello-gateway.yaml`, that defines your `Gateway` object. The `spec.addresses` field is omitted from this configuration to ensure automatic assignment. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. actually, we don't explicitly support address assignment on Gateway API (we don't run conformance tests for it nowadays) so this later claim of "ensure automatic assignment" is not quite true right now. It can (and will) change in a close future, but today it is what it is |
||
| + | ||
| [source,yaml] | ||
| ---- | ||
|
|
@@ -30,18 +30,22 @@ spec: | |
| gatewayClassName: openshift-default | ||
| listeners: | ||
| - name: http | ||
| hostname: "*.gwapi.<cluster_domain>" | ||
| hostname: "*.gwapi.<cluster_domain>" | ||
| port: 80 | ||
| protocol: HTTP | ||
| allowedRoutes: | ||
| namespaces: | ||
| from: All | ||
| from: Selector | ||
| selector: | ||
| matchLabels: | ||
| shared-gateway-access: "true" | ||
| ---- | ||
| + | ||
| * `metadata.name`: The name of your `Gateway` object. The name must consist of a maximum of 63 lowercase alphanumeric characters or hyphens (`-`). The name must also start and end with an alphanumeric character. | ||
| * Replace `<cluster_domain>` with your actual cluster ingress domain (for example, `example.com`). | ||
| * The `spec.addresses` field is omitted from this configuration to ensure automatic assignment. | ||
| * The `gatewayClassName` dictates which controller provisions the address and populates the `status.addresses` field. | ||
| * `metadata.name`: Specify the name of your `Gateway` object. The name must consist of a maximum of 63 lowercase alphanumeric characters or hyphens (`-`). The name must also start and end with an alphanumeric character. | ||
| * `spec.listeners.hostname`: Replace `<cluster_domain>` with your actual cluster ingress domain (for example, `example.com`). | ||
| * `listeners.hostname`: Specifies which hostnames can attach to this listener. Set a hostname so that only routes under that domain can receive traffic. | ||
| * `allowedRoutes.namespaces.from`: Allows route attachment only from namespaces that have the `shared-gateway-access: "true"` label. | ||
| * `spec.gatewayClassName`: The controller that provisions the address and populates the `status.addresses` field. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think the comment here is out of alignment with others. On the others above, you use a verb like "Specifies, Allows" but here you just say "the controller that provisions...". Maybe we should align the format here (and on the spec.listeners.hostname as well).
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. maybe a "what and how" phrasing would be better? like you did on metadata.name and listeners.hostname which btw I just figured out you are duplicating the meaning (line 45 and line 46)
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @rikatz, agreed. I was just too exhausted today to properly parallelize it here. I'll have to find some way to apply consistency. |
||
|
|
||
| . Apply the `Gateway` configuration by running the following command: | ||
| + | ||
|
|
@@ -64,4 +68,4 @@ NAME CLASS ADDRESS PROGRAMMED AGE | |
| sample-gateway openshift-default <gateway_address> True 6m16s | ||
| ---- | ||
| + | ||
| The `ADDRESS` column in the output displays the dynamically provisioned network address for your gateway. | ||
| The `ADDRESS` column in the output displays the dynamically provisioned network address for your gateway. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -9,44 +9,53 @@ | |
| [role="_abstract"] | ||
| To ensure that your applications receive only authenticated and authorized traffic, you must specify the allowed protocols and ports for your gateway. If you are routing secure traffic, you must also configure TLS settings. You can define these parameters by configuring the `spec.listeners` field in your `Gateway` custom resource (CR). | ||
|
|
||
| [IMPORTANT] | ||
| ==== | ||
| If your gateway is accessible from other namespaces, always configure the `spec.listeners[].allowedRoutes[].namespaces.selector` field with a selector for the permitted namespaces. By specifying a namespace selector, you prevent possible misuse or hijacking of the gateway from other namespaces. | ||
| ==== | ||
|
|
||
| .Procedure | ||
|
|
||
| . Create or edit a `Gateway` YAML file to include your desired listener configuration. | ||
| . Create or edit a `Gateway` YAML file to include your listener configuration. | ||
| + | ||
| -- | ||
| The following example demonstrates a `Gateway` CR with two listeners, one for HTTP and one for HTTPS. For detailed descriptions of the listener fields, see xref:gateway-listener-configuration-reference_{context}[]. | ||
| The following example demonstrates a `Gateway` CR with two listeners, one for HTTP and one for HTTPS. For detailed descriptions of the listener fields, see "Gateway listener configuration reference". | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't really know how this works, but is this new reference link right?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @rikatz, so because reasons we have very specific usage limitations on |
||
|
|
||
| [source,yaml] | ||
| ---- | ||
| kind: Gateway | ||
| apiVersion: gateway.networking.k8s.io/v1 | ||
| metadata: | ||
| name: <example_gateway> | ||
| namespace: openshift-ingress | ||
| name: <example_gateway> | ||
| namespace: openshift-ingress | ||
| spec: | ||
| gatewayClassName: openshift-default | ||
| listeners: | ||
| - protocol: HTTP | ||
| port: 80 | ||
| name: http | ||
| allowedRoutes: | ||
| namespaces: | ||
| from: Selector | ||
| selector: | ||
| matchLabels: | ||
| env: "dev" | ||
| - protocol: HTTPS | ||
| port: 443 | ||
| name: https | ||
| hostname: "*.<example_domain.tld>" | ||
| tls: | ||
| mode: Terminate | ||
| certificateRefs: | ||
| - name: <listener_cert> | ||
| kind: Secret | ||
| allowedRoutes: | ||
| namespaces: | ||
| from: All | ||
| gatewayClassName: openshift-default | ||
| listeners: | ||
| - protocol: HTTP | ||
| port: 80 | ||
| name: http | ||
| hostname: "*.<example_domain.tld>" | ||
| allowedRoutes: | ||
| namespaces: | ||
| from: Selector | ||
| selector: | ||
| matchLabels: | ||
| env: "dev" | ||
| - protocol: HTTPS | ||
| port: 443 | ||
| name: https | ||
| hostname: "*.<example_domain.tld>" | ||
| tls: | ||
| mode: Terminate | ||
| certificateRefs: | ||
| - name: <listener_cert> | ||
| kind: Secret | ||
| allowedRoutes: | ||
| namespaces: | ||
| from: Selector | ||
| selector: | ||
| matchLabels: | ||
| env: "dev" | ||
| ---- | ||
| -- | ||
|
|
||
|
|
@@ -55,4 +64,4 @@ spec: | |
| [source,terminal] | ||
| ---- | ||
| $ oc apply -f <gateway_cr>.yaml | ||
| ---- | ||
| ---- | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -39,7 +39,8 @@ spec: | |
| ---- | ||
| + | ||
| * You must include application hostnames and a `backendRef` rule that points to your backend service. | ||
| * If your `HTTPRoute` and `Gateway` CRs are deployed in different namespaces, the `Gateway` CR listener must be configured to allow cross-namespace routes. You must set `allowedRoutes.namespaces.from: All` in the `Gateway` CR. | ||
| * If your `HTTPRoute` and `Gateway` CRs are deployed in different namespaces, the `Gateway` CR listener must allow routes from the `HTTPRoute` namespace. You must configure the `spec.listeners.allowedRoutes.namespaces` field in the `Gateway` CR and specify a selector for trusted namespaces. | ||
| * The listener `hostname` on the parent `Gateway` CR must cover the hostnames in your `HTTPRoute` CR. For example, a listener hostname of `*.gwapi.apps.example.com` accepts an `HTTPRoute` hostname such as `app.gwapi.apps.example.com`, but rejects hostnames outside that domain. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. this is more a recommendation (should) than an enforcement (must) fwiw here and on every other place of this document, the hostname definition on the Gateway and on the xRoutes is less about permission and more about routing and union definition. For example, if a Gateway listener has no hostname definition, but a route does and attaches to that listener (eg.: www.some.tld), every call to www.some.tld will be directed to that route. If a Gateway defines a hostname (like *.some.tld) and the route doesn't but attach to that Gateway, and no other route defines *.some.tld every traffic from *.some.tld will be directed to that route. If you need some better definition for this hostname intersection, I recommend taking a look at https://gateway-api.sigs.k8s.io/docs/concepts/hostnames/ |
||
|
|
||
| . Apply the `HTTPRoute` CR file to your cluster: | ||
| + | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -9,18 +9,20 @@ | |
| [role="_abstract"] | ||
| To determine if Gateway API is the right routing solution for your cluster, review its benefits and limitations. The project is an effort to provide a standardized ecosystem by using a portable API with broad community support. Understanding these factors ensures your networking infrastructure aligns with your organizational needs and technical capabilities. | ||
|
|
||
| == Benefits | ||
| [id="gateway-api-benefits_{context}"] | ||
| == Gateway API Benefits | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I have no idea of the process here, but is there a reason to change this section and the section below to add "Gateway API" given this is already a Gateway API page?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @rikatz, institutional memory. I've always done my sections this way for the OpenShift networking content on the SDN side. Or at least I think I did. I'll have to go back and see what I've done historically. |
||
|
|
||
| Gateway API provides the following benefits: | ||
|
|
||
| * Portability: Where {product-title} uses HAProxy to improve Ingress performance, Gateway API does not rely on vendor-specific annotations to provide certain behavior. To get comparable performance to HAProxy, the `Gateway` objects need to be horizontally scaled or their associated nodes need to be vertically scaled. | ||
| * Separation of concerns: Gateway API uses a role-based approach to its resources, and more neatly fits into how a large organization structures its responsibilities and teams. Platform engineers might focus on `GatewayClass` resources, cluster administrators might focus on configuring `Gateway` resources, and application developers might focus on routing their services with `HTTPRoute` resources. | ||
| * Extensibility: Additional functionality is developed as a standardized CRD. | ||
|
|
||
| == Limitations | ||
| [id="gateway-api-limitations_{context}"] | ||
| == Gateway API Limitations | ||
|
|
||
| Gateway API has the following limitations: | ||
|
|
||
| * Version incompatibilities: The Gateway API ecosystem changes rapidly, and some implementations do not work with others because their featureset is based on differing versions of Gateway API. | ||
| * Resource overhead: While more flexible, Gateway API uses multiple resource types to achieve an outcome. For smaller applications, the simplicity of traditional Ingress might be a better fit. | ||
| * On-premise infrastructure dependencies: On-premise deployments, such as bare metal or VMware vSphere environments, do not automatically provision network load balancers or manage external DNS records. To use Gateway API on an on-premise platform, you must explicitly deploy a load balancer controller, such as MetalLB, and manually map your DNS records to the provisioned gateway address. | ||
| * On-premise infrastructure dependencies: On-premise deployments, such as bare metal or VMware vSphere environments, do not automatically provision network load balancers or manage external DNS records. To use Gateway API on an on-premise platform, you must explicitly deploy a load balancer controller, such as MetalLB, and manually map your DNS records to the provisioned gateway address. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -18,10 +18,10 @@ Defines the list of listeners for the gateway. You can customize this field with | |
| Defines the network port and protocol. For example, `protocol: HTTP` accepts HTTP traffic on port `80`, and `protocol: HTTPS` accepts HTTPS traffic on port `443`. | ||
|
|
||
| `listeners.hostname`:: | ||
| Defines the hostnames that the listener matches for incoming requests. For example, it can match only requests for hostnames ending in `<example_domain.tld>` (such as `www.<example_domain.tld>`). If no hostname is specified, the gateway routes any traffic that can attach to it. | ||
| Defines the hostnames that the listener matches for incoming requests. For example, it can match only requests for hostnames ending in `<example_domain.tld>` (such as `www.<example_domain.tld>` or `*.gwapi.<example_domain.tld>`). Always set a hostname on each listener. If no hostname is specified, any route that can attach to the listener can claim arbitrary hostnames, which can allow hostname or domain hijacking. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. unless the user has a good reason to not set a hostname, like if they want to define a catch-all route. Anyway, on managed cloud environments (AWS, GCP, Azure), not setting a hostname on the listener will cause a DNSRecord to not be created (not a problem/bug, but the Gateway will not have an automatic DNS entry created) |
||
|
|
||
| `listeners.tls`:: | ||
| Specifies the TLS settings for secure communication, including the mode (currently, only `Terminate` is supported on {product-title}) and the Kubernetes secret containing the certificate keypair. For example, TLS is terminated at the gateway using a certificate stored in a Kubernetes `Secret` called `<listener_cert>`. You must ensure that the specified Kubernetes secret exists before you create the `Gateway` CR. | ||
|
|
||
| `listeners.allowedRoutes`:: | ||
| Controls which `Route` resources can attach to this listener. For example, an HTTP listener might only allow `HTTPRoute` resources from namespaces that have the `env: "dev"` label (using `Selector`), while an HTTPS listener might allow `HTTPRoute` resources from any namespace to attach (using `All`). Setting `allowedRoutes.namespaces.from: Same` is not supported; routes from the same namespace as the gateway are always allowed. | ||
| Controls which `Route` resources can attach to this listener. For example, an HTTP listener might only allow `HTTPRoute` resources from namespaces that have the `env: "dev"` label (using `Selector`). Setting `allowedRoutes.namespaces.from: Same` is not supported; routes from the same namespace as the gateway are always allowed. Do not use `.spec.listeners[].allowedRoutes.namespaces.from=All` because this is insecure and can allow hostname or domain hijacking. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hum, I think the "custom" word here may be unnecessary, let me explain why: in Kubernetes, a resource means "an instance of a specific type".
So this is similar to say "a
Routeresource, or aFeatureGateresource" and so one (there are pedantic differences here which I don't think will add anything to my review)Anyway, IMO we can remove the "custom" word
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@rikatz, interesting; I'd always considered any resource that is implemented via the CRD mechanism to be a CR, and thus custom resource, and this usage is what we've historically had throughout the OCP documentation. So that's why I changed it, even though I could have left it alone; I couldn't help it.