Skip to content

Inference Operator AddOn, NodeAffinity and Version Update 3.0 #371

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
nargokul merged 6 commits into from
Feb 16, 2026
+217 −5
Merged

Inference Operator AddOn, NodeAffinity and Version Update 3.0 #371

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
fe4cbeb
inferenceendpointconfigs remove tab to fix syntax
Jan 30, 2026
83077ff
Merge branch 'aws:main' into main
XuanCS Feb 13, 2026
a25d116
comment
Feb 13, 2026
c64edae
Add addOn nodeAffinity CRD and helm version update
Feb 15, 2026
9b8af44
Add Migration README
Feb 15, 2026
474cb83
Seperate migration into follow up PR
Feb 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions helm_chart/HyperPodHelmChart/charts/inference-operator/Chart.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,11 @@ type: application
# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
# Versions are expected to follow Semantic Versioning (https://semver.org/)
version: 1.3.0
version: 2.0.0

# This is the version number of the application being deployed. Keep this aligned
# with operator image MAJOR.MINOR version.
appVersion: "2.3"
appVersion: "3.0"

dependencies:
- name: aws-mountpoint-s3-csi-driver
Expand Down
203 changes: 201 additions & 2 deletions ...ence-operator/config/crd/inference.sagemaker.aws.amazon.com_inferenceendpointconfigs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -353,8 +353,7 @@ spec:
instanceType:
description: |-
Single instance type to deploy the model on.
This field is mutually exclusive with instanceTypes.
Use this when you want to deploy on a specific instance type.
This field is mutually exclusive with instanceTypes. Use this when you want to deploy on a specific instance type.
pattern: ^ml\..*
type: string
instanceTypes:
Expand Down Expand Up @@ -824,6 +823,206 @@ spec:
modelVersion:
description: Version of the model used in creating sagemaker endpoint
type: string
nodeAffinity:
description: |-
Custom node affinity configuration for advanced scheduling.
This field is mutually exclusive with instanceType and instanceTypes fields.
properties:
preferredDuringSchedulingIgnoredDuringExecution:
description: |-
The scheduler will prefer to schedule pods to nodes that satisfy
the affinity expressions specified by this field, but it may choose
a node that violates one or more of the expressions. The node that is
most preferred is the one with the greatest sum of weights, i.e.
for each node that meets all of the scheduling requirements (resource
request, requiredDuringScheduling affinity expressions, etc.),
compute a sum by iterating through the elements of this field and adding
"weight" to the sum if the node matches the corresponding matchExpressions; the
node(s) with the highest sum are the most preferred.
items:
description: |-
An empty preferred scheduling term matches all objects with implicit weight 0
(i.e. it's a no-op). A null preferred scheduling term matches no objects (i.e. is also a no-op).
properties:
preference:
description: A node selector term, associated with the corresponding
weight.
properties:
matchExpressions:
description: A list of node selector requirements by
node's labels.
items:
description: |-
A node selector requirement is a selector that contains values, a key, and an operator
that relates the key and values.
properties:
key:
description: The label key that the selector applies
to.
type: string
operator:
description: |-
Represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.
type: string
values:
description: |-
An array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. If the operator is Gt or Lt, the values
array must have a single element, which will be interpreted as an integer.
This array is replaced during a strategic merge patch.
items:
type: string
type: array
x-kubernetes-list-type: atomic
required:
- key
- operator
type: object
type: array
x-kubernetes-list-type: atomic
matchFields:
description: A list of node selector requirements by
node's fields.
items:
description: |-
A node selector requirement is a selector that contains values, a key, and an operator
that relates the key and values.
properties:
key:
description: The label key that the selector applies
to.
type: string
operator:
description: |-
Represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.
type: string
values:
description: |-
An array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. If the operator is Gt or Lt, the values
array must have a single element, which will be interpreted as an integer.
This array is replaced during a strategic merge patch.
items:
type: string
type: array
x-kubernetes-list-type: atomic
required:
- key
- operator
type: object
type: array
x-kubernetes-list-type: atomic
type: object
x-kubernetes-map-type: atomic
weight:
description: Weight associated with matching the corresponding
nodeSelectorTerm, in the range 1-100.
format: int32
type: integer
required:
- preference
- weight
type: object
type: array
x-kubernetes-list-type: atomic
requiredDuringSchedulingIgnoredDuringExecution:
description: |-
If the affinity requirements specified by this field are not met at
scheduling time, the pod will not be scheduled onto the node.
If the affinity requirements specified by this field cease to be met
at some point during pod execution (e.g. due to an update), the system
may or may not try to eventually evict the pod from its node.
properties:
nodeSelectorTerms:
description: Required. A list of node selector terms. The
terms are ORed.
items:
description: |-
A null or empty node selector term matches no objects. The requirements of
them are ANDed.
The TopologySelectorTerm type implements a subset of the NodeSelectorTerm.
properties:
matchExpressions:
description: A list of node selector requirements by
node's labels.
items:
description: |-
A node selector requirement is a selector that contains values, a key, and an operator
that relates the key and values.
properties:
key:
description: The label key that the selector applies
to.
type: string
operator:
description: |-
Represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.
type: string
values:
description: |-
An array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. If the operator is Gt or Lt, the values
array must have a single element, which will be interpreted as an integer.
This array is replaced during a strategic merge patch.
items:
type: string
type: array
x-kubernetes-list-type: atomic
required:
- key
- operator
type: object
type: array
x-kubernetes-list-type: atomic
matchFields:
description: A list of node selector requirements by
node's fields.
items:
description: |-
A node selector requirement is a selector that contains values, a key, and an operator
that relates the key and values.
properties:
key:
description: The label key that the selector applies
to.
type: string
operator:
description: |-
Represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.
type: string
values:
description: |-
An array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. If the operator is Gt or Lt, the values
array must have a single element, which will be interpreted as an integer.
This array is replaced during a strategic merge patch.
items:
type: string
type: array
x-kubernetes-list-type: atomic
required:
- key
- operator
type: object
type: array
x-kubernetes-list-type: atomic
type: object
x-kubernetes-map-type: atomic
type: array
x-kubernetes-list-type: atomic
required:
- nodeSelectorTerms
type: object
x-kubernetes-map-type: atomic
type: object
replicas:
default: 1
description: The desired number of inference server replicas. Default
Expand Down
10 changes: 10 additions & 0 deletions helm_chart/HyperPodHelmChart/charts/inference-operator/templates/_validateRequiredValues.tpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,16 @@
{{- fail "executionRoleArn must be set via the --set executionRoleArn=<arn> flag" -}}
{{- end}}

{{/* Validate that the execution role name has the required prefix for the managed policy restrictions */}}
{{/* This validation only applies when isAddOn is true */}}
{{- if $.Values.isAddOn -}}
{{- $roleNameParts := splitList "/" $.Values.executionRoleArn -}}
{{- $roleName := last $roleNameParts -}}
{{- if not (hasPrefix "SageMakerHyperPodInference" $roleName) -}}
{{- fail "executionRoleArn must have a role name with prefix 'SageMakerHyperPodInference'" -}}
{{- end}}
{{- end}}

{{- if not $.Values.tlsCertificateS3Bucket}}
{{- fail "tlsCertificateS3Bucket must be set via the --set tlsCertificateS3Bucket=<s3 Bucket> flag" -}}
{{- else -}}
Expand Down
5 changes: 4 additions & 1 deletion helm_chart/HyperPodHelmChart/charts/inference-operator/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@ shortPrefix: "hyperpod-inference"
region: ""
eksClusterName: ""

# Set to true when deploying as an EKS add-on (enables execution role name prefix validation)
isAddOn: false

image:
repositoryDomainMap:
us-east-2: 061051259071.dkr.ecr.us-east-2.amazonaws.com
Expand All @@ -21,7 +24,7 @@ image:
ap-southeast-4: 311141544681.dkr.ecr.ap-southeast-4.amazonaws.com
ap-southeast-3: 158128612970.dkr.ecr.ap-southeast-3.amazonaws.com
eu-south-2: 025050981094.dkr.ecr.eu-south-2.amazonaws.com
tag: v2.3
tag: v3.0
pullPolicy: Always
repository:
hyperpodClusterArn:
Expand Down