Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,11 @@ All notable changes to this project will be documented in this file.
Previously, the Spark application pods contained the same TLS volume twice, which could not be applied to the API server.
- The spark-submit job now sets the correct `-Djavax.net.ssl.trustStore` properties ([#655]).
- Spark application jobs can now have pod/node affinities. This was an omission as the application driver and executors already had this field for a long time. ([#664]).
- Fix "404 page not found" error for the initial object list ([#666]).

### Changed

- Bump stackable-operator to 0.106.2, snafu to 0.9, strum to 0.28 ([#663]).
- Bump stackable-operator to 0.108.0, snafu to 0.9, strum to 0.28 ([#663], [#666]).
- Gracefully shutdown all concurrent tasks by forwarding the SIGTERM signal ([#651]).
- Remove the Spark application owner reference from the executor pods.
This allows Kubernetes to garbage collect them early when the driver or the submit job fail ([#648]).
Expand All @@ -51,6 +52,7 @@ All notable changes to this project will be documented in this file.
[#660]: https://github.com/stackabletech/spark-k8s-operator/pull/660
[#663]: https://github.com/stackabletech/spark-k8s-operator/pull/663
[#664]: https://github.com/stackabletech/spark-k8s-operator/pull/664
[#666]: https://github.com/stackabletech/spark-k8s-operator/pull/666

## [25.11.0] - 2025-11-07

Expand Down
22 changes: 11 additions & 11 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

55 changes: 27 additions & 28 deletions Cargo.nix

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ repository = "https://github.com/stackabletech/spark-k8s-operator"

[workspace.dependencies]
product-config = { git = "https://github.com/stackabletech/product-config.git", tag = "0.8.0" }
stackable-operator = { git = "https://github.com/stackabletech/operator-rs.git", tag = "stackable-operator-0.106.2", features = ["telemetry", "versioned", "webhook"] }
stackable-operator = { git = "https://github.com/stackabletech/operator-rs.git", tag = "stackable-operator-0.108.0", features = ["webhook"] }

anyhow = "1.0"
built = { version = "0.8", features = ["chrono", "git2"] }
Expand Down
18 changes: 9 additions & 9 deletions crate-hashes.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions deploy/helm/spark-k8s-operator/templates/roles.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,9 @@ rules:
{{- if .Values.maintenance.customResourceDefinitions.maintain }}
- create
- patch
# Required for startup condition
- list
- watch
{{- end }}
- apiGroups:
- events.k8s.io
Expand Down
35 changes: 31 additions & 4 deletions rust/operator-binary/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ use stackable_operator::{
core::v1::{ConfigMap, Pod, Service},
},
kube::{
CustomResourceExt as _,
core::DeserializeGuard,
runtime::{
Controller,
Expand All @@ -28,7 +29,7 @@ use stackable_operator::{
logging::controller::report_controller_reconciled,
shared::yaml::SerializeOptions,
telemetry::Tracing,
utils::signal::SignalWatcher,
utils::signal::{self, SignalWatcher},
};
use tracing::info_span;
use tracing_futures::Instrument;
Expand Down Expand Up @@ -356,13 +357,39 @@ async fn main() -> anyhow::Result<()> {
)
.map(anyhow::Ok);

let delayed_history_controller = async {
signal::crd_established(
&client,
crd::history::v1alpha1::SparkHistoryServer::crd_name(),
None,
)
.await?;
history_controller.await
};

let delayed_connect_controller = async {
signal::crd_established(
&client,
connect::crd::v1alpha1::SparkConnectServer::crd_name(),
None,
)
.await?;
connect_controller.await
};

let delayed_app_controller = async {
signal::crd_established(&client, crd::v1alpha1::SparkApplication::crd_name(), None)
.await?;
app_controller.await
};

// kube-runtime's Controller will tokio::spawn each reconciliation, so this only concerns the internal watch machinery
futures::try_join!(
pod_driver_controller,
history_controller,
connect_controller,
delayed_history_controller,
delayed_connect_controller,
delayed_app_controller,
webhook_server,
app_controller,
eos_checker,
)?;
}
Expand Down
Loading