[ST] Enable AuthST, fix several tests, performance improvements by jankalinic · Pull Request #2776 · streamshub/console

jankalinic · 2026-07-16T17:16:35Z

No description provided.

jankalinic · 2026-07-20T08:43:14Z

/systemtests run --help

streamshub-bot · 2026-07-20T08:43:25Z

ℹ️ Systemtests Help ℹ️

You can run system tests with the following parameters:

--testcase=<name> (optional): Run a specific test case. Use Maven style, e.g. [TopicST | KafkaST#testKafkaTwo...]
--profile=<profile> (optional): Profile to use, e.g. [regression]
--env=<VAR1=val1;VAR2=val2> (optional): Set environment variables for the test
--install-type=<olm|yaml> (optional): Choose installation type. Default is [olm]
--retry-count=<number> (optional): Number of Maven retries. Default is [0]
--help: Show this help message

Note: Use either --testcase or --profile to select tests to run, setting both will result in
github actions prioritizing testcase over profile. If neither is provided, no systemtest will start.

Example usage:

/systemtests run --profile=regression --env=MY_ENV=one;SECOND=two
/systemtests run --testcase=TopicST --install-type=olm --retry-count=3

jankalinic · 2026-07-20T08:44:08Z

/systemtests run --testcase=AuthST --install-type=olm --retry-count=2

jankalinic · 2026-07-29T11:27:16Z

/systemtests run --testcase=AuthST --install-type=olm --retry-count=2

jankalinic · 2026-08-03T14:22:50Z

/systemtests run --testcase=AuthST --install-type=olm --retry-count=2

MikeEdgar · 2026-08-20T09:23:09Z

@jankalinic I'm looking into this. I may expand the scope to make sure all of the tests are running OK.

jankalinic · 2026-08-20T09:48:55Z

Thank you @MikeEdgar. Feel free to modify or decide the future of this PR.

Signed-off-by: jkalinic <jkalinic@redhat.com>

Signed-off-by: Michael Edgar <medgar@redhat.com>

MikeEdgar · 2026-08-20T18:22:48Z

    --driver=${MINIKUBE_DRIVER:-kvm2} \
    --cpus=${MINIKUBE_CPU_COUNT:-6} \
    --memory=${MINIKUBE_MEMORY:-16384} \
+    --disk-size=${MINIKUBE_DISK_SIZE:-50g} \


The default disk allocation in minikube is insufficient for the Kafka Connect image build process.

This change has no effect in GitHub actions, but is useful for local testing.

MikeEdgar · 2026-08-20T18:23:30Z

+                    if (method.trySetAccessible()) {
+                        method.invoke(testMethodContext.getRequiredTestInstance());
+                    } else {
+                        throw new IllegalStateException("Unable to access @SetupTestBucket method " + method);
+                    }


This allows us to keep the test classes as package-private (JUnit best practice) with minimal fuss.

MikeEdgar · 2026-08-20T18:24:45Z

                .setName(STDOUT)
                .setLayout(PatternLayout.newBuilder()
-                    .withPattern("%d{yyyy-MM-dd HH:mm:ss}{GMT} [%thread] %highlight{%-5p} [%c{1}:%L] %m%n")
+                    .setPattern("%d{yyyy-MM-dd HH:mm:ss}{GMT} [%thread] %highlight{%-5p} [%c{1}:%L] %m%n")


These with methods are deprecated in favor of the set equivalents.

MikeEdgar · 2026-08-20T18:25:24Z

        maybeCreateCatalogSource();

-        String newVersion = Environment.getConsoleOperatorVersion();
+        String newVersion = Environment.getConsoleOperatorVersion(String::toLowerCase);


The version from the POM needs to be lower-cased for the OLM upgrade where the images and OLM artifacts use lowercase -snapshot as the suffix.

MikeEdgar · 2026-08-20T18:27:01Z

    }

-    private static KafkaClients buildClients(String namespace, String kafkaName, String kafkaUser, String topicName, int messageCount) {
-        return new KafkaClientsBuilder()


I think we can (over time) migrate the KafkaClients and Strimzi kafka client image Jobs to just use the Kafka Admin, Consumer, and Producer clients instead. It's significantly faster and minimizes the number of moving parts in the tests.

MikeEdgar · 2026-08-20T18:29:07Z

        }
    }
+
+    public static CompletableFuture<Void> runAsyncWithContext(Runnable runnable) {


These methods allow us to run more of the test setup concurrently on worker threads while using the same test context. The Kubetest4j libraries relies on ThreadLocal a lot, so copying the context across threads is necessary.

MikeEdgar · 2026-08-20T18:30:45Z

+                    clusterName, 
+                    KafkaNamingUtils.kafkaUserName(clusterName),
+                    Admin::create)) {
+                Wait.until("Groups deleted", 2_000, 120_000, () -> {


This long timeout is needed because the test clients Job appears to not call close on consumers. Only after a timeout does Kafka transition the groups from Stable to Empty, required for deletion. This won't be needed if/when we migrated to just using Producers/Consumers in-process.

MikeEdgar · 2026-08-20T18:32:15Z

/systemtests run --install-type=olm

Signed-off-by: Michael Edgar <medgar@redhat.com>

MikeEdgar · 2026-08-21T12:19:49Z

+            LOGGER.info("CONNECT_DEPLOY_CONCURRENT is false, deploying {} and {} sequentially", KAFKA_CONNECT_SRC_NAME, KAFKA_CONNECT_SINK_NAME);
+            connect = srcConn.thenComposeAsync(runComposableAsyncWithContext(deployConnectSink));


Apparently, GitHub runners do not have enough disk space for the concurrent connect image builds.

MikeEdgar · 2026-08-21T12:20:51Z

        consoleOperatorSetup.setInstallConfig(olmConfig);
        LOGGER.info("Installing Console Operator via OLM into namespace '{}'", Constants.CO_NAMESPACE);
-        consoleOperatorSetup.install(false);
+        consoleOperatorSetup.install(true);


When this test runs with other test classes, any previously-installed operator needs to be removed first, false -> true

MikeEdgar · 2026-08-21T12:22:57Z

+        });
    }

    public static String getClusterDomain() {


This reworks cluster domain lookup now respects CONSOLE_CLUSTER_DOMAIN primarily, then if not set it will derive from the OCP base domain as before (if OpenShift), or try to use the Kubernetes master URL's IP address to construct a .nip.io domain. This should help in local environments where, for example, minikube's IP is changing each time the cluster is created.

MikeEdgar · 2026-08-21T12:29:52Z

/systemtests run --install-type=olm

Signed-off-by: Michael Edgar <medgar@redhat.com>

MikeEdgar · 2026-08-21T17:14:02Z

/systemtests run --install-type=olm

sonarqubecloud · 2026-08-21T17:26:44Z

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

streamshub-bot · 2026-08-21T19:19:03Z

✅ Systemtests run finished - success ✅

Test Summary:

TOTAL: 43
PASS: 38
FAIL: 0
SKIP: 5

Used parameters:

TEST_CASE:
PROFILE:
INSTALL_TYPE: olm
RETRY_COUNT: 0
ENVS:

MikeEdgar · 2026-08-21T19:24:34Z

@jankalinic please take a look next week and let me know if I've done anything objectionable with this. Aside from make some of the tests pass, this PR now has some performance improvements and a start to migrating away from the kafka clients jobs to just using the Kafka clients directly.

jankalinic self-assigned this Jul 16, 2026

jankalinic requested a review from MikeEdgar July 16, 2026 17:16

jankalinic marked this pull request as ready for review July 16, 2026 17:16

MikeEdgar reviewed Jul 16, 2026

View reviewed changes

Comment thread systemtests/src/main/java/com/github/streamshub/systemtests/utils/playwright/PwPageUrls.java Outdated

jankalinic force-pushed the enable-fixed-auth-sts branch from e535e24 to a81c0fa Compare July 20, 2026 08:22

jankalinic force-pushed the enable-fixed-auth-sts branch 2 times, most recently from c2a999a to ff51014 Compare July 29, 2026 11:06

jankalinic and others added 9 commits August 20, 2026 14:19

enable authst checks after fix

4804cbf

Signed-off-by: jkalinic <jkalinic@redhat.com>

added specific groupID, removed blank members page check

e49d1dd

Signed-off-by: jkalinic <jkalinic@redhat.com>

Fix AuthST, performance improvements

1fe628b

Signed-off-by: Michael Edgar <medgar@redhat.com>

Support package-private test classes with TestBucketExtension

5e2d256

Signed-off-by: Michael Edgar <medgar@redhat.com>

Fix failing MessagesST, refine messages table column selector methods

ab51151

Signed-off-by: Michael Edgar <medgar@redhat.com>

Speed up ConnectST, increase default minikube storage (for KC build)

a9f8ac5

Signed-off-by: Michael Edgar <medgar@redhat.com>

Fix failing TopicST tests, performance improvements

eaba08d

Signed-off-by: Michael Edgar <medgar@redhat.com>

Fix OlmUpgradeST

e63107a

Signed-off-by: Michael Edgar <medgar@redhat.com>

Remove previous operator for OlmUpgradeST, suppress Kafka client logs

b378bd5

Signed-off-by: Michael Edgar <medgar@redhat.com>

MikeEdgar force-pushed the enable-fixed-auth-sts branch from ff51014 to b378bd5 Compare August 20, 2026 18:19

MikeEdgar changed the title ~~[ST] Enable authst checks after fix~~ [ST] Enable AuthST, fix several tests, performance improvements Aug 20, 2026

MikeEdgar reviewed Aug 20, 2026

View reviewed changes

MikeEdgar added 2 commits August 21, 2026 08:14

Add Connect concurrent deployment configurable (sequential by default)

ae65a93

Signed-off-by: Michael Edgar <medgar@redhat.com>

Cache ClusterUtils, derive Kubernetes domain if not configured

68bca22

Signed-off-by: Michael Edgar <medgar@redhat.com>

MikeEdgar reviewed Aug 21, 2026

View reviewed changes

MikeEdgar added 2 commits August 21, 2026 12:23

Re-factor GroupsST, performance improvements

537db71

Signed-off-by: Michael Edgar <medgar@redhat.com>

Fix TopicST message range boundary

d4adfa0

Signed-off-by: Michael Edgar <medgar@redhat.com>

MikeEdgar added this to the 0.14.0 milestone Aug 21, 2026

		LOGGER.info("CONNECT_DEPLOY_CONCURRENT is false, deploying {} and {} sequentially", KAFKA_CONNECT_SRC_NAME, KAFKA_CONNECT_SINK_NAME);
		connect = srcConn.thenComposeAsync(runComposableAsyncWithContext(deployConnectSink));

Uh oh!

Conversation

jankalinic commented Jul 16, 2026

Uh oh!

Uh oh!

jankalinic commented Jul 20, 2026

Uh oh!

streamshub-bot commented Jul 20, 2026

ℹ️ Systemtests Help ℹ️

Example usage:

Uh oh!

jankalinic commented Jul 20, 2026

Uh oh!

jankalinic commented Jul 29, 2026

Uh oh!

jankalinic commented Aug 3, 2026

Uh oh!

MikeEdgar commented Aug 20, 2026

Uh oh!

jankalinic commented Aug 20, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MikeEdgar commented Aug 20, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MikeEdgar commented Aug 21, 2026

Uh oh!

MikeEdgar commented Aug 21, 2026

Uh oh!

sonarqubecloud Bot commented Aug 21, 2026

Quality Gate passed

Uh oh!

streamshub-bot commented Aug 21, 2026

✅ Systemtests run finished - success ✅

Test Summary:

Used parameters:

Uh oh!

MikeEdgar commented Aug 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants