Skip to content

Trt 1989 benchmarking#3557

Open
neisw wants to merge 3 commits into
openshift:mainfrom
neisw:trt-1989-benchmarking
Open

Trt 1989 benchmarking#3557
neisw wants to merge 3 commits into
openshift:mainfrom
neisw:trt-1989-benchmarking

Conversation

@neisw
Copy link
Copy Markdown
Contributor

@neisw neisw commented May 25, 2026

Updates benchmarking queries and optionally writes report to file

Summary by CodeRabbit

  • Tests

    • Expanded benchmark coverage with many additional cases measuring query and API performance across varied scenarios; tests now record and log row counts per case.
  • Chores

    • Improved benchmark reporting: results include a short connection identifier, render as a single consolidated table, and can optionally emit timestamped benchmark report files when configured.

@openshift-merge-bot
Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

@openshift-ci openshift-ci Bot requested review from smg247 and xueqzhan May 25, 2026 21:04
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 25, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: b8bd772e-1011-41b9-8221-d9ffdbe16fc6

📥 Commits

Reviewing files that changed from the base of the PR and between 117d9e3 and 9b3122d.

📒 Files selected for processing (1)
  • pkg/flags/postgres_benchmarking_test.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • pkg/flags/postgres_benchmarking_test.go

Walkthrough

This PR adds DSN-derived short connection naming, refactors benchmark summary output to build and optionally persist timestamped reports, expands getBenchmarkCases with multiple new report query benchmarks, and updates DB client and benchmark tests to thread the connection name into reporting.

Changes

Benchmark Harness Enhancements

Layer / File(s) Summary
Reporting helpers and output refactor
pkg/flags/postgres_benchmarking_test.go
Adds extractConnectionName(dsn), updates printSummaryTable(t, results, connName) to use strings.Builder + fmt.Fprintf formatting, prints once, and optionally writes a timestamped benchmark-<connName>-<ts>.txt when benchmarking_file_path is set.
Expanded benchmark cases
pkg/flags/postgres_benchmarking_test.go
Adds multiple benchmark cases to getBenchmarkCases(asOf) (VariantReports, JobReports, BuildClusterHealth, RecentTestFailures, PullRequestReport, RepositoryReport, JobsRunsReport, ProwJobHistoricalTestCounts) that call corresponding query/api functions and log row counts.
DB client returns connection name
pkg/flags/postgres_benchmarking_test.go
Refactors getBenchmarkDBClient(t) to return (*db.DB, connName) and derives connName from the DSN via extractConnectionName.
Test wiring to thread connName
pkg/flags/postgres_benchmarking_test.go
Updates Test_BenchmarkIndividual, Test_BenchmarkFindTestsByRelease, Test_BenchmarkCombined, and Test_BenchmarkGroup to capture connName from getBenchmarkDBClient and pass it to printSummaryTable.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

  • openshift/sippy#3533: Introduces the benchmark harness foundation that this PR builds upon with additional benchmark cases and connection tracking enhancements.

Suggested labels

lgtm

Suggested reviewers

  • deepsm007
  • petr-muller
🚥 Pre-merge checks | ✅ 16 | ❌ 4

❌ Failed checks (3 warnings, 1 inconclusive)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Go Error Handling ⚠️ Warning Multiple errors returned without context wrapping: database query errors at lines 140, 277, 302, 414 and function errors throughout benchmark cases not wrapped with fmt.Errorf %w. Wrap all returned errors with fmt.Errorf using %w to add context, e.g., return fmt.Errorf("query failed: %w", res.Error) at lines 140, 277, 302, 414, 391 and similar locations.
Test Coverage For New Features ⚠️ Warning Pure function extractConnectionName lacks dedicated unit tests. Custom check requires pure functions to always be tested. Add unit tests for extractConnectionName(dsn string) string covering PostgreSQL DSN formats and edge cases (malformed DSNs, missing dots, empty strings).
Title check ❓ Inconclusive The title 'Trt 1989 benchmarking' is vague and contains unclear abbreviations. While it references the ticket number (TRT 1989), it lacks specificity about what benchmarking work was done and does not clearly convey the main changes. Consider a more descriptive title like 'Add benchmarking report file output and expand benchmark query coverage' to clearly communicate the main objectives of the PR.
✅ Passed checks (16 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Sql Injection Prevention ✅ Passed All SQL queries use parameterized placeholders (? or @name); all parameters are hardcoded constants or computed values, never user input. No string concatenation in SQL construction found.
Excessive Css In React Should Use Styles ✅ Passed Custom check for React CSS styling (useStyles pattern) is not applicable to this PR, which only modifies Go benchmarking test code with no React components or inline CSS styling.
Single Responsibility And Clear Naming ✅ Passed Functions have single responsibilities, structs are well-scoped (2-6 fields), names are specific/action-oriented with no generic Manager/Handler patterns, and no excessive parameter overloading found.
Stable And Deterministic Test Names ✅ Passed All test names are static and deterministic. No Ginkgo framework is used. Test functions and benchmark case names contain no dynamic information like timestamps, UUIDs, or generated identifiers.
Test Structure And Quality ✅ Passed The custom check applies to Ginkgo test code, but the modified file (postgres_benchmarking_test.go) uses standard Go testing package tests with testing.T, not Ginkgo. Check is not applicable.
Microshift Test Compatibility ✅ Passed This PR modifies database benchmarking tests in the sippy repository, not Ginkgo e2e tests. No new Ginkgo tests (It, Describe, Context, When) are added, and no Kubernetes/OpenShift APIs are used.
Single Node Openshift (Sno) Test Compatibility ✅ Passed PR modifies database benchmarking tests using standard Go testing.T, not Ginkgo e2e tests. SNO check applies only to Ginkgo e2e tests on OpenShift clusters, not sippy unit/integration tests.
Topology-Aware Scheduling Compatibility ✅ Passed PR modifies only postgres_benchmarking_test.go, a test file. Contains no deployment manifests, operator code, controllers, or scheduling constraints. Check is not applicable.
Ote Binary Stdout Contract ✅ Passed The modified file is a standard Go test (not an OTE binary), containing only test-level functions with stdout writes inside Test_* functions, which are allowed by the check guidelines.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed PR modifies a Go benchmark test file using standard testing.T, not Ginkgo e2e tests. No It(), Describe(), Context(), or When() calls present. Check only applies to Ginkgo e2e tests.
No-Weak-Crypto ✅ Passed The modified file contains benchmark test code with no cryptographic operations, weak crypto algorithms, custom crypto, or non-constant-time secret comparisons.
Container-Privileges ✅ Passed PR modifies only postgres_benchmarking_test.go (Go test code), not container/K8s manifests. No privileged settings, hostPID, hostNetwork, hostIPC, SYS_ADMIN, or allowPrivilegeEscalation flags present.
No-Sensitive-Data-In-Logs ✅ Passed No passwords, tokens, API keys, or PII are logged. Code extracts only hostname subdomains from DSN for file naming; benchmark metrics logged are non-sensitive counts/durations.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 25, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: neisw

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 25, 2026
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pkg/flags/postgres_benchmarking_test.go`:
- Around line 39-49: The extractConnectionName function currently slices the DSN
after "@" but can include path or port and returns empty for hosts without dots;
update it to first isolate the authority portion (stop at the first "/" or "?"
after the "@"), then strip any ":port" if present, and finally return the host
portion — if a dot exists in that host return the substring before the first
dot, otherwise return the full host (so "localhost" yields "localhost" rather
than ""). Modify the function extractConnectionName to implement this parsing
order to avoid picking up dots from the DB path and to preserve common hosts
without dots.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 918a58a0-21dd-4ca6-83f0-aec0a76e4979

📥 Commits

Reviewing files that changed from the base of the PR and between aeebab0 and 9681e27.

📒 Files selected for processing (1)
  • pkg/flags/postgres_benchmarking_test.go

Comment on lines +39 to +49
func extractConnectionName(dsn string) string {
atIdx := strings.Index(dsn, "@")
if atIdx < 0 {
return ""
}
host := dsn[atIdx+1:]
if dotIdx := strings.Index(host, "."); dotIdx > 0 {
return host[:dotIdx]
}
return ""
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Connection-name parsing drops common DSNs and can generate invalid filename prefixes.

extractConnectionName returns "" for hosts without a dot (e.g. localhost), so file reporting is silently skipped. It can also accidentally include :port/ when a dot appears later in the DB path.

Proposed fix
 func extractConnectionName(dsn string) string {
-	atIdx := strings.Index(dsn, "@")
+	atIdx := strings.LastIndex(dsn, "@")
 	if atIdx < 0 {
 		return ""
 	}
-	host := dsn[atIdx+1:]
-	if dotIdx := strings.Index(host, "."); dotIdx > 0 {
-		return host[:dotIdx]
+
+	hostPort := dsn[atIdx+1:]
+	if slashIdx := strings.Index(hostPort, "/"); slashIdx >= 0 {
+		hostPort = hostPort[:slashIdx]
+	}
+
+	host := hostPort
+	if colonIdx := strings.Index(host, ":"); colonIdx >= 0 {
+		host = host[:colonIdx]
+	}
+	if host == "" {
+		return ""
+	}
+
+	if dotIdx := strings.Index(host, "."); dotIdx > 0 {
+		return host[:dotIdx]
 	}
-	return ""
+	return host
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
func extractConnectionName(dsn string) string {
atIdx := strings.Index(dsn, "@")
if atIdx < 0 {
return ""
}
host := dsn[atIdx+1:]
if dotIdx := strings.Index(host, "."); dotIdx > 0 {
return host[:dotIdx]
}
return ""
}
func extractConnectionName(dsn string) string {
atIdx := strings.LastIndex(dsn, "@")
if atIdx < 0 {
return ""
}
hostPort := dsn[atIdx+1:]
if slashIdx := strings.Index(hostPort, "/"); slashIdx >= 0 {
hostPort = hostPort[:slashIdx]
}
host := hostPort
if colonIdx := strings.Index(host, ":"); colonIdx >= 0 {
host = host[:colonIdx]
}
if host == "" {
return ""
}
if dotIdx := strings.Index(host, "."); dotIdx > 0 {
return host[:dotIdx]
}
return host
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/flags/postgres_benchmarking_test.go` around lines 39 - 49, The
extractConnectionName function currently slices the DSN after "@" but can
include path or port and returns empty for hosts without dots; update it to
first isolate the authority portion (stop at the first "/" or "?" after the
"@"), then strip any ":port" if present, and finally return the host portion —
if a dot exists in that host return the substring before the first dot,
otherwise return the full host (so "localhost" yields "localhost" rather than
""). Modify the function extractConnectionName to implement this parsing order
to avoid picking up dots from the DB path and to preserve common hosts without
dots.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (1)
pkg/flags/postgres_benchmarking_test.go (1)

39-49: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Fix DSN host parsing before deriving connName.

Line 44 currently takes everything after @, so host may include :port//db, and hosts like localhost produce empty connName. That causes report writing to be silently skipped and can generate malformed filename prefixes.

Proposed fix
 func extractConnectionName(dsn string) string {
-	atIdx := strings.Index(dsn, "@")
+	atIdx := strings.LastIndex(dsn, "@")
 	if atIdx < 0 {
 		return ""
 	}
-	host := dsn[atIdx+1:]
-	if dotIdx := strings.Index(host, "."); dotIdx > 0 {
-		return host[:dotIdx]
+
+	hostPort := dsn[atIdx+1:]
+	if slashIdx := strings.Index(hostPort, "/"); slashIdx >= 0 {
+		hostPort = hostPort[:slashIdx]
+	}
+	if qIdx := strings.Index(hostPort, "?"); qIdx >= 0 {
+		hostPort = hostPort[:qIdx]
+	}
+
+	host := hostPort
+	if colonIdx := strings.Index(host, ":"); colonIdx >= 0 {
+		host = host[:colonIdx]
+	}
+	if host == "" {
+		return ""
+	}
+	if dotIdx := strings.Index(host, "."); dotIdx > 0 {
+		return host[:dotIdx]
 	}
-	return ""
+	return host
 }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/flags/postgres_benchmarking_test.go` around lines 39 - 49, The
extractConnectionName function incorrectly sets host to everything after "@" so
it may include ":port", "/db", or URL path and returns empty for plain hosts
like "localhost"; update extractConnectionName to first isolate the host portion
by trimming any leading scheme/userinfo if present and then stripping port
(after ':') and any path/query (after '/'), then derive and return the
connection name by taking the substring before the first '.' (use variables
atIdx, host, colonIdx, slashIdx and the function name extractConnectionName to
locate the logic) so report filenames are generated correctly and no writes are
skipped.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Duplicate comments:
In `@pkg/flags/postgres_benchmarking_test.go`:
- Around line 39-49: The extractConnectionName function incorrectly sets host to
everything after "@" so it may include ":port", "/db", or URL path and returns
empty for plain hosts like "localhost"; update extractConnectionName to first
isolate the host portion by trimming any leading scheme/userinfo if present and
then stripping port (after ':') and any path/query (after '/'), then derive and
return the connection name by taking the substring before the first '.' (use
variables atIdx, host, colonIdx, slashIdx and the function name
extractConnectionName to locate the logic) so report filenames are generated
correctly and no writes are skipped.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 64110f1e-09bd-4e10-95ba-5a5781462e10

📥 Commits

Reviewing files that changed from the base of the PR and between 9681e27 and aa8bb16.

📒 Files selected for processing (1)
  • pkg/flags/postgres_benchmarking_test.go

@neisw neisw force-pushed the trt-1989-benchmarking branch from 7752706 to 117d9e3 Compare May 25, 2026 23:07
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (1)
pkg/flags/postgres_benchmarking_test.go (1)

40-50: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Connection-name parsing still drops common DSNs and can generate invalid filename prefixes.

This issue was flagged in a previous review and remains unaddressed. extractConnectionName returns "" for hosts without a dot (e.g., localhost), causing file reporting to be silently skipped. It can also accidentally include :port/ when a dot appears later in the DB path.

Proposed fix from previous review
 func extractConnectionName(dsn string) string {
-	atIdx := strings.Index(dsn, "@")
+	atIdx := strings.LastIndex(dsn, "@")
 	if atIdx < 0 {
 		return ""
 	}
-	host := dsn[atIdx+1:]
-	if dotIdx := strings.Index(host, "."); dotIdx > 0 {
-		return host[:dotIdx]
+
+	hostPort := dsn[atIdx+1:]
+	if slashIdx := strings.Index(hostPort, "/"); slashIdx >= 0 {
+		hostPort = hostPort[:slashIdx]
+	}
+
+	host := hostPort
+	if colonIdx := strings.Index(host, ":"); colonIdx >= 0 {
+		host = host[:colonIdx]
+	}
+	if host == "" {
+		return ""
+	}
+
+	if dotIdx := strings.Index(host, "."); dotIdx > 0 {
+		return host[:dotIdx]
 	}
-	return ""
+	return host
 }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/flags/postgres_benchmarking_test.go` around lines 40 - 50,
extractConnectionName currently slices from the first "@" to the first "." which
drops hosts like "localhost" and can include ":port/" or path segments; update
extractConnectionName to first extract the host portion between the "@" and the
next "/" (or end), strip any trailing ":port" if present, then if the host
contains a dot return the first label before the dot otherwise return the whole
host; additionally sanitize the returned string to be a valid filename prefix
(e.g., replace non-alphanumeric characters with underscores) so callers get a
safe, non-empty connection name from extractConnectionName.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Duplicate comments:
In `@pkg/flags/postgres_benchmarking_test.go`:
- Around line 40-50: extractConnectionName currently slices from the first "@"
to the first "." which drops hosts like "localhost" and can include ":port/" or
path segments; update extractConnectionName to first extract the host portion
between the "@" and the next "/" (or end), strip any trailing ":port" if
present, then if the host contains a dot return the first label before the dot
otherwise return the whole host; additionally sanitize the returned string to be
a valid filename prefix (e.g., replace non-alphanumeric characters with
underscores) so callers get a safe, non-empty connection name from
extractConnectionName.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 4f74fb03-f283-43fb-97cd-f7e8a0063b1a

📥 Commits

Reviewing files that changed from the base of the PR and between 7752706 and 117d9e3.

📒 Files selected for processing (1)
  • pkg/flags/postgres_benchmarking_test.go

@neisw
Copy link
Copy Markdown
Contributor Author

neisw commented May 26, 2026

/hold

@openshift-ci openshift-ci Bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 26, 2026
@neisw neisw force-pushed the trt-1989-benchmarking branch from 117d9e3 to 9b3122d Compare May 27, 2026 19:34
@openshift-merge-bot
Copy link
Copy Markdown
Contributor

Scheduling required tests:
/test e2e

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 27, 2026

@neisw: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant