feat(scorecard): Implement exclude metrics annotation

[alizard0]

User description

Hey, I just made a Pull Request!

It adds an annotation that allows the developer to exclude any scorecard metric.
More details: https://issues.redhat.com/browse/RHIDP-12183

apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
  name: openssf-scorecard-only
  annotations:
    exclude/metrics: 'openssf.maintained,openssf.code_review'
spec:
  type: service
  owner: guests
  lifecycle: experimental 

✔️ Checklist

• A changeset describing the change and affected packages. (more info)
• Added or Updated documentation
• Tests for new functionality and regression tests for bug fixes
• Screenshots attached (for UI changes)

---

PR Type

Enhancement

---

Description

• Add support for excluding OpenSSF checks via annotation

• Implement check filtering in OpenSSFClient based on exclude-checks annotation

• Inject LoggerService dependency into client and metric provider classes

• Update module initialization to pass logger to metric provider factory

• Enhance test coverage with exclude checks functionality

---

Diagram Walkthrough

flowchart LR
  A["Entity with<br/>exclude-checks annotation"] -->|"scorecard-location"| B["OpenSSFClient"]
  B -->|"filters checks"| C["OpenSSFResponse"]
  C -->|"excluded checks removed"| D["Filtered scorecard data"]
  E["LoggerService"] -->|"injected"| B
  E -->|"injected"| F["OpenSSFMetricProvider"]
  F -->|"uses"| B

Loading

File Walkthrough

	Relevant files
Enhancement	OpenSSFClient.ts Add check exclusion filtering with logging                              workspaces/scorecard/plugins/scorecard-backend-module-openssf/src/clients/OpenSSFClient.ts Add LoggerService dependency injection via constructor Implement check filtering logic based on openssf/exclude-checks annotation Filter out excluded checks from the OpenSSF API response before returning Add debug logging when checks are excluded +15/-0    OpenSSFMetricProvider.ts Inject logger into metric provider                                              workspaces/scorecard/plugins/scorecard-backend-module-openssf/src/metricProviders/OpenSSFMetricProvider.ts Add LoggerService parameter to constructor Pass logger to OpenSSFClient initialization Update createOpenSSFMetricProvider factory function to accept and pass logger Add JSDoc comment documenting logger parameter +8/-3
Tests	OpenSSFClient.test.ts Add tests for check exclusion functionality                            workspaces/scorecard/plugins/scorecard-backend-module-openssf/src/clients/OpenSSFClient.test.ts Add excludeChecks parameter to createEntity test helper function Create mock logger object for testing Add test case for excluding Maintained check via annotation Update existing test to pass logger to OpenSSFClient constructor Expand mock OpenSSF response with additional Code-Review check +37/-2    OpenSSFMetricProvider.test.ts Update tests with logger dependency                                            workspaces/scorecard/plugins/scorecard-backend-module-openssf/src/metricProviders/OpenSSFMetricProvider.test.ts Create mock logger object for all test cases Update all OpenSSFMetricProvider instantiations to pass mockLogger Ensure consistent logger injection across all test scenarios +19/-0
Configuration changes	module.ts Wire logger dependency in module initialization                    workspaces/scorecard/plugins/scorecard-backend-module-openssf/src/module.ts Import coreServices from @backstage/backend-plugin-api Add logger to module dependencies Pass logger to createOpenSSFMetricProvider factory function Update init function to receive and use logger parameter +7/-3
Documentation	openssf-scorecard-only.yaml Add exclude-checks annotation example                                        workspaces/scorecard/examples/openssf-scorecard-only.yaml Rename component from openssf-scorecard-only to openssf-scorecard Add openssf/exclude-checks annotation with example values Demonstrate exclusion of Maintained and Code-Review checks +2/-1

---

[rhdh-gh-app]

Missing Changesets

The following package(s) are changed by this PR but do not have a changeset:

• @red-hat-developer-hub/backstage-plugin-scorecard-backend-module-openssf
• @red-hat-developer-hub/backstage-plugin-scorecard-backend

See CONTRIBUTING.md for more information about how to add changesets.

Changed Packages

Package Name	Package Path	Changeset Bump	Current Version
@red-hat-developer-hub/backstage-plugin-scorecard-backend-module-openssf	workspaces/scorecard/plugins/scorecard-backend-module-openssf	none	v0.1.6
@red-hat-developer-hub/backstage-plugin-scorecard-backend	workspaces/scorecard/plugins/scorecard-backend	none	v2.4.0

[rhdh-qodo-merge]

CI Feedback 🧐

A test triggered by this PR failed. Here is an AI-generated analysis of the failure:

Action: check all required jobs

Failed stage: Run exit 1 [❌]

Failed test name: ""

Failure summary: The action failed because the workflow explicitly ran exit 1 in a bash step (shell: /usr/bin/bash -e), which forces the job to terminate with a non-zero status. - The failure occurs at the step shown as Run exit 1 (log lines 41-45), resulting in Process completed with exit code 1.

Relevant error logs: 1: ##[group]Runner Image Provisioner 2: Hosted Compute Agent ... 30: Packages: write 31: Pages: write 32: PullRequests: write 33: RepositoryProjects: write 34: SecurityEvents: write 35: Statuses: write 36: ##[endgroup] 37: Secret source: Actions 38: Prepare workflow directory 39: Prepare all required actions 40: Complete job name: check all required jobs 41: ##[group]Run exit 1 42: �[36;1mexit 1�[0m 43: shell: /usr/bin/bash -e {0} 44: ##[endgroup] 45: ##[error]Process completed with exit code 1. 46: Cleaning up orphan processes

[rhdh-qodo-merge]

PR Compliance Guide 🔍

Below is a summary of compliance checks for this PR:

Security Compliance
🟢	No security concerns identified No security vulnerabilities detected by AI analysis. Human verification advised for critical code.
Ticket Compliance
⚪	🎫 No ticket provided Create ticket/issue
Codebase Duplication Compliance
🟢	No codebase code duplication found New Components Detected: - it: excludes Maintained when excludeChecks annotation is present
Custom Compliance
🟢	Generic: Comprehensive Audit Trails Objective: To create a detailed and reliable record of critical system actions for security analysis and compliance. Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Meaningful Naming and Self-Documenting Code Objective: Ensure all identifiers clearly express their purpose and intent, making code self-documenting Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Secure Error Handling Objective: To prevent the leakage of sensitive system information through error messages while providing sufficient detail for internal debugging. Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Secure Logging Practices Objective: To ensure logs are useful for debugging and auditing without exposing sensitive information like PII, PHI, or cardholder data. Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
🔴	Generic: Robust Error Handling and Edge Case Management Objective: Ensure comprehensive error handling that provides meaningful context and graceful degradation Status: Missing input handling: The code assumes entity.metadata.annotations['openssf/exclude-checks'] is an array and uses .length/.includes without validating/parsing, which can break when annotations are provided as strings. Referred Code const excludeChecks = entity.metadata.annotations?.['openssf/exclude-checks']; if (excludeChecks && excludeChecks.length > 0) { this.logger.debug( `Excluding checks: ${excludeChecks} for entity ${entity.metadata.name}`, ); data.checks = data.checks.filter( check => !excludeChecks.includes(check.name), ); } Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Security-First Input Validation and Data Handling Objective: Ensure all data inputs are validated, sanitized, and handled securely to prevent vulnerabilities Status: Unvalidated annotation format: The new openssf/exclude-checks annotation is shown as a comma-separated string (Maintained,Code-Review) while the implementation expects an array, indicating missing validation/parsing of external input. Referred Code name: openssf-scorecard annotations: openssf/scorecard-location: https://api.securityscorecards.dev/projects/github.com/alizard0/rhdh-plugins openssf/exclude-checks: Maintained,Code-Review spec: Learn more about managing compliance generic rules or creating your own custom rules
Update

Compliance status legend

🟢 - Fully Compliant
🟡 - Partial Compliant
🔴 - Not Compliant
⚪ - Requires Further Human Verification
🏷️ - Compliance label

[rhdh-qodo-merge]

PR Code Suggestions ✨

Latest suggestions up to 84b2172

Category	Suggestion	Impact
Possible issue	✅ Return filtered copy safely Suggestion Impact: The commit removed the entire exclude-checks filtering logic (including the in-place mutation of data.checks) and now returns response.json() directly, eliminating the mutation concern but not implementing the suggested safe filtered-copy behavior. code diff: - const data: OpenSSFResponse = await response.json(); - - const excludeChecksRaw = - entity.metadata.annotations?.['openssf/exclude-checks']; - const excludeChecks = excludeChecksRaw - ? excludeChecksRaw - .split(',') - .map(s => s.trim()) - .filter(Boolean) - : []; - - if (excludeChecks.length > 0) { - this.logger.debug( - `Excluding checks: ${excludeChecks.join(', ')} for entity ${ - entity.metadata.name - }`, - ); - data.checks = data.checks.filter( - check => !excludeChecks.includes(check.name), - ); - } - - return data; + return await response.json(); } Refactor the code to avoid in-place mutation of the API response. Instead, create a new object with the filtered checks and defensively handle cases where data.checks might not be an array. workspaces/scorecard/plugins/scorecard-backend-module-openssf/src/clients/OpenSSFClient.ts [49-71] const data: OpenSSFResponse = await response.json(); const excludeChecksRaw = entity.metadata.annotations?.['openssf/exclude-checks']; const excludeChecks = excludeChecksRaw ? excludeChecksRaw .split(',') .map(s => s.trim()) .filter(Boolean) : []; +const checks = Array.isArray((data as any).checks) ? data.checks : []; + if (excludeChecks.length > 0) { this.logger.debug( `Excluding checks: ${excludeChecks.join(', ')} for entity ${ entity.metadata.name }`, ); - data.checks = data.checks.filter( + + const filteredChecks = checks.filter( check => !excludeChecks.includes(check.name), ); + + return { ...data, checks: filteredChecks }; } -return data; +return { ...data, checks }; [Suggestion processed] Suggestion importance[1-10]: 7 __ Why: This suggestion improves code robustness by defensively checking if data.checks is an array and avoids mutating the original data object, which are good practices for handling external API responses.	Medium
	✅ Add safe default logger Suggestion Impact: The committed change removed the LoggerService dependency and the logger constructor parameter entirely, eliminating the need to pass a logger (an alternative way to restore backward compatibility), instead of adding a default no-op logger. code diff: -import type { LoggerService } from '@backstage/backend-plugin-api'; import type { Entity } from '@backstage/catalog-model'; import { OpenSSFResponse } from './types'; export class OpenSSFClient { - constructor(private readonly logger: LoggerService) {} - Make the logger parameter in the OpenSSFClient constructor optional by providing a default no-op logger implementation. This ensures backward compatibility for any existing code that instantiates the client without a logger. workspaces/scorecard/plugins/scorecard-backend-module-openssf/src/clients/OpenSSFClient.ts [22-23] export class OpenSSFClient { - constructor(private readonly logger: LoggerService) {} + constructor( + private readonly logger: LoggerService = { + child: () => this.logger, + debug: () => {}, + info: () => {}, + warn: () => {}, + error: () => {}, + } as unknown as LoggerService, + ) {} [Suggestion processed] Suggestion importance[1-10]: 6 __ Why: The suggestion correctly identifies that changing the OpenSSFClient constructor is a breaking change and proposes a good solution to maintain backward compatibility, which is a valuable improvement for library code.	Low
Update

---

Previous suggestions

✅ Suggestions up to commit aea0d64

Category	Suggestion	Impact
Possible issue	✅ Fix incorrect partial string matching Suggestion Impact: The commit now parses the openssf/exclude-checks annotation by splitting on commas, trimming entries (and filtering empty ones), logs the joined list, and uses array .includes(check.name) to perform exact match filtering instead of partial string matching. code diff: - const excludeChecks = + const excludeChecksRaw = entity.metadata.annotations?.['openssf/exclude-checks']; + const excludeChecks = excludeChecksRaw + ? excludeChecksRaw + .split(',') + .map(s => s.trim()) + .filter(Boolean) + : []; - if (excludeChecks && excludeChecks.length > 0) { + if (excludeChecks.length > 0) { this.logger.debug( - `Excluding checks: ${excludeChecks} for entity ${entity.metadata.name}`, + `Excluding checks: ${excludeChecks.join(', ')} for entity ${ + entity.metadata.name + }`, ); data.checks = data.checks.filter( check => !excludeChecks.includes(check.name), To prevent incorrect partial matches when filtering, split the comma-separated openssf/exclude-checks annotation string into an array and use it for exact matching. workspaces/scorecard/plugins/scorecard-backend-module-openssf/src/clients/OpenSSFClient.ts [51-61] -const excludeChecks = +const excludeChecksAnnotation = entity.metadata.annotations?.['openssf/exclude-checks']; -if (excludeChecks && excludeChecks.length > 0) { +if (typeof excludeChecksAnnotation === 'string' && excludeChecksAnnotation.length > 0) { + const checksToExclude = excludeChecksAnnotation.split(',').map(s => s.trim()); this.logger.debug( - `Excluding checks: ${excludeChecks} for entity ${entity.metadata.name}`, + `Excluding checks: ${checksToExclude.join(', ')} for entity ${entity.metadata.name}`, ); data.checks = data.checks.filter( - check => !excludeChecks.includes(check.name), + check => !checksToExclude.includes(check.name), ); } [Suggestion processed] Suggestion importance[1-10]: 9 __ Why: The suggestion correctly identifies a critical bug in the filtering logic that would lead to incorrect behavior due to partial string matching, and the proposed fix is accurate.	High
General	✅ Align test helper with annotation type Suggestion Impact: The commit changed createEntity's excludeChecks parameter type from list to string and updated tests to pass a string (including a new test for comma-separated values). It also conditionally adds the annotation only when excludeChecks is provided. code diff: function createEntity( scorecardLocation: string, - excludeChecks?: list<string>, + excludeChecks?: string, ): Entity { return { apiVersion: 'backstage.io/v1beta1', @@ -33,7 +33,9 @@ name: 'my-service', annotations: { 'openssf/scorecard-location': scorecardLocation, - 'openssf/exclude-checks': excludeChecks, + ...(excludeChecks !== undefined && { + 'openssf/exclude-checks': excludeChecks, + }), }, }, spec: {}, @@ -103,7 +105,7 @@ statusText: 'Not Found', }); - const client = new OpenSSFClient(); + const client = new OpenSSFClient(mockLogger as any); await expect(client.getScorecard(entity)).rejects.toThrow( 'OpenSSF API request failed with status 404: Not Found', @@ -115,17 +117,18 @@ new Error('Network error'), ); - const client = new OpenSSFClient(); + const client = new OpenSSFClient(mockLogger as any); await expect(client.getScorecard(entity)).rejects.toThrow( 'Network error', ); }); - it('excludes Maintained when excludeChecks annotation is present', async () => { - const entityWithExcludeChecks = createEntity(mockScorecardLocation, [ + it('excludes Maintained when exclude-checks annotation is present', async () => { + const entityWithExcludeChecks = createEntity( + mockScorecardLocation, 'Maintained', - ]); + ); (globalThis.fetch as jest.Mock).mockResolvedValue({ ok: true, json: jest.fn().mockResolvedValue(mockOpenSSFResponse), @@ -137,6 +140,22 @@ expect(result.checks).toHaveLength(1); expect(result.checks[0].name).toBe('Code-Review'); }); + + it('excludes multiple checks when exclude-checks is comma-separated', async () => { + const entityWithExcludeChecks = createEntity( + mockScorecardLocation, + 'Maintained, Code-Review', // added space to test out the trim() + ); + (globalThis.fetch as jest.Mock).mockResolvedValue({ + ok: true, + json: jest.fn().mockResolvedValue(mockOpenSSFResponse), + }); + + const client = new OpenSSFClient(mockLogger as any); + const result = await client.getScorecard(entityWithExcludeChecks); + + expect(result.checks).toHaveLength(0); + }); Update the createEntity test helper to accept excludeChecks as a string instead of list to accurately reflect the real-world annotation format. workspaces/scorecard/plugins/scorecard-backend-module-openssf/src/clients/OpenSSFClient.test.ts [25-41] function createEntity( scorecardLocation: string, - excludeChecks?: list<string>, + excludeChecks?: string, ): Entity { return { apiVersion: 'backstage.io/v1beta1', kind: 'Component', metadata: { name: 'my-service', annotations: { 'openssf/scorecard-location': scorecardLocation, 'openssf/exclude-checks': excludeChecks, }, }, spec: {}, } as Entity; } [Suggestion processed] Suggestion importance[1-10]: 7 __ Why: The suggestion correctly points out that the test helper's signature is incorrect and doesn't reflect the actual annotation type, improving test quality and realism.	Medium

[dzemanov]

@alizard0 I see scorecards not being excluded:

With error: openssf check 'Maintained' not found in scorecard

[dzemanov]

@alizard0 to make this work, I think it makes sense to implement this for all metrics. You can update code in scorecard-backend PullMetricsByProviderTask to not call calculate metric if check is disabled. It would be also great if you used metricId for exclude checks definition as it is a unique identifier across all metrics. Although I understand why you used Name for openssf.

It would be nice to allow administrators to be able to define exclude checks also in app-config. Make sure to add option to disable exclude checks via annotations if administrators want to, since we also have this feature requested: https://issues.redhat.com/browse/RHDHPLAN-901

cc @christoph-jerolimov, @PatAKnight

[alizard0]

Added excluded/metrics annotation
PullMetricsByProviderTask can now exclude metrics using the annotation excluded/metrics

example component yaml

---
apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
  name: openssf-scorecard-only
  annotations:
    openssf/scorecard-location: https://api.securityscorecards.dev/projects/github.com/alizard0/rhdh-plugins
    exclude/metrics: openssf.maintained,openssf.code_review
spec:
  type: service
  owner: group:development/guests
  lifecycle: experimental

localhost logs

2026-03-02T14:07:14.113Z scorecard info Loaded excluded/metrics annotation: openssf.maintained,openssf.code_review class="PullMetricsByProviderTask" taskId="openssf.maintained" taskInstanceId="4223f4fa-090f-4b7c-b9e2-dfb5ccd9f544"
2026-03-02T14:07:14.114Z scorecard info Excluded metric: openssf.maintained class="PullMetricsByProviderTask" taskId="openssf.maintained" taskInstanceId="4223f4fa-090f-4b7c-b9e2-dfb5ccd9f544"

[dzemanov]

Hi @alizard0,
I have added some comments.
Thinking about our scorecard work yesterday, and how users will eventually create their own via UI or YAML, I am wondering if the metric exclusion feature might become unnecessary then. However, the timeline for scorecard creation hasn't been defined yet, so it could make sense to implement this.

cc @PatAKnight, @christoph-jerolimov

[alizard0]

Added exclude/include metrics by providerId in app-config.yaml.

component yaml

...
metadata:
  name: openssf-scorecard-only
  annotations:
    openssf/scorecard-location: https://api.securityscorecards.dev/projects/github.com/alizard0/rhdh-plugins
    scorecard.io/exclude_metrics: openssf.maintained,openssf.packaging
...

app-config.yaml

scorecard:
  plugins:
    openssf:
      enabled: true
    jira:
      enabled: false
    github:
      enabled: false
    dependabot:
      enabled: true
  exclude_metrics:
    - openssf.code_review
  include_metrics:
    - openssf.packaging

execution logs

2026-03-04T10:01:06.062Z scorecard info Excluded metric by app-config: openssf.code_review class="PullMetricsByProviderTask" taskId="openssf.code_review" taskInstanceId="5e1fcb1e-a195-4abc-928e-5dababeff672"
...
2026-03-04T10:01:06.076Z scorecard info Exclusion override: metric excluded by annotation but INCLUDED by app-config: openssf.packaging class="PullMetricsByProviderTask" taskId="openssf.packaging" taskInstanceId="2c66ea8a-e289-4561-b4fd-06fed96da81d"
...

Result code_review and maintained excluded, packaging forced included

With no exclude/include rules

[sonarqubecloud]

Quality Gate passed

Issues
2 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[dzemanov]

@alizard0, can you please fix CI?
You will need to update scorecard backend config.d.ts with the new schema. Also, I am not sure if this is working, but I haven't looked into it closely:

 plugins:
    openssf:
      enabled: true

I was thinking about what the best syntax for this feature would be. For annotations, usually - is used as a separator. In app-config, we usually use lower camel case.

For app-config.yaml, not sure if key include_metrics clearly shows that it actually means users can not override excluding metrics by entity annotations.
I suggest we can use something like this, so it can be consistent once we add disabling thresholds overrides:

scorecard:
  entityOverrides:
    thresholds:
      enabled: true
      except:
        - openssf.packaging
    disabledMetrics:
      enabled: false
      except:
        - openssf.packaging

Then, for syntax for disabling some metric runs, I was thinking disable might be clearer, since exclude might mean check results are just filtered out instead of not run at all. It would also mean admins can define it this way:

scorecard:
  disabledMetrics:
    - openssf.packaging

Annotation will then be:

scorecard.io/disabled-metrics: openssf.maintained,openssf.code_review

This is of course up for debate. @imykhno, @PatAKnight WDYT?