Skip to content

Inventory views#2195

Merged
jdobes merged 3 commits intoRedHatInsights:masterfrom
MichaelMraka:pr1
Feb 12, 2026
Merged

Inventory views#2195
jdobes merged 3 commits intoRedHatInsights:masterfrom
MichaelMraka:pr1

Conversation

@MichaelMraka
Copy link
Copy Markdown
Contributor

@MichaelMraka MichaelMraka commented Feb 2, 2026

Secure Coding Practices Checklist GitHub Link

Secure Coding Checklist

  • Input Validation
  • Output Encoding
  • Authentication and Password Management
  • Session Management
  • Access Control
  • Cryptographic Practices
  • Error Handling and Logging
  • Data Protection
  • Communication Security
  • System Configuration
  • Database Security
  • File Management
  • Memory Management
  • General Coding Practices

Summary by Sourcery

Publish host vulnerability evaluation summaries to the inventory views Kafka topic and enrich CVE caching and messaging to support severity and exploitability metadata.

New Features:

  • Emit per-host CVE summary messages to the platform.inventory.host-apps topic for inventory views consumption.

Enhancements:

  • Extend evaluator CVE cache to include impact and exploitability metadata for richer evaluation statistics.
  • Propagate Kafka headers (including request IDs) through the MQWriter so downstream topics receive contextual metadata.
  • Wire a new inventory views MQ writer and topic configuration into the evaluator service deployment and configuration.

@jira-linking
Copy link
Copy Markdown

jira-linking Bot commented Feb 2, 2026

Referenced Jiras:
https://issues.redhat.com/browse/RHINENG-23481

@sourcery-ai
Copy link
Copy Markdown

sourcery-ai Bot commented Feb 2, 2026

Reviewer's Guide

Adds per-host vulnerability evaluation summaries enriched with CVE impact and exploitability to a new Kafka topic for inventory views, and extends messaging/configuration to support headers and the new topic across evaluator, common utilities, and deployment artifacts.

Sequence diagram for sending inventory host vulnerability summary to Kafka

sequenceDiagram
    participant Evaluator
    participant EvaluatorProcessor
    participant EvaluatorLogic
    participant CveCache
    participant MQWriter_inventoryViews
    participant Kafka_inventoryViews

    Evaluator->>EvaluatorProcessor: evaluate_system(msg, inventory_id, org_id, request_id, request_timestamp)
    EvaluatorProcessor->>EvaluatorProcessor: _evaluate_system(inventory_id, org_id, request_id, request_timestamp, recalc_event_id)

    loop over system_vulnerabilities
        EvaluatorProcessor->>EvaluatorLogic: evaluator_logic.cve_cache.get(cve)
        EvaluatorLogic->>CveCache: lookup(cve)
        CveCache-->>EvaluatorLogic: CveCache(id, impact_id, exploitable)
        EvaluatorLogic-->>EvaluatorProcessor: CveCache
        EvaluatorProcessor->>EvaluatorProcessor: update total_cves, critical_cves, high_severity_cves, cves_with_security_rules, cves_with_known_exploits
    end

    EvaluatorProcessor->>MQWriter_inventoryViews: send_one(summary_msg, key=None, headers=[application=vulnerability, request_id])
    MQWriter_inventoryViews->>Kafka_inventoryViews: send_and_wait(topic=platform.inventory.host-apps, value=summary_msg, headers)
    Kafka_inventoryViews-->>MQWriter_inventoryViews: ack
    MQWriter_inventoryViews-->>EvaluatorProcessor: result
    EvaluatorProcessor-->>Evaluator: evaluation complete
Loading

ER diagram for updated cve_metadata usage in CVE cache

erDiagram
    cve_metadata {
        bigint id PK
        varchar cve
        smallint impact_id
        jsonb exploit_data
    }

    cve_metadata ||--o{ cve_cache_entry : populates

    cve_cache_entry {
        bigint id
        smallint impact_id
        boolean exploitable
    }
Loading

Updated class diagram for evaluator messaging and CVE cache

classDiagram
    class MQWriter {
        -topic
        -client
        +__init__(topic, bootstrap_servers, loop, kwargs)
        +send_one(msg, key, headers)
        +send_many(msg_list, key, headers)
        +send_raw(msg, key, headers)
        -_serialize_key(key)
    }

    class CveCache {
        +id
        +impact_id
        +exploitable
    }

    class EvaluatorLogic {
        -db_pool
        +cve_cache: Dict~str, CveCache~
        +_load_cve_cache() Dict~str, CveCache~
    }

    class Evaluator {
        -loop: BaseEventLoop
        -db_pool: AsyncConnectionPool
        +payload_tracker: MQWriter
        +remediations_results: MQWriter
        +evaluator_results: MQWriter
        +inventory_views_results: MQWriter
        +processor: EvaluatorProcessor
        +__init__(db_pool, loop)
    }

    class EvaluatorProcessor {
        +remediations_results: MQWriter
        +evaluator_results: MQWriter
        +inventory_views_results: MQWriter
        +payload_tracker: MQWriter
        +db_pool: AsyncConnectionPool
        +evaluator_logic: EvaluatorLogic
        +recent_recalc_events: Dict~int, RecalcEvent~
        +__init__(remediations_results, evaluator_results, inventory_views_results, payload_tracker, db_pool, loop)
        +evaluate_system(msg, inventory_id, org_id, request_id, request_timestamp, recalc_event_id)
        -_evaluate_system(inventory_id, org_id, request_id, request_timestamp, recalc_event_id)
    }

    EvaluatorLogic --> CveCache : uses
    Evaluator --> EvaluatorProcessor : composes
    Evaluator --> MQWriter : creates
    EvaluatorProcessor --> MQWriter : sends messages via
    EvaluatorProcessor --> EvaluatorLogic : uses CVE cache
Loading

File-Level Changes

Change Details Files
Emit per-host vulnerability evaluation summaries to the inventory views Kafka topic with CVE and exploitability metrics.
  • Introduce send_inventory_views helper to build per-host summary payload including total, critical, high, rule-backed, and exploitable CVE counts and to send it with Kafka headers
  • Guard sending by requiring org_id and log attempts with unknown org_id
  • Include request_id and application headers on the emitted Kafka message
common/utils.py
Extend evaluator processing to compute per-system vulnerability metrics using enriched CVE cache and publish to the new inventory views topic.
  • Update EvaluatorProcessor constructor to accept an inventory_views_results MQWriter and store it
  • Augment _evaluate_system to accept request_id, derive fixable vulnerabilities, and compute total, critical, high severity, rule-backed, and exploitable CVE counts from sys_vuln_rows and CVE cache
  • Change _mark_system_evaluated call to use total CVE count instead of only fixable CVEs
  • Send inventory views summary message alongside existing remediation and notification messages
  • Plumb request_id through evaluate_system into _evaluate_system
evaluator/processor.py
Allow MQWriter to attach Kafka headers to sent messages.
  • Extend send_one, send_many, and send_raw to accept an optional headers parameter
  • Pass headers through to AIOKafkaProducer.send_and_wait calls for single, batched, and raw messages
common/mqueue.py
Wire evaluator to use the new inventory views topic and enriched CVE metadata cache.
  • Instantiate an MQWriter for the inventory views topic in Evaluator and pass it into EvaluatorProcessor
  • Expand CVE cache loading query to include impact_id and a derived exploitable flag from exploit_data presence
  • Adjust CveCache namedtuple to carry impact_id and exploitable alongside id
evaluator/evaluator.py
evaluator/logic.py
evaluator/common.py
Add configuration and deployment for the new inventory host-apps Kafka topic, including local platform mock setup.
  • Register platform.inventory.host-apps in ClowdApp KafkaTopics with partitions/replicas
  • Expose inventory_views_topic in configuration for both Clowder and legacy env var modes
  • Ensure platform_mock creates the platform.inventory.host-apps topic at startup
deploy/clowdapp.yaml
common/config.py
platform_mock/entrypoint.sh
conf/evaluator.env

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@MichaelMraka MichaelMraka force-pushed the pr1 branch 2 times, most recently from dc684e5 to d2ed2b1 Compare February 2, 2026 10:51
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Feb 2, 2026

Codecov Report

❌ Patch coverage is 9.30233% with 39 lines in your changes missing coverage. Please review.
✅ Project coverage is 62.41%. Comparing base (da22b40) to head (9ae16e3).
⚠️ Report is 72 commits behind head on master.

Files with missing lines Patch % Lines
evaluator/processor.py 0.00% 24 Missing ⚠️
common/utils.py 12.50% 7 Missing ⚠️
common/mqueue.py 50.00% 3 Missing ⚠️
evaluator/evaluator.py 0.00% 2 Missing ⚠️
evaluator/logic.py 0.00% 2 Missing ⚠️
evaluator/common.py 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2195      +/-   ##
==========================================
- Coverage   65.55%   62.41%   -3.15%     
==========================================
  Files          56       58       +2     
  Lines        6640     6994     +354     
==========================================
+ Hits         4353     4365      +12     
- Misses       2287     2629     +342     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@MichaelMraka MichaelMraka force-pushed the pr1 branch 2 times, most recently from 05fb781 to 9ae16e3 Compare February 2, 2026 11:49
@MichaelMraka MichaelMraka marked this pull request as ready for review February 2, 2026 12:44
Copy link
Copy Markdown

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 3 issues, and left some high level feedback:

  • In _evaluate_system, cve = await self.evaluator_logic.cve_cache.get(cve) is awaiting a dict lookup and overwriting the cve key variable; cve_cache entries are loaded synchronously, so this should be a plain lookup (e.g. cve_data = self.evaluator_logic.cve_cache[cve]) using a different variable name.
  • In send_inventory_views, the log messages still refer to 'notification' / 'notificator' even though this function publishes to inventory views; consider updating the log text to reflect the new target to avoid confusion when debugging.
  • send_inventory_views assumes request_id is always a non-empty ASCII string (bytes(request_id, 'ascii')); if there is any chance it can be None or contain non-ASCII characters, consider guarding or normalizing it before constructing the header.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- In `_evaluate_system`, `cve = await self.evaluator_logic.cve_cache.get(cve)` is awaiting a dict lookup and overwriting the `cve` key variable; `cve_cache` entries are loaded synchronously, so this should be a plain lookup (e.g. `cve_data = self.evaluator_logic.cve_cache[cve]`) using a different variable name.
- In `send_inventory_views`, the log messages still refer to 'notification' / 'notificator' even though this function publishes to inventory views; consider updating the log text to reflect the new target to avoid confusion when debugging.
- `send_inventory_views` assumes `request_id` is always a non-empty ASCII string (`bytes(request_id, 'ascii')`); if there is any chance it can be `None` or contain non-ASCII characters, consider guarding or normalizing it before constructing the header.

## Individual Comments

### Comment 1
<location> `evaluator/processor.py:363` </location>
<code_context>
+                        total_cves += 1
+                        if row.rule_id is not None:
+                            cves_with_security_rules += 1
+                        cve = await self.evaluator_logic.cve_cache.get(cve)
+                        if cve.impact_id == 7:
+                            critical_cves += 1
</code_context>

<issue_to_address>
**issue (bug_risk):** Using `await` on `cve_cache.get` is likely incorrect and will fail if `cve_cache` is a plain dict.

`_load_cve_cache` builds `cve_cache` as a dict of `CveCache` instances, so `.get` is a normal dict lookup, not async. `await` on this will raise `TypeError` at runtime. This should be a plain lookup, e.g. `cve = self.evaluator_logic.cve_cache.get(cve)`, with appropriate handling for a missing key.
</issue_to_address>

### Comment 2
<location> `common/utils.py:372-374` </location>
<code_context>
+    cves_with_known_exploits,
+):
+    """Sends kafka message to inventory views"""
+    if not org_id:
+        LOGGER.info("Attempt to send notification to unknown org_id: %s", rh_account_id)
+        return
+
</code_context>

<issue_to_address>
**suggestion:** Log message for missing `org_id` logs `rh_account_id`, which is potentially misleading.

Since this branch handles a missing `org_id`, the log should highlight `org_id` as the unknown value, and optionally include `rh_account_id` as context. For example: `"Attempt to send inventory views message with unknown org_id: %s (account: %s)"`.

```suggestion
    if not org_id:
        LOGGER.info(
            "Attempt to send inventory views message with unknown org_id: %s (account: %s)",
            org_id,
            rh_account_id,
        )
        return
```
</issue_to_address>

### Comment 3
<location> `evaluator/processor.py:370` </location>
<code_context>
+                            high_severity_cves += 1
+                        if cve.exploitable:
+                            cves_with_known_exploits += 1
+                await self._mark_system_evaluated(total_cves, system_platform, conn)

                 send_remediations_update(self.remediations_results, inventory_id, fixable_sys_vuln_rows)
</code_context>

<issue_to_address>
**nitpick:** `total_cves` only counts fixable CVEs, which may be confusing given the name and usage.

Given it’s only incremented when `row.remediation_type_id > 1`, this value is effectively `fixable_cves_count` and matches `len(fixable_sys_vuln_rows)`. Renaming to something like `fixable_cves` (and/or directly using `len(fixable_sys_vuln_rows)`) would better reflect its meaning and reduce confusion in metrics/logs.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread evaluator/processor.py Outdated
Comment thread common/utils.py
Comment thread evaluator/processor.py
@MichaelMraka MichaelMraka changed the title Pr1 Inventory views Feb 4, 2026
@jdobes jdobes merged commit 6df823c into RedHatInsights:master Feb 12, 2026
8 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants