RUST-117 Add Agent Quality Profile for AI agents#188
RUST-117 Add Agent Quality Profile for AI agents#188
Conversation
hashicorp-vault-sonar-prod
Bot
changed the title
Create a quality profile for AI agents focused on reliability, security, and complexity with high-severity rules only (Major, Critical, Blocker). This profile activates 61 high-severity rules: - Security vulnerabilities and hotspots: 0 rules (Rust's memory safety provides built-in security) - Reliability issues (bugs): 59 rules - Code complexity metrics: 3 rules The Agent Quality Profile helps AI agents maintain code quality by enforcing critical reliability and complexity standards for Rust code. Changes: - Add Agent_quality_profile.json with curated high-severity rules - Add RustAgentQualityProfile.java profile loader - Register profile in RustPlugin.java 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
benzonico
force-pushed
the
feature/agent-quality-profile
branch
from
3abdb97 to
6cd1b34
Compare
Wohops
left a comment
Wohops
left a comment
There was a problem hiding this comment.
Hello Nico,
The build is not passing, your agent didn't modified RustPluginTest.java accordingly, and didn't generate any tests corresponding to the new feature you are adding. According to the javadoc of the RustAgentQualityProfile.java class, I would expect that there is some logic that have been encoded to select rules and I'm surprised not to see related tests. Can you please remediate to this? I would also expect some end-2-end (ITs?) test to validate that the profile can be leveraged correctly and that the rules are triggered.
I also have a question related to the feature itself. The corresponding ticket (RUST-117) is empty (it has been generated when opening the PR), not tied to any Epic or Initiative, not documented and as far as I can tell no communication has been done around this initiative to the RAD squad... So I don't know what to do with it.
So, what are your expectation with this work? Is it a one shot or an experiment? Is it part of a bigger initiative? Do you expect us to ship this with the next SQS release? (code freeze is this Friday for next SQS release)
Cheers,
Michael
| LcovSensor.class, | ||
| RustLanguage.class, | ||
| RustProfile.class, | ||
| RustAgentQualityProfile.class, |
There was a problem hiding this comment.
You are adding a new extension, unit tests of this class should be updated accordingly.
| @@ -0,0 +1,66 @@ | |||
| { | |||
| "name": "Agent Quality Profile", | |||
There was a problem hiding this comment.
These quality profiles are usually populated leveraging metadata from RSPEC and rule-api, so we don't have to maintain them by hand and the knowledge is centralized, and shared within languages.
From your previous commit, I feel that you are generating a list using an agent, and this does not feel sustainable if this logic is not centralized in RSPEC (it's a "one shot").
For instance, for S1656, I would expect to see in its metadata the presence of the new Quality profile. Without it, how to control on the long which new rule would be enabled or not?
| "S7462", | ||
| "S7463", | ||
| "S7464", | ||
| "S905" |
There was a problem hiding this comment.
In the equivalent SonarWay profile, S905 is placed before S1XXX. Rules are ordered by key numbers, can you fix it? It helps review what is "a new rule" from "an old rule". Old rule keys are more likely to be shared with other languages.
| @@ -0,0 +1,66 @@ | |||
| { | |||
| "name": "Agent Quality Profile", | |||
| "ruleKeys": [ | |||
There was a problem hiding this comment.
I didn't review what rules are part of this quality profile and which one are not compared to SonarWay. Should I?
The Agent Quality Profile adds one more extension to the plugin, so the test now expects 19 extensions instead of 18.
|
2 participants
Summary
Create a quality profile for AI agents focused on reliability, security, and complexity with high-severity rules only (Major, Critical, Blocker).
Profile Statistics
Severity Distribution
Why Fewer Security Rules?
Rust's ownership system and borrow checker provide memory safety guarantees at compile time, eliminating entire classes of security vulnerabilities (buffer overflows, use-after-free, data races, etc.).
Changes
Files added/modified:
Agent_quality_profile.json- Curated high-severity rule listRustAgentQualityProfile.java- Profile loader implementationRustPlugin.java- Profile registrationWhy This Matters
AI agents benefit from focused guidance on critical code quality issues. This minimalist profile enforces only the most severe reliability and complexity standards for Rust code.
🤖 Generated with Claude Code
Co-Authored-By: Claude noreply@anthropic.com