Summary
The security risk score for a skill is computed entirely from hardcoded package-level rules in forge-skills/analyzer/scoring.go. Skill authors and operators have no supported way to override the score, configure the scoring tables, or load a custom security policy when running forge skills audit.
This is limiting for custom skills that legitimately need a high-risk binary, a non-builtin egress domain, or a script, but are reviewed/approved at the operator level and should not be flagged repeatedly by the default policy.
Current behavior
AnalyzeSkillEntry (forge-skills/analyzer/scoring.go:70) sums RiskFactor points from four categories using package-level constants:
| Category |
Rule |
Source |
egress |
+2 per trusted domain, +10 per unknown, +15 if >5 domains |
scoring.go:116-152 |
binary |
+15 per high-risk binary, +3 per standard |
scoring.go:154-172 |
env |
+10 per sensitive env name, +5 otherwise |
scoring.go:174-197 |
script |
+20 if skill has a script |
scoring.go:199-205 |
The trustedDomains, highRiskBinaries, and sensitiveEnvPatterns lists (scoring.go:11-47) are unexported package vars — not reachable from SKILL.md, forge.yaml, or any CLI flag.
Gaps in the existing policy plumbing
The SecurityPolicy struct (forge-skills/analyzer/types.go:63-71) already has fields that look like they should influence scoring, but the wiring is incomplete:
SecurityPolicy.TrustedDomains exists, but scoreEgress is always invoked with extraTrusted = nil (scoring.go:53, scoring.go:100). The policy's trusted list never reaches the scorer.
SecurityPolicy.MaxRiskScore is checked only inside CheckPolicy to emit a max_risk_score violation (policy.go:99-109); it doesn't influence the score value itself.
- The
forge skills audit command hardcodes analyzer.DefaultPolicy() (forge-cli/cmd/skills.go:377) with no flag to load an alternative policy.
So even an operator who builds a SecurityPolicy today cannot make scoring honor it through the CLI.
Why a static list in analyzer/scoring.go is not the fix
Adding a new domain to trustedDomains, a new env pattern to sensitiveEnvPatterns, or a new binary to highRiskBinaries requires a code change to forge-skills/analyzer/scoring.go every time a custom skill needs different scoring treatment. This does not scale and is the same anti-pattern called out in #48 for runner.go's hardcoded knownKeys.
The solution must be dynamic: a custom skill author or operator must be able to influence scoring without editing any Go file in the forge repo.
Proposed options (any one or combination)
- Honor
SecurityPolicy.TrustedDomains in scoring. Thread the policy through AnalyzeSkillEntry / AnalyzeSkillDescriptor so scoreEgress receives policy.TrustedDomains as extraTrusted. This closes the existing gap without any new schema.
--policy <path> flag on forge skills audit. Load a SecurityPolicy YAML, replacing DefaultPolicy() at forge-cli/cmd/skills.go:377. Combined with (1), this lets operators expand trusted domains, binary lists, and sensitive-env patterns per-project.
forge.security block in SKILL.md frontmatter. Let a skill declare its own scoring inputs:
trusted_domains: [<extra domains specific to this skill>]
acknowledged_high_risk_bins: [<list>] with a justification field — does not silence the factor, but caps its points or marks it as accepted.
- Optionally a
score_override with a max_value and required justification so the override is auditable.
forge.yaml project-level policy block. A security_policy: section in forge.yaml that loads the same SecurityPolicy shape, applied to all skills in the project. Avoids per-invocation flags.
Approach (1)+(2) is the minimum to fix the existing wiring gap. (3) and (4) are user-facing enhancements that build on top.
Affected files (pointers, not a fix list)
forge-skills/analyzer/scoring.go:11-47 — hardcoded tables
forge-skills/analyzer/scoring.go:50-114 — AnalyzeSkillDescriptor / AnalyzeSkillEntry (no policy parameter today)
forge-skills/analyzer/scoring.go:116 — scoreEgress accepts extraTrusted but is always called with nil
forge-skills/analyzer/types.go:63-71 — SecurityPolicy (existing fields not threaded into scoring)
forge-skills/analyzer/policy.go:11-24 — DefaultPolicy
forge-cli/cmd/skills.go:377 — hardcoded analyzer.DefaultPolicy() in audit
forge-cli/cmd/skills.go:91-93 — audit command flags (no --policy)
Acceptance criteria
- A custom skill can have its egress domain, env var, or binary scored as non-risky without editing any file under
forge-skills/analyzer/.
forge skills audit accepts (or reads) a configurable policy that affects both scoring and policy checks, not only the latter.
- Any score override path is auditable — i.e. it leaves a trace in the audit report (factor entry, justification, or similar) so operators can see the override was applied rather than the score being silently lower.
- Default behavior is unchanged for skills that declare no overrides and run under no custom policy.
Summary
The security risk score for a skill is computed entirely from hardcoded package-level rules in
forge-skills/analyzer/scoring.go. Skill authors and operators have no supported way to override the score, configure the scoring tables, or load a custom security policy when runningforge skills audit.This is limiting for custom skills that legitimately need a high-risk binary, a non-builtin egress domain, or a script, but are reviewed/approved at the operator level and should not be flagged repeatedly by the default policy.
Current behavior
AnalyzeSkillEntry(forge-skills/analyzer/scoring.go:70) sumsRiskFactorpoints from four categories using package-level constants:egressscoring.go:116-152binaryscoring.go:154-172envscoring.go:174-197scriptscoring.go:199-205The
trustedDomains,highRiskBinaries, andsensitiveEnvPatternslists (scoring.go:11-47) are unexported package vars — not reachable fromSKILL.md,forge.yaml, or any CLI flag.Gaps in the existing policy plumbing
The
SecurityPolicystruct (forge-skills/analyzer/types.go:63-71) already has fields that look like they should influence scoring, but the wiring is incomplete:SecurityPolicy.TrustedDomainsexists, butscoreEgressis always invoked withextraTrusted = nil(scoring.go:53,scoring.go:100). The policy's trusted list never reaches the scorer.SecurityPolicy.MaxRiskScoreis checked only insideCheckPolicyto emit amax_risk_scoreviolation (policy.go:99-109); it doesn't influence the score value itself.forge skills auditcommand hardcodesanalyzer.DefaultPolicy()(forge-cli/cmd/skills.go:377) with no flag to load an alternative policy.So even an operator who builds a
SecurityPolicytoday cannot make scoring honor it through the CLI.Why a static list in
analyzer/scoring.gois not the fixAdding a new domain to
trustedDomains, a new env pattern tosensitiveEnvPatterns, or a new binary tohighRiskBinariesrequires a code change toforge-skills/analyzer/scoring.goevery time a custom skill needs different scoring treatment. This does not scale and is the same anti-pattern called out in #48 forrunner.go's hardcodedknownKeys.The solution must be dynamic: a custom skill author or operator must be able to influence scoring without editing any Go file in the forge repo.
Proposed options (any one or combination)
SecurityPolicy.TrustedDomainsin scoring. Thread the policy throughAnalyzeSkillEntry/AnalyzeSkillDescriptorsoscoreEgressreceivespolicy.TrustedDomainsasextraTrusted. This closes the existing gap without any new schema.--policy <path>flag onforge skills audit. Load aSecurityPolicyYAML, replacingDefaultPolicy()atforge-cli/cmd/skills.go:377. Combined with (1), this lets operators expand trusted domains, binary lists, and sensitive-env patterns per-project.forge.securityblock inSKILL.mdfrontmatter. Let a skill declare its own scoring inputs:trusted_domains: [<extra domains specific to this skill>]acknowledged_high_risk_bins: [<list>]with a justification field — does not silence the factor, but caps its points or marks it asaccepted.score_overridewith amax_valueand requiredjustificationso the override is auditable.forge.yamlproject-level policy block. Asecurity_policy:section inforge.yamlthat loads the sameSecurityPolicyshape, applied to all skills in the project. Avoids per-invocation flags.Approach (1)+(2) is the minimum to fix the existing wiring gap. (3) and (4) are user-facing enhancements that build on top.
Affected files (pointers, not a fix list)
forge-skills/analyzer/scoring.go:11-47— hardcoded tablesforge-skills/analyzer/scoring.go:50-114—AnalyzeSkillDescriptor/AnalyzeSkillEntry(no policy parameter today)forge-skills/analyzer/scoring.go:116—scoreEgressacceptsextraTrustedbut is always called withnilforge-skills/analyzer/types.go:63-71—SecurityPolicy(existing fields not threaded into scoring)forge-skills/analyzer/policy.go:11-24—DefaultPolicyforge-cli/cmd/skills.go:377— hardcodedanalyzer.DefaultPolicy()in auditforge-cli/cmd/skills.go:91-93— audit command flags (no--policy)Acceptance criteria
forge-skills/analyzer/.forge skills auditaccepts (or reads) a configurable policy that affects both scoring and policy checks, not only the latter.