FEAT: Adding PyRITInitializer parameters#1456
Conversation
Add a dict[str, str] params system for PyRIT initializers that works uniformly across YAML config, CLI, and the programmatic API. Key changes: - Add InitializerParameter dataclass for declaring supported params - Add params: dict[str, str] | None to initialize_async signature - Migrate TargetInitializer/ScorerInitializer from constructor tags to params-based (tags as comma-separated string) - Update ConfigurationLoader to set _params instead of constructor kwargs - Update FrontendCore with _initializer_configs and name:key=val;key2=val2 CLI syntax for passing params - Add param validation (ValueError on unknown/missing required params) - Backward-compatible with old-style initializer scripts (no params arg) - Update InitializerMetadata with supported_parameters for discovery - Add comprehensive tests (294 passing) - Update documentation and .pyrit_conf_example Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…list[str]] Each parameter value is now a list of strings, which maps naturally to: - YAML: native list syntax under args - CLI: comma-separated values (tags=default,scorer -> ['default', 'scorer']) - Programmatic API: direct list passing This eliminates the need for comma-separated string parsing inside initializers and provides a cleaner, more uniform API. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add name:key=val1,val2 syntax hints to --initializers help in shell. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
| - Simple name: "simple" → {"name": "simple"} | ||
| - Name with params: "target:tags=default,scorer" → {"name": "target", "args": {"tags": "default,scorer"}} | ||
|
|
||
| For multiple params, separate with semicolons: "name:key1=val1;key2=val2" |
There was a problem hiding this comment.
maybe this is not common and I don't have a solution for this but this has the potential to get long esp if a user has many initializers with many tags
There was a problem hiding this comment.
Potentially! But I hope people use the config, and also I could see this shortening things in some cases (e.g. something that would otherwise take several initializers can use an argument).
|
|
||
| Run Command Options: | ||
| --initializers <name> ... Built-in initializers to run before the scenario | ||
| --initializers <name> ... Built-in initializers (supports name:key=val1,val2 syntax) |
There was a problem hiding this comment.
I'm confused by that syntax. Do you have an example?
There was a problem hiding this comment.
Found target:tags=default,scorer below. But what if there are multiple?
| # - simple | ||
| # - name: airt |
There was a problem hiding this comment.
should this be name: simple?
Also, why is airt removed?
| tags: | ||
| - default | ||
| - scorer |
There was a problem hiding this comment.
these should be for target, not airt right?
| await self.initialize_async() | ||
| params = self.params if self.params else None | ||
| try: | ||
| await self.initialize_async(params=params) |
There was a problem hiding this comment.
I'm a bit confused on these lines. We're passing an instance variable as an arg to the initializer's own method. Curious as to the reason we need to store the params as both an internal attribute of the instance and allow it to be passed in as an arg to initialize_async?
| # - A dictionary with 'name' and optional 'args' for parameters | ||
| # | ||
| # Parameters are lists of strings. Use the CLI command | ||
| # `pyrit_scan --list-initializers` to see available parameters. |
There was a problem hiding this comment.
Should we implement something like pyrit_scan --list-initializers --help or pyrit_scan --list-parameters simple instead to see available parameters? I imagine that users would want a dedicated interface for figuring out parameters once they choose one or more initializers and don't need to list them all
| # Instantiate with args if provided | ||
| instance = initializer_class(**config.args) if config.args else initializer_class() | ||
| # Instantiate and set params if provided | ||
| instance = initializer_class() | ||
| if config.args: | ||
| instance.params = { | ||
| k: [str(i) for i in v] if isinstance(v, list) else [str(v)] for k, v in config.args.items() | ||
| } |
There was a problem hiding this comment.
Nit: might be wrong, but the old implementation would implicitly validate the args passed, since by instantiating initializer_class the constructor would validate or at least consume the arguments before appending the instance to the resolved list. This just adds them as an instance attribute.
Also, neither SimpleInitializer nor AIRTInitializer take any arguments in their constructors. I'm probably misunderstanding this method but that implies to me that this code is functionally broken with our current initializer classes?
5 participants
Previously, initializer configuration (like target tags) was passed through constructor arguments, which didn't
translate cleanly to YAML config or CLI usage. This unifies the approach so parameters flow the same way regardless
of entry point.
This change adds a uniform params system for PyRIT initializers that works consistently across YAML config, CLI, and the
programmatic API.
What changed:
InitializerParameterdataclass for initializers to declare their supported parameters (name, description,required, default)
initialize_async()now accepts an optional params: dict[str, list[str]] argumentAPI usage