FIX: Aligning Scenarios with Technique Registry

.github/instructions/scenarios.instructions.md

@@ -16,7 +16,7 @@ All scenarios inherit from `Scenario` (ABC) and must:
 2. **Optionally declare `BASELINE_ATTACK_POLICY`** (defaults to `BaselineAttackPolicy.Enabled` — a baseline `PromptSendingAttack` is prepended and callers can opt out per run by setting `include_baseline=False` in the run params, see "Run Parameters" below):
    - `BaselineAttackPolicy.Disabled` — baseline supported but off by default (e.g. `Jailbreak`, where templates dominate the run).
    - `BaselineAttackPolicy.Forbidden` — baseline is meaningless for this scenario's comparison axis (e.g. `AdversarialBenchmark`, which compares against gold-standard answers). Supplying `include_baseline=True` raises `ValueError`.
-3. **Pass `technique_class`, `default_technique`, and `default_dataset_config` to `super().__init__()`:**
+3. **Pass `technique_class` and `default_dataset_config` to `super().__init__()`:**
 
 ```python
 class MyScenario(Scenario):
@@ -28,13 +28,16 @@ class MyScenario(Scenario):
         super().__init__(
             version=self.VERSION,
             technique_class=MyTechnique,
-            default_technique=MyTechnique.ALL,
             default_dataset_config=DatasetConfiguration(dataset_names=["my_dataset"]),
             objective_scorer=objective_scorer or self._get_default_objective_scorer(),
             scenario_result_id=scenario_result_id,
         )
 ```
 
+The default technique (what runs when the caller passes no `scenario_techniques`) is
+**not** a constructor argument — it is owned by the technique catalog and read from
+`technique_class.default()`. See "Default Technique" below.
+
 For scenarios whose technique enum is built dynamically (RapidResponse pattern), build the
 technique class in a module-level `@cache`-decorated function and pass the result through
 the constructor — no classmethod indirection required.
@@ -65,7 +68,6 @@ def __init__(
     super().__init__(
         version=self.VERSION,
         technique_class=MyTechnique,
-        default_technique=MyTechnique.ALL,
         default_dataset_config=DatasetConfiguration(dataset_names=["my_dataset"]),
         objective_scorer=objective_scorer,
     )
@@ -78,7 +80,7 @@ Requirements:
   `pyrit/common/brick_contract.py`). Violators raise `TypeError` at
   import time.
 - **All constructor parameters must be optional** (default to `None`) so the registry can instantiate the scenario with no arguments for metadata introspection. Defer required-input validation to `initialize_async()` or `_build_atomic_attacks_async()`. `ScenarioRegistry._build_metadata` raises `TypeError` if `scenario_class()` cannot be called with no arguments.
-- `super().__init__()` called with `version`, `technique_class`, `default_technique`, `default_dataset_config`, `objective_scorer`
+- `super().__init__()` called with `version`, `technique_class`, `default_dataset_config`, `objective_scorer` (the default technique is owned by the catalog, not passed here — see "Default Technique")
 - complex objects like `adversarial_chat` or `objective_scorer` should be passed into the constructor.
 
 ## Run Parameters
@@ -153,12 +155,34 @@ class MyTechnique(ScenarioTechnique):
     @classmethod
     def get_aggregate_tags(cls) -> set[str]:
         return {"all", "default", "single_turn", "multi_turn"}
+
+    @classmethod
+    def default(cls) -> "MyTechnique":
+        return cls.DEFAULT  # what runs when the caller selects nothing
 ```
 
 - `ALL` aggregate is always required
 - Each member: `NAME = ("string_value", {tag_set})`
 - Aggregates expand to all techniques matching their tag
 
+### Default Technique
+
+The default technique — what runs when no `scenario_techniques` are passed — is a property
+of the **technique catalog**, not the scenario. It is *not* passed to `super().__init__()`;
+the base `Scenario` reads it from `technique_class.default()`.
+
+- **Statically declared enums:** override the `default()` classmethod to return the default
+  member (e.g. `return cls.DEFAULT`, `return cls.EASY`). If the default is `ALL`, don't
+  override — the base `ScenarioTechnique.default()` already falls back to `ALL`.
+- **Dynamically built enums** (`build_technique_class_from_factories`, the RapidResponse
+  pattern): declare the default via exactly one of `default_tags={...}` (every pool technique
+  carrying any of those tags) or `default_names={...}` (exact technique names). Both build the
+  synthetic `DEFAULT` aggregate and record it so `default()` returns it. Passing both raises
+  `ValueError`. When neither is given (or nothing matches), the default falls back to `ALL`.
+
+The default is scenario-relative: the same catalog technique can be the default for one
+scenario and not another, so there is deliberately no catalog-wide `default` tag.
+
 ### Result grouping (`display_group`)
 
 `display_group` controls how attack results are aggregated for display. It is set per
@@ -248,9 +272,11 @@ AttackTechniqueFactory(
 
 Key points:
 - `name` is required and must match the technique enum value the scenario looks up.
-- `technique_tags` on the factory drives `TagQuery` filters used by
-  `AttackTechniqueRegistry.build_technique_class_from_factories(...)`. This is **distinct**
-  from the per-entry `tags` argument passed to `registry.register_technique(...)`.
+- `technique_tags` on the factory drives the aggregates and default selection built by
+  `AttackTechniqueRegistry.build_technique_class_from_factories(...)`: every tag present in
+  the pool auto-promotes to a selectable aggregate, and `default_tags` picks the default set
+  by tag. This is **distinct** from the per-entry `tags` argument passed to
+  `registry.register_technique(...)`.
 - `uses_adversarial` is auto-derived from the attack class signature (presence of
   `attack_adversarial_config`) and seed shape; pass `False` explicitly to opt out, or
   `True` to force opt-in.

doc/code/framework.md

@@ -177,7 +177,7 @@ If you are contributing to PyRIT, that work will most likely land in one of the
 - Techniques are self-describing `AttackTechniqueFactory` instances (a `name`, `attack_class`, `attack_kwargs`, and `technique_tags`). They read like configuration even though they are code, so adding one — or bringing your own — is easy. The canonical catalog lives under `pyrit/setup/initializers/techniques/`: `core.py` (a small, curated standard set, auto-loaded on a bare `initialize_pyrit_async`), `extra.py` (the broader collection, opt-in), and per-source modules like `airt.py` (owned by a source/scenario but reusable, tagged with their owner and kept out of the default pool).
 - `core` stays deliberately small so a default run doesn't print 200 techniques or take forever; the wider catalog lives in `extra` and is selected on demand. Users pick subsets by passing initializer tags (e.g. `core`, `extra`, `all`) or writing their own initializer, so different runs — including from the CLI — can register different technique sets without changing the catalog.
 - A technique tied to one scenario is fine; if it's pinned and non-reusable it can stay local to that scenario, but if another scenario could reuse it, promote it to a catalog module and tag it.
-- Tags describe a technique (behavioral tags like `single_turn`/`multi_turn`, owner tags like `airt`); they don't decide what a scenario runs. There is deliberately **no global `default` tag** — a default is scenario-relative, declared per scenario via `build_technique_class_from_factories` (`available` selects the pool, aggregates are named presets, `default` / `default_technique_names` set what runs when nothing is chosen).
+- Tags describe a technique (behavioral tags like `single_turn`/`multi_turn`, owner tags like `airt`); they don't decide what a scenario runs. There is deliberately **no global `default` tag** — a default is scenario-relative, declared per scenario via `build_technique_class_from_factories` (the `factories` list is the pool, catalog tags become named aggregate presets, and `default_tags` / `default_names` set what runs when nothing is chosen).
 - **Does not own**: the conversation algorithm itself. Branching, turn management, and scoring decisions live in the executor it wraps — a technique only selects and configures existing components, and shouldn't implement new sending, scoring, or branching logic.
 
 **Framework Plans**:

doc/code/scenarios/0_scenarios.ipynb

@@ -57,10 +57,10 @@
     "   - Each enum member represents an **attack technique** (the *how* of an attack)\n",
     "   - Each member is defined as `(value, tags)` where value is a string and tags is a set of strings\n",
     "   - Include an `ALL` aggregate technique that expands to all available techniques\n",
+    "   - The default technique (what runs when the caller selects nothing) is owned by the catalog, not the scenario: override the `default()` classmethod to return the default member (omit it to fall back to `ALL`)\n",
     "\n",
     "2. **Scenario Class**: Extend `Scenario` and pass these to `super().__init__()`:\n",
     "   - `technique_class`: Your technique enum class\n",
-    "   - `default_technique`: The default technique (typically `YourTechnique.ALL` or `YourTechnique.DEFAULT`)\n",
     "   - Implement `_build_atomic_attacks_async(context)` — the single abstract extension point.\n",
     "     Matrix-shaped scenarios delegate to `build_matrix_atomic_attacks(context=...)` in one line.\n",
     "\n",
@@ -72,7 +72,6 @@
     "   - `name`: Descriptive name for your scenario\n",
     "   - `version`: Integer version number\n",
     "   - `technique_class`: The technique enum class for this scenario\n",
-    "   - `default_technique`: The default technique member (typically `YourTechnique.ALL` or `YourTechnique.DEFAULT`)\n",
     "   - `default_dataset_config`: A `DatasetConfiguration` specifying the scenario's default datasets\n",
     "   - `objective_scorer`: The scorer used to judge responses\n",
     "   - `scenario_result_id`: Optional ID to resume an existing scenario (optional)\n",
@@ -141,6 +140,10 @@
     "    PromptSending = (\"prompt_sending\", {\"single_turn\", \"default\"})\n",
     "    RolePlay = (\"role_play_movie_script\", {\"single_turn\"})\n",
     "\n",
+    "    @classmethod\n",
+    "    def default(cls) -> \"MyTechnique\":\n",
+    "        return cls.DEFAULT\n",
+    "\n",
     "\n",
     "class MyScenario(Scenario):\n",
     "    \"\"\"Quick-check scenario for testing model behavior across harm categories.\"\"\"\n",
@@ -162,7 +165,6 @@
     "            version=self.VERSION,\n",
     "            objective_scorer=self._objective_scorer,\n",
     "            technique_class=MyTechnique,\n",
-    "            default_technique=MyTechnique.DEFAULT,\n",
     "            default_dataset_config=DatasetConfiguration(dataset_names=[\"dataset_name\"], max_dataset_size=4),\n",
     "            scenario_result_id=scenario_result_id,\n",
     "        )\n",

doc/code/scenarios/0_scenarios.py

@@ -59,10 +59,12 @@
 #    - Each enum member represents an **attack technique** (the *how* of an attack)
 #    - Each member is defined as `(value, tags)` where value is a string and tags is a set of strings
 #    - Include an `ALL` aggregate technique that expands to all available techniques
+#    - The default technique (what runs when the caller selects nothing) is owned by the
+#      catalog, not the scenario: override the `default()` classmethod to return the default
+#      member (omit it to fall back to `ALL`)
 #
 # 2. **Scenario Class**: Extend `Scenario` and pass these to `super().__init__()`:
 #    - `technique_class`: Your technique enum class
-#    - `default_technique`: The default technique (typically `YourTechnique.ALL` or `YourTechnique.DEFAULT`)
 #    - Implement `_build_atomic_attacks_async(context)` — the single abstract extension point.
 #      Matrix-shaped scenarios delegate to `build_matrix_atomic_attacks(context=...)` in one line.
 #
@@ -74,7 +76,6 @@
 #    - `name`: Descriptive name for your scenario
 #    - `version`: Integer version number
 #    - `technique_class`: The technique enum class for this scenario
-#    - `default_technique`: The default technique member (typically `YourTechnique.ALL` or `YourTechnique.DEFAULT`)
 #    - `default_dataset_config`: A `DatasetConfiguration` specifying the scenario's default datasets
 #    - `objective_scorer`: The scorer used to judge responses
 #    - `scenario_result_id`: Optional ID to resume an existing scenario (optional)
@@ -118,6 +119,10 @@ class MyTechnique(ScenarioTechnique):
     PromptSending = ("prompt_sending", {"single_turn", "default"})
     RolePlay = ("role_play_movie_script", {"single_turn"})
 
+    @classmethod
+    def default(cls) -> "MyTechnique":
+        return cls.DEFAULT
+
 
 class MyScenario(Scenario):
     """Quick-check scenario for testing model behavior across harm categories."""
@@ -139,7 +144,6 @@ def __init__(
             version=self.VERSION,
             objective_scorer=self._objective_scorer,
             technique_class=MyTechnique,
-            default_technique=MyTechnique.DEFAULT,
             default_dataset_config=DatasetConfiguration(dataset_names=["dataset_name"], max_dataset_size=4),
             scenario_result_id=scenario_result_id,
         )

doc/scanner/1_pyrit_scan.ipynb

@@ -941,7 +941,6 @@
     "            version=1,\n",
     "            objective_scorer=TrueFalseInverterScorer(scorer=SelfAskRefusalScorer(chat_target=OpenAIChatTarget())),\n",
     "            technique_class=MyCustomTechnique,\n",
-    "            default_technique=MyCustomTechnique.ALL,\n",
     "            default_dataset_config=DatasetAttackConfiguration(dataset_names=[\"harmbench\"]),\n",
     "            scenario_result_id=scenario_result_id,\n",
     "        )\n",

doc/scanner/1_pyrit_scan.py

@@ -185,7 +185,6 @@ def __init__(self, *, scenario_result_id=None, **kwargs):
             version=1,
             objective_scorer=TrueFalseInverterScorer(scorer=SelfAskRefusalScorer(chat_target=OpenAIChatTarget())),
             technique_class=MyCustomTechnique,
-            default_technique=MyCustomTechnique.ALL,
             default_dataset_config=DatasetAttackConfiguration(dataset_names=["harmbench"]),
             scenario_result_id=scenario_result_id,
         )