Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
90 commits
Select commit Hold shift + click to select a range
2808205
MAINT: fix docstrings for `/prompt_target` (#1263)
paulinek13 Dec 19, 2025
016e145
FIX add transphobia awareness dataset back (#1264)
romanlutz Dec 19, 2025
a63a984
FEAT add UV support (#1226)
hannahwestra25 Dec 19, 2025
8e0ef51
TEST: integration test fixes (#1265)
rlundeen2 Dec 19, 2025
b6380ce
MAINT Breaking: Modifying attack params (#1260)
rlundeen2 Dec 20, 2025
37f2ef8
FEAT: Refactor and Enhance Scorer Identifier for Evaluations (#1262)
jsong468 Dec 20, 2025
5260c47
FIX add OPENAI_CHAT_MODEL as required in docs, initializers (#1267)
romanlutz Dec 21, 2025
fb84425
FIX: Add ARM64/Apple Silicon support for devcontainer builds (#1251)
riyosha Dec 21, 2025
fce5d51
FIX: use max_iterations in CharSwapConverter perturbation loop (#1269)
KutalVolkan Dec 21, 2025
fc6ff44
FIX activate env (#1274)
hannahwestra25 Dec 22, 2025
72504a3
FIX make bash default and remove volume mount for venv in devcontaine…
romanlutz Dec 22, 2025
12397c8
MAINT add py.typed to help with mypy type checking for consuming pack…
romanlutz Dec 23, 2025
485be33
MAINT CONTROVERSIAL: Make env files configurable (#1253)
rlundeen2 Dec 23, 2025
26ffba3
FIX fix permission denied error when creating env (#1279)
hannahwestra25 Dec 23, 2025
5fc9ce1
MAINT remove dispose memory engine calls in docs (#1278)
romanlutz Dec 23, 2025
f51235e
FIX: Updating Pipelines (#1282)
rlundeen2 Dec 23, 2025
defb46f
MAINT: Updating AttackExecutor to more generically call attacks (#1270)
rlundeen2 Dec 24, 2025
aec14c4
FIX: set virtual env in docker dev setup (#1281)
hannahwestra25 Dec 26, 2025
f525ee2
FIX/FEAT: Enable multi-modal pieces for SelfAskTrueFalseScorer scorin…
jsong468 Dec 29, 2025
c8dcf57
FEAT: Adding simulated_conversation and adding prepended_conversation…
rlundeen2 Dec 29, 2025
a7be973
FIX: Bug with prepended_conversation system prompt (#1289)
rlundeen2 Dec 29, 2025
233a1e4
FEAT: adding underlying_model for target identification (#1234)
jsong468 Dec 29, 2025
a4939a5
MAINT change devcontainer base image to python from MCR, mount .env* …
romanlutz Dec 30, 2025
97ebac5
DOC reorganize converter docs (#1268)
romanlutz Dec 30, 2025
4748750
FIX: integration tests and ImageTarget refactor (#1293)
rlundeen2 Dec 30, 2025
b62672f
[FEAT] Support JSON Schema in Responses (#1177)
riedgar-ms Dec 30, 2025
59108c1
FIX remove obsolete assertions from image target integration tests (#…
romanlutz Dec 30, 2025
202fa8c
MAINT update ignored notebook index (#1295)
romanlutz Dec 30, 2025
65159c4
FIX: integration test fixes (#1297)
rlundeen2 Dec 31, 2025
3813661
MAINT add skeleton frontend (#1290)
romanlutz Dec 31, 2025
b76704a
MAINT: Adding simulated assistant role (#1292)
rlundeen2 Dec 31, 2025
e8ec417
MAINT Breaking: Message Normalizer Refactor (#1296)
rlundeen2 Dec 31, 2025
698d848
FEAT: Scenario DatasetConfiguration (#1288)
rlundeen2 Jan 1, 2026
7e8518a
[MAINT BREAKING]: Renaming scenarios (#1301)
rlundeen2 Jan 2, 2026
4125fb5
FEAT add skeleton backend for the GUI/frontend (#1298)
romanlutz Jan 2, 2026
d7c893f
[FIX Breaking]: PrependedConversationConfig and Attack Param Consiste…
rlundeen2 Jan 6, 2026
fb8e223
FEAT: New Scenario - Scams (#1202)
nina-msft Jan 6, 2026
5ba83cc
MAINT add deprecation instructions (#1303)
romanlutz Jan 7, 2026
8229302
MAINT remove flake8, black and consolidate under ruff (including copy…
romanlutz Jan 7, 2026
f9c6ba5
MAINT: Enhance type hinting across auth and analytics modules (#1300)
ytc338 Jan 7, 2026
f70f000
FEAT: Add NegationTrapConverter and ChunkedRequestConverter (#1261)
fitzpr Jan 7, 2026
d2adef1
FIX example filename in Docker setup instructions (#1305)
fukusuket Jan 7, 2026
4d9d72f
FEAT [BREAKING]: Scorer evaluation refactor (#1280)
jsong468 Jan 7, 2026
520257e
FEAT: SeedSimulatedConversation to generate simulated conversations i…
rlundeen2 Jan 8, 2026
0af160c
MAINT: Fixing deprecated usage (#1306)
rlundeen2 Jan 8, 2026
b029f02
FEAT Breaking: Registry protocol + ScorerRegistry (#1308)
rlundeen2 Jan 11, 2026
af011a0
MAINT strict mypy checking on the whole repository (#1310)
romanlutz Jan 13, 2026
7a7f8b6
MAINT: fix docstrings for `/prompt_converter` (#1314)
paulinek13 Jan 14, 2026
68fa348
FEAT: Added VLSU Multimodal Dataset (#1309)
riyosha Jan 14, 2026
4a40aa3
FEAT: Add binary_path data type (#1315)
jsong468 Jan 15, 2026
c2bedb3
FIX MAINT: Improved Attack reliability and maintainability (#1317)
rlundeen2 Jan 17, 2026
b530252
FEAT: Adding audio and tool support to chat completions (#1311)
rlundeen2 Jan 17, 2026
6aaae74
FEAT: Leakage Scenario - New (#1284)
varunj-msft Jan 23, 2026
22755a2
MAINT: Registry Metadata Refactor (#1323)
rlundeen2 Jan 23, 2026
10538c3
FIX: commit `blank_canvas.png` required by `LeakageScenario` (#1330)
paulinek13 Jan 27, 2026
74e16a1
DOC: Update `doc/code/scenarios/1_configuring_scenarios` Notebook (#1…
nina-msft Jan 27, 2026
ca0107f
DOC: Add Response Converters section to converters documentation (#1326)
Copilot Jan 27, 2026
d2af298
MAINT: Refactoring General Identifiers and ScorerIdentifier (#1328)
rlundeen2 Jan 27, 2026
9200250
FEAT: More Informative Attack Exceptions (#1318)
rlundeen2 Jan 27, 2026
737cc54
FEAT: add WebSocket-based prompt target for Microsoft Copilot (#1275)
paulinek13 Jan 27, 2026
d89872c
TEST add frontend unit and e2e tests (#1331)
romanlutz Jan 28, 2026
26989b8
FIX Auto-wrap synchronous token providers for AsyncOpenAI compatibili…
Copilot Jan 28, 2026
90d3270
FEAT: Adding ConverterIdentifier and minor Identifiable refactor (#1333)
rlundeen2 Jan 28, 2026
42c96ff
FEAT Support baseline-only execution in Scenario (#1321)
slister1001 Jan 28, 2026
a8b1fac
[FEAT]: Psychosocial Scenario (#1266)
jbolor21 Jan 28, 2026
6745ec2
FEAT: Support image edition/remix in OpenAIImageTarget (#1322)
fdubut Jan 28, 2026
bd858ae
FIX update encoding default data configuration (#1335)
hannahwestra25 Jan 29, 2026
5c945b4
FEAT: Adding pyrit_version to identifiers (#1334)
rlundeen2 Jan 29, 2026
393fbca
FEAT: Adding `TargetIdentifier` (#1336)
jsong468 Jan 30, 2026
6f9ca6a
FIX use message in xpia website notebook (#1339)
hannahwestra25 Jan 30, 2026
4603387
FEAT Use TASK_ACHIEVED as fallback for SelfAskTrueFalseScorer (#1342)
fdubut Feb 2, 2026
fa406e8
FEAT: Jailbreak Scenario (#1329)
ValbuenaVC Feb 5, 2026
80f05ee
MAINT post-0.11.0 release update to 0.11.1dev0 (#1352)
hannahwestra25 Feb 5, 2026
b102045
FEAT: Add JsonStringConverter for JSON-safe string conversion (#454) …
akashWhoCodes Feb 6, 2026
bd3dcf7
FIX MS Package Signing Key issue #1353 to enable DevContainers after …
nmolivo Feb 6, 2026
3604ab6
MAINT address code scanning alert on url sanitization (#1361)
romanlutz Feb 6, 2026
44b6e10
MAINT bump package versions to address dependabot alerts (#1360)
romanlutz Feb 6, 2026
159e452
MAINT mypy fixes (#1359)
romanlutz Feb 6, 2026
d47d4a8
MAINT upgrade vite in response to npm audit (#1358)
romanlutz Feb 6, 2026
24dd603
FEAT: Target Registry and AIRT Targets Initializer (#1320)
jsong468 Feb 6, 2026
9a2e036
MAINT address remaining dependabout issues with uv.lock (#1363)
romanlutz Feb 9, 2026
7094c5e
FEAT local Docker setup for running GUI or Jupyter (#1357)
romanlutz Feb 10, 2026
ad7f52a
FEAT: Adding audio scoring (#1337)
jbolor21 Feb 11, 2026
f2c32c0
MAINT: PEP 8: Uppercase class constants (VERSION, _DEFAULT_VALIDATOR)…
Copilot Feb 11, 2026
4039c2d
FEAT: PyRIT Config (#1343)
ValbuenaVC Feb 12, 2026
c0ce883
FEAT: Attack Identifier (#1364)
rlundeen2 Feb 13, 2026
a1ebb12
FEAT Add backend APIs (#1354)
romanlutz Feb 14, 2026
2484292
Decoupled ScenarioMetadata and InitializerMetadata from Identity by i…
bashirpartovi Feb 17, 2026
d495d46
feat(#6a): plumb media output through adversarial feedback loop
Feb 18, 2026
c2723ff
fix: handle error data type in multimodal chat messages
Feb 19, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
60 changes: 44 additions & 16 deletions .devcontainer/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,28 +1,39 @@
FROM mcr.microsoft.com/devcontainers/anaconda:3
FROM --platform=linux/amd64 mcr.microsoft.com/devcontainers/python:3.11

# Makes installation faster
ENV UV_COMPILE_BYTECODE=1

SHELL ["/bin/bash", "-c"]

USER root

# Remove the Yarn repository (has expired GPG key and we don't use Yarn)
RUN rm -f /etc/apt/sources.list.d/yarn.list 2>/dev/null || true

# Install required system packages + ODBC prerequisites
RUN apt-get update && apt-get install -y \
sudo \
unixodbc \
unixodbc-dev \
libgl1-mesa-glx \
libgl1 \
git \
curl \
xdg-utils \
build-essential \
&& apt-get clean && rm -rf /var/lib/apt/lists/*

# Install the Azure CLI, Microsoft ODBC Driver 18 & SQL tools
# Note: Debian Trixie's sqv rejects SHA1 signatures, so we use gpg directly to import the Microsoft key
RUN apt-get update && apt-get install -y \
apt-transport-https \
ca-certificates \
gnupg \
lsb-release \
&& curl -sL https://packages.microsoft.com/config/debian/12/packages-microsoft-prod.deb \
-o packages-microsoft-prod.deb \
&& dpkg -i packages-microsoft-prod.deb \
&& rm packages-microsoft-prod.deb \
&& curl -sL https://packages.microsoft.com/keys/microsoft.asc \
| gpg --dearmor \
> /usr/share/keyrings/microsoft-archive-keyring.gpg \
&& echo "deb [arch=amd64 signed-by=/usr/share/keyrings/microsoft-archive-keyring.gpg] https://packages.microsoft.com/debian/12/prod bookworm main" \
> /etc/apt/sources.list.d/microsoft.list \
&& apt-get update \
&& ACCEPT_EULA=Y apt-get install -y \
msodbcsql18 \
Expand All @@ -41,25 +52,41 @@ RUN apt-get update \
libpulse0 \
&& rm -rf /var/lib/apt/lists/*

# Create conda env and install pyodbc into it
RUN conda create -n pyrit-dev python=3.11 -y && \
conda install -n pyrit-dev -c conda-forge pyodbc -y && \
chown -R vscode:vscode /opt/conda/envs/pyrit-dev
# Install uv system-wide and create pyrit-dev venv
RUN curl -LsSf https://astral.sh/uv/install.sh | sh \
&& mv /root/.local/bin/uv /usr/local/bin/uv \
&& rm -rf /opt/venv \
&& uv venv /opt/venv --python 3.11 --prompt pyrit-dev \
&& chown -R vscode:vscode /opt/venv \
&& ls -la /opt/venv/bin/activate
ENV PATH="/opt/venv/bin:$PATH"

# vscode user already exists in the base image, just ensure sudo access
RUN echo "vscode ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers

# Install Node.js 20.x and npm for frontend development
RUN curl -fsSL https://deb.nodesource.com/setup_20.x | bash - \
&& apt-get install -y nodejs \
&& npm install -g npm@latest \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*

# Pre-create common user caches and fix permissions
RUN mkdir -p /home/vscode/.cache/pre-commit \
&& mkdir -p /home/vscode/.vscode-server \
&& mkdir -p /home/vscode/.cache/pip \
&& mkdir -p /home/vscode/.cache/conda \
&& mkdir -p /home/vscode/.cache/uv \
&& mkdir -p /home/vscode/.cache/venv \
&& mkdir -p /home/vscode/.cache/pylance \
&& chown -R vscode:vscode /home/vscode/.cache /home/vscode/.vscode-server \
&& chmod -R 777 /home/vscode/.cache/conda /home/vscode/.cache/pip /home/vscode/.cache/pylance /opt/conda/pkgs/cache/ \
&& chmod -R 777 /home/vscode/.cache/pip /home/vscode/.cache/pylance /home/vscode/.cache/venv /home/vscode/.cache/uv\
&& chmod -R 755 /home/vscode/.vscode-server

USER vscode
RUN /opt/conda/bin/conda init bash && \
echo "conda activate pyrit-dev" >> /home/vscode/.bashrc
RUN echo "source /opt/conda/etc/profile.d/conda.sh && conda activate pyrit-dev" >> /home/vscode/.bash_profile
# Create bash configuration files and activate the venv in bash sessions
RUN touch /home/vscode/.bashrc /home/vscode/.bash_profile \
&& echo "[ -f /opt/venv/bin/activate ] && source /opt/venv/bin/activate" >> /home/vscode/.bashrc \
&& echo "[ -f /opt/venv/bin/activate ] && source /opt/venv/bin/activate" >> /home/vscode/.bash_profile

# Configure Git for better performance with bind mounts
RUN git config --global core.preloadindex true \
Expand All @@ -68,5 +95,6 @@ RUN git config --global core.preloadindex true \
&& git config --global status.showUntrackedFiles all \
&& git config --global core.fsmonitor true

# Set pip’s cache directory so it can be mounted
# Set cache directories so they can be mounted
ENV PIP_CACHE_DIR="/home/vscode/.cache/pip"
ENV UV_CACHE_DIR="/home/vscode/.cache/uv"
21 changes: 15 additions & 6 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,12 @@
"containerEnv": {
"PYTHONPATH": "/workspace"
},
"containerUser": "vscode",
"customizations": {
"vscode": {
"settings": {
"python.defaultInterpreterPath": "/opt/conda/envs/pyrit-dev/bin/python",
"terminal.integrated.defaultProfile.linux": "bash",
"python.defaultInterpreterPath": "/opt/venv/bin/python",
"python.analysis.extraPaths": [
"/workspace"
],
Expand All @@ -31,7 +33,7 @@
"pyrit/**"
],
"python.analysis.exclude": [
"/opt/conda/envs/**",
"/opt/venv/**",
"**/.venv/**",
"**/site-packages/**",
"**/doc/**",
Expand All @@ -50,7 +52,10 @@
"**/dist/**": true,
"**/pyrit/auxiliary_attacks/gcg/attack/**": true,
"**/doc/**": true,
"**/.mypy_cache/**": true
"**/.mypy_cache/**": true,
"**/frontend/node_modules/**": true,
"**/frontend/dist/**": true,
"**/dbdata/**": true
},
"search.exclude": {
"**/node_modules": true,
Expand All @@ -66,7 +71,7 @@
"**/build": true,
"**/__pycache__": true
},
"explorer.autoReveal": false,
"explorer.autoReveal": true,
"files.maxMemoryForLargeFilesMB": 4096,
"files.useExperimentalFileWatcher": true,
"git.showUntrackedFiles": true
Expand All @@ -75,10 +80,14 @@
"ms-python.python",
"ms-toolsai.jupyter",
"ms-azuretools.vscode-docker",
"tamasfe.even-better-toml"
"tamasfe.even-better-toml",
"dbaeumer.vscode-eslint",
"esbenp.prettier-vscode",
"ms-playwright.playwright",
"orta.vscode-jest"
]
}
},
"postCreateCommand": "/bin/bash -i .devcontainer/devcontainer_setup.sh",
"forwardPorts": [4213, 5000, 8888]
"forwardPorts": [3000, 4213, 5000, 8000, 8888]
}
48 changes: 37 additions & 11 deletions .devcontainer/devcontainer_setup.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
set -e

MYPY_CACHE="/workspace/.mypy_cache"
VIRTUAL_ENV="/opt/venv"
# Create the mypy cache directory if it doesn't exist
if [ ! -d "$MYPY_CACHE" ]; then
echo "Creating mypy cache directory..."
Expand Down Expand Up @@ -30,16 +31,11 @@ fi
sudo rm -rf /vscode/vscode-server/extensionsCache/github.copilot-*
rm -rf /home/vscode/.vscode-server/extensions/{*,.[!.]*,..?*}

# Path to store the hash
HASH_FILE="/home/vscode/.cache/pip/pyproject_hash"
# Activate the uv venv created in the Dockerfile
source /opt/venv/bin/activate

# Make sure the hash file is writable if it exists; if not, it will be created
if [ -f "$HASH_FILE" ]; then
chmod 666 "$HASH_FILE"
fi

source /opt/conda/etc/profile.d/conda.sh
conda activate pyrit-dev
# Store hash inside venv so it's tied to the venv lifecycle
HASH_FILE="/opt/venv/pyproject_hash"

# Compute current hash
CURRENT_HASH=$(sha256sum /workspace/pyproject.toml | awk '{print $1}')
Expand All @@ -49,13 +45,43 @@ if [ ! -f "$HASH_FILE" ] || [ "$(cat $HASH_FILE)" != "$CURRENT_HASH" ]; then
echo "📦 pyproject.toml has changed, installing environment..."

# Install dependencies
conda install ipykernel -y
pip install -e '.[dev,all]'
uv pip install ipykernel
uv pip install -e ".[dev,all]"
# Register the kernel with Jupyter
python -m ipykernel install --user --name=pyrit-dev --display-name="Python (pyrit-dev)"

# Save the new hash
echo "$CURRENT_HASH" > "$HASH_FILE"
else
echo "✅ pyproject.toml has not changed, skipping installation."
fi

# Install frontend dependencies
echo "📦 Installing frontend dependencies..."

# Fix node_modules permissions (volume is owned by root)
if [ -d "/workspace/frontend/node_modules" ]; then
echo "Fixing node_modules permissions..."
sudo chown -R vscode:vscode /workspace/frontend/node_modules
fi

cd /workspace/frontend
if [ -f "package.json" ]; then
npm install

# Install Playwright browsers and system dependencies for E2E testing
echo "📦 Installing Playwright browsers..."

# Remove third-party repos with SHA1 signature issues (rejected since 2026-02-01)
# Playwright deps come from Debian main repos, these aren't needed
sudo rm -f /etc/apt/sources.list.d/yarn.list \
/etc/apt/sources.list.d/nodesource.list \
/etc/apt/sources.list.d/microsoft.list 2>/dev/null || true

npx playwright install --with-deps chromium

echo "✅ Frontend dependencies installed."
fi
cd /workspace

echo "🚀 Dev container setup complete!"
10 changes: 6 additions & 4 deletions .devcontainer/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
services:
devcontainer:
platform: linux/amd64
build:
context: ..
dockerfile: .devcontainer/Dockerfile
Expand All @@ -10,22 +11,23 @@ services:
memory: "16G"
volumes:
- ..:/workspace:delegated
- pyrit-env:/opt/conda/envs/pyrit-dev:cached
- pip-cache:/home/vscode/.cache/pip:cached
- uv-cache:/home/vscode/.cache/uv:cached
- precommit-cache:/home/vscode/.cache/pre-commit:cached
- conda-cache:/home/vscode/.cache/conda:cached
- mypy-cache:/workspace/.mypy_cache:cached
- pylance-cache:/home/vscode/.cache/pylance:cached
- node-modules:/workspace/frontend/node_modules:cached
- ~/.pyrit:/home/vscode/.pyrit:cached
network_mode: "host"
# Note: ports section is not needed with host network mode
# The container will have direct access to all host network interfaces
# Keep the container running so the post-create command can execute.
command: "sleep infinity"

volumes:
pyrit-env:
pip-cache:
uv-cache:
precommit-cache:
conda-cache:
mypy-cache:
pylance-cache:
node-modules:
27 changes: 27 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,30 @@ build/
**/*$py.class
**/.pytest_cache/
**/.mypy_cache/

# Environment files with secrets
.env
.env.*
*.env

# Database files with conversation history
dbdata/
results/
default_memory.json.memory
*.db
*.sqlite

# Azure and other credentials
.azure/
*.pem
*.key
*.pfx
*.p12

# Frontend build artifacts (will be built inside Docker)
frontend/node_modules/
frontend/dist/
frontend/.vite/

# Backend frontend directory (generated during packaging)
pyrit/backend/frontend/
Loading
Loading