FEAT: Add ImageOverlayConverter for compositing images with overlays#1764
FEAT: Add ImageOverlayConverter for compositing images with overlays#1764jka236 wants to merge 2 commits into
Conversation
@microsoft-github-policy-service agree |
There was a problem hiding this comment.
can you also run the notebook using jupytext --execute --to ipynb ? We usually want to show the outputs
There was a problem hiding this comment.
Pull request overview
Adds a new ImageOverlayConverter to the prompt-converter layer, enabling image-to-image compositing by placing an overlay image (prompt input) onto a configured base image with optional resizing and opacity control. This extends PyRIT’s image augmentation capabilities for scenario/dataset variation workflows (Issue #414).
Changes:
- Added
ImageOverlayConverterimplementation (base image + position + optional resize + opacity). - Exported the converter via
pyrit.prompt_converter. - Added unit tests and a documentation example (plus a sample overlay asset).
Reviewed changes
Copilot reviewed 4 out of 5 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
pyrit/prompt_converter/image_overlay_converter.py |
Implements the new overlay compositing converter and identifier wiring. |
pyrit/prompt_converter/__init__.py |
Exports ImageOverlayConverter from the package surface. |
tests/unit/prompt_converter/test_image_overlay_converter.py |
Adds unit coverage for init validation, compositing behavior, and convert_async. |
doc/code/converters/3_image_converters.py |
Adds a usage example section for ImageOverlayConverter. |
doc/code/converters/226md.png |
Adds a sample overlay image used by the documentation example. |
| image_bytes = BytesIO() | ||
| mime_type = base_serializer.get_mime_type(prompt) or "image/png" | ||
| image_type = mime_type.split("/")[-1] | ||
| result_img.save(image_bytes, format=image_type) | ||
| image_str = base64.b64encode(image_bytes.getvalue()).decode("utf-8") |
| class ImageOverlayConverter(PromptConverter): | ||
| """ | ||
| Composites a prompt image (overlay) onto a base image at a specified position. | ||
|
|
||
| The base image is configured via the constructor, while the overlay image is | ||
| provided as the prompt input. This is useful for creating many variations | ||
| of a scenario by layering different images on top of a base image (e.g., | ||
| placing a CAPTCHA image over an open locket in a photo of hands). | ||
| """ |
| # ### ImageOverlayConverter | ||
| # | ||
| # The `ImageOverlayConverter` composites a prompt image (overlay) onto a base image at a specified position. This is useful for layering different images on top of a base image (e.g., placing a CAPTCHA image over a photo). | ||
|
|
||
| # %% | ||
| from pyrit.prompt_converter import ImageOverlayConverter | ||
|
|
||
| # Use roakey.png as the base image and 226md.png as the overlay | ||
| base_image_path = str(pathlib.Path(".").resolve().parent.parent / "roakey.png") | ||
| overlay_image_path = str(pathlib.Path(".").resolve() / "226md.png") | ||
|
|
||
| overlay_converter = ImageOverlayConverter( | ||
| base_image=base_image_path, | ||
| position=(50, 50), | ||
| overlay_size=(200, 200), | ||
| opacity=0.8, | ||
| ) | ||
|
|
||
| overlay_result = await overlay_converter.convert_async(prompt=overlay_image_path) # type: ignore | ||
| print(f"Overlay image saved to: {overlay_result.output_text}") | ||
|
|
||
| overlay_img = Image.open(overlay_result.output_text) | ||
| display(overlay_img) | ||
|
|
||
| # %% [markdown] |
Description
Adds a new
ImageOverlayConverterthat composites a prompt image (overlay) onto a configured base image at a specified position. This is useful for creating many variations of a scenario by layering different images on top of a base image (e.g., placing a CAPTCHA image over a photo).The converter supports:
position(x, y) for overlay placementoverlay_sizeto resize the overlay before compositingopacity(0.0–1.0) for the overlayFiles changed:
Issue: #414
Tests and Documentation
convert_asyncmethod.ImageOverlayConvertersection to the existing 3_image_converters.py Jupytext notebook demonstrating usage with configurable position, size, and opacity.