Image inpainting, outpainting by atobiszei · Pull Request #4047 · openvinotoolkit/model_server

atobiszei · 2026-03-09T08:19:31Z

Add inpainting and image-to-image pipeline support via /v3/images/edits endpoint (multipart form with image + optional mask)
Add mask parsing in multipart request handler
Propagate extraQuantizationParams to all optimum export functions (embeddings, TTS, STT, rerank, image_gen, text_gen)
Demo

Ticket:CVS-181770

Copilot

Pull request overview

This PR extends OVMS image-generation support to cover inpainting/outpainting via the OpenAI-compatible /v3/images/edits endpoint (using an uploaded mask), and updates pipeline initialization to include OpenVINO GenAI’s InpaintingPipeline.

Changes:

Add InpaintingPipeline to the image-generation pipeline bundle and initialize pipelines with weight-sharing + fallbacks.
Route /v3/images/edits requests to inpainting when mask is provided; expand request-option validation to accept mask.
Extend multipart parser interface with a “files array” accessor and update demos/docs/assets accordingly.

Reviewed changes

Copilot reviewed 13 out of 20 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
`src/image_gen/http_image_gen_calculator.cc`	Routes edit requests to inpainting when `mask` is present; removes per-request pipeline cloning.
`src/image_gen/pipelines.hpp`	Adds `InpaintingPipeline` member and include.
`src/image_gen/pipelines.cpp`	Builds/compiles inpainting pipeline and derives other pipelines with fallbacks.
`src/image_gen/imagegenutils.cpp`	Accepts `mask` in allowed fields for generation/edit option parsing.
`src/multi_part_parser.hpp`	Adds `getFilesArrayByFieldName()` to the multipart parser API.
`src/http_frontend/multi_part_parser_drogon_impl.hpp/.cpp`	Implements `getFilesArrayByFieldName()` for Drogon multipart parser.
`src/test/text2image_test.cpp`	Updates `MockedMultiPartParser` to match the new interface.
`src/test/test_http_utils.hpp`	Updates multipart parser mock to match the new interface.
`src/test/pull_hf_model_test.cpp`	Adds test coverage for image-generation export command with extra params.
`src/pull_module/optimum_export.cpp`	Appends `extraQuantizationParams` for image-generation export command.
`demos/image_generation/README.md`	Documents inpainting/outpainting usage via cURL and OpenAI Python client.
`demos/image_generation/cat_mask.png`	Demo asset for inpainting mask.
`demos/image_generation/outpaint_mask.png`	Demo asset for outpainting mask.
`.gitignore`	Reorders/adds Bazel output directories to ignore list.
`.dockerignore`	Expands ignored local/dev and build artifacts; ignores `models/` and logs.

You can also share your feedback on Copilot code review. Take the survey.

src/image_gen/imagegenutils.cpp

src/pull_module/optimum_export.cpp

src/image_gen/http_image_gen_calculator.cc

mask field should only be accepted in image edit (inpainting) requests, not in text-to-image generation requests.

…ting

dkalinowski · 2026-03-17T14:08:29Z

demos/image_generation/README.md

+![cat](./cat.png) ![cat_mask](./cat_mask.png)
+
+Linux
+```bash


Add tab for sphinx? Can you generate sphinx doc so we can see how it is going to look? @atobiszei

atobiszei · 2026-03-17T14:09:05Z

demos/image_generation/README.md

+
+![cat](./cat.png) ![cat_mask](./cat_mask.png)
+
+Linux


Follow switch formating with sphinx, as in earlier part of readme.

dkalinowski · 2026-03-17T15:52:12Z

src/pull_module/optimum_export.cpp

    oss << " --weight-format " << this->exportSettings.precision;
+    if (this->exportSettings.extraQuantizationParams.has_value()) {
+        oss << " " << this->exportSettings.extraQuantizationParams.value();
+    }  // TODO FIXME check if its not needed to propagate to other exports


who is going to do this? FIXME are treated as errors during release

src/image_gen/pipelines.cpp

…s and demo review update

…ting

Copilot

Pull request overview

Adds OpenAI-compatible image edit support (image-to-image + inpainting/outpainting) to OVMS image generation, extends multipart parsing, and propagates Optimum export extra quantization parameters across tasks. This fits into the image generation REST API surface (/v3/images/*) and the HF pull/export flow.

Changes:

Add inpainting routing for /v3/images/edits when a mask multipart file is provided, including pipeline initialization and request serialization.
Extend multipart parser interface to support arrays of file parts; add unit tests for edits option validation (including mask).
Propagate extraQuantizationParams into additional Optimum export command builders and update related tests; refresh demo docs/assets.

Reviewed changes

Copilot reviewed 14 out of 21 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
src/test/text2image_test.cpp	Adds unit tests around `/v3/images/edits` request-option validation (mask accepted, unknown/rejected fields, defaults, bounds).
src/test/test_http_utils.hpp	Updates multipart parser mock with new `getFilesArrayByFieldName()` API.
src/test/pull_hf_model_test.cpp	Updates expected Optimum export commands to include extra quantization params; adds image-gen specific coverage.
src/pull_module/optimum_export.cpp	Appends `extraQuantizationParams` to additional Optimum export command builders (embeddings/TTS/STT/rerank/image-gen).
src/multi_part_parser.hpp	Extends multipart parser interface with `getFilesArrayByFieldName()`.
src/image_gen/pipelines.hpp	Adds inpainting pipeline + queue serialization guard for inpainting requests.
src/image_gen/pipelines.cpp	Builds pipeline chain with fallbacks and initializes single-slot queue for inpainting serialization.
src/image_gen/imagegenutils.cpp	Treats `mask` as an accepted multipart field for edits.
src/image_gen/http_image_gen_calculator.cc	Routes `/v3/images/edits` to inpainting when `mask` is present; adds pipeline availability checks and queue-guarded execution.
src/image_gen/BUILD	Adds dependency on `//src:libovms_queue` for inpainting serialization.
src/http_frontend/multi_part_parser_drogon_impl.hpp	Declares new `getFilesArrayByFieldName()` implementation.
src/http_frontend/multi_part_parser_drogon_impl.cpp	Implements `getFilesArrayByFieldName()` using Drogon parser’s files list.
demos/image_generation/outpaint_mask.png	Adds demo asset for outpainting mask example.
demos/image_generation/cat_mask.png	Adds demo asset for inpainting mask example.
demos/image_generation/README.md	Documents inpainting/outpainting usage and updates model-status endpoint example.
.gitignore	Adjusts Bazel output ignore entries.
.dockerignore	Expands ignored dev/build artifacts to reduce Docker build context size.

You can also share your feedback on Copilot code review. Take the survey.

src/image_gen/pipelines.hpp

+    InpaintingQueueGuard(Queue<int>& queue, std::chrono::seconds timeout) :
+        queue_(queue) {
+        auto future = queue_.getIdleStream();
+        if (future.wait_for(timeout) == std::future_status::ready) {
+            streamId_ = future.get();
+            acquired_ = true;
+        }
+    }


src/image_gen/pipelines.hpp

+    // Serializes concurrent inpainting requests (InpaintingPipeline lacks clone()).
+    // Queue size = 1: only one inpainting inference runs at a time.
+    std::unique_ptr<Queue<int>> inpaintingQueue;
+    static constexpr std::chrono::seconds DEFAULT_INPAINTING_TIMEOUT{300};  // 5 minutes
+


src/multi_part_parser.hpp

    // API for MP calculators to get the multipart file content by field name.
    // Returns empty string if file is not found.
    virtual std::string_view getFileContentByFieldName(const std::string& name) const = 0;

+    // API for MP calculators to get all file contents for a given array field name (e.g. "image[]").
+    virtual std::vector<std::string_view> getFilesArrayByFieldName(const std::string& name) const = 0;


src/image_gen/http_image_gen_calculator.cc

+                ov::Tensor maskTensor;
+                SPDLOG_LOGGER_DEBUG(llm_calculator_logger, "ImageGenCalculator [Node: {}] Inpainting: decoding mask tensor", cc->NodeName());
+                status = makeTensorFromString(std::string(mask.value()), maskTensor);
+                if (!status.ok()) {


…n, fix docs and includes

atobiszei added 2 commits March 2, 2026 16:28

WIP

3ddabdc

Inpainting/outpainting CPU

b5ed7d8

atobiszei added the WIP Do not merge until resolved label Mar 9, 2026

atobiszei added 3 commits March 11, 2026 11:31

Update dockerignore

c2ba9d8

Demo

af51476

Change mask

5f89fc2

atobiszei marked this pull request as ready for review March 13, 2026 14:12

Copilot AI review requested due to automatic review settings March 13, 2026 14:13

Copilot started reviewing on behalf of atobiszei March 13, 2026 14:13 View session

atobiszei removed the WIP Do not merge until resolved label Mar 13, 2026

Copilot AI reviewed Mar 13, 2026

View reviewed changes

atobiszei added 3 commits March 13, 2026 15:19

Fix

102f8af

Fix: remove mask from accepted fields in text2image request options

330374f

mask field should only be accepted in image edit (inpainting) requests, not in text-to-image generation requests.

Merge remote-tracking branch 'origin/main' into atobisze_image_inpain…

0a65e68

…ting

atobiszei requested a review from dkalinowski March 17, 2026 13:26

dkalinowski reviewed Mar 17, 2026

View reviewed changes

atobiszei commented Mar 17, 2026

View reviewed changes

dkalinowski reviewed Mar 17, 2026

View reviewed changes

src/image_gen/pipelines.cpp Show resolved Hide resolved

Fix concurrent request inpainting issue, propagate quantization param…

eac3932

…s and demo review update

atobiszei force-pushed the atobisze_image_inpainting branch from c818d8b to eac3932 Compare March 18, 2026 10:28

atobiszei added 2 commits March 18, 2026 12:45

Merge remote-tracking branch 'origin/main' into atobisze_image_inpain…

f4be12a

…ting

Add tests & review fix

a2476ee

atobiszei requested a review from Copilot March 18, 2026 13:43

Copilot started reviewing on behalf of atobiszei March 18, 2026 13:44 View session

Copilot AI reviewed Mar 18, 2026

View reviewed changes

Address PR review: blocking inpainting guard, string_view optimizatio…

15dd08a

…n, fix docs and includes

atobiszei requested a review from rasapala March 18, 2026 15:26

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Image inpainting, outpainting#4047

Image inpainting, outpainting#4047
atobiszei wants to merge 12 commits intomainfrom
atobisze_image_inpainting

atobiszei commented Mar 9, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dkalinowski Mar 17, 2026

Uh oh!

atobiszei Mar 17, 2026

Uh oh!

dkalinowski Mar 17, 2026

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants


		![cat](./cat.png) ![cat_mask](./cat_mask.png)

		Linux

Conversation

atobiszei commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dkalinowski Mar 17, 2026

Choose a reason for hiding this comment

Uh oh!

atobiszei Mar 17, 2026

Choose a reason for hiding this comment

Uh oh!

dkalinowski Mar 17, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

atobiszei commented Mar 9, 2026 •

edited

Loading