Fix concurrent request inpainting issue and demo review update

- Add clone() for Text2ImagePipeline and Image2ImagePipeline to enable
  thread-safe concurrent requests
- Add InpaintingQueueGuard with Queue<int>(1) to serialize concurrent
  inpainting requests (InpaintingPipeline lacks clone())
- Propagate extraQuantizationParams to all optimum export functions
- Add mask acceptance test for image edits endpoint
- Fix Sphinx tab formatting in image generation README
- Remove CPU from dedicated inpainting deployment section in README

Author: atobiszei

demos/image_generation/README.md

@@ -370,24 +370,20 @@ ovms --rest_port 8000 ^
 
 Wait for the model to load. You can check the status with a simple command:
 ```console
-curl http://localhost:8000/v1/config
+curl http://localhost:8000/v3/models
 ```
 
 ```json
 {
- "OpenVINO/stable-diffusion-v1-5-int8-ov" :
- {
-  "model_version_status": [
-   {
-    "version": "1",
-    "state": "AVAILABLE",
-    "status": {
-     "error_code": "OK",
-     "error_message": "OK"
+  "object": "list",
+  "data": [
+    {
+      "id": "OpenVINO/stable-diffusion-v1-5-int8-ov",
+      "object": "model",
+      "created": 0,
+      "owned_by": "openvinotoolkit"
     }
-   }
   ]
- }
 }
 ```
 
@@ -399,7 +395,7 @@ A single servable exposes the following endpoints:
 - **Inpainting**: `images/edits` — multipart form with `image` + `mask` + `prompt`
 - **Outpainting**: `images/edits` — multipart form with `image` + `mask` + `prompt` (image placed on larger canvas, mask marks the area to fill)
 
-> **Note:** For inpainting/outpainting, dedicated inpainting models (e.g. `stable-diffusion-v1-5/stable-diffusion-inpainting`) only support the `images/edits` endpoint. Base models (e.g. `stable-diffusion-v1-5/stable-diffusion-v1-5`) support all endpoints.
+> **Note:** For inpainting/outpainting, dedicated inpainting models (e.g. `stable-diffusion-v1-5/stable-diffusion-inpainting`) only support the `images/edits` endpoint. Check [supported models](https://openvinotoolkit.github.io/openvino.genai/docs/supported-models/#image-generation-models).
 
 All requests are processed in unary format, with no streaming capabilities.
 
@@ -532,7 +528,9 @@ Inpainting replaces a masked region in an image based on the prompt. The `mask`
 
 ![cat](./cat.png) ![cat_mask](./cat_mask.png)
 
-Linux
+::::{tab-set}
+:::{tab-item} Linux
+:sync: linux
 ```bash
 curl http://localhost:8000/v3/images/edits \
   -F "model=diffusers/stable-diffusion-xl-1.0-inpainting-0.1" \
@@ -542,8 +540,10 @@ curl http://localhost:8000/v3/images/edits \
   -F "num_inference_steps=50" \
   -F "size=1024x1024" | jq -r '.data[0].b64_json' | base64 --decode > inpaint_output.png
 ```
+:::
 
-Windows Command Prompt
+:::{tab-item} Windows Command Prompt
+:sync: windows
 ```bat
 curl http://localhost:8000/v3/images/edits ^
   -F "model=diffusers/stable-diffusion-xl-1.0-inpainting-0.1" ^
@@ -553,6 +553,9 @@ curl http://localhost:8000/v3/images/edits ^
   -F "num_inference_steps=50" ^
   -F "size=1024x1024"
 ```
+:::
+
+::::
 
 Expected output (`inpaint_output.png`):
 
@@ -596,7 +599,9 @@ Outpainting extends an image beyond its original borders. Prepare two images:
 
 ![outpaint_input](./outpaint_input.png) ![outpaint_mask](./outpaint_mask.png)
 
-Linux
+::::{tab-set}
+:::{tab-item} Linux
+:sync: linux
 ```bash
 curl http://localhost:8000/v3/images/edits \
   -F "model=stable-diffusion-v1-5/stable-diffusion-inpainting" \
@@ -606,8 +611,10 @@ curl http://localhost:8000/v3/images/edits \
   -F "num_inference_steps=50" \
   -F "size=768x768" | jq -r '.data[0].b64_json' | base64 --decode > outpaint_output.png
 ```
+:::
 
-Windows Command Prompt
+:::{tab-item} Windows Command Prompt
+:sync: windows
 ```bat
 curl http://localhost:8000/v3/images/edits ^
   -F "model=stable-diffusion-v1-5/stable-diffusion-inpainting" ^
@@ -617,6 +624,9 @@ curl http://localhost:8000/v3/images/edits ^
   -F "num_inference_steps=50" ^
   -F "size=768x768"
 ```
+:::
+
+::::
 
 Expected output (`outpaint_output.png`):
 
@@ -665,22 +675,6 @@ For the full list see [supported image generation models](https://openvinotoolki
 > **Note:** Dedicated inpainting models only expose the `images/edits` endpoint (with mask). Text-to-image and image-to-image requests will return an error indicating the pipeline is not available for this model. Base models (e.g. `stable-diffusion-v1-5/stable-diffusion-v1-5`) support all endpoints including inpainting.
 
 ::::{tab-set}
-:::{tab-item} Docker (Linux) — CPU
-:sync: docker
-```bash
-mkdir -p models
-
-docker run -d --rm --user $(id -u):$(id -g) -p 8000:8000 -v $(pwd)/models:/models/:rw \
-  -e http_proxy=$http_proxy -e https_proxy=$https_proxy -e no_proxy=$no_proxy \
-  openvino/model_server:latest \
-    --rest_port 8000 \
-    --model_repository_path /models/ \
-    --task image_generation \
-    --source_model stable-diffusion-v1-5/stable-diffusion-inpainting \
-    --weight-format int8
-```
-:::
-
 :::{tab-item} Docker (Linux) — GPU
 :sync: docker-gpu
 ```bash
@@ -708,7 +702,8 @@ ovms --rest_port 8000 ^
   --model_repository_path ./models/ ^
   --task image_generation ^
   --source_model stable-diffusion-v1-5/stable-diffusion-inpainting ^
-  --weight-format int8
+  --weight-format int8 ^
+  --target_device GPU
 ```
 :::
 

src/image_gen/BUILD

@@ -24,6 +24,7 @@ ovms_cc_library(
     deps = [
         "imagegenpipelineargs",
         "//src:libovmslogging",
+        "//src:libovms_queue",
         "//src:libovmsstring_utils",
         "//third_party:genai",],
     visibility = ["//visibility:public"],

src/image_gen/http_image_gen_calculator.cc

@@ -179,10 +179,10 @@ class ImageGenCalculator : public CalculatorBase {
             SET_OR_RETURN(std::string, prompt, getPromptField(*payload.parsedJson));
             SET_OR_RETURN(ov::AnyMap, requestOptions, getImageGenerationRequestOptions(*payload.parsedJson, pipe->args));
 
-            // single request assumption - use pipeline instance directly
             if (!pipe->text2ImagePipeline)
                 return absl::FailedPreconditionError("Text-to-image pipeline is not available for this model");
-            auto status = generateTensor(*pipe->text2ImagePipeline, prompt, requestOptions, images);
+            auto t2i = pipe->text2ImagePipeline->clone();
+            auto status = generateTensor(t2i, prompt, requestOptions, images);
             if (!status.ok()) {
                 return status;
             }
@@ -218,13 +218,18 @@ class ImageGenCalculator : public CalculatorBase {
                 if (!status.ok()) {
                     return status;
                 }
-                SPDLOG_LOGGER_DEBUG(llm_calculator_logger, "ImageGenCalculator [Node: {}] Inpainting: mask tensor decoded, invoking generate()", cc->NodeName());
+                SPDLOG_LOGGER_DEBUG(llm_calculator_logger, "ImageGenCalculator [Node: {}] Inpainting: mask tensor decoded, acquiring inpainting queue slot", cc->NodeName());
+                InpaintingQueueGuard inpaintingGuard(*pipe->inpaintingQueue, ImageGenerationPipelines::DEFAULT_INPAINTING_TIMEOUT);
+                if (!inpaintingGuard.acquired()) {
+                    return absl::DeadlineExceededError("Inpainting pipeline is busy, request timed out");
+                }
+                SPDLOG_LOGGER_DEBUG(llm_calculator_logger, "ImageGenCalculator [Node: {}] Inpainting: queue slot acquired, invoking generate()", cc->NodeName());
                 status = generateTensorInpainting(*pipe->inpaintingPipeline, prompt, imageTensor, maskTensor, requestOptions, images);
             } else {
                 if (!pipe->image2ImagePipeline)
                     return absl::FailedPreconditionError("Image-to-image pipeline is not available for this model");
-                // image-to-image path - single pipeline instance, no clone needed
-                status = generateTensorImg2Img(*pipe->image2ImagePipeline, prompt, imageTensor, requestOptions, images);
+                auto i2i = pipe->image2ImagePipeline->clone();
+                status = generateTensorImg2Img(i2i, prompt, imageTensor, requestOptions, images);
             }
             if (!status.ok()) {
                 return status;

src/image_gen/pipelines.cpp

@@ -138,6 +138,12 @@ ImageGenerationPipelines::ImageGenerationPipelines(const ImageGenPipelineArgs& a
         throw std::runtime_error("Failed to create any image generation pipeline from: " + args.modelsPath);
     }
 
+    // InpaintingPipeline does not support clone(), so concurrent inpainting
+    // requests must be serialized.  Queue size = 1 acts as a mutex.
+    if (inpaintingPipeline) {
+        inpaintingQueue = std::make_unique<Queue<int>>(1);
+    }
+
     SPDLOG_INFO("Image Generation Pipelines ready — T2I: {} | I2I: {} | INP: {}",
         text2ImagePipeline ? "OK" : "N/A",
         image2ImagePipeline ? "OK" : "N/A",

src/image_gen/pipelines.hpp

@@ -15,22 +15,61 @@
 //*****************************************************************************
 #pragma once
 
+#include <chrono>
+#include <future>
 #include <memory>
+#include <optional>
 #include <string>
 
 #include <openvino/genai/image_generation/image2image_pipeline.hpp>
 #include <openvino/genai/image_generation/inpainting_pipeline.hpp>
 #include <openvino/genai/image_generation/text2image_pipeline.hpp>
 
 #include "imagegenpipelineargs.hpp"
+#include "src/queue.hpp"
 
 namespace ovms {
+
+// RAII guard that acquires a slot from a Queue<int>(1) on construction
+// and returns it on destruction, serializing concurrent inpainting requests.
+class InpaintingQueueGuard {
+public:
+    // Acquires a slot or sets acquired_ = false on timeout.
+    InpaintingQueueGuard(Queue<int>& queue, std::chrono::seconds timeout) :
+        queue_(queue) {
+        auto future = queue_.getIdleStream();
+        if (future.wait_for(timeout) == std::future_status::ready) {
+            streamId_ = future.get();
+            acquired_ = true;
+        }
+    }
+    ~InpaintingQueueGuard() {
+        if (acquired_) {
+            queue_.returnStream(streamId_);
+        }
+    }
+    bool acquired() const { return acquired_; }
+
+    InpaintingQueueGuard(const InpaintingQueueGuard&) = delete;
+    InpaintingQueueGuard& operator=(const InpaintingQueueGuard&) = delete;
+
+private:
+    Queue<int>& queue_;
+    int streamId_ = -1;
+    bool acquired_ = false;
+};
+
 struct ImageGenerationPipelines {
     std::unique_ptr<ov::genai::Image2ImagePipeline> image2ImagePipeline;
     std::unique_ptr<ov::genai::Text2ImagePipeline> text2ImagePipeline;
     std::unique_ptr<ov::genai::InpaintingPipeline> inpaintingPipeline;
     ImageGenPipelineArgs args;
 
+    // Serializes concurrent inpainting requests (InpaintingPipeline lacks clone()).
+    // Queue size = 1: only one inpainting inference runs at a time.
+    std::unique_ptr<Queue<int>> inpaintingQueue;
+    static constexpr std::chrono::seconds DEFAULT_INPAINTING_TIMEOUT{300};  // 5 minutes
+
     ImageGenerationPipelines() = delete;
     ImageGenerationPipelines(const ImageGenPipelineArgs& args);
     ImageGenerationPipelines(const ImageGenerationPipelines&) = delete;

src/pull_module/optimum_export.cpp

@@ -54,6 +54,9 @@ std::string OptimumDownloader::getExportCmdEmbeddings() {
     oss << "--disable-convert-tokenizer --task feature-extraction --library sentence_transformers";
     oss << " --model " << this->sourceModel << " --trust-remote-code ";
     oss << " --weight-format " << this->exportSettings.precision;
+    if (this->exportSettings.extraQuantizationParams.has_value()) {
+        oss << " " << this->exportSettings.extraQuantizationParams.value();
+    }
     oss << " " << this->downloadPath;
     // clang-format on
 
@@ -69,6 +72,9 @@ std::string OptimumDownloader::getExportCmdTextToSpeech() {
     }
     oss << "--model " << this->sourceModel << " --trust-remote-code ";
     oss << " --weight-format " << this->exportSettings.precision;
+    if (this->exportSettings.extraQuantizationParams.has_value()) {
+        oss << " " << this->exportSettings.extraQuantizationParams.value();
+    }
     oss << " " << this->downloadPath;
     // clang-format on
 
@@ -81,6 +87,9 @@ std::string OptimumDownloader::getExportCmdSpeechToText() {
     oss << this->OPTIMUM_CLI_EXPORT_COMMAND;
     oss << "--model " << this->sourceModel << " --trust-remote-code ";
     oss << " --weight-format " << this->exportSettings.precision;
+    if (this->exportSettings.extraQuantizationParams.has_value()) {
+        oss << " " << this->exportSettings.extraQuantizationParams.value();
+    }
     oss << " " << this->downloadPath;
     // clang-format on
 
@@ -95,6 +104,9 @@ std::string OptimumDownloader::getExportCmdRerank() {
     oss << " --trust-remote-code ";
     oss << " --weight-format " << this->exportSettings.precision;
     oss << " --task text-classification ";
+    if (this->exportSettings.extraQuantizationParams.has_value()) {
+        oss << " " << this->exportSettings.extraQuantizationParams.value();
+    }
     oss << " " << this->downloadPath;
     // clang-format on
 
@@ -109,7 +121,7 @@ std::string OptimumDownloader::getExportCmdImageGeneration() {
     oss << " --weight-format " << this->exportSettings.precision;
     if (this->exportSettings.extraQuantizationParams.has_value()) {
         oss << " " << this->exportSettings.extraQuantizationParams.value();
-    }  // TODO FIXME check if its not needed to propagate to other exports
+    }
     oss << " " << this->downloadPath;
     // clang-format on
 

src/test/pull_hf_model_test.cpp

@@ -345,7 +345,7 @@ class TestOptimumDownloaderSetup : public ::testing::Test {
         inHfSettings.sourceModel = "model/name";
         inHfSettings.downloadPath = "/path/to/Download";
         inHfSettings.exportSettings.precision = "fp64";
-        inHfSettings.exportSettings.extraQuantizationParams = "--param --param value";
+        inHfSettings.exportSettings.extraQuantizationParams = "--someOptimumParam --anotherOptParam value";
         inHfSettings.task = ovms::TEXT_GENERATION_GRAPH;
         inHfSettings.downloadType = ovms::OPTIMUM_CLI_DOWNLOAD;
 #ifdef _WIN32
@@ -371,7 +371,7 @@ class TestOptimumDownloaderSetupWithFile : public TestOptimumDownloaderSetup {
 TEST_F(TestOptimumDownloaderSetup, Methods) {
     std::unique_ptr<TestOptimumDownloader> optimumDownloader = std::make_unique<TestOptimumDownloader>(inHfSettings);
     std::string expectedPath = inHfSettings.downloadPath + "/" + inHfSettings.sourceModel;
-    std::string expectedCmd = "optimum-cli export openvino --model model/name --trust-remote-code  --weight-format fp64 --param --param value \\path\\to\\Download\\model\\name";
+    std::string expectedCmd = "optimum-cli export openvino --model model/name --trust-remote-code  --weight-format fp64 --someOptimumParam --anotherOptParam value \\path\\to\\Download\\model\\name";
     std::string expectedCmd2 = "convert_tokenizer model/name --with-detokenizer -o \\path\\to\\Download\\model\\name";
 #ifdef _WIN32
     std::replace(expectedPath.begin(), expectedPath.end(), '/', '\\');
@@ -388,7 +388,7 @@ TEST_F(TestOptimumDownloaderSetup, Methods) {
 TEST_F(TestOptimumDownloaderSetup, RerankExportCmd) {
     inHfSettings.task = ovms::RERANK_GRAPH;
     std::unique_ptr<TestOptimumDownloader> optimumDownloader = std::make_unique<TestOptimumDownloader>(inHfSettings);
-    std::string expectedCmd = "optimum-cli export openvino --disable-convert-tokenizer --model model/name --trust-remote-code  --weight-format fp64 --task text-classification  \\path\\to\\Download\\model\\name";
+    std::string expectedCmd = "optimum-cli export openvino --disable-convert-tokenizer --model model/name --trust-remote-code  --weight-format fp64 --task text-classification  --someOptimumParam --anotherOptParam value \\path\\to\\Download\\model\\name";
     std::string expectedCmd2 = "convert_tokenizer model/name -o \\path\\to\\Download\\model\\name";
 #ifdef __linux__
     std::replace(expectedCmd.begin(), expectedCmd.end(), '\\', '/');
@@ -401,7 +401,7 @@ TEST_F(TestOptimumDownloaderSetup, RerankExportCmd) {
 TEST_F(TestOptimumDownloaderSetup, ImageGenExportCmd) {
     inHfSettings.task = ovms::IMAGE_GENERATION_GRAPH;
     std::unique_ptr<TestOptimumDownloader> optimumDownloader = std::make_unique<TestOptimumDownloader>(inHfSettings);
-    std::string expectedCmd = "optimum-cli export openvino --model model/name --weight-format fp64 --param --param value \\path\\to\\Download\\model\\name";
+    std::string expectedCmd = "optimum-cli export openvino --model model/name --weight-format fp64 --someOptimumParam --anotherOptParam value \\path\\to\\Download\\model\\name";
     std::string expectedCmd2 = "";
 #ifdef __linux__
     std::replace(expectedCmd.begin(), expectedCmd.end(), '\\', '/');
@@ -426,7 +426,7 @@ TEST_F(TestOptimumDownloaderSetup, ImageGenExportCmdNoExtraParams) {
 TEST_F(TestOptimumDownloaderSetup, EmbeddingsExportCmd) {
     inHfSettings.task = ovms::EMBEDDINGS_GRAPH;
     std::unique_ptr<TestOptimumDownloader> optimumDownloader = std::make_unique<TestOptimumDownloader>(inHfSettings);
-    std::string expectedCmd = "optimum-cli export openvino --disable-convert-tokenizer --task feature-extraction --library sentence_transformers --model model/name --trust-remote-code  --weight-format fp64 \\path\\to\\Download\\model\\name";
+    std::string expectedCmd = "optimum-cli export openvino --disable-convert-tokenizer --task feature-extraction --library sentence_transformers --model model/name --trust-remote-code  --weight-format fp64 --someOptimumParam --anotherOptParam value \\path\\to\\Download\\model\\name";
     std::string expectedCmd2 = "convert_tokenizer model/name -o \\path\\to\\Download\\model\\name";
 #ifdef __linux__
     std::replace(expectedCmd.begin(), expectedCmd.end(), '\\', '/');
@@ -440,7 +440,7 @@ TEST_F(TestOptimumDownloaderSetup, TextToSpeechExportCmd) {
     inHfSettings.task = ovms::TEXT_TO_SPEECH_GRAPH;
     inHfSettings.exportSettings.vocoder = "microsoft/speecht5_hifigan";
     std::unique_ptr<TestOptimumDownloader> optimumDownloader = std::make_unique<TestOptimumDownloader>(inHfSettings);
-    std::string expectedCmd = "optimum-cli export openvino --model-kwargs \"{\"vocoder\": \"microsoft/speecht5_hifigan\"}\" --model model/name --trust-remote-code  --weight-format fp64 \\path\\to\\Download\\model\\name";
+    std::string expectedCmd = "optimum-cli export openvino --model-kwargs \"{\"vocoder\": \"microsoft/speecht5_hifigan\"}\" --model model/name --trust-remote-code  --weight-format fp64 --someOptimumParam --anotherOptParam value \\path\\to\\Download\\model\\name";
     std::string expectedCmd2 = "convert_tokenizer model/name -o \\path\\to\\Download\\model\\name";
 #ifdef __linux__
     std::replace(expectedCmd.begin(), expectedCmd.end(), '\\', '/');
@@ -453,7 +453,7 @@ TEST_F(TestOptimumDownloaderSetup, TextToSpeechExportCmd) {
 TEST_F(TestOptimumDownloaderSetup, SpeechToTextExportCmd) {
     inHfSettings.task = ovms::SPEECH_TO_TEXT_GRAPH;
     std::unique_ptr<TestOptimumDownloader> optimumDownloader = std::make_unique<TestOptimumDownloader>(inHfSettings);
-    std::string expectedCmd = "optimum-cli export openvino --model model/name --trust-remote-code  --weight-format fp64 \\path\\to\\Download\\model\\name";
+    std::string expectedCmd = "optimum-cli export openvino --model model/name --trust-remote-code  --weight-format fp64 --someOptimumParam --anotherOptParam value \\path\\to\\Download\\model\\name";
     std::string expectedCmd2 = "convert_tokenizer model/name -o \\path\\to\\Download\\model\\name";
 #ifdef __linux__
     std::replace(expectedCmd.begin(), expectedCmd.end(), '\\', '/');

src/test/text2image_test.cpp

@@ -797,6 +797,14 @@ TEST(Image2ImageTest, getImageEditRequestOptionsRejectedFields) {
     ASSERT_FALSE(std::holds_alternative<ov::AnyMap>(requestOptions));
 }
 
+TEST(Image2ImageTest, getImageEditRequestOptionsMaskAccepted) {
+    MockedMultiPartParser multipartParser;
+    ON_CALL(multipartParser, getFieldByName("prompt")).WillByDefault(Return("test prompt"));
+    ON_CALL(multipartParser, getAllFieldNames()).WillByDefault(Return(std::set<std::string>{"prompt", "mask"}));
+    auto requestOptions = ovms::getImageEditRequestOptions(multipartParser, DEFAULTIMAGE_GEN_ARGS);
+    ASSERT_TRUE(std::holds_alternative<ov::AnyMap>(requestOptions));
+}
+
 using mediapipe::CalculatorGraphConfig;
 using ovms::ImageGenPipelineArgs;
 TEST(ImageGenCalculatorOptionsTest, PositiveAllfields) {