Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 0 additions & 17 deletions demos/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,8 @@ ovms_demos_continuous_batching_vlm
ovms_demos_image_generation
ovms_demo_clip_image_classification
ovms_demo_age_gender_guide
ovms_demo_horizontal_text_detection
ovms_demo_optical_character_recognition
ovms_demo_face_detection
ovms_demo_face_blur_pipeline
ovms_demo_capi_inference_demo
ovms_demo_single_face_analysis_pipeline
ovms_demo_multi_faces_analysis_pipeline
ovms_docs_demo_ensemble
ovms_docs_demo_mediapipe_image_classification
ovms_docs_demo_mediapipe_multi_model
Expand All @@ -32,7 +27,6 @@ ovms_docs_image_classification
ovms_demo_using_onnx_model
ovms_demo_tf_classification
ovms_demo_person_vehicle_bike_detection
ovms_demo_vehicle_analysis_pipeline
ovms_demo_real_time_stream_analysis
ovms_demo_using_paddlepaddle_model
ovms_demo_bert
Expand Down Expand Up @@ -92,17 +86,6 @@ Check out the list below to see complete step-by-step examples of using OpenVINO
|[Iris demo](./mediapipe/object_detection/README.md)| A pipeline implementing iris detection |
|[Holistic demo](./mediapipe/holistic_tracking/README.md)| A complex pipeline linking several image analytical models and image transformations |

## With DAG Pipelines
| Demo | Description |
|---|---|
|[Horizontal Text Detection in Real-Time](horizontal_text_detection/python/README.md) | Run prediction on camera stream using a horizontal text detection model via gRPC API. This demo uses [pipeline](../docs/dag_scheduler.md) with [horizontal_ocr custom node](https://github.com/openvinotoolkit/model_server/tree/main/src/custom_nodes/horizontal_ocr) and [demultiplexer](../docs/demultiplexing.md). |
|[Optical Character Recognition Pipeline](optical_character_recognition/python/README.md) | Run prediction on a JPEG image using a pipeline of text recognition and text detection models with a custom node for intermediate results processing via gRPC API. This demo uses [pipeline](../docs/dag_scheduler.md) with [east_ocr custom node](https://github.com/openvinotoolkit/model_server/tree/main/src/custom_nodes/east_ocr) and [demultiplexer](../docs/demultiplexing.md). |
|[Single Face Analysis Pipeline](single_face_analysis_pipeline/python/README.md)|Run prediction on a JPEG image using a simple pipeline of age-gender recognition and emotion recognition models via gRPC API to analyze image with a single face. This demo uses [pipeline](../docs/dag_scheduler.md) |
|[Multi Faces Analysis Pipeline](multi_faces_analysis_pipeline/python/README.md)|Run prediction on a JPEG image using a pipeline of age-gender recognition and emotion recognition models via gRPC API to extract multiple faces from the image and analyze all of them. This demo uses [pipeline](../docs/dag_scheduler.md) with [model_zoo_intel_object_detection custom node](https://github.com/openvinotoolkit/model_server/tree/main/src/custom_nodes/model_zoo_intel_object_detection) and [demultiplexer](../docs/demultiplexing.md) |
|[Model Ensemble Pipeline](model_ensemble/python/README.md)|Combine multiple image classification models into one [pipeline](../docs/dag_scheduler.md) and aggregate results to improve classification accuracy. |
|[Face Blur Pipeline](face_blur/python/README.md)|Detect faces and blur image using a pipeline of object detection models with a custom node for intermediate results processing via gRPC API. This demo uses [pipeline](../docs/dag_scheduler.md) with [face_blur custom node](https://github.com/openvinotoolkit/model_server/tree/main/src/custom_nodes/face_blur). |
|[Vehicle Analysis Pipeline](vehicle_analysis_pipeline/python/README.md)|Detect vehicles and recognize their attributes using a pipeline of vehicle detection and vehicle attributes recognition models with a custom node for intermediate results processing via gRPC API. This demo uses [pipeline](../docs/dag_scheduler.md) with [model_zoo_intel_object_detection custom node](https://github.com/openvinotoolkit/model_server/tree/main/src/custom_nodes/model_zoo_intel_object_detection). |

## With C++ Client
| Demo | Description |
|---|---|
Expand Down
5 changes: 0 additions & 5 deletions demos/image_classification/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,3 @@ ovms_demo_image_classification_go
| Demo | Description |
|---|---|
|[Image Classification](python/README.md)|Run prediction on a JPEG image using image classification model via gRPC API.|

## Go
| Demo | Description |
|---|---|
|[Image Classification](go/README.md)|Run prediction on a JPEG image using image classification model via gRPC API.|
1 change: 0 additions & 1 deletion demos/real_time_stream_analysis/python/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,6 @@ The helper class `StreamClient` supports using unary gRPC calls. In that case it
It sends the frames to the model server asynchronously but each of them is stateless and each request can be processed independently.
The key advantage of that mode is easier load balancing and scalability, because each request could be routed to a different instance of the model server or a different compute node.

Such use case with the unary calls with a horizontal text analysis can be followed based on [this document](../../horizontal_text_detection/python/README.md).


> **Note** Depending on the output format, there might be needed a custom postprocessing function implementation.
Expand Down
12 changes: 0 additions & 12 deletions docs/dag_scheduler.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ This guide gives information about:
* [Node Types](#node-types)
* [Configuration file](#configuration-file)
* [Using the pipelines](#using-pipelines)
* [Pipelines examples](#pipelines-examples)
* [Current Limitations](#current-limitations)


Expand Down Expand Up @@ -205,19 +204,8 @@ and [REST Model Ready](model_server_rest_api_kfs.md)
The only difference in using the pipelines and individual models is in version management. In all calls to the pipelines,
the version parameter is ignored. Pipelines are not versioned. Though, they can reference a particular version of the models in the graph.

## Pipelines Examples

[Single face analysis with combined models](../demos/single_face_analysis_pipeline/python/README.md)

[Multiple vehicles analysis using demultiplexer with model_zoo_object_detection example custom node](../demos/vehicle_analysis_pipeline/python/README.md)

[Optical Character Recognition pipeline with east_ocr example custom node](../demos/optical_character_recognition/python/README.md)

[Horizontal Text Detection pipeline with horizontal_ocr example custom node](../demos/horizontal_text_detection/python/README.md)

## Current limitations

- Models with "auto" [batch size](dynamic_bs_auto_reload.md) or [shape](dynamic_shape_auto_reload.md) cannot be referenced in pipeline
- Connected inputs and output for subsequent node models need to match each other in terms of data shape, precision and layout -
there is no automatic conversion between input/output model precisions or layouts. This limitation can be addressed with `--shape` and `--layout` model configuration or with a custom node to transform the data as required to match the expected data format.
- REST requests with no named format (JSON body with one unnamed input) are not supported
Expand Down
1 change: 0 additions & 1 deletion docs/demultiplexing.md
Original file line number Diff line number Diff line change
Expand Up @@ -176,7 +176,6 @@ Output: (N,1,1001)

Because of this additional dimension, demultiplexing implementation is generic and can support any input and output data layout.

Exemplary usage of this feature is available in [dynamic batch size guide](./dynamic_bs_demultiplexer.md).

*Note:* You can use additional parameters in model config ('nireq' and 'NUM_STREAMS') to fine-tune your performance for dynamic batch. 'NUM_STREAMS' allows for multiple parallel
inferences processing in OpenVINO™ which may increase throughput at the cost of latency of predict requests. 'nireq' specifies how many inference requests can be prepared.
Expand Down
3 changes: 0 additions & 3 deletions docs/dynamic_input.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,13 +21,10 @@ There are multiple ways of enabling dynamic inputs depending on the kind of serv

- [dynamic input shape with binary input format](./dynamic_shape_binary_inputs.md) - send data in binary format (JPEG or PNG encoded), so the Model Server will adjust the input during data decoding. Consider this option in case of slower networks to minimize amount of data transferred over the network and fit image size to the size accepted by endpoint.

- [**DEPRECATED**] [dynamic batch size with automatic model reloading](./dynamic_bs_auto_reload.md) - configure the Model Server to reload the model each time it receives a request with a batch size other than what is currently set. Consider using this option when request batch size may change, but usually stays the same. Each request with varying batch size will impact the performance due to model reloading.

- [**DEPRECATED**] [dynamic shape with automatic model reloading](./dynamic_shape_auto_reload.md) - configure the Model Server to reload the model each time the model receives a request with data in shape other than what is currently set. Consider using this option when request shape may change, but usually stays the same. Each request with varying shape will impact the performance due to model reloading.

**DAG Pipelines**:

- [dynamic batch size with a demultiplexer](./dynamic_bs_demultiplexer.md) - create a simple pipeline that splits data of any batch size and performs inference on each element in the batch separately. Consider using this option if incoming requests will be containing various batch size. This option does not need to reload underlying model, therefore there is no model reloading impact on the performance.

- [dynamic input shape with a custom node](./dynamic_shape_custom_node.md) - create a simple pipeline by pairing your model with a custom node that performs data preprocessing and provides the model with data in an acceptable shape. Consider this option if you want to fit the image into model shape by performing image resize operation before inference. This may affect accuracy.

Expand Down
2 changes: 1 addition & 1 deletion docs/shape_batch_size_and_layout.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ It accepts also a value `auto` - this command makes the served model set the bat
This feature is useful for sequential inference requests of the same batch size.

*Note:* In case of frequent batch size changes in predict requests, consider using [demultiplexing feature](./demultiplexing.md) from [Directed Acyclic Graph Scheduler](./dag_scheduler.md) which is more
performant in such situations because it is not adding an extra overhead with model reloading between requests like --batch_size auto setting. Exemplary usage of this feature can be found in [dynamic_batch_size](./dynamic_bs_demultiplexer.md) document.
performant in such situations because it is not adding an extra overhead with model reloading between requests like --batch_size auto setting.

- OpenVINO™ Model Server determines the batch size based on the size of the first dimension in the first input by default.
For example with the input shape (1, 3, 225, 225), the batch size is set to 1. With input shape (8, 3, 225, 225) the batch size is set to 8.
Expand Down
1 change: 0 additions & 1 deletion src/custom_nodes/east_ocr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ DAG pipeline.
Additionally to the detected text boxes, in the two additional outputs are returned their coordinates with information about geometry
and confidence levels for the filtered list of detections.

**NOTE** Exemplary [configuration file](https://github.com/openvinotoolkit/model_server/blob/main/demos/optical_character_recognition/python/config.json) is available in [optical character recognition demo](https://github.com/openvinotoolkit/model_server/blob/main/demos/optical_character_recognition/python/).

# Building custom node library

Expand Down
1 change: 0 additions & 1 deletion src/custom_nodes/face_blur/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@ All [OpenVINO Model Zoo](https://github.com/openvinotoolkit/open_model_zoo/tree/
- vehicle-license-plate-detection
- pedestrian-and-vehicle-detector

**NOTE** Exemplary [configuration file](https://github.com/openvinotoolkit/model_server/blob/main/demos/face_blur/python/config.json) is available in [face_blur demo](https://github.com/openvinotoolkit/model_server/blob/main/demos/face_blur/python/).

# Building custom node library

Expand Down
2 changes: 0 additions & 2 deletions src/custom_nodes/horizontal_ocr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,7 @@ Each image in the output will be resized to the predefined target size to fit th
DAG pipeline.
Additionally to the detected text boxes, in the two additional outputs are returned their coordinates and confidence levels.

This custom node can be used to process video frames via [camera example](../../../demos/horizontal_text_detection/python/README.md).

**NOTE** Exemplary [configuration file](https://github.com/openvinotoolkit/model_server/blob/main/demos/horizontal_text_detection/python/config.json) is available in [demo with camera](https://github.com/openvinotoolkit/model_server/blob/main/demos/horizontal_text_detection/python/).

# Building custom node library

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,6 @@ All [OpenVINO Model Zoo](https://github.com/openvinotoolkit/open_model_zoo/tree/
Public [OpenVINO Model Zoo](https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public) object detection models with output tensor shape: `[1, 1, 100, 7]`:
- ssdlite_mobilenet_v2

**NOTE** Exemplary configuration files are available in [vehicle analysis pipeline demo](https://github.com/openvinotoolkit/model_server/blob/main/demos/horizontal_text_detection/python/config.json) and [multiple faces analysis demo](https://github.com/openvinotoolkit/model_server/blob/main/demos/multi_faces_analysis_pipeline/python/config.json).

# Building custom node library

Expand Down