Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added int support for OpenVINO dnn backend #25458

Merged
merged 6 commits into from May 15, 2024

Conversation

alexlyulkov
Copy link
Contributor

@alexlyulkov alexlyulkov commented Apr 19, 2024

Modified dnn OpenVINO integration to support type inference and int operations.

Added OpenVINO support to Cast, CumSum, Expand, Gather, GatherElements, Scatter, ScatterND, Tile layers.
I tried to add Reduce layer, but looks like OpenVINO uses float values inside Reduce operation so it can't pass our int tests.

OpenVINO uses int32 precision for int64 operations, so I've modified input values for int64 tests when backend is OpenVINO.

OpenVINO has a strange behavior with custom layers and int64 values. After model compilation OpenVINO may change types, so the model can have different output type. That's why these tests were disabled:

  • Test_ArgMax_Int.random/0, where GetParam() = (4, NGRAPH/CPU)
  • Test_ArgMax_Int.random/6, where GetParam() = (11, NGRAPH/CPU)
  • Test_Reduce_Int.random/6, where GetParam() = (11, NGRAPH/CPU)
  • Test_Reduce_Int.two_axes/6, where GetParam() = (11, NGRAPH/CPU)

Also these tests were temporary disabled, they didn't work on both 4.x and 5.x branches:

  • Test_Caffe_layers.layer_prelu_fc/0, where GetParam() = NGRAPH/CPU
  • Test_ONNX_layers.LSTM_Activations/0, where GetParam() = NGRAPH/CPU
  • Test_ONNX_layers.Quantized_Convolution/0, where GetParam() = NGRAPH/CPU
  • Test_ONNX_layers.Quantized_Eltwise_Scalar/0, where GetParam() = NGRAPH/CPU
  • Test_TFLite.EfficientDet_int8/0, where GetParam() = NGRAPH/CPU

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
  • The PR is proposed to the proper branch
  • There is a reference to the original bug report and related work
  • There is accuracy test, performance test and test data in opencv_extra repository, if applicable
    Patch to opencv_extra has the same branch name.
  • The feature is well documented and sample code can be built with the project CMake

@asmorkalov asmorkalov added this to the 5.0 milestone Apr 19, 2024
@alexlyulkov alexlyulkov marked this pull request as ready for review April 25, 2024 16:52
modules/dnn/src/net_impl.cpp Outdated Show resolved Hide resolved
@asmorkalov
Copy link
Contributor

[ RUN      ] Test_ONNX_conformance.Layer_Test/test_scatter_elements_with_negative_indices_NGRAPH_CPU, where GetParam() = (test_scatter_elements_with_negative_indices, NGRAPH/CPU)
Exception during net.forward() call!
unknown file: Failure
C++ exception with description "Check 'false' failed at src/inference/src/infer_request.cpp:215:
ScatterElementsUpdate node with name 'y' have indices value that points to non-existing output tensor element
" thrown in the test body.
[  FAILED  ] Test_ONNX_conformance.Layer_Test/test_scatter_elements_with_negative_indices_NGRAPH_CPU, where GetParam() = (test_scatter_elements_with_negative_indices, NGRAPH/CPU) (10 ms)

@asmorkalov
Copy link
Contributor

Other failures are related to CI. I'll fix it and return back today.

@dkurt dkurt added the backport is needed Label for maintainers. Authors of PR can ignore this label Apr 26, 2024
@@ -210,6 +210,9 @@ TEST_P(Test_TFLite, max_unpooling)
}

TEST_P(Test_TFLite, EfficientDet_int8) {
if (backend == DNN_BACKEND_INFERENCE_ENGINE_NGRAPH)
applyTestTag(CV_TEST_TAG_DNN_SKIP_IE_NGRAPH); // TODO: fix this test for OpenVINO
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did test work before the patch?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test doesn't work in both 4.x and 5.x branches. Probably because I use the newest OpenVINO version

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test definitely worked on 2023 release of OpenVINO. I will check 2024 version. I think ok to skip on this PR.

@asmorkalov asmorkalov added port/backport done Label for maintainers. Authors of PR can ignore this and removed backport is needed Label for maintainers. Authors of PR can ignore this labels May 13, 2024
@asmorkalov
Copy link
Contributor

Backport: #25524

@asmorkalov
Copy link
Contributor

@dkurt could you take a look again?

@asmorkalov asmorkalov assigned asmorkalov and unassigned dkurt May 15, 2024
@asmorkalov asmorkalov merged commit 6af0394 into opencv:5.x May 15, 2024
24 of 25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category: dnn port/backport done Label for maintainers. Authors of PR can ignore this
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

None yet

3 participants