Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fixed OpenVINO gemm layer #25518

Merged
merged 6 commits into from May 14, 2024
Merged

Fixed OpenVINO gemm layer #25518

merged 6 commits into from May 14, 2024

Conversation

alexlyulkov
Copy link
Contributor

Fixed OpenVINO gemm layer
The problem was that our layer didn't properly handle all the possible gemm options in OpenVINO mode
Fixes #25472

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
  • The PR is proposed to the proper branch
  • There is a reference to the original bug report and related work
  • There is accuracy test, performance test and test data in opencv_extra repository, if applicable
    Patch to opencv_extra has the same branch name.
  • The feature is well documented and sample code can be built with the project CMake

@asmorkalov asmorkalov added this to the 4.10.0 milestone May 1, 2024
@asmorkalov
Copy link
Contributor

The related ONNX conformance test was skipped in Abduragim's PR. Please enable it back.

@dkurt
Copy link
Member

dkurt commented May 2, 2024

Please change a target branch to 4.x

@asmorkalov asmorkalov added the pr: needs rebase Rebase patch (and squash fixup commits) on the top of target branch label May 2, 2024
@alexlyulkov alexlyulkov changed the base branch from 5.x to 4.x May 3, 2024 07:10
Alexander Lyulkov added 2 commits May 3, 2024 13:15
@asmorkalov asmorkalov removed the pr: needs rebase Rebase patch (and squash fixup commits) on the top of target branch label May 3, 2024
@asmorkalov
Copy link
Contributor

The PR brings several test regressions with OpenVINO 2022.1.1:

[  FAILED  ] 26 tests, listed below:
[  FAILED  ] Test_Model.Keypoints_face/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_Model.TextRecognitionWithCTCPrefixBeamSearch/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_layers.Linear/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_layers.Multiplication/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_layers.Constant/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_layers.LinearWithConstant/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_layers.MatmulWithTwoInputs/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.Alexnet/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.Googlenet/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.CaffeNet/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.RCNN_ILSVRC13/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.ZFNet/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.ResNet18v1/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.ResNet50v1/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.MobileNet_v2_FP16/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.LResNet100E_IR/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.Inception_v2/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.Inception_v1/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.Shufflenet/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.Resnet34_kinetics/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.ViT_B_32/0, where GetParam() = NGRAPH/CPU

@asmorkalov
Copy link
Contributor

Regressions with OpenVINO 2024.0:

[  FAILED  ] Test_Int8_nets.Inception_v2/1, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_Int8_nets.Shufflenet/1, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_Model.Keypoints_face/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_Model.TextRecognition/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_Model.TextRecognitionWithCTCPrefixBeamSearch/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_layers.Linear/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_layers.Multiplication/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_layers.Constant/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_layers.LinearWithConstant/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_layers.MatmulWithTwoInputs/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.Alexnet/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.Googlenet/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.CaffeNet/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.RCNN_ILSVRC13/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.ZFNet/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.ResNet18v1/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.ResNet50v1/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.MobileNet_v2_FP16/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.LResNet100E_IR/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.Inception_v2/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.Inception_v1/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.Shufflenet/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.Resnet34_kinetics/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.ViT_B_32/0, where GetParam() = NGRAPH/CPU

@asmorkalov
Copy link
Contributor

Failure example:

[----------] 1 test from Test_ONNX_nets
[ RUN      ] Test_ONNX_nets.Alexnet/0, where GetParam() = NGRAPH/CPU
unknown file: Failure
C++ exception with description "Check 'TRShape::broadcast_merge_into(output_shape, input_shapes[1], autob)' failed at src/core/shape_inference/include/eltwise_shape_inference.hpp:28:
While validating node 'opset1::Add Add_51 (opset1::MatMul MatMul_49[0]:f32[4096,1000], opset1::Constant Constant_50[0]:f32[1000,1]) -> (dynamic[...])' with friendly_name 'Add_51':
Argument shapes are inconsistent.
" thrown in the test body.
[  FAILED  ] Test_ONNX_nets.Alexnet/0, where GetParam() = NGRAPH/CPU (705 ms)
[----------] 1 test from Test_ONNX_nets (705 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (705 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] Test_ONNX_nets.Alexnet/0, where GetParam() = NGRAPH/CPU

asmorkalov
asmorkalov reviewed May 13, 2024
View reviewed changes
modules/dnn/test/test_onnx_conformance_layer_parser_denylist.inl.hpp
Comment on lines -124 to -125
"test_gemm_default_matrix_bias",
"test_gemm_default_no_bias",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove.

asmorkalov
asmorkalov approved these changes May 13, 2024
View reviewed changes
Copy link
Contributor

@asmorkalov asmorkalov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@asmorkalov asmorkalov self-assigned this May 14, 2024
@asmorkalov asmorkalov merged commit 03507e0 into opencv:4.x May 14, 2024
27 of 28 checks passed
klatism pushed a commit to klatism/opencv that referenced this pull request May 17, 2024
@alexlyulkov @klatism
Merge pull request opencv#25518 from alexlyulkov:al/fixed-gemm-openvino
1746e55 
Fixed OpenVINO gemm layer opencv#25518

Fixed OpenVINO gemm layer
The problem was that our layer didn't properly handle all the possible gemm options in OpenVINO mode
Fixes opencv#25472

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@fengyuentau fengyuentau fengyuentau left review comments

@asmorkalov asmorkalov asmorkalov approved these changes

@dkurt dkurt Awaiting requested review from dkurt

Assignees

@asmorkalov asmorkalov

Labels
category: dnn feature
Projects
None yet
Milestone
4.10.0
Development

Successfully merging this pull request may close these issues.

Test_ONNX_conformance.Layer_Test/test_gemm_default_matrix_bias_NGRAPH_CPU crashes
4 participants
@alexlyulkov @asmorkalov @dkurt @fengyuentau