You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We test inference on openvino 2023.0.1 and it works good for face recognition.
However, we switch to use openvino 2024.1.0 and it's first inference delayed about several minutes.
We found compile_model not compile full resource for inference, so first called infer() will take several minutes to prepare opencl related resource.
Step-by-step reproduction
No response
Relevant log output
No response
Issue submission checklist
I'm reporting an issue. It's not a question.
I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
There is reproducer code and related data files such as images, videos, models, etc.
The text was updated successfully, but these errors were encountered:
We found this issue may be caused by ov::set_batch.
hihapaddy
changed the title
[Bug]: First Inference on GPU device is slow for openvino 2024
[Bug]: First Inference on GPU device is slow when using ov::set_batch for openvino 2024
May 10, 2024
@ilya-lavrenov
We provide a jupyter notebook to test set_batch feature.
By the way, we found another issue that set_batch will also influence outputs of some model.
For example, the result of face-reidentification-retail-0095 on GPU and CPU only the same in first inference. Thus, the face recognition on GPU will not work when switching to openvino 2024.1.
OpenVINO Version
2024.1.0
Operating System
Ubuntu 20.04 (LTS)
Device used for inference
GPU
Framework
None
Model used
face-reidentification-retail-0095
Issue description
We test inference on openvino 2023.0.1 and it works good for face recognition.
However, we switch to use openvino 2024.1.0 and it's first inference delayed about several minutes.
We found compile_model not compile full resource for inference, so first called infer() will take several minutes to prepare opencl related resource.
Step-by-step reproduction
No response
Relevant log output
No response
Issue submission checklist
The text was updated successfully, but these errors were encountered: