Optimize Inference Time and Memory usage #5745
Unanswered
Deepanshu-iiits-it
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi team, I have Keras models which are causing memory leakage on calling prediction method. To overcome this issue, we converted these keras models into onnx format using below function-
onnx_model,_ = tf2onnx.convert.from_keras(keras_model, opset=13)
It has reduced memory usage significantly, but increased Inference time. I tried with different opset values like 16, but that didn't seem to help.
Please propose few ways to reduce inference time for onnx models.
Thanks in advance!
Beta Was this translation helpful? Give feedback.
All reactions