Improved yolov6 to have the same trt model inference speed under int8 and fp16

[xiaoche-24]

Ask a Question

Question

I improved yolov6. After converting to tensorrt, the improved model's inference speed is faster than the original yolov6 under fp32 and fp16. However, after converting to int8, the speed of the original yolov6 has doubled (fp16:66fps -> int8:122fps ), and the speed of the improved yolov6 is (fp16: 100fps -> int8: 102fps). The speed is almost not improved. What is the reason? My improvement module includes split, concat, and DropPath operations. Onnx.opset is set to 12 because 13 will report an error.Because when using opset13 to convert tensorrt on NX, the following error will be reported:

[TensorRT] VERBOSE: ModelImporter.cpp:119: Searching for input: backbone.ERBlock_2.1.spatial_mixing.partial_conv3.0.weight
[TensorRT] VERBOSE: ModelImporter.cpp:125: Conv_6 [Conv] inputs: [input.8 -> (1, 32, 160, 160)], [backbone.ERBlock_2.1.spatial_mixing.partial_conv3.0.weight -> (16, 16, 3, 3)], 
[TensorRT] VERBOSE: builtin_op_importers.cpp:450: Convolution input dimensions: (1, 32, 160, 160)
ERROR: ONNX Parse Failed
In node -1 (importConv): INVALID_NODE: Assertion failed: nchan == -1 || kernelWeights.shape.d[1] * ngroup == nchan
ERROR: failed to build the TensorRT engine!

Further information

• Relevant Area:

• Is this issue related to a specific model?
  Model name:
  Model opset: 12

Notes

[xadupre]

Which runtime are you using? This issue should probably be moved there.

[justinchuby]

I would move the issue to https://github.com/NVIDIA/TensorRT for more help. Thanks!