Not able to see performance gain for the Optimized model?

[mwangms]

After export a PyTorch model, run optimizer to get an optimized model (), then run it on RTX8000, was unable to see if any performance gain.

The optimizing code is as below:
optimizer = ORTOptimizer.from_pretrained(model)
optimization_config = AutoOptimizationConfig.O4(disable_shape_inference=True)
optimizer.optimize(save_dir=onnx_optimized_output_path, optimization_config=optimization_config)

The code to run inference:
ort_session = ort.InferenceSession(onnx_optimized_model_path, providers='CUDAExecutionProvider')
outputs = self._inference_session.run(["logits"], {"input_ids": input_ids, "attention_mask": attention_mask})

Anyone can share some insights about what's going wrong? Is that specific GPU card required for this optimization? Thanks.