I am surprised that this problem does not show up more often. There are other ops where attributes are promoted to become inputs. The test-creation logic, however, has to explicitly indicate which are inputs, which are attributes etc. Eg. the Reduce* ops.

I think, in the end, users creating a test will probably need to specify actual versions, like in the case of Reshape. May be with other utilities like all_versions('Reshape', least_version=5)

@gramalingam The PR #5263 is trying to solve the problem to increase test coverage. Before the PR, onnx/test/shape_inference_test.py only covers the latest version of Ops. When developers introduced incompatible change, developers often deleted the old test code and only kept the latest test code. The PR introduced a scheme that requires test function to cover all versions of Ops. If incompatible changes are introduced, the developers need to add new test code and keep the old test code.

I'm now trying to solve the inference test for OpVersion <= 5. I fixed the Reshape creation problem. But I found that when OpVersion=1, the shape inference seems not inference the output shape(showed in following figure). I found that many Ops had the same problem when OpVersion==1. I don't know the root cause. Maybe some C++ code needs to be changed.