As reported by #1360, SGVO should guarantee that at least the default settings produce a lossless optimization. On the other hand for the rest of the plugins, we should still require that the changes do not introduce visual differences that are too noticeable.

In #1357 @TrySound started the work for adding such regression testing using the W3C SVG 1.1 Test Suite. And @sk- has been working on a similar effort using a dataset taken from Wikimedia Commons.

I think we should have both a minimal dataset for doing some regression testing on CI (as the operation is rather expensive) an an extensive collection to be used offline before releasing. (To reduce the chance of errors we could instruct contributors to run that test suite on their machines before sending a PR, it could be a checkbox in the PR template).

In my testing the files from Wikimedia commons are quite a good source as they unveil quite a few errors. Also we should use the Oxygen icon them as at least svgcleaner claims svgo breaks many files.

For comparing the images I think reg-suit is a good alternative, as it integrates with github and has a visual ui, which makes it easy to see the differences (see these screenshots).