Global element versioning

[AntonKhorev]

Problem

Sometimes the code tries to reproduce the sequence of element modifications but the data to do this is not directly available.

Changeset downloads rely primarily on timestamps, and then additionally sort by version, type etc and it's done differently in osm-website and cgimap. This additional ordering may not be correct. For example, it makes sense to reverse ordering by type for deletions. Relying on the additional ordering becomes necessary if timestamps are truncated, and that may happen if the data is restored from a dump.

Changeset webpages show lists elements. Those also reproduce the modification sequence in most of the cases but that happens as a side-effect of them not being sorted. This lack of sorting causes problems elsewhere and so does assuming that the elements are not sorted, see #4571.

Description

Is it feasible to do this thing?

openstreetmap-website/app/controllers/api/changesets_controller.rb

Lines 154 to 157 in 0c4cbda

	# sort the elements by timestamp and version number, as this is the
	# almost sensible ordering available. this would be much nicer if
	# global (SVN-style) versioning were used - then that would be
	# unambiguous.

1. Create one sequence to serve three old element tables.
2. Start it at a large enough number.
3. Add a column to each old element table with a default value taken from this sequence. This should work with db clients that don't yet know about this column but insert elements.
4. Backfill the column according to this order. It's not necessarily correct, maybe update it later.
5. Use this global version column for ordering (old) elements inside changesets on osm website and in other places.

Screenshots

No response

[mmd-osm]

Backfill the column according to this order. It's not necessarily correct, maybe update it later.

I'm not sure how feasible this is from a total runtime point of view. In any case, mass updating old element tables is probably bad news for our minutely diff replication, since it would end up generating billions of entries in our replication log files.

I haven't tested it, but I believe it would bring minutely diff replication to a standstill for quite some time. The original use case for old element updates was to support redaction, which is rather low data volume. Also, we're not using this information at the moment.

By the way, I'm well aware that the additional sorting in both implementations would not necessarily match the original upload sequence. However, I never really had the expectation that it should.