Some recently updated images are missing license_url in the meta_data field

[krysal]

Description

On 2024-05-08 UTC the batched_update DAG was triggered¹ to fill the license_url in the meta_data field with its corresponding value for rows WHERE license = 'by' AND license_version = '2.0', and it reported a successful end on 2024-05-09, 17:00:18 UTC updating 746,571 records. However, after triggering a run of the add_license_url DAG on 2024-05-10, it reported the same row number missing said license, which indicates that some workflows may not be filling this field or are overwriting it.

Flicker is confirmed to be on the set of rows missing this value.

SELECT source, provider, created_on, updated_on FROM image 
 WHERE license = 'by' AND license_version = '2.0' AND meta_data->>'license_url' IS NULL LIMIT 2;

+--------+----------+-------------------------------+-------------------------------+
| source | provider | created_on                    | updated_on                    |
|--------+----------+-------------------------------+-------------------------------|
| flickr | flickr   | 2020-04-28 07:20:32.183578+00 | 2024-05-12 03:13:46.696867+00 |
| flickr | flickr   | 2020-04-28 07:08:30.821693+00 | 2024-05-12 03:13:46.696867+00 |
+--------+----------+-------------------------------+-------------------------------+

If there are more, it is to be confirmed. It is known the Flickr DAG was running those days, as well as Europeana, the Finnish Museum, Wikimedia Commons, and the Metropolitan Museum.

Screenshot of DAG reports on Thursday, May 9th. Time is in VET.

Additional context

Discovered while working on #3885.

Footnotes

1. Link only available to maintainers. ↩