New Zealand parser broke in new arrow

[jarek]

arrow library used to manipulate times has been upgraded, and apparently parsers/NZ.py has been depending on some removed functionality.

In Kibana:

for production:

  File "/home/src/electricitymap/contrib/parsers/NZ.py", line 61, in fetch_production
    datetime = arrow.get(obj['soPgenGraph']['timestamp']).datetime
arrow.parser.ParserError: Could not match input '1587818770' to any of the following formats: YYYY-MM-DD, YYYY-M-DD, YYYY-M-D, YYYY/MM/DD, YYYY/M/DD, YYYY/M/D, YYYY.MM.DD, YYYY.M.DD, YYYY.M.D, YYYYMMDD, YYYY-DDDD, YYYYDDDD, YYYY-MM, YYYY/MM, YYYY.MM, YYYY

for exchange:

  File "/home/src/electricitymap/contrib/parsers/NZ.py", line 121, in fetch_exchange
    datetime = datetime_start.replace(minutes=+item[0])
AttributeError: unknown attribute: "minutes"

[peterhughgriffin]

Seems to be two different problems.

The production problem was brought in with 0.15.0. The arrow.get() method no longer accepts string without a format string. This is really easily fixed, as described in the Arrow documentation:

[CHANGE] Timestamps of type str are no longer supported without a format string in the arrow.get() method. This change was made to support the ISO 8601 basic format and to address bugs such as #447.

The following will NOT work in v0.15.0:

>>> arrow.get("1565358758")
>>> arrow.get("1565358758.123413")

The following will work in v0.15.0:

>>> arrow.get("1565358758", "X")
>>> arrow.get("1565358758.123413", "X")
>>> arrow.get(1565358758)
>>> arrow.get(1565358758.123413)

There's a warning about this if you run in older versions:

ArrowParseWarning: The .get() parsing method with a format string will parse more strictly in version 0.15.0.See arrow-py/arrow#612 for more details.

The exchange problem is a little older, from 0.14.5:

[CHANGE] Removed deprecated replace shift functionality. Users looking to pass plural properties to the replace function to shift values should use shift instead.

Very easily solved too, just change replace to shift.

I'll get on with making a pull request for both of these.

[robertahunt]

fixed by #2420