refactor(opennem) upgrade OPENNEM parser with event classes and replace arrow #6614
Conversation
github-actions
bot
added
parser
python
| "capacity": { | ||
| "coal": sum_vector(capacities, OPENNEM_PRODUCTION_CATEGORIES["coal"]), | ||
| "gas": sum_vector(capacities, OPENNEM_PRODUCTION_CATEGORIES["gas"]), | ||
| "oil": sum_vector(capacities, OPENNEM_PRODUCTION_CATEGORIES["oil"]), | ||
| "hydro": sum_vector(capacities, OPENNEM_PRODUCTION_CATEGORIES["hydro"]), | ||
| "wind": sum_vector(capacities, OPENNEM_PRODUCTION_CATEGORIES["wind"]), | ||
| "biomass": sum_vector( | ||
| capacities, OPENNEM_PRODUCTION_CATEGORIES["biomass"] | ||
| ), | ||
| "solar": sum_vector(capacities, OPENNEM_PRODUCTION_CATEGORIES["solar"]), | ||
| "hydro storage": capacities.get(OPENNEM_STORAGE_CATEGORIES["hydro"][0]), | ||
| "battery storage": capacities.get( | ||
| OPENNEM_STORAGE_CATEGORIES["battery"][0] | ||
| ), |
There was a problem hiding this comment.
question: @VIKTORVAV99 what is the recommended way to deal with this capacity data given that it cannot be captured in a ProductionMix?
I see references to OPENNEM.fetch_production_capacity in some configs but I can't seem to find it. Is that a function that I need to write or was it dynamically generated from something else?
There was a problem hiding this comment.
So this is a small mess right now and I don't really know why but the capacity parsers live in an entirely different folder.
And yes as you mention there is no way to currently return the live capacity data. Honestly I'd leave it as a draft for now until I have discussed with the team on how to handle this.
But I'll probably end up creating a capacity mix class or something similar so we can keep parsing this data.
There was a problem hiding this comment.
So I have been trying to figure out how to do this, what I thought was going to be pretty straight forward is not as the space in the "hydro storage" and "battery storage" modes. This means they can't be used as variables which is kind of needed for it to plug into the existing solution we have. All workarounds I have create more errors than I can count 😅
There is a TODO about removing that space and simplifying things a bit and I think I'll have to do that first before working on the "capacity mixes".
There was a problem hiding this comment.
No worries!! Thanks for the update
|
Just adding this as a suggestion here but it's by no means required. If you want you can split this PR up by removing arrow in one and migrating it to parser classes in the other while we figure out how to handle live capacities with the new system. PS: You can also add a PR that adds you as a contributor to all to zones you have helped upgrade. It's optional but you deserve the credit! |
Issue
#6011
#6135
Description
This PR upgrades the OPENNEM parser to use event classes, and drops usage of
arrowin favour of thedatetimelibrary.Double check
poetry run test_parser "zone_key"pnpx prettier@2 --write .andpoetry run formatin the top level directory to format my changes.