Fix spending over time data for animated map

[travissouthard]

Overview

These changes fix how spending over months for each state is calculated and adds a total spending amount to get the AnimatedTotalSpendingBucket to "drain" correctly. Before, the aggregated spending was per capita by state and the total it was subtracting from was the raw total in the BIL, meaning it didn't go down by very much at all.

There was also a bug in the aggregation math that was adding too little each month and thus the coloring on the map's states didn't match the above-the-fold map at the end of the animation.

Closes #104

Demo

Notes

Not sure if it's because of the particular start time we are using but the bucket starts at 418B instead of 550B

Testing Instructions

• In this branch delete src/app/sr/data/monthly.spending.json (This may be unnecessary but the fetch data script would skip if the file existed, unless there is a flag I didn't find, which, if so, do that instead)
• run ./scripts/fetch-data (I sometimes had to do this twice because of an exit with code 1, but no output?)
• run ./scripts/server and navigate to the homepage
• Scroll down and hit the play button on the map and ensure:
  • The app plays as it normally does
  • The bucket drains and matches the expected remaining amount at the end (268B)
  • At the end of the animation, the animated map has the same coloring as the top map

Checklist

• fixup! commits have been squashed
• CHANGELOG.md updated with summary of features or fixes, following Keep a Changelog guidelines
• README.md updated if necessary to reflect the changes
• CI passes after rebase

[github-actions]

🦙 MegaLinter status: ✅ SUCCESS

Descriptor	Linter	Files	Fixed	Errors	Elapsed time
✅ ACTION	actionlint	1		0	0.01s
✅ BASH	shfmt	11		0	0.01s
✅ JAVASCRIPT	eslint	45		0	3.55s
✅ JAVASCRIPT	prettier	45		0	2.47s
✅ JSON	eslint-plugin-jsonc	4		0	1.16s
✅ JSON	jsonlint	4		0	0.14s
✅ SPELL	misspell	90		0	0.21s
✅ TERRAFORM	terraform-fmt	5		0	0.31s
✅ YAML	yamllint	5		0	0.23s

See detailed report in MegaLinter reports

MegaLinter is graciously provided by

[jacobtylerwalls]

Looking good. Did you look into why the bucket starts at less than the total? There shouldn't be any infrastructure spending before the infrastructure bill was passed!

There was also a bug in the aggregation math that was adding too little each month and thus the coloring on the map's states didn't match the above-the-fold map at the end of the animation.

Also, do you mind referring me to where you made that change ☝️ ? I thought that was already addressed in #96.

Some cosmetic feedback at your option :-)

[jacobtylerwalls]

Suggested change

	sum += (stateResults && stateResults?.total) || 0;
	sum += stateResults?.total ?? 0;

[travissouthard]

Looking good. Did you look into why the bucket starts at less than the total? There shouldn't be any infrastructure spending before the infrastructure bill was passed!

It looks to be a data problem. The data we are seeing in the map is set to { "fiscal_year" : 2021, "month" : 1 }and in the monthly.spending.json the earliest I see money in Alaska is month 12, 2016:

{
    "per_capita": 0,
    "total": 0,
    "time_period": {
        "fiscal_year": 2016,
        "month": 11
    }
},
{
    "per_capita": 0.4374303699704051,
    "total": 320000,
    "time_period": {
        "fiscal_year": 2016,
        "month": 12
    }
},

Money in Idaho starts in month 4 of 2008:

{
    "per_capita": 0,
    "total": 0,
    "time_period": {
        "fiscal_year": 2008,
        "month": 3
    }
},
{
    "per_capita": 53.97577155279746,
    "total": 96458212.19,
    "time_period": {
        "fiscal_year": 2008,
        "month": 4
    }
},

On develop it's also showing this in Idaho:

{
    "per_capita": 53.97577155279746,
    "time_period": {
        "fiscal_year": 2008,
        "month": 4
    }
},
{
    "per_capita": 170.90665833083855,
    "time_period": {
        "fiscal_year": 2008,
        "month": 5
    }
},

And in Alaska ion develop:

{
    "per_capita": 0,
    "time_period": {
        "fiscal_year": 2016,
        "month": 11
    }
},
{
    "per_capita": 0.4374303699704051,
    "time_period": {
        "fiscal_year": 2016,
        "month": 12
    }
},

[travissouthard]

@jacobtylerwalls if we are seeing this on develop, is it worth spinning that out to its own card?

[jacobtylerwalls]

Yeah, that's wise. The spending codes "Z" and "1" are supposed to exclusively deal with BIL spending, so we need an investigation card to see if we botched something or if the API is returning bad results (or if a rogue agency submitted bad contract data, hah). Would you mind opening the card?

[travissouthard]

Not at all. Will do so first thing.

[jacobtylerwalls]

Follow-up card makes sense for the starting level in the bucket. Nice work.