Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SWY QFi sum is producing coarse results with unexpected value ranges #1541

Closed
dcdenu4 opened this issue Mar 21, 2024 · 3 comments · Fixed by #1550
Closed

SWY QFi sum is producing coarse results with unexpected value ranges #1541

dcdenu4 opened this issue Mar 21, 2024 · 3 comments · Fixed by #1550
Assignees
@dcdenu4
Labels
bug Something isn't working in progress This issue is actively being worked on
Milestone
3.14.2

Comments

@dcdenu4
Copy link
Member

dcdenu4 commented Mar 21, 2024

@jagoldstein and @nadinetrahan both reported seeing this bug. Jesse believes that a bug might have been introduced in InVEST 3.8.9, as earlier InVEST results seemed okay.

Jesse:

I noticed mine looks buggy in my latest run with v3.14.1. It doesn't match my ArcPro raster math result (summing the 12 monthly QFs... which looks good).
QF outputs from the previous times I used this model look good with earlier versions of InVEST, so I'm wondering if a bug was introduced. Otherwise, something must be off with my inputs.

the range of values is way too high too: 5.4e7 - 4.2e5 versus the range of sums I calculated 0 - 4156

[ QFi summed via ArcGIS | QFi summed from InVEST | QFi summed from InVEST]

qfi-sum-arc-good qfi-bug qfi-bug-coarse
@dcdenu4 dcdenu4 added the bug Something isn't working label Mar 21, 2024
@jagoldstein
Copy link
Member

Here's my input data that created the QF results shown above in purple: https://drive.google.com/file/d/1E5hkhbmqPbehosmxpKf3ygorI770wbd_/view?usp=sharing .

dcdenu4 added a commit to dcdenu4/invest that referenced this issue Mar 21, 2024
@dcdenu4
I. natcap#1541 Fix a pixelwise sum op for QF. Remove unneeded task de…
aa67a9c 
…pendency.
@dcdenu4 dcdenu4 self-assigned this Mar 22, 2024
@dcdenu4
Copy link
Member Author

dcdenu4 commented Mar 22, 2024

Working fix in my branch here: https://github.com/dcdenu4/invest/tree/bugfix/1541-swy-qfi-bad-sum

@dcdenu4 dcdenu4 added the in progress This issue is actively being worked on label Mar 22, 2024
@dcdenu4 dcdenu4 added this to the 3.14.2 milestone Mar 22, 2024
@dcdenu4
Copy link
Member Author

dcdenu4 commented Mar 22, 2024

I haven't looked into verifying when this bug might have been introduced, but the current problem was with a numpy.sum call used by pgp.raster_map. Without specifying the axis for the numpy.sum call, it was summing across the whole array for the block and not doing a pixelwise sum.

Here's a working build I shared with Jesse and others. Requires Stanford google drive access. https://drive.google.com/file/d/1-IkYgn856GvBwImJjHD1SZbAt7pAfkbn/view?usp=sharing

This was referenced Apr 3, 2024
Open
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dcdenu4 dcdenu4

Labels
bug Something isn't working in progress This issue is actively being worked on
Projects
None yet
Milestone
3.14.2
Development

Successfully merging a pull request may close this issue.

Fix a quickflow summation bug in SWY dcdenu4/invest
2 participants
@dcdenu4 @jagoldstein