Unhandled gdal error silently failing HQ model

[dcdenu4]

A user posted about experiencing an error with HQ in 3.14.0. The logfiles revelead:

2023-11-30 19:02:22,959 (natcap.invest.habitat_quality) habitat_quality.execute(471) INFO Aligning, resizing, and reprojecting raster inputs to that of the current land cover.
2023-11-30 19:02:23,146 (pygeoprocessing.geoprocessing) geoprocessing.align_and_resize_raster_stack(996) INFO 1 of 5 aligned: LULC_reclass_habitat_pic_aligned_pic_New.tif
2023-11-30 19:02:23,214 (pygeoprocessing.geoprocessing) geoprocessing.align_and_resize_raster_stack(996) INFO 2 of 5 aligned: AGRI_aligned_pic_New.tif
2023-11-30 19:02:23,218 (osgeo) utils._log_gdal_errors(97) ERROR [errno 6] Cannot find coordinate operations from `ENGCRS["NAD83 / Quebec Lambert",EDATUM[""],CS[Cartesian,2],AXIS["(E)",east,ORDER[1],LENGTHUNIT["metre",1,ID["EPSG",9001]]],AXIS["(N)",north,ORDER[2],LENGTHUNIT["metre",1,ID["EPSG",9001]]]]' to `EPSG:32198'
2023-11-30 19:02:23,218 (pygeoprocessing.geoprocessing) geoprocessing.align_and_resize_raster_stack(996) INFO 3 of 5 aligned: ROAD_aligned_pic_New.tif
2023-11-30 19:02:23,275 (pygeoprocessing.geoprocessing) geoprocessing.align_and_resize_raster_stack(996) INFO 4 of 5 aligned: BUILT_aligned_pic_New.tif
2023-11-30 19:02:23,334 (pygeoprocessing.geoprocessing) geoprocessing.align_and_resize_raster_stack(996) INFO 5 of 5 aligned: EDGE_aligned_pic_New.tif
2023-11-30 19:02:23,335 (pygeoprocessing.geoprocessing) geoprocessing.align_and_resize_raster_stack(1000) INFO aligned all 5 rasters.
2023-11-30 19:02:23,337 (taskgraph.Task) Task.add_task(706) ERROR Something went wrong when adding task align_input_rasters (4), terminating taskgraph.
Traceback (most recent call last):
  File "taskgraph/Task.py", line 674, in add_task
  File "taskgraph/Task.py", line 1109, in _call
RuntimeError: In Task: align_input_rasters (4)
Missing expected target path results.

I'm not sure I've ever seen this before, but it looks like the aligned step succeeded when in fact, I don't think that aligned output ever gets created. It would be great to better handle an error message like this.

[dcdenu4]

Similar issue popping up in NDR with the same user: https://community.naturalcapitalproject.org/t/ndr-output-version-3-10-2-vs-3-14-0/4351/2

According to their logs, this does not happen when running 3.10.2 with the same data.

2023-11-30 20:07:06,685 (pygeoprocessing.geoprocessing) geoprocessing.align_and_resize_raster_stack(1000) INFO aligned all 3 rasters.
2023-11-30 20:07:06,973 (osgeo) utils._log_gdal_errors(97) WARNING [errno 1] Too few points in geometry component at or near point 931.63784461152864 -814.10501253132861
2023-11-30 20:07:06,974 (osgeo) utils._log_gdal_errors(97) ERROR [errno 1] Cutline polygon is invalid.
2023-11-30 20:07:07,224 (osgeo) utils._log_gdal_errors(97) WARNING [errno 1] Too few points in geometry component at or near point 931.63784461152864 -814.10501253132861
2023-11-30 20:07:07,224 (osgeo) utils._log_gdal_errors(97) ERROR [errno 1] Cutline polygon is invalid.
2023-11-30 20:07:07,419 (osgeo) utils._log_gdal_errors(97) WARNING [errno 1] Too few points in geometry component at or near point 931.63784461152864 -814.10501253132861
2023-11-30 20:07:07,419 (osgeo) utils._log_gdal_errors(97) ERROR [errno 1] Cutline polygon is invalid.

[emlys]

I think this situation would be handled better if we enable gdal.UseExceptions(). Then the GDAL error would be raised as a python error and stop execution where the issue happens. Rather than just logging the error message and continuing until another problem results from it downstream.