A problem converting a grib2 file to a netCDF file using cfgrib and xarray

[happymvd]

My test code -

import cfgrib
import xarray as xr

# Path to the GRIB2 file
grib_file = 'GFS_Wind_000.grib2'

# Path for the output netCDF file
netcdf_file = 'output.nc'

# Open the GRIB2 file using cfgrib
ds = cfgrib.open_dataset(grib_file)

# Convert the GRIB2 dataset to xarray dataset
xds = xr.Dataset.from_dataframe(ds.to_dataframe())

# Save the xarray dataset to netCDF format
xds.to_netcdf(netcdf_file)

this works in python from the terminal window, but when I compile the code using the following pyinstaller command line -

pyinstaller --onefile --copy-metadata numpy --collect-data gribapi TryGrib.py

and then run the exe I get the following error message -

Traceback (most recent call last):
  File "TryGrib.py", line 11, in <module>
    ds = cfgrib.open_dataset(grib_file)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "cfgrib\xarray_store.py", line 39, in open_dataset
  File "xarray\backends\api.py", line 557, in open_dataset
  File "xarray\backends\plugins.py", line 205, in get_backend
ValueError: unrecognized engine cfgrib must be one of: ['netcdf4', 'scipy', 'store']

any help will be greatly appreciated

[rokm]

Looks like plugins are registed via xarray.backends entry point, so that needs to be collected.

For now, try creating a custom hook that will override the one from pyinstaller-hooks-contrib; i.e., create an extra-hooks directory, and inside it, file called hook-xarray.py, with the following contents:

# hook-xarray.py
from PyInstaller.utils.hooks import copy_metadata, collect_entry_point

# Collect additional backend plugins
datas, hiddenimports = collect_entry_point('xarray.backends')

# `xarray` requires `numpy` metadata due to several calls to its `xarray.core.utils.module_available` with specified
# `minversion` argument, which end up calling `importlib.metadata.version`.
datas += copy_metadata('numpy')

Then add --additional-hooks-dir extra-hooks to your PyInstaller command.

[happymvd]

@rokm
Thank you for your quick response
I did the following -

1. created a subfolder called "extra-hooks"
2. in that folder I created a file with the name "hook-xarray.py" and I copied your code into this file
3. when I compiled I issued the following command
   pyinstaller --onefile --additional-hooks-dir extra-hooks TryGrib.py (the name of my short test script)
4. the TryGrib.exe was created in the \dist folder
5. when I run the TryGrib.exe I get the following message in the terminal window -
   (VE) PS C:\Data\GitHub\SearchWell\dist> .\TryGrib.exe
   Traceback (most recent call last):
   File "TryGrib.py", line 1, in
   File "", line 1176, in _find_and_load
   File "", line 1147, in _find_and_load_unlocked
   File "", line 690, in load_unlocked
   File "PyInstaller\loader\pyimod02_importers.py", line 419, in exec_module
   File "cfgrib_init.py", line 20, in
   File "", line 1176, in _find_and_load
   File "", line 1147, in _find_and_load_unlocked
   File "", line 690, in _load_unlocked
   File "PyInstaller\loader\pyimod02_importers.py", line 419, in exec_module
   File "cfgrib\cfmessage.py", line 29, in
   File "", line 1176, in _find_and_load
   File "", line 1147, in _find_and_load_unlocked
   File "", line 690, in _load_unlocked
   File "PyInstaller\loader\pyimod02_importers.py", line 419, in exec_module
   File "cfgrib\messages.py", line 28, in
   File "", line 1176, in _find_and_load
   File "", line 1147, in _find_and_load_unlocked
   File "", line 690, in load_unlocked
   File "PyInstaller\loader\pyimod02_importers.py", line 419, in exec_module
   File "eccodes_init.py", line 13, in
   File "", line 1176, in _find_and_load
   File "", line 1147, in _find_and_load_unlocked
   File "", line 690, in _load_unlocked
   File "PyInstaller\loader\pyimod02_importers.py", line 419, in exec_module
   File "eccodes\eccodes.py", line 12, in
   File "", line 1176, in _find_and_load
   File "", line 1147, in _find_and_load_unlocked
   File "", line 690, in load_unlocked
   File "PyInstaller\loader\pyimod02_importers.py", line 419, in exec_module
   File "gribapi_init.py", line 13, in
   File "", line 1176, in _find_and_load
   File "", line 1147, in _find_and_load_unlocked
   File "", line 690, in _load_unlocked
   File "PyInstaller\loader\pyimod02_importers.py", line 419, in exec_module
   File "gribapi\gribapi.py", line 34, in
   File "", line 1176, in _find_and_load
   File "", line 1147, in _find_and_load_unlocked
   File "", line 690, in _load_unlocked
   File "PyInstaller\loader\pyimod02_importers.py", line 419, in exec_module
   File "gribapi\errors.py", line 16, in
   File "", line 1176, in _find_and_load
   File "", line 1147, in _find_and_load_unlocked
   File "", line 690, in _load_unlocked
   File "PyInstaller\loader\pyimod02_importers.py", line 419, in exec_module
   File "gribapi\bindings.py", line 41, in
   File "pkgutil.py", line 640, in get_data
   File "PyInstaller\loader\pyimod02_importers.py", line 228, in get_data
   FileNotFoundError: [Errno 2] No such file or directory: 'C:\Users\Dell\AppData\Local\Temp\_MEI152482\gribapi\grib_api.h'
   [13404] Failed to execute script 'TryGrib' due to unhandled exception!

[rokm]

You omitted --collect-data gribapi that you had in your original PyInstaller command; you need that to collect that gribapi\grib_api.h file. So:

pyinstaller --onefile --additional-hooks-dir extra-hooks TryGrib.py --collect-data gribapi

[happymvd]

@rokm
With great gratitude, I salute you and your knowledge.
I have spent many hours on this ...... thank you very very much