How to pass environment variable to GitHub actions in ci.yml in a way that pipx run recognizes

[sgbaird]

I was expecting MP_API_KEY environment variable (stored as secret) to be recognized, but realized it probably has to do with using pipx and its virtual env behavior. Trying to get it working by removing the usage of pipx within the tests step.

...
      - name: Run tests
        env:
          MP_API_KEY: ${{ secrets.MP_API_KEY }}
        run: >-
          pipx run tox
          --installpkg '${{ needs.prepare.outputs.wheel-distribution }}'
          -- -rFEx --durations 10 --color yes
...

GitHub actions output:

Run pipx run tox --installpkg 'dist/mp_time_split-0.0.post1.dev21+g[8](https://github.com/sparks-baird/mp-time-split/runs/6717197397?check_suite_focus=true#step:5:9)662[9](https://github.com/sparks-baird/mp-time-split/runs/6717197397?check_suite_focus=true#step:5:10)dd-py3-none-any.whl' -- -rFEx --durations [10](https://github.com/sparks-baird/mp-time-split/runs/6717197397?check_suite_focus=true#step:5:11) --color yes
default create: /home/runner/work/mp-time-split/mp-time-split/.tox/default
default inst: /home/runner/work/mp-time-split/mp-time-split/.tox/.tmp/package/1/mp_time_split-0.0.post1.dev21+g86629dd-py3-none-any.whl
default installed: aioitertools==0.10.0,anyio==3.6.1,asttokens==2.0.5,attrs==21.4.0,Automat==20.2.0,backcall==0.2.0,bcrypt==3.2.2,boltons==21.0.0,boto3==1.24.2,botocore==1.27.2,bravado==[11](https://github.com/sparks-baird/mp-time-split/runs/6717197397?check_suite_focus=true#step:5:12).0.3,bravado-core==5.17.0,certifi==2022.5.18.1,cffi==1.15.0,charset-normalizer==2.0.[12](https://github.com/sparks-baird/mp-time-split/runs/6717197397?check_suite_focus=true#step:5:13),click==8.1.3,constantly==15.1.0,coverage==6.4.1,crochet==2.0.0,cryptography==37.0.2,cycler==0.11.0,decorator==5.1.1,dnspython==2.2.1,emmet-core==0.27.6,executing==0.8.3,fastapi==0.78.0,fido==4.2.2,filetype==1.0.[13](https://github.com/sparks-baird/mp-time-split/runs/6717197397?check_suite_focus=true#step:5:14),Flask==2.1.2,flatten-dict==0.4.2,fonttools==4.33.3,future==0.18.2,hyperlink==21.0.0,idna==3.3,importlib-metadata==4.11.4,incremental==21.3.0,inflect==5.6.0,iniconfig==1.1.1,ipython==8.4.0,itsdangerous==2.1.2,jedi==0.18.1,Jinja2==3.1.2,jmespath==1.0.0,joblib==1.1.0,json2html==1.3.0,jsonpointer==2.3,jsonref==0.2,jsonschema==3.2.0,kiwisolver==1.4.2,latexcodec==2.0.1,maggma==0.47.2,MarkupSafe==2.1.1,matminer==0.7.4,matplotlib==3.5.2,matplotlib-inline==0.1.3,mongogrant==0.3.3,mongomock==4.0.0,monotonic==1.6,monty==2022.4.26,mp-api==0.23.6,mp-time-split @ file:///home/runner/work/mp-time-split/mp-time-split/.tox/.tmp/package/1/mp_time_split-0.0.post1.dev21%2Bg86629dd-py3-none-any.whl,mpcontribs-client==4.2.10,mpmath==1.2.1,msgpack==1.0.3,networkx==2.8.2,numpy==1.22.4,orjson==3.6.9,packaging==21.3,palettable==3.3.0,pandas==1.4.2,paramiko==2.11.0,parso==0.8.3,pexpect==4.8.0,pickleshare==0.7.5,Pillow==9.1.1,Pint==0.19.2,plotly==5.8.0,pluggy==1.0.0,prompt-toolkit==3.0.29,ptyprocess==0.7.0,PubChemPy==1.0.4,pure-eval==0.2.2,py==1.11.0,pyasn1==0.4.8,pyasn1-modules==0.2.8,pybtex==0.24.0,pycparser==2.21,pydantic==1.8.2,pydash==5.1.0,Pygments==2.12.0,pyIsEmail==1.4.0,pymatgen==2022.5.26,pymongo==4.1.1,PyNaCl==1.5.0,pyOpenSSL==22.0.0,pyparsing==3.0.9,pyrsistent==0.18.1,pytest==7.1.2,pytest-cov==3.0.0,python-dateutil==2.8.2,pytz==2022.1,PyYAML==6.0,pyzmq==22.3.0,requests==2.27.1,requests-futures==1.0.0,rfc3987==1.3.8,robocrys==0.2.7,ruamel.yaml==0.17.21,ruamel.yaml.clib==0.2.6,s3transfer==0.6.0,scikit-learn==1.1.1,scipy==1.8.1,sentinels==1.0.0,service-identity==21.1.0,simplejson==3.17.6,six==1.16.0,sniffio==1.2.0,spglib==1.16.5,sshtunnel==0.4.0,stack-data==0.2.0,starlette==0.19.1,strict-rfc3339==0.7,swagger-spec-validator==2.7.3,sympy==1.10.1,tabulate==0.8.9,tenacity==8.0.1,threadpoolctl==3.1.0,tomli==2.0.1,tqdm==4.64.0,traitlets==5.2.2.post1,Twisted==22.4.0,typing_extensions==4.2.0,ujson==5.3.0,uncertainties==3.1.6,urllib3==1.26.9,wcwidth==0.2.5,webcolors==1.12,Werkzeug==2.1.2,wrapt==1.14.1,yelp-bytes==0.3.0,yelp-encodings==1.0.0,zipp==3.8.0,zope.interface==5.4.0
default run-test-pre: PYTHONHASHSEED='112770253'
default run-test: commands[0] | pytest -rFEx --durations 10 --color yes
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0 -- /home/runner/work/mp-time-split/mp-time-split/.tox/default/bin/python
cachedir: .tox/default/.pytest_cache
rootdir: /home/runner/work/mp-time-split/mp-time-split, configfile: setup.cfg, testpaths: tests
plugins: anyio-3.6.1, cov-3.0.0
collecting ... collected 0 items / 1 error

==================================== ERRORS ====================================
_________________ ERROR collecting tests/test_mp_time_split.py _________________
tests/test_mp_time_split.py:1: in <module>
    from mp_time_split.core import MPTimeSplit
.tox/default/lib/python3.8/site-packages/mp_time_split/core.py:269: in <module>
    df = retrieve_data(nsites=nsites, elements=elements, use_theoretical=use_theoretical)
.tox/default/lib/python3.8/site-packages/mp_time_split/core.py:226: in retrieve_data
    results = mpr.summary.search(
.tox/default/lib/python3.8/site-packages/mp_api/core/client.py:786: in search
    return self._get_all_documents(
.tox/default/lib/python3.8/site-packages/mp_api/core/client.py:835: in _get_all_documents
    results = self._query_resource(
.tox/default/lib/python3.8/site-packages/mp_api/core/client.py:288: in _query_resource
    data = self._submit_requests(
.tox/default/lib/python3.8/site-packages/mp_api/core/client.py:387: in _submit_requests
    initial_data_tuples = self._multi_thread(use_document_model, initial_params_list)
.tox/default/lib/python3.8/site-packages/mp_api/core/client.py:587: in _multi_thread
    data, subtotal = future.result()
/usr/lib/python3.8/concurrent/futures/_base.py:437: in result
    return self.__get_result()
/usr/lib/python3.8/concurrent/futures/_base.py:389: in __get_result
    raise self._exception
/usr/lib/python3.8/concurrent/futures/thread.py:57: in run
    result = self.fn(*self.args, **self.kwargs)
.tox/default/lib/python3.8/site-packages/mp_api/core/client.py:652: in _submit_request_and_process
    raise MPRestError(
E   mp_api.core.client.MPRestError: REST query returned with error status code 401 on URL https://api.materialsproject.org/summary/?nsites=1&nsites=4&elements=V&_limit=1000&_fields=structure%2Cmaterial_id%2Ctheoretical with message:
E   Response {
E     "message":"No API key found in request"
E   }
=============================== warnings summary ===============================
.tox/default/lib/python3.8/site-packages/past/builtins/misc.py:45
  the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses

.tox/default/lib/python3.8/site-packages/matminer/featurizers/site/fingerprint.py:32
.tox/default/lib/python3.8/site-packages/matminer/featurizers/site/fingerprint.py:34
  
  safe_load will be removed, use
  
    yaml=YAML(typ='safe', pure=True)
    yaml.load(...)
  
  instead

.tox/default/lib/python3.8/site-packages/ruamel/yaml/main.py:11[18](https://github.com/sparks-baird/mp-time-split/runs/6717197397?check_suite_focus=true#step:5:19)
.tox/default/lib/python3.8/site-packages/ruamel/yaml/main.py:1118
  
  load will be removed, use
  
    yaml=YAML(typ='unsafe', pure=True)
    yaml.load(...)
  
  instead

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

---------- coverage: platform linux, python 3.8.10-final-0 -----------
Name                            Stmts   Miss Branch BrPart  Cover   Missing
---------------------------------------------------------------------------
src/mp_time_split/__init__.py       6      0      0      0   100%
src/mp_time_split/core.py         100     68     40      3    25%   66-70, 105-127, 2[19](https://github.com/sparks-baird/mp-time-split/runs/6717197397?check_suite_focus=true#step:5:20)->225, 2[21](https://github.com/sparks-baird/mp-time-split/runs/6717197397?check_suite_focus=true#step:5:22), [22](https://github.com/sparks-baird/mp-time-split/runs/6717197397?check_suite_focus=true#step:5:23)3, 230-266, 271-359
---------------------------------------------------------------------------
TOTAL                             106     68     40      3    28%

=========================== short test summary info ============================
ERROR tests/test_mp_time_split.py - mp_api.core.client.MPRestError: REST quer...
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
======================== 5 warnings, 1 error in 11.[43](https://github.com/sparks-baird/mp-time-split/runs/6717197397?check_suite_focus=true#step:5:44)s =========================
ERROR: InvocationError for command /home/runner/work/mp-time-split/mp-time-split/.tox/default/bin/pytest -rFEx --durations 10 --color yes (exited with code 2)
___________________________________ summary ____________________________________
ERROR:   default: commands failed
Error: Process completed with exit code 1.

[sgbaird]

Fixed by modifying tox.ini:

[testenv]
description = Invoke pytest to run automated tests
setenv =
    TOXINIDIR = {toxinidir}
passenv =
    HOME
    MP_API_KEY

Was able to leave pipx run as is

[abravalheri]

Thank you very much for providing the example for the community.