bump aiobotocore to 1.4.2

[raybellwaves]

bumped aiobotocore to 1.4.2 and updated the ci to point to a yml file (with aiobotocore==1.4.2).

In discovery of #553 when working on a VM I got a botocore.exceptions.NoCredentialsError: Unable to locate credentials when I installed s3fs using conda. The tail of the Trackback looks like:

  File "/opt/userenvs/ray.bell/test_env2/lib/python3.9/site-packages/fsspec/asyn.py", line 91, in wrapper
    return sync(self.loop, func, *args, **kwargs)
  File "/opt/userenvs/ray.bell/test_env2/lib/python3.9/site-packages/fsspec/asyn.py", line 71, in sync
    raise return_result
  File "/opt/userenvs/ray.bell/test_env2/lib/python3.9/site-packages/fsspec/asyn.py", line 25, in _runner
    result[0] = await coro
  File "/opt/userenvs/ray.bell/test_env2/lib/python3.9/site-packages/s3fs/core.py", line 1128, in _isdir
    return bool(await self._lsdir(path))
  File "/opt/userenvs/ray.bell/test_env2/lib/python3.9/site-packages/s3fs/core.py", line 580, in _lsdir
    async for i in it:
  File "/opt/userenvs/ray.bell/test_env2/lib/python3.9/site-packages/aiobotocore/paginate.py", line 32, in __anext__
    response = await self._make_request(current_kwargs)
  File "/opt/userenvs/ray.bell/test_env2/lib/python3.9/site-packages/aiobotocore/client.py", line 141, in _make_api_call
    http, parsed_response = await self._make_request(
  File "/opt/userenvs/ray.bell/test_env2/lib/python3.9/site-packages/aiobotocore/client.py", line 161, in _make_request
    return await self._endpoint.make_request(operation_model, request_dict)
  File "/opt/userenvs/ray.bell/test_env2/lib/python3.9/site-packages/aiobotocore/endpoint.py", line 77, in _send_request
    request = await self.create_request(request_dict, operation_model)
  File "/opt/userenvs/ray.bell/test_env2/lib/python3.9/site-packages/aiobotocore/endpoint.py", line 70, in create_request
    await self._event_emitter.emit(event_name, request=request,
  File "/opt/userenvs/ray.bell/test_env2/lib/python3.9/site-packages/aiobotocore/hooks.py", line 27, in _emit
    response = await handler(**kwargs)
  File "/opt/userenvs/ray.bell/test_env2/lib/python3.9/site-packages/aiobotocore/signers.py", line 16, in handler
    return await self.sign(operation_name, request)
  File "/opt/userenvs/ray.bell/test_env2/lib/python3.9/site-packages/aiobotocore/signers.py", line 63, in sign
    auth.add_auth(request)
  File "/opt/userenvs/ray.bell/test_env2/lib/python3.9/site-packages/botocore/auth.py", line 373, in add_auth
    raise NoCredentialsError()

However, when installing from pip which grabbed aiobotocore 1.4.2 I was able to do pd.read_parquet("s3://bucket/file.parquet"). This solves my use case and I hope doesn't break anything for others.

[martindurant]

With this change, fsspec will default to the latest released version. I would prefer to install the current main branch in CI.

[raybellwaves]

should be fixed: https://github.com/fsspec/s3fs/runs/4239254873?check_suite_focus=true#step:4:558

[martindurant]

Sorry if I'm being dense, but does this now actually install the local version of s3fs? Also, I notice lines like this apparently changing the base python version in the environment, which doesn't seem like a great idea. That particular line installs py39, even though it's the py38 run.

[raybellwaves]

Sorry if I'm being dense, but does this now actually install the local version of s3fs? Also, I notice lines like this apparently changing the base python version in the environment, which doesn't seem like a great idea. That particular line installs py39, even though it's the py38 run.

Thanks for the comments. I agree more thought is needed for the best way to do this. I added a comment on an issue to move the convo there (#528).

Opened up #555 as a minimal example example of what i'm trying to achieve here.

Thanks for the discussion here. I'm going to close this one as I feel it the changes to ci.yml file too much.

[martindurant]

We tend to consider the current version of aiobotocore at release time; however, a PR bumping the requirement will be gratefully received any time, and hopefully nothing breaks.