File not found when running docker + python + tox

[dandunckelman]

• Your Windows build number: (Type ver at a Windows Command Prompt)

  • Microsoft Windows [Version 10.0.17692.1004]

• What you're doing and what's happening: (Copy&paste specific commands and their output, or include screen shots)

  • I'm trying to get this to work: https://github.com/dandunckelman/docker-python-tox
  • It should start a single container running python v3.6.6, install tox, run tests, and succeed. Instead, I get this error:

python_1  | py36 create: /tox-testing/.tox/py36
python_1  | Traceback (most recent call last):
python_1  |   File "/usr/local/lib/python3.6/site-packages/py/_error.py", line 66, in checked_call
python_1  |     return func(*args, **kwargs)
python_1  | FileNotFoundError: [Errno 2] No such file or directory: '/tox-testing/.tox/py36/log/py36-0.log'
python_1  |
python_1  | During handling of the above exception, another exception occurred:
python_1  |
python_1  | Traceback (most recent call last):
python_1  |   File "/usr/local/bin/tox", line 11, in <module>
python_1  |     sys.exit(cmdline())
python_1  |   File "/usr/local/lib/python3.6/site-packages/tox/session.py", line 39, in cmdline
python_1  |     main(args)
python_1  |   File "/usr/local/lib/python3.6/site-packages/tox/session.py", line 45, in main
python_1  |     retcode = Session(config).runcommand()
python_1  |   File "/usr/local/lib/python3.6/site-packages/tox/session.py", line 422, in runcommand
python_1  |     return self.subcommand_test()
python_1  |   File "/usr/local/lib/python3.6/site-packages/tox/session.py", line 611, in subcommand_test
python_1  |     if self.setupenv(venv):
python_1  |   File "/usr/local/lib/python3.6/site-packages/tox/session.py", line 504, in setupenv
python_1  |     status = venv.update(action=action)
python_1  |   File "/usr/local/lib/python3.6/site-packages/tox/venv.py", line 170, in update
python_1  |     self.hook.tox_testenv_create(action=action, venv=self)
python_1  |   File "/usr/local/lib/python3.6/site-packages/pluggy/__init__.py", line 617, in __call__
python_1  |     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
python_1  |   File "/usr/local/lib/python3.6/site-packages/pluggy/__init__.py", line 222, in _hookexec
python_1  |     return self._inner_hookexec(hook, methods, kwargs)
python_1  |   File "/usr/local/lib/python3.6/site-packages/pluggy/__init__.py", line 216, in <lambda>
python_1  |     firstresult=hook.spec_opts.get('firstresult'),
python_1  |   File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 201, in _multicall
python_1  |     return outcome.get_result()
python_1  |   File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 76, in get_result
python_1  |     raise ex[1].with_traceback(ex[2])
python_1  |   File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 180, in _multicall
python_1  |     res = hook_impl.function(*args)
python_1  |   File "/usr/local/lib/python3.6/site-packages/tox/venv.py", line 475, in tox_testenv_create
python_1  |     venv._pcall(args, venv=False, action=action, cwd=basepath)
python_1  |   File "/usr/local/lib/python3.6/site-packages/tox/venv.py", line 427, in _pcall
python_1  |     return action.popen(args, cwd=cwd, env=env, redirect=redirect, ignore_ret=ignore_ret)
python_1  |   File "/usr/local/lib/python3.6/site-packages/tox/session.py", line 144, in popen
python_1  |     fin = outpath.open("rb")
python_1  |   File "/usr/local/lib/python3.6/site-packages/py/_path/local.py", line 361, in open
python_1  |     return py.error.checked_call(open, self.strpath, mode)
python_1  |   File "/usr/local/lib/python3.6/site-packages/py/_error.py", line 86, in checked_call
python_1  |     raise cls("%s%r" % (func.__name__, args))
python_1  | py.error.ENOENT: [No such file or directory]: open('/tox-testing/.tox/py36/log/py36-0.log', 'rb')
dockerpythontox_python_1 exited with code 1

Despite it saying that the /tox-testing/.tox/py36/log/py36-0.log file doesn't exist, I can see it does exist.

• What's wrong / what should be happening instead:
  • There seems to be a permission issue or something specific to how this works with the underlying file system.
  • If I clone the project via Powershell, then I can run docker-compose up, and it works as expected.
  • Cloning it in Ubuntu results in the above error when I run docker-compose up.

Please let me know if more info is needed.

(WIP) Full Process to Reproduce

• This is based on this stackoverflow answer

• Install Docker for Windows: https://docs.docker.com/docker-for-windows/install/

• Configure Docker for Windows:

  • Check Expose daemon on tcp://localhost:2375 without TLS
  • Enable drive sharing and select the drive you wish to share
  • Disable experimental features

• Follow https://docs.microsoft.com/en-us/windows/wsl/install-win10 to install Ubuntu

• Launch the app and setup username/password

• Upgrade packages:

  apt update
  apt upgrade
  

• Setup Docker client

  cd # ensure we're at home
  wget https://download.docker.com/linux/static/stable/x86_64/docker-18.03.1-ce.tgz
  tar -xzvf docker-*.tgz
  mkdir ~/bin
  mv ~/docker/docker ~/bin
  

• Configure the docker server and add the docker binaries to $PATH

  echo "export DOCKER_HOST=tcp://0.0.0.0:2375" >> ~/.bashrc
  echo "export PATH=$PATH:~/bin" >> ~/.bashrc
  

• Install docker-compose

  sudo -i
  curl -L https://github.com/docker/compose/releases/download/1.21.2/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose
  chmod +x /usr/local/bin/docker-compose
  

• Enable proper volume mapping by writing /etc/wsl.conf on the Ubuntu filesystem w/ this:

  # Enable extra metadata options by default
  [automount]
  enabled = true
  root = /
  options = "metadata,umask=22,fmask=11"
  mountFsTab = false
  

• Setup mount point for windows drive:

  sudo mkdir /c
  sudo mount --bind /mnt/c /c
  

• Clone the project into your Windows user's home directory:

  cd /c/Users/USERNAME
  git clone https://github.com/dandunckelman/docker-python-tox
  

• Run docker-compose to see it fail

  cd docker-python-tox/
  docker-compose up
  

Here's the strace logs from running strace -ff -o docker-python-tox-strace.log docker-compose up:

• https://gist.github.com/dandunckelman/73b3500182789dc1591f27b613513fd9

[therealkenc]

It should start a single container

#2114 #3317 et al landing zone is #2291

[dandunckelman]

@therealkenc I must be missing something. After reading those other issues, I'm not sure why this is closed.

I could very well be wrong, but the crux of the issue to me is that it works in the WSL when I clone the project from Windows but fails when I clone from the WSL.

I'm not running the docker engine from the WSL (though the client and docker-compose are specific to the WSL). I'm running Docker on Windows w/ version 18.03.1-ce-win65 (17513).

I then tell the WSL to use the docker engine running from Windows:

echo "export DOCKER_HOST=tcp://0.0.0.0:2375" >> ~/.bashrc

Maybe my /etc/wsl.conf is somehow related to the issue?

# Enable extra metadata options by default
[automount]
enabled = true
root = /
options = "metadata,umask=22,fmask=11"
mountFsTab = false

Could you please explain how my issue is directly related to the linked issues? I'll be happy to follow them if they may lead to fixing my problem.

Thanks for the help.

[therealkenc]

I'm running Docker on Windows w/ version 18.03.1-ce-win65 (17513).

Appreciated; that was the missing bit in your OP.

You'll get better chances of a hit on this one with CLI repro steps from clean install on your WSL side. Think something a five year old can cut and paste into their terminal. The Windows docker server side you can probably take as a given. Linking an strace(1) log in a gist will help too, although if it is massive (and it probably is) the repro is more important.

There seems to be a permission issue or something specific to how this works with the underlying file system.

Yeah there be dragons there. Might be perennial #1529 (or it might not, I just like linking it). [ed] Still smells like #3317 tho, which is three weeks chirping crickets for lack of aforementioned repro steps.

[dandunckelman]

@therealkenc There definitely are more steps to properly reproduce this. I'll put more into the issue description.

And this definitely smells like it could be #1529.

Thanks.

[dandunckelman]

@therealkenc I've updated the description. While I don't see a 5 year old being able to easily walk thru it successfully, I think it covers all the steps to get where I am.

I'll get a strace now.

EDIT

Strace added.

[therealkenc]

I think it covers all the steps to get where I am.

That's about the best you can do. In a 30 second looksee there isn't a single rename(2) in there, so it's not #1529.

One last straw-grasp. In your wsl.conf, do:

options = "metadata,umask=22,fmask=11,case=off"

Delete everything on your C:/Users/USERNAME/docker-python-tox, or even safer, start over in C:/some/where/else. Reboot for good measure (there's been two unconfirmed reports this is necessary). Then try the WSL-side git clone again. The hail Mary here is ref #3356.

[dandunckelman]

@therealkenc add case=off to wsl.conf did it!

I updated /etc/wsl.conf, quit the shell, opened a shell to Ubuntu again, created a new directory on the windows side, cloned the project in that directory (from the ubuntu shell), and the docker-compose up worked!

Thanks so much! I'll call this closed!

[therealkenc]

Great. If I was a little quicker on the uptake I would have clued in on "If I clone the project via Powershell, then I can run docker-compose up, and it works as expected" sooner. Thank you for taking the time to post a repro; that helps contribute on its own merit.

[dandunckelman]

@therealkenc no worries. Glad we got it working.