Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

os.scandir does not work with filedescriptor #723

Closed
albrycht opened this issue Sep 30, 2022 · 5 comments · Fixed by #724
Closed

os.scandir does not work with filedescriptor #723

albrycht opened this issue Sep 30, 2022 · 5 comments · Fixed by #724
Labels

Comments

@albrycht
Copy link

Describe the bug
I'm using a code that is reading directory content in such a way:

os.chdir('/test-volume')
fd = os.open(''/test-volume'', flags=os.O_DIRECTORY)
content = list(os.scandir(fd))

This code should return content of /test-volume directory. I get:
[Errno 20] Not a directory in the fake filesystem: '/test-volume/df01d42e-1f05-41fa-9ef3-8463b03ff8f0'

It would be nice to add support for file descriptors or at least throw NotImplementedError when path is int.

How To Reproduce
Shown above.

Your environment

Linux-5.15.0-48-generic-x86_64-with-glibc2.35
Python 3.10.4 (main, Apr 28 2022, 10:58:00) [GCC 11.2.0]
pyfakefs 4.7.0
@mrbean-bremen
Copy link
Member

Actually it should support this, at least the code suggests that. Can you please add a callstack? I also don't understand where it takes this uid from - is this a path in the fake filesystem?

@mrbean-bremen
Copy link
Member

And can you please show a reproducible example? Your snippet does not look like real code, as it uses an invalid string notation.

@albrycht
Copy link
Author

albrycht commented Sep 30, 2022

Here is an example that fails for me:

with Patcher() as patcher:
    patcher.fs.create_dir('/tmp/dir')
    patcher.fs.create_file('/tmp/dir/file1')
    patcher.fs.create_file('/tmp/dir/file2')
    patcher.fs.create_dir('/tmp/dir/subdir')

    os.chdir('/tmp/dir')
    fd = os.open('/tmp/dir', flags=os.O_DIRECTORY)
    children = [dir_entry.name for dir_entry in os.scandir(fd)]
    children.sort()
    assert children == ['file1', 'file2', 'subdir']

It throws error: NotADirectoryError: [Errno 20] Not a directory in the fake filesystem: '/tmp/dir/b8ec8d70-c3e6-4a14-94b9-f5c78f2c85c6' while executing os.scandir

Here is full stacktrace:

agent/tests/crawler/incremental_test.py:931 (CrawlerWorkerPerfTest.test_scandir_with_fd)
self = <crawler.incremental_test.CrawlerWorkerPerfTest testMethod=test_scandir_with_fd>

    def test_scandir_with_fd(self):
        with Patcher() as patcher:
            patcher.fs.create_dir('/tmp/dir')
            patcher.fs.create_file('/tmp/dir/file1')
            patcher.fs.create_file('/tmp/dir/file2')
            patcher.fs.create_dir('/tmp/dir/subdir')
    
            os.chdir('/tmp/dir')
            fd = os.open('/tmp/dir', flags=os.O_DIRECTORY)
>           children = [dir_entry.name for dir_entry in os.scandir(fd)]

agent/tests/crawler/incremental_test.py:941: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../.virtualenvs/sf/lib/python3.10/site-packages/pyfakefs/fake_filesystem.py:4995: in wrapped
    return f(*args, **kwargs)
../../.virtualenvs/sf/lib/python3.10/site-packages/pyfakefs/fake_filesystem.py:4334: in scandir
    return scandir(self.filesystem, path)
../../.virtualenvs/sf/lib/python3.10/site-packages/pyfakefs/fake_scandir.py:180: in scandir
    return ScanDirIter(filesystem, path)
../../.virtualenvs/sf/lib/python3.10/site-packages/pyfakefs/fake_scandir.py:136: in __init__
    entries = self.filesystem.confirmdir(self.abspath).entries
../../.virtualenvs/sf/lib/python3.10/site-packages/pyfakefs/fake_filesystem.py:3318: in confirmdir
    self.raise_os_error(errno.ENOTDIR, target_directory, 267)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <pyfakefs.fake_filesystem.FakeFilesystem object at 0x7f4f6b042380>
err_no = 20, filename = '/tmp/dir/b8ec8d70-c3e6-4a14-94b9-f5c78f2c85c6'
winerror = 267

    def raise_os_error(self, err_no: int,
                       filename: Optional[AnyString] = None,
                       winerror: Optional[int] = None) -> NoReturn:
        """Raises OSError.
        The error message is constructed from the given error code and shall
        start with the error string issued in the real system.
        Note: this is not true under Windows if winerror is given - in this
        case a localized message specific to winerror will be shown in the
        real file system.
    
        Args:
            err_no: A numeric error code from the C variable errno.
            filename: The name of the affected file, if any.
            winerror: Windows only - the specific Windows error code.
        """
        message = os.strerror(err_no) + ' in the fake filesystem'
        if (winerror is not None and sys.platform == 'win32' and
                self.is_windows_fs):
            raise OSError(err_no, message, filename, winerror)
>       raise OSError(err_no, message, filename)
E       NotADirectoryError: [Errno 20] Not a directory in the fake filesystem: '/tmp/dir/b8ec8d70-c3e6-4a14-94b9-f5c78f2c85c6'

@mrbean-bremen
Copy link
Member

Thank you, I will try to reproduce this.

mrbean-bremen added a commit to mrbean-bremen/pyfakefs that referenced this issue Oct 1, 2022
- the check for the flag was incorrect
- fixes pytest-dev#723
mrbean-bremen added a commit to mrbean-bremen/pyfakefs that referenced this issue Oct 1, 2022
- the check for the flag was incorrect
- change PyPy version in CI to 3.7
- fixes pytest-dev#723
mrbean-bremen added a commit that referenced this issue Oct 2, 2022
- the check for the flag was incorrect
- change PyPy version in CI to 3.7
- fixes #723
github-actions bot pushed a commit that referenced this issue Oct 2, 2022
…s incorrect - change PyPy version in CI to 3.7 - fixes #723
@mrbean-bremen
Copy link
Member

Shall be fixed in master now. That was actually a bug I had accidentally introduced myself a couple of years ago, that never came up so far - good catch, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants