lots of ways to trigger unhandled errors via filesystem permissions

[minrk]

Description

There are a number of places where we call things like stat on paths without catching errors, resulting in 500: Unhandled Errors. They largely stem from unprotected calls to stat (e.g. in is_hidden) without catching OSError, which can be EPERM, etc. All of these errors are pretty unlikely in well-behaved situations, but messing with file permissions and/or requesting invalid paths can trigger 500 errors.

It's not quite clear to me how much of this fix belongs in server vs core, but there is certainly a disagreement in expectation, because jupyter_core.paths.is_hidden definitely can raise ValueError and OSError in many cases, but server definitely assumes it can only return True and False and can never raise.

Reproduce

One example is to try to rename a file into a directory without read permissions:

touch foo.txt
mkdir bad-dir
sudo chown root:root bad-dir
sudo chmod 700 bad-dir

and try to rename a file into a directory without write permission:

requests.patch("http://127.0.0.1:8888/api/contents/foo.txt", data=json.dumps({"path": "bad-dir/bar.txt"})

which fails with:

[E 2023-11-24 13:09:09.141 minrk] Uncaught exception PATCH /api/contents/foo.txt (::1)
    HTTPServerRequest(protocol='http', host='localhost:8888', method='PATCH', uri='/api/contents/foo.txt', version='HTTP/1.1', remote_ip='::1')
    Traceback (most recent call last):
      File "/Users/minrk/conda/lib/python3.10/site-packages/tornado/web.py", line 1786, in _execute
        result = await result
      File "/Users/minrk/conda/lib/python3.10/site-packages/jupyter_server/services/contents/handlers.py", line 146, in patch
        await ensure_async(cm.is_hidden(path)) or await ensure_async(cm.is_hidden(old_path))
      File "/Users/minrk/conda/lib/python3.10/site-packages/jupyter_core/utils/__init__.py", line 189, in ensure_async
        result = await obj
      File "/Users/minrk/conda/lib/python3.10/site-packages/jupyter_server/services/contents/filemanager.py", line 1042, in is_hidden
        return is_hidden(os_path, self.root_dir)
      File "/Users/minrk/conda/lib/python3.10/site-packages/jupyter_core/paths.py", line 484, in is_hidden
        if is_file_hidden(abs_path):
      File "/Users/minrk/conda/lib/python3.10/site-packages/jupyter_core/paths.py", line 433, in is_file_hidden_posix
        stat_res = os.stat(abs_path)
    PermissionError: [Errno 13] Permission denied: '/Users/minrk/dev/simula/testpaper/bad-dir/hidden.txt'

Expected behavior

Errors are handled as 404, 403, 400, etc.

Context

• Jupyter Server version: 2.10.0
• jupyter-core: 5.5.0