More robust path-traversal check in StaticFiles app

[tomchristie]

Closes #981

[char]

Looks like this causes an issue where you can get 404s for every static file except if you create your StaticFiles object with StaticFiles(directory=os.path.realpath(...)).

Also (and potentially should be its own issue), could we consider allowing following symbolic links with a flag that we pass in?

[norefle]

The latest changes break previous behaviour from 0.13.4.
Assume we have a following layout locally:

/ project-root
    - app.py
    / static
        - example.txt

Based on the documentation we can register static files as following: Mount('/static', app=StaticFiles(directory='static'), name="static")
Then requests like GET /static/example.txt should be served. And they used to be served, until these changes.

If we use the layout from above and the recommended way from documentation then the code from this PR will produce the following:

>>> import os
>>> full_path = os.path.join("static", "/static/example.txt")
>>> full_path
'/static/example.txt'
>>> os.path.commonprefix([os.path.realpath(full_path), "static"])
''
>>> os.path.commonprefix([os.path.realpath(full_path), "static"]) != "static"
True

@tomchristie what is the recommended way to register StaticFiles now?

Especially in case of different layouts:

• Local dev: /home/whatev/whatev/projects/static/ -> static
• In a docker container: /app/static -> static

[i-Ching]

At a local layout, I convert realpath+static to string in order to fix the error 404 for the version 0.13.5 as below:

from pathlib import Path
...
Mount("/static", app=StaticFiles( directory=str(Path(Path.cwd()).joinpath('static')) ), name="static")

[indeets-vasily]

I have tested a little bit on my Windows 10 Notebook and it seems that StaticFiles return 404 if directory parameter is pathlib class. When you pass the string of correct format (Windows for Windows system) it works:

• StaticFiles(directory=Path("C:\\path\\to\\directory")) -- 404
• StaticFiles(directory=Path("C:/path/to/directory")) -- 404
• StaticFiles(directory=str(Path("C:\\path\\to\\directory"))) -- OK
• StaticFiles(directory=str(Path("C:/path/to/directory"))) -- OK, Path automatically converts Linux path to Windows format
• StaticFiles(directory=StaticFiles(directory="C:\\path\\to\\directory") -- OK
• StaticFiles(directory=StaticFiles(directory="C:/path/to/directory") -- 404, because this string is in Linux format

os.path.* functions work because they return string values, even if their argument is Path:

• StaticFiles(directory=os.path.realpath("C:\\path\\to\\directory")) -- OK
• StaticFiles(directory=os.path.realpath("C:/path/to/directory")) -- OK
• StaticFiles(directory=os.path.realpath(Path("C:\\path\\to\\directory"))) -- OK

Most interestingly that there are different results depending on whether directory passed as Path exists or not:
Both StaticFiles(directory=Path("C:\\non\\existent\\directory")) and StaticFiles(directory="C:\\non\\existent\\directory") return same:
RuntimeError: Directory 'C:/non/existent/directory' does not exist

[tomchristie]

Resolved in 0.13.6