Contents API fetches are cached for too long

[sebbov]

Description

Without cache directives in the response headers from the Contents API, Chrome appears to cache fetch responses for a duration proportional to the time since the reported last modification of the content. This prevents fetches from seeing updated contents when contents (on first fetch) haven't seen recent modifications.

Reproduce

Start the server:

jupyter server \
  --ServerApp.port=8892 \
  --ServerApp.tornado_settings='{"headers": { "Content-Security-Policy": "frame-ancestors self http://127.0.0.1:8892/; connect-src '"'"'self'"'"' " }}' \
  --ServerApp.token=''

From the server's root dir (same cwd as previous command) make a dir to list contents from. Pretend it was created a few months ago (but using just a few hours ago repros as well):

mkdir some_dir
touch -t 202301010000 some_dir

In an incognito Chrome window, clear browser cache, then navigate to the server root url (http://127.0.0.1:8892/). Be welcomed by "A Jupyter Server is running.".

In the Developer Tools console, exercise the Contents API for the created dir:

fetch('http://127.0.0.1:8892/api/contents/some_dir').then(res => res.json()).then(console.log)

Response headers should look like:

HTTP/1.1 200 OK
Server: TornadoServer/6.3.3
Content-Type: application/json
Date: Tue, 24 Oct 2023 18:10:12 GMT
X-Content-Type-Options: nosniff
Content-Security-Policy: frame-ancestors self http://127.0.0.1:8892/; connect-src 'self' ; default-src 'none'
Access-Control-Allow-Origin: *
Last-Modified: Sun, 01 Jan 2023 08:00:00 GMT
Etag: "d2879b009a893042a30501118aea5e98986eac8e"
Content-Length: 227

Run the fetch again and notice the response is obtained from disk cache.

Make a modification to the directory:

touch some_dir/a

Run the fetch again and notice the response is obtained from disk cache, and hence doesn't include some_dir/a. Repeatable for minutes.

Expected behavior

A fetch with default options should show the new directory entry. (Caching this for a few seconds seems acceptable, but not 10m+.)

The problem can be avoided by specifying cache use behavior in the fetch options (e.g. no-store). I don't know if it is expected to have to specify this in the frontend, vs. having the server control the caching behavior better. I found fe730a6 which disables caching for unversioned static files and suggests server-side handling was at least preferred there.

Trying a simple version of that:

diff --git a/jupyter_server/services/contents/handlers.py b/jupyter_server/services/contents/handlers.py
index 50e7703db..b193eb0d0 100644
--- a/jupyter_server/services/contents/handlers.py
+++ b/jupyter_server/services/contents/handlers.py
@@ -89,6 +89,9 @@ class ContentsHandler(ContentsAPIHandler):
             self.set_header("Location", location)
         self.set_header("Last-Modified", model["last_modified"])
         self.set_header("Content-Type", "application/json")
+        # disable browser caching, rely in 304 replies for savings
+        if "v" not in self.request.arguments:
+          self.add_header("Cache-Control", "no-cache")
         self.finish(json.dumps(model, default=json_default))
 
     @web.authenticated

, conditional requests are performed for every fetch, problem no longer observed. IDK enough about browser caching to know if this is the right fix though.

I haven't seen this Chrome behavior with page loads, only fetches.

Context

• Operating System and version: Debian based
• Browser and version: Chrome 118.0.5993.88
• Jupyter Server version: head github repo

Troubleshoot Output

$PATH:
	/home/seb/venv/jupyter_server.1/bin
	/usr/local/buildtools/java/jdk/bin
	/usr/local/sbin
	/usr/local/bin
	/usr/sbin
	/usr/bin
	/sbin
	/bin
	/home/seb/bin
	/home/seb/bin
	/home/seb/bin
sys.path:

/home/seb/venv/jupyter_server.1/bin

/usr/local/buildtools/current/sitecustomize

/usr/lib/python311.zip

/usr/lib/python3.11

/usr/lib/python3.11/lib-dynload

/home/seb/venv/jupyter_server.1/lib/python3.11/site-packages

/home/seb/src/github.com/jupyter_server/jupyter_server

/home/seb/src/github.com/jupyter_server/jupyter_server/examples/simple
sys.executable:

/home/seb/venv/jupyter_server.1/bin/python3
sys.version:

3.11.5 (main, Aug 25 2023, 07:43:52) [GCC 12.2.0]
platform.platform():

Linux-6.5.3-1rodete1-amd64-x86_64-with-glibc2.37
which -a jupyter:

/home/seb/venv/jupyter_server.1/bin/jupyter
pip list:

Package                   Version    Editable project location

------------------------- ---------- ---------------------------------------------------------------------------------------

anyio                     4.0.0

argon2-cffi               23.1.0

argon2-cffi-bindings      21.2.0

arrow                     1.3.0

asttokens                 2.4.0

attrs                     23.1.0

backcall                  0.2.0

beautifulsoup4            4.12.2

bleach                    6.1.0

certifi                   2023.7.22

cffi                      1.16.0

cfgv                      3.4.0

charset-normalizer        3.3.1

comm                      0.1.4

debugpy                   1.8.0

decorator                 5.1.1

defusedxml                0.7.1

distlib                   0.3.7

executing                 2.0.0

fastjsonschema            2.18.1

filelock                  3.12.4

flaky                     3.7.0

fqdn                      1.5.1

identify                  2.5.30

idna                      3.4

iniconfig                 2.0.0

ipykernel                 6.25.2

ipython                   8.16.1

isoduration               20.11.0

jedi                      0.19.1

Jinja2                    3.1.2

jsonpointer               2.4

jsonschema                4.19.1

jsonschema-specifications 2023.7.1

jupyter_client            8.4.0

jupyter_core              5.4.0

jupyter-events            0.8.0

jupyter_server            2.9.0.dev0 /home/seb/src/github.com/jupyter_server/jupyter_server

jupyter-server-example    0.0.1      /home/seb/src/github.com/jupyter_server/jupyter_server/examples/simple

jupyter_server_terminals  0.4.4

jupyterlab-pygments       0.2.2

MarkupSafe                2.1.3

matplotlib-inline         0.1.6

mistune                   3.0.2

nbclient                  0.8.0

nbconvert                 7.9.2

nbformat                  5.9.2

nest-asyncio              1.5.8

nodeenv                   1.8.0

overrides                 7.4.0

packaging                 23.2

pandocfilters             1.5.0

parso                     0.8.3

pexpect                   4.8.0

pickleshare               0.7.5

pip                       23.1.2

platformdirs              3.11.0

pluggy                    1.3.0

pre-commit                3.5.0

prometheus-client         0.17.1

prompt-toolkit            3.0.39

psutil                    5.9.6

ptyprocess                0.7.0

pure-eval                 0.2.2

pycparser                 2.21

Pygments                  2.16.1

pytest                    7.4.2

pytest-console-scripts    1.4.1

pytest-jupyter            0.7.0

pytest-timeout            2.2.0

python-dateutil           2.8.2

python-json-logger        2.0.7

PyYAML                    6.0.1

pyzmq                     25.1.1

referencing               0.30.2

requests                  2.31.0

rfc3339-validator         0.1.4

rfc3986-validator         0.1.1

rpds-py                   0.10.6

Send2Trash                1.8.2

setuptools                67.8.0

six                       1.16.0

sniffio                   1.3.0

soupsieve                 2.5

stack-data                0.6.3

terminado                 0.17.1

tinycss2                  1.2.1

tornado                   6.3.3

traitlets                 5.11.2

types-python-dateutil     2.8.19.14

uri-template              1.3.0

urllib3                   2.0.7

virtualenv                20.24.6

wcwidth                   0.2.8

webcolors                 1.13

webencodings              0.5.1

websocket-client          1.6.4

[welcome]

Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! 🤗

If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively.

You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! 👋

Welcome to the Jupyter community! 🎉