Added loading of sitemap to filter instant loading requests

[squidfunk]

This PR is a fix attempt for #1653. When instant loading is enabled, the current master will intercept all links that are relative to the base_url and assume that they can be loaded via XHR. However, as described in the linked issue, static pages may be included as part of the build, but not part of the documentation as such. For this reason, we need a way to distinguish URLs to be loaded via instant loading.

With this PR, before setting up instant loading, the sitemap.xml is fetched, which is built by MkDocs and includes all URLs to all documents that are part of the navigation tree. Before dispatching via XHR, the application now checks whether the URL is part of this list. If it is part of the list, the request is loaded via XHR, if it isn't, it is dispatched regularly.

As this is a pretty fundamental change, I'd want to get some feedback before merging this into master. Also, locally, the URL must now match the development server host, or instant loading won't work, e.g. when the docs are served with -a 0.0.0.0, URLs including localhost won't be dispatched. If somebody has a good idea of how to get a canonical URL out of all the different localhost stuff, I'd be happy to integrate it.

[netlify]

Deploy preview for mkdocs-material-preview ready!

Built with commit 6a9495b

https://deploy-preview-1668--mkdocs-material-preview.netlify.app

[squidfunk]

As I received no feedback, we just release it and fix bugs as we go.

[sdague]

Something about this change has made it so that instant mode with our site goes completely unresponsive. I downgraded to 5.1.5 and it works again.

It's so unresponsive that pulling up firefox dev tools doesn't even work, and the browser asks if I want to Wait or Stop the script that's taking the page too long to load.

[squidfunk]

Thanks for reporting. Could you open an issue for that including the necessary details? In the meantime you can disable instant loading and everything should be back to normal.