Build time is increased by 6x after upgrading from >=5.1.1 to 5.2.0+

[surya-sigtech]

Describe the bug

When upgrading sphinx-build to the latest version (7.3.7), I noticed a significant increase in build time for generating HTML pages in our project. Upon further investigation, I found that the build time increases noticeably after upgrading to any version 5.2.0+.

Observations:
The build time for the exact same project and same configuration is:

• With version 5.1.1, it is approx 2min 30sec
• With version 5.2.0, it is approx 15mins (6 times more)

Root Cause:
Upon deeper investigation, I found that the commit f57177d is the cause of the issue.
Specifically, the changes made to the function build_toc in file sphinx/environment/collectors/toctree.py.

Cc: @AA-Turner

How to Reproduce

1. Upgrade sphinx-build to version 7.3.7 or any version 5.2.0+.
2. Generate HTML pages using sphinx-build
   Command:

sphinx-build -b html -j auto ${base_path}/docs/source ${base_path}/html

Environment Information

Platform:              darwin; (macOS-14.2.1-arm64-arm-64bit)
Python version:        3.11.9 (main, Apr 19 2024, 11:43:47) [Clang 14.0.6 ])
Python implementation: CPython
Sphinx version:        7.4.0+/a66546895
Docutils version:      0.21.2
Jinja2 version:        3.1.3
Pygments version:      2.17.2

Sphinx extensions

No response

Additional context

No response

[AA-Turner]

What happens if you disable it?

https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-toc_object_entries

toc_object_entries = False

A

[surya-sigtech]

Just tried disabling it by adding it to my conf.py.
Using latest sphinx version, it's working 👍
Thanks man!! @AA-Turner

Build time now is approx 4min 50sec (lot better then before)

However, It's taking a bit more time while reading sources with new versions (not sure after which one).

[surya-sigtech]

I'm wondering if there are any additional configurations we could adjust to further improve the build time.

With disabling the toc_object_entries flag for my project,

• Prior to version <6.0.0, the build time averaged around 2min 30 sec
• However, after version >=6.0.0, it increased ranging from 4min 50sec to 5min 50sec (approximately twice as before).

Any insights on potential configuration adjustments or optimizations would be greatly appreciated.

[chrisjsewell]

I'm wondering if there are any additional configurations we could adjust to further improve the build time.

Been looking into this a bit for sphinx-needs: useblocks/sphinx-needs#967 (comment)

One thing is to set: smartquotes = False as that is a big time kill (although in the newest docutils release I added some improvements to this)

Also looking to eventually improve multi-processing performance: #11746

Happy to hear about more reports like this though!

[surya-sigtech]

Appreciate the pointers, @chrisjsewell

I've taken a look at the other issues you mentioned, yet I'm still struggling to pinpoint what's causing such a drastic increase in build time with the latest versions. I've noticed a substantial slowdown with sphinx-build post version 6.0.0, with the build process now taking more than twice as long as it did previously.

This slowdown is concerning, as we'd like to leverage the latest features, but the performance hit is making it impractical. Any further insights will be really helpful.