Update docs requirements list #74956

acozine · 2021-06-09T21:06:28Z

SUMMARY

In the Docs Working Group meeting on 25 May 2021 we agreed to maintain two requirements files for the docs build:

one with as few version pins as possible
a second one pinned to known good versions for use with the publication pipeline

The plan is to "unleash" the known good versions, test the most recent available packages, and update the. known_good_reqs. file on a quarterly basis going forward, so the versions don't get too stale.

This PR:

creates the second requirements file
removes the pin on the Sphinx version

Related to #74318
Closes #74194
Closes #71395

Tested on a mac running macOS Catalina. Pip package versions listed below. Test if you have time and report any issues you find!

ISSUE TYPE

Docs Pull Request

COMPONENT NAME

docs.ansible.com

ADDITIONAL INFORMATION

Versions installed from the main (unpinned) requirements.txt:

Package                       Version
----------------------------- -----------
aiofiles                      0.7.0
aiohttp                       3.7.4.post0
alabaster                     0.7.12
antsibull                     0.33.0
antsibull-changelog           0.10.0
async-timeout                 3.0.1
asyncio-pool                  0.5.2
attrs                         21.2.0
Babel                         2.9.1
certifi                       2021.5.30
cffi                          1.14.5
chardet                       4.0.0
click                         8.0.1
cryptography                  3.4.7
docutils                      0.16
idna                          2.10
imagesize                     1.2.0
Jinja2                        3.0.1
MarkupSafe                    2.0.1
multidict                     5.1.0
packaging                     20.9
perky                         0.5.5
pip                           21.0.1
pycparser                     2.20
pydantic                      1.8.2
Pygments                      2.9.0
pyparsing                     2.4.7
pytz                          2021.1
PyYAML                        5.4.1
requests                      2.25.1
resolvelib                    0.5.4
rstcheck                      3.3.1
semantic-version              2.8.5
setuptools                    54.2.0
sh                            1.14.2
six                           1.16.0
snowballstemmer               2.1.0
Sphinx                        4.0.2
sphinx-ansible-theme          0.6.0
sphinx-intl                   2.0.1
sphinx-notfound-page          0.7.1
sphinx-rtd-theme              0.5.2
sphinxcontrib-applehelp       1.0.2
sphinxcontrib-devhelp         1.0.2
sphinxcontrib-htmlhelp        2.0.0
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          1.0.3
sphinxcontrib-serializinghtml 1.1.5
straight.plugin               1.5.0
Twiggy                        0.5.1
typing-extensions             3.10.0.0
urllib3                       1.26.5
yarl                          1.6.3

webknjaz · 2021-06-09T21:46:59Z

docs/docsite/requirements.txt

 sphinx-notfound-page >= 0.6
 sphinx-intl
-sphinx_ansible_theme === 0.6.0


I'd like to have an exact pin here. == matches .post releases while === matches the exact spec.

Ah, thanks @webknjaz, that is not syntax I knew before. I'll restore that.

It's documented here: https://www.python.org/dev/peps/pep-0440/#arbitrary-equality

fixed in a4fb538

samccann · 2021-06-10T15:30:01Z

Tested locally (Fedora 34) in new venv - all permutations work:

requirements.txt - works for make webdocs and make coredocs
known)_good_reqs.txt - works for make webdocs and make coredocs

samccann · 2021-06-10T15:35:09Z

docs/docsite/rst/community/documentation_contributions.rst

@@ -74,8 +74,7 @@ Setting up your environment to build documentation locally

 To build documentation locally, ensure you have a working :ref:`development environment <environment_setup>`.

-To work with documentation on your local machine, you need to have python-3.5 or greater and the
-following packages installed:
+To work with documentation on your local machine, you need to have python-3.5 or greater and the following packages installed:


There was a comment in an older PR that suggests we just remove this list and say something like see xx requirements.txt for the packages and versions required (maybe even link to that file in github.)

thanks, I've updated based on this comment

webknjaz · 2021-06-11T18:39:34Z

Tested locally (Fedora 34) in new venv - all permutations work:

requirements.txt - works for make webdocs and make coredocs

known)_good_reqs.txt - works for make webdocs and make coredocs

I've mentioned this in Slack but let me document it here. It's easier and more reliable to test such a setup with pip-tools.

ansibot · 2021-06-14T20:32:00Z

The test ansible-test sanity --test docs-build [explain] failed with 1 error:

docs/docsite/rst/community/documentation_contributions.rst:77:0: warning: Unknown target name: "documentation dependencies<https://github.com/ansible/ansible/blob/devel/docs/docsite/requirements.txt>".

The test ansible-test sanity --test rstcheck [explain] failed with 1 error:

docs/docsite/rst/community/documentation_contributions.rst:77:0: Unknown target name: "documentation dependencies<https://github.com/ansible/ansible/blob/devel/docs/docsite/requirements.txt>".

click here for bot help

…ents file

acozine · 2021-06-14T20:49:19Z

@webknjaz I'm sure pip-tools is an excellent option, but adding another tool seems like an over-engineered solution for the problem we're addressing

webknjaz · 2021-06-19T07:19:58Z

@acozine it's not really overengineering, its a pretty standard practice of keeping the builds reproducible. Currently, this PR almost solves a part of the problem not taking into account transitive deps that may break things while my proposal is to actually address the root of the issue. But I agree that the current patch is better than nothing, of course. And automatic pinning could be implemented separately.

webknjaz · 2021-06-19T15:25:45Z

docs/docsite/known_good_reqs.txt

+sphinx==4.0.2
+sphinx-notfound-page==0.7.1 # must be >= 0.6
+sphinx-intl==2.0.1
+sphinx_ansible_theme===0.6.0


Taking into account #75059, it should now be

Suggested change

sphinx_ansible_theme===0.6.0

sphinx_ansible_theme===0.7.0

webknjaz · 2021-06-19T15:26:22Z

docs/docsite/requirements.txt

 PyYAML
 rstcheck
-sphinx==2.1.2
+sphinx
 sphinx-notfound-page >= 0.6
 sphinx-intl
 sphinx_ansible_theme === 0.6.0


With #75059 it's now

Suggested change

sphinx_ansible_theme === 0.6.0

sphinx_ansible_theme >= 0.7.0

I would really avoid >= here.

I think it's okay for direct unpinned deps. Anybody wanting the pins should do pip install -r docs/docsite/requirements.txt -c docs/docsite/known_good_reqs.txt

webknjaz · 2021-06-19T15:27:08Z

docs/docsite/requirements.txt

 jinja2
+Pygments >= 2.4.0


This is not necessary among the direct deps anymore because the theme pulls in this dep and we no longer use it directly.

Suggested change

Pygments >= 2.4.0

acozine · 2021-06-22T18:54:18Z

The goal we agreed on in the DaWGs meeting is to offer two options: one set of requirements with the least possible restrictions on versions, and a second set of requirements with a known-good set that updates infrequently. I think the current PR meets those requirements. I'm fine with the known-good list getting stale, as long as it continues to work.

acozine · 2021-06-22T18:56:18Z

We can update again in future if we all agree on a goal and an implementation strategy.

* removes upper bound on sphinx version * updates versions of docs build dependencies, adds known good requirements file * adds instructions for using known_good_reqs file (cherry picked from commit 58f2638)

webknjaz · 2021-06-23T11:07:06Z

docs/docsite/rst/community/documentation_contributions.rst

+    pip install --user -r requirements.txt
+    pip install --user -r docs/docsite/known_good_reqs.txt


@acozine @samccann I missed this but it should be something like pip install -r docs/docsite/requirements.txt -c docs/docsite/known_good_reqs.txt. Which basically constraints what should be installed.

webknjaz · 2021-06-23T11:16:35Z

The goal we agreed on in the DaWGs meeting is to offer two options: one set of requirements with the least possible restrictions on versions, and a second set of requirements with a known-good set that updates infrequently. I think the current PR meets those requirements. I'm fine with the known-good list getting stale, as long as it continues to work.

The point is that it won't continue to work if some indirect dependency changes in an incompatible way. This will be a silent change with no instrumentation to notice it automatically until a human complains. This is because the current pins list is incomplete.

* Use `sphinx_ansible_theme` Sphinx theme in docs (#74318) (cherry picked from commit 346c7a7) * 🔥 Drop unused `core.css` file This is a forgotten leftover from #74318 that should've been removed earlier. (cherry picked from commit ec408a6) * Update docs requirements list (#74956) * removes upper bound on sphinx version * updates versions of docs build dependencies, adds known good requirements file * adds instructions for using known_good_reqs file (cherry picked from commit 58f2638) Co-authored-by: Sviatoslav Sydorenko <webknjaz@redhat.com> Co-authored-by: Sviatoslav Sydorenko <wk@sydorenko.org.ua> Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>

acozine mentioned this pull request Jun 9, 2021

remove limitation on sphinx version #74372

Closed

acozine changed the title ~~Update docs requirements list~~ WIP: Update docs requirements list Jun 9, 2021

ansibot added WIP This issue/PR is a work in progress. Nevertheless it was shared for getting input from peers. and removed core_review In order to be merged, this PR must follow the core review workflow. labels Jun 9, 2021

webknjaz reviewed Jun 9, 2021

View reviewed changes

ansibot added the docsite This issue/PR relates to the documentation website. label Jun 10, 2021

samccann reviewed Jun 10, 2021

View reviewed changes

jborean93 removed the needs_triage Needs a first human triage before being processed. label Jun 10, 2021

ansibot added the ci_verified Changes made in this PR are causing tests to fail. label Jun 14, 2021

acozine added 7 commits June 14, 2021 15:42

removes upper bound on sphinx version

3a21652

updates versions of docs build dependencies, adds known good requirem…

2f9ee22

…ents file

pin ansible sphinx theme version to exact release

c142f95

adds instructions for using known_good_reqs file

88be25c

remove list of required packages, point at requirements files

8b24256

alphabetize and tidy files

a9f4101

no inline link syntax allowed?

987be2a

acozine force-pushed the update_docs_reqs branch from a0cff42 to 987be2a Compare June 14, 2021 20:46

ansibot removed the ci_verified Changes made in this PR are causing tests to fail. label Jun 14, 2021

acozine mentioned this pull request Jun 15, 2021

DaWGs (Documentation Working Group) Meeting Agenda 2021 ansible/community#579

Closed

samccann mentioned this pull request Jun 16, 2021

Add requirements.txt file for building documentation #73401

Closed

acozine changed the title ~~WIP: Update docs requirements list~~ Update docs requirements list Jun 18, 2021

ansibot added core_review In order to be merged, this PR must follow the core review workflow. has_issue and removed WIP This issue/PR is a work in progress. Nevertheless it was shared for getting input from peers. labels Jun 18, 2021

felixfontein mentioned this pull request Jun 19, 2021

Use antsibull sphinx extension #73170

Merged

webknjaz reviewed Jun 19, 2021

View reviewed changes

acozine merged commit 58f2638 into ansible:devel Jun 22, 2021

acozine deleted the update_docs_reqs branch June 22, 2021 18:58

felixfontein mentioned this pull request Jun 22, 2021

[2.11] Use sphinx_ansible_theme Sphinx theme in docs #75058

Merged

webknjaz reviewed Jun 23, 2021

View reviewed changes

ansible locked and limited conversation to collaborators Jul 20, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update docs requirements list #74956

Update docs requirements list #74956

acozine commented Jun 9, 2021

webknjaz Jun 9, 2021

acozine Jun 10, 2021

webknjaz Jun 10, 2021

acozine Jun 10, 2021

samccann commented Jun 10, 2021

samccann Jun 10, 2021

acozine Jun 14, 2021

webknjaz commented Jun 11, 2021

ansibot commented Jun 14, 2021

acozine commented Jun 14, 2021

webknjaz commented Jun 19, 2021

webknjaz Jun 19, 2021

webknjaz Jun 19, 2021

felixfontein Jun 22, 2021

webknjaz Jun 23, 2021

webknjaz Jun 19, 2021

acozine commented Jun 22, 2021

acozine commented Jun 22, 2021

webknjaz Jun 23, 2021

webknjaz commented Jun 23, 2021

		pip install --user -r requirements.txt
		pip install --user -r docs/docsite/known_good_reqs.txt

Update docs requirements list #74956

Update docs requirements list #74956

Conversation

acozine commented Jun 9, 2021

SUMMARY

ISSUE TYPE

COMPONENT NAME

ADDITIONAL INFORMATION

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

samccann commented Jun 10, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

webknjaz commented Jun 11, 2021

ansibot commented Jun 14, 2021

acozine commented Jun 14, 2021

webknjaz commented Jun 19, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

acozine commented Jun 22, 2021

acozine commented Jun 22, 2021

Choose a reason for hiding this comment

webknjaz commented Jun 23, 2021