Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Linkcheck crashes in 3.4.0 #8565

Closed
nijel opened this issue Dec 20, 2020 · 3 comments
Closed

Linkcheck crashes in 3.4.0 #8565

nijel opened this issue Dec 20, 2020 · 3 comments
Assignees
@francoisfreitag
Labels
builder:linkcheck type:bug
Milestone
3.4.1

Comments

@nijel
Copy link
Contributor

nijel commented Dec 20, 2020

Describe the bug

When running linkcheck in Weblate docs, it crashes with:

 Exception in thread Thread-2:
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.8.6/x64/lib/python3.8/threading.py", line 932, in _bootstrap_inner

Exception occurred:
    self.run()
  File "/opt/hostedtoolcache/Python/3.8.6/x64/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/hostedtoolcache/Python/3.8.6/x64/lib/python3.8/site-packages/sphinx/builders/linkcheck.py", line 298, in check_thread
    self.wqueue.task_done()
  File "/opt/hostedtoolcache/Python/3.8.6/x64/lib/python3.8/queue.py", line 74, in task_done
Error:     raise ValueError('task_done() called too many times')
ValueError: task_done() called too many times
  File "/opt/hostedtoolcache/Python/3.8.6/x64/lib/python3.8/queue.py", line 233, in _put
    heappush(self.queue, item)
TypeError: '<' not supported between instances of 'int' and 'NoneType'

To Reproduce
Steps to reproduce the behavior:

<Paste your command-line here which cause the problem>

$ git clone https://github.com/WeblateOrg/weblate.git
$ cd weblate
$ pip install -r docs/requirements.txt
$ cd docs
$ make linkcheck

Expected behavior
No crash :-)

Your project
https://github.com/WeblateOrg/weblate/tree/master/docs

Screenshots
CI failure: https://github.com/WeblateOrg/weblate/runs/1585580811?check_suite_focus=true

Environment info

  • OS: Linux
  • Python version: 3.8.6
  • Sphinx version: 3.4.0
  • Sphinx extensions: several, but should not be relevant here
  • Extra tools: none involved

Additional context
Add any other context about the problem here.

  • [e.g. URL or Ticket]
@nijel nijel added the type:bug label Dec 20, 2020
nijel added a commit to WeblateOrg/weblate that referenced this issue Dec 20, 2020
@nijel
Docs: Avoid Sphinx 3.4.0
8d57b86 
It crashes CI, see sphinx-doc/sphinx#8565
@francoisfreitag
Copy link
Contributor

Hi,
Thanks for the report! I can reproduce the issue.
I’ll be looking into fixing it later today.

@francoisfreitag francoisfreitag self-assigned this Dec 21, 2020
@tk0miya tk0miya added this to the 3.4.1 milestone Dec 21, 2020
@nijel nijel mentioned this issue Dec 21, 2020
Closed
@francoisfreitag
Copy link
Contributor

I understand what is wrong:

Linkcheck organizes the urls to checks in a PriorityQueue. The items are tuples (priority, url, docname, lineno). For some links, the get_node_line() returns None, I’m guessing the line information is not available on that node.
A tuple where the lineno is None is not comparable with tuples that have an integer lineno (None and int aren’t comparable), and PriorityQueue items must be comparable (see https://bugs.python.org/issue31145).

That issue only manifests when a link has no lineno and a document contains two links to the same URL. In Weblate README.rst:

.. image:: https://s.weblate.org/cdn/Logo-Darktext-borders.png
   :alt: Weblate
   :target: https://weblate.org/

And:

Install it, or use the Hosted Weblate service at `weblate.org`_.

.. _weblate.org: https://weblate.org/

I have a minimal regression test and will investigate how the line number is acquired tomorrow. If that’s reasonable, I think it would be more helpful to have the original line number where the URL appeared. If it is too big of a change for a fix release, I’ll probably end-up wrapping the data in a class that handles the comparison between no line number information and a line number information (and its variants).

@LeSuisse LeSuisse mentioned this issue Dec 22, 2020
Closed
francoisfreitag added a commit to francoisfreitag/sphinx that referenced this issue Dec 22, 2020
@francoisfreitag
Ensure linkcheck items are comparable
b9648e5 
Linkcheck organizes the urls to checks in a PriorityQueue. The items are
tuples (priority, url, docname, lineno).

Tuples where the lineno is `None` are not comparable with tuples that
have an integer lineno, and PriorityQueue items must be comparable (see
https://bugs.python.org/issue31145).

Fixes an issue when a document contains two links to the same URL, one
with an int line number and the other without line number metadata (such
as an image :target: attribute).

Close sphinx-doc#8565
francoisfreitag added a commit to francoisfreitag/sphinx that referenced this issue Dec 22, 2020
@francoisfreitag
Ensure linkcheck items are comparable
26347e5 
Linkcheck organizes the urls to checks in a PriorityQueue. The items are
tuples (priority, url, docname, lineno).

Tuples where the lineno is `None` are not comparable with tuples that
have an integer lineno, and PriorityQueue items must be comparable (see
https://bugs.python.org/issue31145).

Fixes an issue when a document contains two links to the same URL, one
with an int line number and the other without line number metadata (such
as an image :target: attribute).

Close sphinx-doc#8565
francoisfreitag added a commit to francoisfreitag/sphinx that referenced this issue Dec 22, 2020
@francoisfreitag
Ensure linkcheck items are comparable
5c36c4f 
Linkcheck organizes the urls to checks in a PriorityQueue. The items are
tuples (priority, url, docname, lineno).

Tuples where the lineno is `None` are not comparable with tuples that
have an integer lineno, and PriorityQueue items must be comparable (see
https://bugs.python.org/issue31145).

Fixes an issue when a document contains two links to the same URL, one
with an int line number and the other without line number metadata (such
as an image :target: attribute).

Using 0 instead of None to represent no line number should not lead to
observable changes, the result logger only logs the line number when it
is truthy.

Close sphinx-doc#8565
francoisfreitag added a commit to francoisfreitag/sphinx that referenced this issue Dec 22, 2020
@francoisfreitag
Ensure linkcheck items are comparable
b469555 
Linkcheck organizes the URLs to checks in a PriorityQueue. The items are
tuples (priority, url, docname, lineno).

Tuples where the lineno is `None` are not comparable with tuples that
have an integer lineno, and PriorityQueue items must be comparable (see
https://bugs.python.org/issue31145).

Fixes an issue when a document contains two links to the same URL, one
with an int line number and the other without line number metadata (such
as an image :target: attribute).

Using 0 instead of None to represent no line number should not lead to
observable changes, the result logger only logs the line number when it
is truthy.

Close sphinx-doc#8565
@francoisfreitag francoisfreitag mentioned this issue Dec 22, 2020
Merged
francoisfreitag added a commit to francoisfreitag/sphinx that referenced this issue Dec 22, 2020
@francoisfreitag
Ensure linkcheck items are comparable
550ad47 
Linkcheck organizes the URLs to checks in a PriorityQueue. The items are
tuples (priority, url, docname, lineno).

Tuples where the lineno is `None` are not comparable with tuples that
have an integer lineno, and PriorityQueue items must be comparable (see
https://bugs.python.org/issue31145).

Fixes an issue when a document contains two links to the same URL, one
with an int line number and the other without line number metadata (such
as an image :target: attribute).

Using 0 instead of None to represent no line number should not lead to
observable changes, the result logger only logs the line number when it
is truthy.

Close sphinx-doc#8565
francoisfreitag added a commit to francoisfreitag/sphinx that referenced this issue Dec 22, 2020
@francoisfreitag
Ensure linkcheck items are comparable
f018c69 
Linkcheck organizes the URLs to checks in a PriorityQueue. The items are
tuples (priority, url, docname, lineno).

Tuples where the lineno is `None` are not comparable with tuples that
have an integer lineno, and PriorityQueue items must be comparable (see
https://bugs.python.org/issue31145).

Fixes an issue when a document contains two links to the same URL, one
with an int line number and the other without line number metadata (such
as an image :target: attribute).

Using 0 instead of None to represent no line number should not lead to
observable changes, the result logger only logs the line number when it
is truthy.

Close sphinx-doc#8565
francoisfreitag added a commit to francoisfreitag/sphinx that referenced this issue Dec 22, 2020
@francoisfreitag
Ensure linkcheck items are comparable
6b80659 
Linkcheck organizes the URLs to checks in a PriorityQueue. The items are
tuples (priority, url, docname, lineno).

Tuples where the lineno is `None` are not comparable with tuples that
have an integer lineno, and PriorityQueue items must be comparable (see
https://bugs.python.org/issue31145).

Fixes an issue when a document contains two links to the same URL, one
with an int line number and the other without line number metadata (such
as an image :target: attribute).

Using 0 instead of None to represent no line number should not lead to
observable changes, the result logger only logs the line number when it
is truthy.

Close sphinx-doc#8565
francoisfreitag added a commit to francoisfreitag/sphinx that referenced this issue Dec 22, 2020
@francoisfreitag
Ensure linkcheck items are comparable
a9a83f0 
Linkcheck organizes the URLs to checks in a PriorityQueue. The items are
tuples (priority, url, docname, lineno).

Tuples where the lineno is `None` are not comparable with tuples that
have an integer lineno, and PriorityQueue items must be comparable (see
https://bugs.python.org/issue31145).

Fixes an issue when a document contains two links to the same URL, one
with an int line number and the other without line number metadata (such
as an image :target: attribute).

Using 0 instead of None to represent no line number should not lead to
observable changes, the result logger only logs the line number when it
is truthy.

Close sphinx-doc#8565
francoisfreitag added a commit to francoisfreitag/sphinx that referenced this issue Dec 22, 2020
@francoisfreitag
Ensure linkcheck items are comparable
a1b8b1f 
Linkcheck organizes the URLs to checks in a PriorityQueue. The items are
tuples (priority, url, docname, lineno).

Tuples where the lineno is `None` are not comparable with tuples that
have an integer lineno, and PriorityQueue items must be comparable (see
https://bugs.python.org/issue31145).

Fixes an issue when a document contains two links to the same URL, one
with an int line number and the other without line number metadata (such
as an image :target: attribute).

Using 0 instead of None to represent no line number should not lead to
observable changes, the result logger only logs the line number when it
is truthy.

Close sphinx-doc#8565
@tk0miya tk0miya closed this as completed Dec 23, 2020
@nijel
Copy link
Contributor Author

nijel commented Dec 23, 2020

Thanks for quickly addressing this issue!

@dependabot dependabot bot mentioned this issue Mar 15, 2021
Closed
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jul 17, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@francoisfreitag francoisfreitag

Labels
builder:linkcheck type:bug
Projects
None yet
Milestone
3.4.1
Development

No branches or pull requests
3 participants
@nijel @tk0miya @francoisfreitag