sphinxupquote and textbackslash: Improper discretionary list.

[JulienPalard]

Describe the bug
Related to: python/python-docs-ja#22

While trying to reproduce python/python-docs-ja#22 I was able to produce:

%% Generated by Sphinx.
\def\sphinxdocclass{jsbook}
\documentclass[a4paper,10pt,dvipdfmx]{sphinxmanual}
\usepackage{sphinx}
\usepackage{hyperref}
\begin{document}
\sphinxupquote{\textbackslash{}x}
\end{document}

It is manually produced from c-api.tex from a cpython documentation japanese build, by manually bisecting the file to find a minimal reproducer.

To Reproduce

• Write the LaTeX example to a foo.tex file
• run /usr/bin/platex -kanji=utf8 -recorder foo.tex

Environment info

• OS: Debian 11
• Python version: 3.9.4
• Sphinx version: 3.2.1
• Extra tools:

$ /usr/bin/platex --version
e-pTeX 3.14159265-p3.8.3-191112-2.6 (utf8.euc) (TeX Live 2020/Debian)
kpathsea version 6.3.2
ptexenc version 1.3.8

Additional context

I don't understand the error message, I don't know how to tell if it's a Sphinx issue (that should not have generated this) or a platex issue (that should understand this), sry.

[JulienPalard]

I can reproduce it with Sphinx 4.0.2.

[jfbu]

Thanks for report. I reduced it to a pure LaTeX minimal example.

\documentclass{book}
%\usepackage[T1]{fontenc}

\usepackage{hyperref}
\begin{document}

\textbackslash\ DOES WORK

\let\savedtextbackslash\textbackslash

\savedtextbackslash\ DOES WORK

\discretionary{}{}{\savedtextbackslash} DOES WORK

\def\textbackslash{\discretionary{}{}{\savedtextbackslash}}%

BUT REDEFINED \textbackslash\ BREAKS IF T1 FONT ENCODING iS NOT LOADED

\end{document}

Japanese Sphinx seems affected from a situation similar to the abvoe if not loading T1 font encoding. In fact the \let\savedtextbackslash\textbackslash is root problem as it does not do what one one expects. The meaning again refers to \textbackslash creating potentially an infinite loop.

The infinite loop raises immediately Improper discretionary list error because it tries to inject a \discretionary inside a \discretionary.

I need to go into the guts of LaTeX so-called new font selection system which I have always considered ill-documented. The source code comments are antequated and one must fetch some information in the "LaTeX Companion" which is not free.

EDITED: the above explanation about infinite loop appears now to me to be wrong. a.t.t.o.w I don't know the complete explanation.

[jfbu]

My explanation "infinite loop" was not correct, but there was some recursion. Rather than trying to fight with LaTeX macros, I have opted to save the backslash as a box.

If you can, can you confirm #9305 fixes it for you?

[JulienPalard]

Hi! Merci Jean-François !

What you did here looks like magic to me!

At first glance, it fixes the issue (I just tried patching the .sty file and running xetex on c-api.tex and it worked), I'm now trying a full build of https://docs.python.org/ja/3/ with the current master to double check and I'll tell you.

Thanks!

[jfbu]

Merci pour la confirmation Julien ! Good that the xetex run worked too.

I hope the patch will work on the full project with uplatex and hope some other problem does not pop up :)

I was a bit confused in my comments. The \textbackslash (like all other \textsomething) is a somewhat complex thing which basically checks the current font encoding to see if the character is available and if not then causes a font in another encoding to be loaded as fallback. Some parts of this process are not allowed to happen inside a "discretionary". The patch uses a box to force the character to prerender. Then from inside the "discretionary" only a box copy to output operation will happen and this is allowed. It is written using primitive TeX syntax which is short and to the point. A priori, if \textbackslash̀ is at all licit by itself this should work.

[JulienPalard]

with the current master to double check and I'll tell you.

Full build went "better", it does no longer hit the same error, so this error is fixed now, thanks a lot!