Close #6239: latex: Support to build Chinese documents

[tk0miya]

Feature or Bugfix

• Feature

Purpose

• refs: Use xeCJK for fontenc as default with XeLaTex and Chinese docs #6239

So far, Sphinx can't build Chinese document correctly by default settings. So Chinese users would customize their settings to build PDF correctly. I'm afraid this breaks their project. But this surely improves Sphinx. So it would be better to add this into next minor release.

[codecov]

Codecov Report

Merging #6398 into 2.0 will increase coverage by 0.05%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##              2.0    #6398      +/-   ##
==========================================
+ Coverage   83.59%   83.64%   +0.05%     
==========================================
  Files         282      280       -2     
  Lines       40429    40404      -25     
  Branches     5956     5953       -3     
==========================================
+ Hits        33796    33797       +1     
+ Misses       5301     5281      -20     
+ Partials     1332     1326       -6

Impacted Files	Coverage Δ
sphinx/writers/latex.py	81.78% <ø> (+0.13%)	⬆️
tests/test_build_latex.py	95.31% <100%> (+0.04%)	⬆️
sphinx/builders/latex/util.py	86.48% <100%> (+6.48%)	⬆️
sphinx/builders/latex/__init__.py	80.85% <100%> (+1.95%)	⬆️
sphinx/__init__.py
sphinx/make_mode.py

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 804d5d8...4ded940. Read the comment docs.

[jfbu]

Should this be applied also for Korean language?

[jfbu]

@tk0miya I wonder if the 'fontpkg' key should also be given better setting? I guess GNU FreeFont does not work for CJK scripts? or perhaps the \setmainfont etc... will apply only to Latin scripts in the document, and package xeCJK takes care of the Chinese ideograms only, thus there is no conflict? sadly I am completely unable to read the xeCJK documentation..

[tk0miya]

Unfortunately, I don't know about both Chinese and Korean languages. So there are nothing to do to me for each language. Indeed, setting some fontpkg would be better. But we need someone's help to improve it...

[zhsj]

from the xeCJK doc, it says its functions are:

1. set font for CJK and English separately.
2. auto ignore blank space between CJK character, and keep other spaces meanwhile. allow line break between alphabet and non-punctuation CJK character.
3. provides a variety of punctuation method.
4. auto adjust blank space between CJK and English.

[zhsj]

xeCJK cares Korean too, but I think the default Korean font should be set manually.(For Chinese, if you don't set default Chinese font, it will fallback to fandol font.)

From https://www.overleaf.com/learn/latex/Korean, it should be

\usepackage{xeCJK}
 
\setCJKmainfont{UnGungseo.ttf}
\setCJKsansfont{UnGungseo.ttf}
\setCJKmonofont{gulim.ttf}

[tk0miya]

@zhsj Are you saying xeCJK and XeLaTeX are also good for default settings for Korean? In addition, have you check the font setting is working fine with Sphinx? If so, I'll add them to next release.

[zhsj]

Nor can I speak/read Korean. So I'm not sure what is better to build PDF in Korean.

I tried to build in a docker container with xeCJK for Korean doc.

docker run --rm -it debian:buster-slim bash
apt install texlive texlive-xetex texlive-lang-korean latexmk xindy wget make python3-venv fonts-freefont-otf
wget -O- https://github.com/python/cpython/archive/3.7.tar.gz | tar xz
wget -O- https://github.com/python/python-docs-ko/archive/3.7.tar.gz | tar xz
cd cpython-3.7/Doc/
mkdir -p locales/ko
ln -sfn /python-docs-ko-3.7 locales/ko/LC_MESSAGES
make venv
make latex SPHINXOPTS='-D language=ko -D gettext_compact=0 -D latex_engine=xelatex -D latex_elements.inputenc= -D latex_elements.fontenc=\\usepackage{xeCJK}\\setCJKmainfont{UnBatang}\\setCJKsansfont{UnDotum}\\setCJKmonofont{UnTaza}'
cd build/latex
make tutorial.pdf

\usepackage{xeCJK}
\setCJKmainfont{UnBatang}
\setCJKsansfont{UnDotum}
\setCJKmonofont{UnTaza}

Another way, which is used for https://docs.python.org/ko/3/download.html, the PDF is built with

https://github.com/python/docsbuild-scripts/blob/c006512/build_docs.py#L74

\usepackage{kotex}
\setmainhangulfont{UnBatang}
\setsanshangulfont{UnDotum}
\setmonohangulfont{UnTaza}

Maybe @JulienPalard and @flowdas can have some comments too.

kotex(left) and xeCJK(right) both work fine.

But I have no idea which fonts are preferred in PDF. I find there are many fonts in texlive-lang-korean

texlive-lang-korean: /usr/share/texlive/texmf-dist/fonts/truetype/public/baekmuk/batang.ttf
texlive-lang-korean: /usr/share/texlive/texmf-dist/fonts/truetype/public/baekmuk/dotum.ttf
texlive-lang-korean: /usr/share/texlive/texmf-dist/fonts/truetype/public/baekmuk/gulim.ttf
texlive-lang-korean: /usr/share/texlive/texmf-dist/fonts/truetype/public/baekmuk/hline.ttf
texlive-lang-korean: /usr/share/texlive/texmf-dist/fonts/truetype/public/unfonts-core/UnBatang.ttf
texlive-lang-korean: /usr/share/texlive/texmf-dist/fonts/truetype/public/unfonts-core/UnBatangBold.ttf
texlive-lang-korean: /usr/share/texlive/texmf-dist/fonts/truetype/public/unfonts-core/UnDinaru.ttf
texlive-lang-korean: /usr/share/texlive/texmf-dist/fonts/truetype/public/unfonts-core/UnDinaruBold.ttf
texlive-lang-korean: /usr/share/texlive/texmf-dist/fonts/truetype/public/unfonts-core/UnDinaruLight.ttf
texlive-lang-korean: /usr/share/texlive/texmf-dist/fonts/truetype/public/unfonts-core/UnDotum.ttf
texlive-lang-korean: /usr/share/texlive/texmf-dist/fonts/truetype/public/unfonts-core/UnDotumBold.ttf
texlive-lang-korean: /usr/share/texlive/texmf-dist/fonts/truetype/public/unfonts-core/UnGraphic.ttf
texlive-lang-korean: /usr/share/texlive/texmf-dist/fonts/truetype/public/unfonts-core/UnGraphicBold.ttf
texlive-lang-korean: /usr/share/texlive/texmf-dist/fonts/truetype/public/unfonts-core/UnGungseo.ttf
texlive-lang-korean: /usr/share/texlive/texmf-dist/fonts/truetype/public/unfonts-core/UnPilgi.ttf
texlive-lang-korean: /usr/share/texlive/texmf-dist/fonts/truetype/public/unfonts-core/UnPilgiBold.ttf
texlive-lang-korean: /usr/share/texlive/texmf-dist/fonts/truetype/public/unfonts-extra/UnJamoBatang.ttf
texlive-lang-korean: /usr/share/texlive/texmf-dist/fonts/truetype/public/unfonts-extra/UnJamoDotum.ttf
texlive-lang-korean: /usr/share/texlive/texmf-dist/fonts/truetype/public/unfonts-extra/UnJamoNovel.ttf
texlive-lang-korean: /usr/share/texlive/texmf-dist/fonts/truetype/public/unfonts-extra/UnJamoSora.ttf
texlive-lang-korean: /usr/share/texlive/texmf-dist/fonts/truetype/public/unfonts-extra/UnPen.ttf
texlive-lang-korean: /usr/share/texlive/texmf-dist/fonts/truetype/public/unfonts-extra/UnPenheulim.ttf
texlive-lang-korean: /usr/share/texlive/texmf-dist/fonts/truetype/public/unfonts-extra/UnPilgia.ttf
texlive-lang-korean: /usr/share/texlive/texmf-dist/fonts/truetype/public/unfonts-extra/UnShinmun.ttf
texlive-lang-korean: /usr/share/texlive/texmf-dist/fonts/truetype/public/unfonts-extra/UnTaza.ttf
texlive-lang-korean: /usr/share/texlive/texmf-dist/fonts/truetype/public/unfonts-extra/UnVada.ttf
texlive-lang-korean: /usr/share/texlive/texmf-dist/fonts/truetype/public/unfonts-extra/UnYetgul.ttf

[tk0miya]

I think helps from some Korean LaTeX users are needed to support Korean. Because it is hard to decide what is good default settings for them to me. How about asking this to users group?