Internationalization Support

[BlueGlassBlock]

Is your feature request related to a problem? Please describe.
mkdocs-material already supports a wide range of languages, should be great if this package supports more language rather than manually modifying all the templates or sticking with hard-coded English :)

Describe the solution you'd like
Allowing user to feed a data file (such as JSON/YAML) should be feasible enough. Original text could be used as a polyfill to ensure backward compatibility.

Describe alternatives you've considered
Utilizing the gettext module is also a considerable alternative, but I doubt it will be a little bit harder for end users to tweak them.

Also, we can centralize the translation context in the repositories to provide support for other languages natively.

[pawamoy]

Agreed, we need internationalization. We can probably reuse the same approach as Material for MkDocs, with a Jinja macro containing a dict with keys and translated values. Each handler would have its own macro, since each handler has its own set of templates.

[BlueGlassBlock]

I tried to tweak the code and made a prototype, however I'm not sure where to place the language files, since multiple themes could utilize the same language file. In that case, how could we provide customizations?

[pawamoy]

Different themes could use different wording, so I believe we should have one language file per theme. WDYT?

[BlueGlassBlock]

That seems reasonable to me. I'll look into it again in the following few days :)

[BlueGlassBlock]

Although the implementation is ugly, I made a working prototype.

Here's the branch: https://github.com/BlueGlassBlock/mkdocstrings-python/tree/master

[pawamoy]

Looks nice to me! Is there something in particular bothering you in your prototype?

[BlueGlassBlock]

Looks nice to me! Is there something in particular bothering you in your prototype?

I'm quite uncertain about handling symbols and capitalization (e.g. description Description Description:), since they might vary much between languages 😕

I'm proposing that we use dotted keys to indicate variants (description.normal description.capitalized, etc.), although I'm not quite certain about their names 😅

[pawamoy]

I see. Yeah I had the same doubts but eventually concluded this:

• having case-sensitive keys for all kinds of capitalization will be a PITA to translate for contributors
• case-sensitiveness might not even be a thing in some languages
• I believe string methods like upper, title and capitalize will do the right thing depending on the locale? at least better than what we could hardcode in translation files?

I have no idea about how to handle colons : though 😕

[pawamoy]

Hmmm...

There is no way to perform case conversions and character classifications according to the locale. For (Unicode) text strings these are done according to the character value only, while for byte strings, the conversions and classifications are done according to the ASCII value of the byte, and bytes whose high bit is set (i.e., non-ASCII bytes) are never converted or considered part of a character class such as letter or whitespace.

https://docs.python.org/3/library/locale.html#background-details-hints-tips-and-caveats

So we might have to actually add all combinations into the translation files...

[pawamoy]

I'm proposing that we use dotted keys to indicate variants (description.normal description.capitalized, etc.), although I'm not quite certain about their names sweat_smile

This, or the keys themselves have the expected casing: description, DESCRIPTION, Description:, etc.

[BlueGlassBlock]

This, or the keys themselves have the expected casing: description, DESCRIPTION, Description:, etc.

Sounds quite interesting, in this way the maintenance burden of the templates could be reduced to minimum :)

Could also possibly benefit translators too (by checking the original file less times)

[pawamoy]

Yeah. It would be a bit cumbersome to translate so many similar keys, but it is not that hard either.

[BlueGlassBlock]

Just submitted mkdocstrings/python#77

However mkdocstrings itself may require a small patch here (didn't find other ways to inject locale value):

diff --git a/src/mkdocstrings/plugin.py b/src/mkdocstrings/plugin.py
index 5233cf4..cde45a4 100644
--- a/src/mkdocstrings/plugin.py
+++ b/src/mkdocstrings/plugin.py
@@ -201,6 +201,7 @@ class MkdocstringsPlugin(BasePlugin):

         extension_config = {
             "theme_name": theme_name,
+            "locale": str(config["theme"]["locale"]),
             "mdx": config["markdown_extensions"],
             "mdx_configs": config["mdx_configs"],
             "mkdocstrings": self.config,

[pawamoy]

Do all themes accept this locale option? It will not always be specified, so it should maybe use config["theme"].get("locale", default_locale) instead.

[BlueGlassBlock]

Do all themes accept this locale option? It will not always be specified, so it should maybe use config["theme"].get("locale", default_locale) instead.

I checked mkdocs' code, it's always provided:

https://github.com/mkdocs/mkdocs/blob/afb66c1bea3eb07a917803a25d5e536edd2b8084/mkdocs/theme.py#L34

Code

class Theme:
    """
    A Theme object.

    Keywords:

        name: The name of the theme as defined by its entrypoint.

        custom_dir: User defined directory for custom templates.

        static_templates: A list of templates to render as static pages.

    All other keywords are passed as-is and made available as a key/value mapping.

    """

    def __init__(self, name: str | None = None, **user_config) -> None:
        self.name = name
        self._vars = {'name': name, 'locale': 'en'}

        # MkDocs provided static templates are always included
        package_dir = os.path.abspath(os.path.dirname(__file__))
        mkdocs_templates = os.path.join(package_dir, 'templates')
        self.static_templates = set(os.listdir(mkdocs_templates))

        # Build self.dirs from various sources in order of precedence
        self.dirs = []

        if 'custom_dir' in user_config:
            self.dirs.append(user_config.pop('custom_dir'))

        if name:
            self._load_theme_config(name)

        # Include templates provided directly by MkDocs (outside any theme)
        self.dirs.append(mkdocs_templates)

        # Handle remaining user configs. Override theme configs (if set)
        self.static_templates.update(user_config.pop('static_templates', []))
        self._vars.update(user_config)

        # Validate locale and convert to Locale object
        self._vars['locale'] = localization.parse_locale(self._vars['locale'])

It's just about whether the theme itself is localized

[BlueGlassBlock]

Ooh, I just found out that mkdocs-material used a different approach(theme.language) from the mkdocs ones(theme.locale), maybe we should provide fallback and respect config["theme"].get("language", "en") in case the locale is not set?

[pawamoy]

Yep, whatever works in all cases 🙂

[BlueGlassBlock]

Sorry for the late update, however I thought I underestimated the pitfalls in i18n implementations across MkDocs :(

So, according to MkDocs Material - Changing the Language, they use a different approach to determine which language the site uses (using dashes (-) and the theme.language key rather than somewhat standard theme.locale)

I tried to use mkdocs.localization.parse_locale to unify the language codes, but it turned out to have some pitfalls.

Original Language Code	Result of Babel	MkDocs Polyfill
ku-IQ	error	ku_IQ
sh	error	sh
pt-BR	pt_BR	pt_BR
tl	fil_PH	tl
zh-Hant	zh_Hant	zh_Hant
zh-TW	zh_Hant_TW	zh_TW
zh	zh	zh

So I guess we should just copy mkdocs-material's approach and stuck with theme.language, forgetting theme.locale.

[ZhiyuanChen]

Is it possible to support multiple languages?
I used to use MkDocs-i18n to generate files for each language. But it won't work if import other site like

  - mkdocstrings:
      handlers:
        python:
          import:
            - https://docs.python.org/3/objects.inv

[BlueGlassBlock]

Is it possible to support multiple languages? I used to use MkDocs-i18n to generate files for each language.

IMO docstrings basically only come in one language, which is what mkdocstrings display. Maybe your request belongs to another issue to allow importing the same namespace with different aliases: e.g. distinguish between English and Chinese version of Python docs.

[ZhiyuanChen]

Is it possible to support multiple languages? I used to use MkDocs-i18n to generate files for each language.

IMO docstrings basically only come in one language, which is what mkdocstrings display. Maybe your request belongs to another issue to allow importing the same namespace with different aliases: e.g. distinguish between English and Chinese version of Python docs.

Yes, but there are other non-mkdocstrings-generated pages, like index page and some tutorial pages.