Skip to content

Commit

Permalink
Prepare 8.2.13+insiders-4.14.0 release
Browse files Browse the repository at this point in the history
  • Loading branch information
squidfunk committed May 5, 2022
1 parent 3666d09 commit 8d2846f
Show file tree
Hide file tree
Showing 7 changed files with 148 additions and 3 deletions.
5 changes: 5 additions & 0 deletions CHANGELOG
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
mkdocs-material-8.2.13+insiders-4.14.0 (2022-05-05)

* Added Chinese language support to built-in search plugin
* Fixed all-numeric page titles raising error in social plugin

mkdocs-material-8.2.13 (2022-05-02)

* Fixed #3865: Tags index links to tagged pages 404 on Windows
Expand Down
91 changes: 91 additions & 0 deletions docs/blog/2022/chinese-search-support.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
---
template: overrides/blog.html
title: Chinese search support
description: >
Insiders adds Chinese language support for the built-in search plugin – a
feature that has been requested many times
hide:
- feedback
---

# Chinese search support – 中文搜索​支持

__Insiders adds experimental Chinese language support for the [built-in search
plugin] – a feature that has been requested for a long time given the large
number of Chinese users.__

<aside class="mdx-author" markdown>
![@squidfunk][@squidfunk avatar]

<span>__Martin Donath__ · @squidfunk</span>
<span>
:octicons-calendar-24: May 5, 2022 ·
:octicons-clock-24: 5 min read ·
[:octicons-tag-24: 8.2.13+insiders-4.14.0][insiders-4.14.0]
</span>
</aside>

[built-in search plugin]: ../../setup/setting-up-site-search.md#built-in-search-plugin
[@squidfunk avatar]: https://avatars.githubusercontent.com/u/932156
[insiders-4.14.0]: ../../insiders/changelog.md#4.14.0

---

After the United States and Germany, the third-largest country of origin of
Material for MkDocs users is China. For a long time, the built-in search plugin
didn't allow for proper segmentation of Chinese characters, mainly due to
missing support in [lunr-languages] which is used for search tokenization and
stemming. The latest Insiders release adds long-awaited Chinese language support
for the built-in search plugin, something that has been requested by many users.

_Material for MkDocs終於​支持​中文​了!文本​被​正確​分割​並且​更​容易​找到。_
{ style="display: inline" }

_This article explains how to set up Chinese language support for the built-in
search plugin in a few minutes._
{ style="display: inline" }

[lunr-languages]: https://github.com/MihaiValentin/lunr-languages

## Configuration

Chinese language support for Material for MkDocs is provided by [jieba], an
excellent Chinese text segmentation library. If [jieba] is installed, the
built-in search plugin automatically detects Chinese characters and runs them
through the segmenter. You can install [jieba] with:

```
pip install jieba
```

The next step is only required if you specified the [separator] configuration
in `mkdocs.yml`. Text is segmented with [zero-width whitespace] characters, so
it renders exactly the same in the search modal. Adjust `mkdocs.yml` so that
the [separator] includes the `\u200b` character:

``` yaml
plugins:
- search:
separator: '[\\s\\u200b\\-]'
```

That's all that is necessary.

## Usage

If you followed the instructions in the configuration guide, Chinese words will
now be tokenized using [jieba]. Try searching for
[:octicons-search-24: 支持][q=支持] to see how it integrates with the
built-in search plugin.

---

Note that this is an experimental feature, and I, @squidfunk, am not
proficient in Chinese (yet?). If you find a bug or think something can be
improved, please [open an issue].

[jieba]: https://pypi.org/project/jieba/
[zero-width whitespace]: https://en.wikipedia.org/wiki/Zero-width_space
[separator]: ../../setup/setting-up-site-search.md#separator
[q=支持]: ?q=支持
[open an issue]: https://github.com/squidfunk/mkdocs-material/issues/new/choose
36 changes: 34 additions & 2 deletions docs/blog/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,40 @@ search:

# Blog

## [Chinese search support – 中文搜索​支持]

__Insiders adds experimental Chinese language support for the [built-in search
plugin] – a feature that has been requested for a long time given the large
number of Chinese users.__

<aside class="mdx-author" markdown>
![@squidfunk][@squidfunk avatar]

<span>__Martin Donath__ · @squidfunk</span>
<span>
:octicons-calendar-24: May 5, 2022 ·
:octicons-clock-24: 5 min read ·
[:octicons-tag-24: 8.2.13+insiders-4.14.0][insiders-4.14.0]
</span>
</aside>

---

After the United States and Germany, the third-largest country of origin of
Material for MkDocs users is China. For a long time, the built-in search plugin
didn't allow for proper segmentation of Chinese characters, mainly due to
missing support in [lunr-languages] which is used for search tokenization and
stemming. The latest Insiders release adds long-awaited Chinese language support
for the built-in search plugin, something that has been requested by many users.

[:octicons-arrow-right-24: Continue reading][Chinese search support – 中文搜索​支持]

[built-in search plugin]: ../setup/setting-up-site-search.md#built-in-search-plugin
[@squidfunk avatar]: https://avatars.githubusercontent.com/u/932156
[insiders-4.14.0]: ../insiders/changelog.md#4.14.0
[lunr-languages]: https://github.com/MihaiValentin/lunr-languages
[Chinese search support – 中文搜索​支持]: 2022/chinese-search-support.md

## [The past, present and future]

__2021 was a fantastic year for this project as we shipped many new awesome
Expand All @@ -29,8 +63,6 @@ project sustainable.__
</span>
</aside>

[@squidfunk avatar]: https://avatars.githubusercontent.com/u/932156

---

Today, together, MkDocs and Material for MkDocs are among the most popular
Expand Down
5 changes: 5 additions & 0 deletions docs/insiders/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,11 @@ template: overrides/main.html

## Material for MkDocs Insiders

### 4.14.0 <small>_ May 5, 2022</small> { id="4.14.0" }

- Added Chinese language support to built-in search plugin
- Fixed all-numeric page titles raising error in social plugin

### 4.13.2 <small>_ April 30, 2022</small> { id="4.13.2" }

- Improved caching of downloaded resources in privacy plugin
Expand Down
5 changes: 4 additions & 1 deletion docs/insiders/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -174,7 +174,8 @@ which are currently exclusively available to sponsors:

<div class="mdx-columns" markdown>

- [x] [Tag icons] :material-new-box:
- [x] [Chinese search support] :material-new-box:
- [x] [Tag icons] :material-new-box:
- [x] [Card grids] :material-new-box:
- [x] [Offline plugin]
- [x] [Privacy plugin]
Expand Down Expand Up @@ -262,12 +263,14 @@ are released for general availability.
#### $ 12,000 – Piri Piri

- [x] [Annotations]
- [x] [Chinese search support]
- [x] [Navigation icons]
- [ ] Navigation status badges
- [ ] Navigation pruning
- [ ] Blog

[Annotations]: ../reference/annotations.md
[Chinese search support]: ../blog/2022/chinese-search-support.md
[Navigation icons]: ../reference/index.md#setting-the-page-icon

#### $ 14,000 – Goat's Horn
Expand Down
7 changes: 7 additions & 0 deletions docs/setup/setting-up-site-search.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,12 @@ The following configuration options are supported:
part of this list by automatically falling back to the stemmer yielding the
best result.

!!! tip "Chinese search support – 中文搜索​支持"

Material for MkDocs recently added __experimental language support for
Chinese__ as part of [Insiders]. [Read the blog article][chinese search]
to learn how to set up search for Chinese in a matter of minutes.

`separator`{ #search-separator }

: :octicons-milestone-24: Default: _automatically set_ – The separator for
Expand Down Expand Up @@ -143,6 +149,7 @@ them at your own risk.
[search support]: https://github.com/squidfunk/mkdocs-material/releases/tag/0.1.0
[lunr]: https://lunrjs.com
[lunr-languages]: https://github.com/MihaiValentin/lunr-languages
[chinese search]: ../blog/2022/chinese-search-support.md
[lunr's default tokenizer]: https://github.com/olivernn/lunr.js/blob/aa5a878f62a6bba1e8e5b95714899e17e8150b38/lunr.js#L413-L456
[site language]: changing-the-language.md#site-language
[tokenizer lookahead]: #tokenizer-lookahead
Expand Down
2 changes: 2 additions & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,8 @@ nav:
- Changelog: insiders/changelog.md
- Blog:
- blog/index.md
- 2022:
- blog/2022/chinese-search-support.md
- 2021:
- blog/2021/the-past-present-and-future.md
- blog/2021/excluding-content-from-search.md
Expand Down

0 comments on commit 8d2846f

Please sign in to comment.