NLog Wiki cannot be googled, because blocked by Github

[snakefoot]

Since May 2020 then Github have blocked the Google-Crawler from indexing any Wiki-pages. Github has taken this step to avoid spam-wiki-pages from reducing their own Google-page-rank.

This means the entire NLog documentation is much more difficult to find. Question is whether the entire Wiki should be moved to GitHub-pages, since they are not affected by the block. But will probably not have the Open-Wiki-Editor.

[nelsonjchen]

Here's a more updated discussion if anyone is interested: community/community#4992

[snakefoot]

Thank you for the link. Seems GitHub-users are left on their own on this one.

I asked if GitHub could not change repository-wiki to become like GitHub-pages (available from sub-domain, so not affecting the github-domain), and received the following reply from Github-support:

If you moved the wiki content of NLog/NLog to a GitHub Pages site published from that repo, it would appear at https://nlog-project.org/nlog

That's because you have an "organization site" on your NLog/NLog.github.io repository that has a custom domain applied, and domains on organization sites automatically apply to all of the org's project sites, too.

You can read more about this here:

https://docs.github.com/en/github/working-with-github-pages/about-github-pages#types-of-github-pages-sites

I'm afraid we don't have any built in tooling to convert repository wikis to GitHub Pages sites.

First you'd need to clone the wiki files locally:

https://docs.github.com/en/communities/documenting-your-project-with-wikis/adding-or-editing-wiki-pages#adding-or-editing-wiki-pages-locally

You can then move them to a location set to publish the Pages site. Either their own branch, or the docs folder:

https://docs.github.com/en/pages/getting-started-with-github-pages/configuring-a-publishing-source-for-your-github-pages-site

You will then need to script or manually change the wiki format URLs to proper markdown format.

GitHub Pages uses relative links, so you won't need to fully qualify all the URLs.

So, for example, you can change:

Read the [[documentation]] or [[Help|get help]].

to
Read the [documentation](documentation.md) or [Help](get%20help.md).

Alternatively, you may be able to use pandoc and the script below to do the conversion for you:

https://github.com/philipashlock/mediawiki-to-markdown

But I haven't tested it myself so I can't guarantee it will work.

I'm afraid GitHub Pages can't be made open in the same way as wikis, no. Edits would need to be made with pull requests.

[nelsonjchen]

I'm afraid GitHub Pages can't be made open in the same way as wikis, no. Edits would need to be made with pull requests.

Yeah, the closest you can get is something like this: tokio-rs/website#109 and maybe an auto-approve and merge action of some sort.

[nelsonjchen]

https://docs.google.com/spreadsheets/d/1xWYtCiAHzrUwwOtINLpyABJbR57tm7vVuUqRCtPTUv8/edit?usp=sharing

[snakefoot]

@nelsonjchen Thank you very much for providing github-wiki-see, since actuallly the only way to perform google-searches of the NLog-Wiki.

[MichelZ]

Would it be a possibility to convert the wiki to static HTML to host on nlog-project.org?
e.g. using mkdocs, it also allows editing of pages by pointing back to the github wiki AFAIK.
mkdocs/mkdocs#2371

[nelsonjchen]

If that's done, please let me know to disable the mirroring on github-wiki-see. The Shellcheck project does this: koalaman/shellcheck#2487. But also please make sure to produce a sitemap and monitor the indexing progress with Google Search Console and similar.

[304NotModified]

Would it be a possibility to convert the wiki to static HTML to host on nlog-project.org? e.g. using mkdocs, it also allows editing of pages by pointing back to the github wiki AFAIK. mkdocs/mkdocs#2371

Pull Request would be great!

[MichelZ]

Pull Request would be great!

I can try. What level of automation are you looking for? Is it OK if I just make it buildable manually, and then it needs to be pushed to GH pages manually, or is the only way to go a fully automated process whenever the wiki gets edited?

[MichelZ]

So far I have this:

Clone the wiki repo, add the following file to the root:

mkdocs.yml

site_name: NLog Wiki
site_url: https://nlog-project.org/wiki
repo_url: https://github.com/NLog/NLog/wiki
docs_dir: ../NLog.Wiki
strict: false
plugins:
  - search
  - ezlinks
  - github-wiki-edit-url
  - redirects:
      redirect_maps:
        index.md: Home.md
theme:
  name: readthedocs
  logo: https://nlog-project.org/images/NLog-logo-only_small.png

Install/Build:
pip install --upgrade pip && pip install mkdocs mkdocs-github-wiki-edit-url mkdocs-ezlinks-plugin mkdocs-redirects
mkdocs build

check it out on a local webserver:
mkdocs serve

It generates a sitemap.xml which can be used for crawlers

Let me know what you think

[snakefoot]

Created NLog/NLog.github.io#224 for anyone that has experience with Github-pages.