Taking MkDocs to the next level

[pawamoy]

Update, April 22

All the discussions created in the private space mentioned below were moved into the public discussions of the MkDocs main repository 🙂

---

Hello everyone,

I'm Timothée aka @pawamoy, you might have met me here and there in the MkDocs ecosystem. I maintain mkdocstrings and had some very nice interactions with some of you :)
I've been participating and collaborating on MkDocs as well for some time now, helping answering questions from users, and sending a few pull requests.

After some discussions with @tomchristie (original MkDocs author) and @squidfunk (Material for MkDocs author), we've come to the realization that it's high time we expanded our team, especially in light of recent events. Given the prolonged periods where MkDocs has relied heavily on individual contributors, we believe it's crucial to onboard more people to ensure the sustainability and growth of the project.

In line with this goal, @tomchristie has created a private space (a repository in the MkDocs GitHub organization with the Discussions feature activated) where we can foster better communication and collaboration among community members, by inviting authors of MkDocs themes and plugins, and authors of Markdown extensions. This space will serve as a platform for suggesting ideas for MkDocs and discussing how we can improve MkDocs to better serve plugin authors like ourselves. It will also be a place to discuss about project organization (like issue management, policies, onboarding new maintainers, etc.), while ensuring progress and maintaining focus on our objectives.

We believe that your involvement would greatly benefit the MkDocs community, and we'd be thrilled to have you on board. If you're interested in joining us, please let us know! This could be a way for us to get more maintainers on the team too.

I will tag some authors and contributors here directly, in alphabetic order:

@Andre601 @athackst @bczsalba @blueswen @byrnereese @chrieke @daizutabi @EddyLuten @facelessuser @fralau @greenape @Guts @JakubAndrysek @koaning @LukeCarrier @lyz-code @mondeja @ofek @ojacques @six-two @smarie @timmeinerzhagen @timvink @unverbuggt @waylan @wilhelmer

[Andre601]

As I stated in another discussion, my knowledge in Python is limited, so I wouldn't be one to provide code contributions or even suggestions (i.e. code improvements) in that area.
At most could I provide contributions and/or suggestions to improving documentation of MkDocs itself. Tho, so far I haven't really see a general need of improving the already existing documentation that much (Outside of one or two changes I already suggested in the past).

I would be open for joining, tho, I hope there are some clear guidelines regarding team organization and specific procedures on stuff like PR reviews, discussing new features/changes, etc. to ensure things are handled in a structured way.

[pawamoy]

Thanks for your interest @Andre601! The space we created is exactly meant to discuss things like how to organize the team around issues, PRs, etc.. You don't need Python skills, we'd very much value your experience in other domains too (such as docs) :)

[trel]

I’d like to follow along as well.

[Andre601]

Personally speaking, I have my mixed feelings on this too. And I say that while I was there myself.

On one hand is it a good idea to collect some info like that, as that way you don't have unrelated people just sliding in, giving feedback and suggestions that may have nothing to do with the topics in question.

But on the other hand does it also create a situation of secrecy, where people are wondering what is being done here. While some can see this as a benefit, as in it can create hype about new MkDocs features and changes, is it also a double-edge sword here as it can also cause people to not feel happy about being left in the dark.

Also, finally, the fact that mostly theme, plugin and extension devs are considered for discussion.
I can understand that you want to gather feedback from those to improve MkDocs in the coding/API aspect, but at the same time do I feel like that this could create a situation where the changes benefit the devs, but not the end-user, which is for sure the majority of the user-base of MkDocs here.
It's a delicate situation to say the least.

I personally am a vendor for openess in a Open source project, tho I can see the issues, and especially limitations (lack of permissions to make discussions view-only for certain people) to avoid major off-topic chats.
Only alternative I could imagine is making a matrix chat where only selected roles have write permissions, while everyone else could read it...
Tho, that is my views here obviously...

[pawamoy]

I don't have a strong opinion, and would be OK with "on request" invites :)

I agree with the concerns about openness. It does feel a bit weird to work in a private space, but this was the only way to gather a few interested authors and contributors. I'll note that this is also a common thing (private/internal space) to have when working as a team. I'm completely open to alternative setups!

[pawamoy]

Let me switch to this "on request" strategy: I've just sent an invite to @trell.

[trel]

received.

[pawamoy]

All discussions were moved into the main, public repository 🙂

[squidfunk]

I'm really psyched to see this happening and will do my very best to contribute as much of my time as I can! Putting maintenance of MkDocs on more shoulders is definitely overdue and the logical next step, given the history and popularity of the project. Let us plugin and extension authors work together and take this project to the moon 🚀

[EddyLuten]

Thanks for tagging me on this! I'd love to be more involved, though I'm not sure how much time I'll be able to dedicate to MkDocs. I'll try to help out where I can.

[pawamoy]

Of course, @squidfunk and I are in a position where we can reliably dedicate some of our time on MkDocs, and we completely understand that others might not be. Thanks a lot for offering your help!

[byrnereese]

I would love to contribute! I can certainly share ideas, time and my talent, but I may not be the best software engineer. I can hack on a mkdocs plugin pretty well, but I am primarily a Product Manager by profession with specialties and expertise in Developer Experience. I would love to contribute those talents if I can.

One specific area I would love to contribute to is the creation of a mkdocs-powered OpenAPI specification renderer. I would like to explore either partnering with a community like Redoc, or Stoplight (or similar) to develop an API Reference module, or develop a native one ourselves.

I have other thoughts and ideas to share, but will save them for a more appropriate forum. Thank you for the invite!

[pawamoy]

Amazing, thanks a lot for joining! These are very nice ideas 🙂 I think other authors would be very interested chatting about this, like @Neoteroi 😄

[pawamoy]

Oh, I'm probably interested too! I wonder if mkdocstrings could be used for REST API / HTTP+JSON API docs 😄
Feel free to create discussions over there, I'll be happy to chat more 🙂

[facelessuser]

Yes, I'm okay with being included.

[pawamoy]

Great to have you!

[bczsalba]

Very in! Haven't had much of a need to touch my MkDocs code in a while, but I'm planning on using it/working on plugins in the near-ish future. Not sure how much direct code I can contribute due to time constraints, but I'd love to be included in discussions!

Sick idea btw!

[pawamoy]

That's awesome @bczsalba, thank you for joining! No worries about time constraints, we know FOSS is not the main occupation of many 🙂

[fralau]

I think it's a good idea, and I feel honored that you asked me to join.

If I can help in some way, I will be happy to.

One of the things I have dreamt about for some time, is some self-hosted web app, which could be a container for static websites, especially MkDoc. Something which had its proper log-in procedure, where we could easily define user groups, with access rights for read and write, and provide a navigation to access the various sites (read/write would translate, in terms of git, as git pull and git push). Basically it would be the "wiki shell", a container for a number of documentation projects. That would look like a very poor man's Git Pages or ReadTheDocs... Writing such a piece of software would be little beyond my ability and resources right now, but just in case somebody liked the idea...

[pawamoy]

So happy to get you on board @fralau!

Interesting idea, yes. It boils down to bringing a collaborative experience to a MkDocs site directly within the browser (user and group support, authentication, authorization, live editing and live previewing). Quite ambitious, love it! There were some previous efforts on this, happy to share more about this once you're in 😉

[koaning]

Interesting!

I've got some ideas in my backpocket to help make general documentation projects a bit more interactive and it's also way past due for me to update the calmcode course (it's 4+ years old ... possibly the 2nd course that I ever made ... and it should be updated to feature the plugins/customisability more).

Can't say for sure how much time I'll have though, but keen to be in this particular loop.

[pawamoy]

Hey @koaning, so cool that you're joining us! I'll send you an invite as soon as I'm in front of my laptop :)

[fralau]

All this sounds exciting, and I am happy to see that this group is full of creative spirit.

@koaning mentioned his calmcode course (kudos), which was evidently produced using a Jupyter notebook. I am currently using Jupyter Lab in a consulting project since it is very adequate to document and illustrate my research work on the go.

As a big fan of Don Knuth's Literate Programming, I just cut the traditional distinction between "research" and "report", and started to tangle and weave as I go).

That also has to do with the philosophy behind Mkdocs-Macros...

Once one has tasted the exciting properties of dynamic Markdown, it's hard to go back to "normal life". Now the Markdown cells in Jupyter seem too plain to me, and I wish they also integrated a Jinja2 environment that exploits the variables of the notebook.

It seems so easy to implement it (that is, for the initiate who understand how Jupyter works, which I am not)... and so elegant.

Actually, somebody thought of implementing a similar functionality to Mkdocs + Macros but frankly it just looks like an usine à gaz compared to the simple elegance of Markdown extensions used in Mkdocs, the beauty of Material, and the powerful elegance of Jinja2... We even know how to do Mermaid diagrams, with or without a dedicated plugin.

What does it have to do with Mkdocs? I am not so sure.

Just some lateral thinking...

[timvink]

Knuth is awesome =)

I do wonder if mkdocs is the right platform for this.. For technical writing I would consider first fastai's https://nbdev.fast.ai/, which is built on top of notebooks with Knuth in mind. For more elaborate writing,https://quarto.org/ is also really nice.

Perhaps mkdocs can learn some things from those approaches..

[mkusz]

Some of this stuff can be achieved by integrating Obsidian.md as an editor for Markdown files. There are a few things that have to be considered since their Markdown flavor is a bit different that one used by MkDocs. Partly it's why I started to build my set of plugins called MkDocs Publisher. Basically it's a bridge that connects Obsidian and MkDocs that allows to publish Obsidian vault as a page with focus on simplicity for setup some publication aspects directly from Markdown files by frontmatter. It's more to talk about it and probably I should put it into the project documentation/web page.

[pawamoy]

Shameless plug with my own contribution to literate programming: markdown-exec (notably the Markdown example here) 😄

[fralau]

Yes, Jupyter and MkDocs are distinct, and at the same time they are like two alpinists attempting to escalate the same mountain from a different face. MkDocs is clearly more batch-oriented (since it aims at creating static sites); whereas Jupyter is more modular in its execution, supporting change of order and more interactivity.

Certainly, the common goal is to present together:

1. High-level concepts, expressed in natural language
2. Actual, formal code
3. The output of the code.

But I believe that each should remain itself and do well what they are doing, because each follows its own path.

When you are content to be simply yourself and don't compare or compete, everyone will respect you. (Lao Tze)

MkDocs has this elegantly simple idea that documentation (Markdown, with all its extensions) is code, part of the documentation can be computer code, and that part of this compiuter code can be executed (either at build time, or at viewing time in the browser).

[fralau]

@pawamoy The idea of using together Mkdocs-Macros and markdown-exec got me interested. 👀

It would be something worth experimenting, and see where it leads to.

[mkusz]

So far I haven't contributed to the main MkDocs repo, but I'm developing my own set of plugins MkDocs Publisher.

I have some ideas to improve documentation related to plugin development and maybe some small things into the code (already implemented in one of my plugin, but kind of fell that this should be part of the core).

It could be grat to be a part of such a great community even deeper.

[pawamoy]

Very nice plugins @mkusz! Ideas for small code improvements in the core to better serve plugin authors is exactly the kind of feedback and contributions we would love to get :)

[lyz-code]

Thanks @pawamoy for thinking in me. I love the idea of fostering collaboration this way and I'm honored to be included <3.

Right now I'm trying to invest the least time in the maintenance of my MkDocs plugin and thus in the MkDocs ecosystem, but I'd love to be in the loop.

One thing that is being in my mind is to support orgmode as well as markdown. I've switched this year to the other side of plaintext and I'm loving it. I was thinking on using pandoc in the CI to do the tranformation but maybe there are better ways to do this.

Thank you so much for the initiative :)

[pawamoy]

Glad to have you here @lyz-code! The ability to use something else than the markdown Python package is a long standing feature request indeed. I myself would like to be able to experiment with other formats/tools like djot :) Happy to discuss more about this (even though it will not be a high priority, unless maybe if someone can commit to it!) 🚀

[athackst]

Thanks for tagging me!

I can help, though it will depend on the level of time commitment you're looking for.

[facelessuser]

I didn't think anyone is looking for an exact amount of participation, only people that are interested in collaborating. I personally give only the time and effort I can. I support other things outside of this project, but there are times where I am able to contribute.

[pawamoy]

You're welcome! Regardless of how much time you can invest, it's probably a good thing to have you in the team, so you can at least follow the discussions that are of interest to you 🙂 I don't think we'd be able to invite a lot of authors if we were looking for specific levels of time commitment, as we know people are usually busy with lots of other things than FOSS 😅 So I'd say, join us and we'll see how things shape up. If everyone can commit even just a tiny amount of time, it's already much better than no time at all, I think.

[athackst]

Sounds good. Count me in then!

[unverbuggt]

Good Idea, count me in. Mkdocs and the plugin system around it is really cool. And compared to other solutions in the field, I am somewhat able to follow how it works under the hood, which is important to me.

Until now, I've never felt the urge to add code or suggestions to mkdocs itself (just some small discussion on issues), because I always thought the big boys (the maintainers) know it better. It's sad seeing a dedicated person like oprypin go away, but let's keep this as a reminder to always try to be excellent to each other.

[pawamoy]

Thank you for joining @unverbuggt 🙂

[smarie]

Thanks @pawamoy for this proposal, the initiative is excellent ! As I currently struggle to maintain all of my packages in parallel with contributions to others' and running professional/family life, I prefer not to commit to core design discussions/maintenance actively, but will be happy to discuss topics directly related with how mkdocs-gallery will be impacted. I also might have involvement in mkdocs-bibtex in the near future for what's worth.

[pawamoy]

Thanks @smarie! Invite sent, happy to chat about your plugins and MkDocs plugin system 🙂

[timvink]

Thanks for the invite! Happy to collaborate. I have not been spending much time on my mkdocs plugins due to family & work. But I am excited to think along. I also like the idea of having an @tag group of mkdocs contributors/plugin authors that can be pulled into relevant discussions.

I think there are opportunities for making plugin development even easier. One idea I have to is enable adding standard options like enabled , enabled_on_serve as well as exclude and include to avoid processing on certain pages. This also helps consistency across plugins.

I also had on my backlog for a long time the wish to build a standardized git 'meta plugin' that processes markdown pages just once and as fast as possible (multithreading, parallel, etc). Then plugins authors that use that information easily and efficiently. I even started an implementation on https://github.com/timvink/mkdocs-git-info-plugin but have never properly dedicated the time required.

[tomchristie]

@pawamoy Sure, yep. I was trying to say, "let's use @mkdocs/core better".

Specifically...

• @mkdocs/core currently has "maintain" permissions on mkdocs, so any users in that group can approve/merge PRs. Having a reasonably large number of individuals empowered to do that seems a good idea.
• We could/should probably have everyone who's invested above be on that team.
• If we do also want a semi-private team space(???), then we can have @mkdocs/core with "maintain" permissions on that space, rather than inviting contributors to the repo directly.

Make sense?

[timvink]

Yes, I meant a team that can be tagged.

I'm fine with open discussions, but I can see why we might want to do that in a separate space. Scikit-learn does that in a separate repo (https://github.com/scikit-learn/enhancement_proposals). I separate repo like mkdocs/enhancement_proposals could work well.

[pawamoy]

@timvink I agree, and I've suggested to make the repo public here: https://github.com/mkdocs/mkdocs-team-space/discussions/12.

@tomchristie

We could/should probably have everyone who's invested above be on that team.
@mkdocs/core currently has "maintain" permissions on mkdocs, so any users in that group can approve/merge PRs. Having a reasonably large number of individuals empowered to do that seems a good idea.

As much as I like reducing the bus factor and empowering people so that they maybe become maintainers, giving merge privilege this easily doesn't sound very sensible regarding security, especially after what we learned from the XZ event. For now, we still don't have a lot of maintainers that can reliably invest a decent amount of time in order to stay up-to-date with everything that happens in this repository. I myself do not feel confident I'm able to prevent malicious changes (or simply low-quality ones) from slipping into MkDocs, by reviewing all PRs and checking the ones that could have been merged without reviews. Some changes have consequences that are particularly hard to fully understand: we need extra, extra care when merging those. More merge access is also more attack surface for external attackers.

I fully trust some of the people who participated here, others have a online presence and reputation that is enough to vouch for them, but some others I don't know at all. Most of them have also said they don't have much time to spare on MkDocs. In short, I would prefer we don't apply the "on-request" strategy for giving merge privilege 🙂 Allowing them to approve PRs could be a good thing though, if it's not already possible.

EDIT: I'm however OK with having a second team, with less privileges than the core one. This would ease managing access/privileges for all these nice people 🙂

[waylan]

In short, I would prefer we don't apply the "on-request" strategy for giving merge privilege 🙂 Allowing them to approve PRs could be a good thing though, if it's not already possible.

I agree with @pawamoy here. I would suggest two levels for team members. New members should be able to review PRs and mark them (perhaps with tags) as 'ready to merge' or 'needs changes' or whatever is appropriate for that PR. Then the seasoned members with additional privileges can merge the 'ready to merge' PRs. This gives the team an opportunity to observe how new team members handle various situations and provide feedback so that they can grow and improve. In time, the new member can be given additional privileges if it is deemed appropriate.

[Guts]

I also had on my backlog for a long time the wish to build a standardized git 'meta plugin' that processes markdown pages just once and as fast as possible (multithreading, parallel, etc). Then plugins authors that use that information easily and efficiently. I even started an implementation on https://github.com/timvink/mkdocs-git-info-plugin but have never properly dedicated the time required.

I haven't completely forgotten that idea either @timvink! I remember that we talked about it in an issue in one of your git based plugin.

[ofek]

I'd be down to assist if everything is public.

[pawamoy]

Thanks for your feedback 🙂 I'll see with other members of the org what alternative setup we could use instead of a private repository.

[pawamoy]

All discussions were moved into the main, public repository 🙂

[ojacques]

Thanks for inviting me, I am honored!
mkdocs and the plugins is how I am doing anything tech docs related, and how I inject "documentation engineering" at work / with customers.
I am no Python wizard, but I'd love to continue the discussion, contribute back, and support where I can.

[pawamoy]

Thanks for joining and helping @ojacques 😄

[wilhelmer]

Thanks for inviting me! I changed jobs and my new employer uses Docusaurus, so I no longer have time to collaborate on MkDocs. But I wish this project all the best, it deserves all the success in the world 🚀

[pawamoy]

Thanks so much @wilhelmer! All the best to you too!

[blueswen]

Thanks for inviting me! I'm excited to join this group and help improve the MkDocs community. I've learned so much about managing open-source projects from both you and @squidfunk.

I'm currently preparing a talk for COSCUP(Conference for Open Source Coders, Users & Promoters) in Taiwan (fingers crossed it gets selected, haha) on how MkDocs can enhance documentation for open-source projects. I hope this presentation will help more people see how awesome MkDocs is.

[pawamoy]

Amazing, I hope you get selected too :) I'll send you an invite asap!

[LukeCarrier]

Hey @pawamoy, many thanks for inviting me, and apologies for the delayed response. I can try and commit some time to maintenance 🙂

I'm in agreement with others above that it's likely better for the project to adopt an open development model which plans in public, and encourages developers within the community to collaborate on reviews. A smaller, more privileged group of core developers that emerges from such a group could be granted the ability to merge/reject PRs as appropriate.

I struggle to see a benefit to non-public project coordination and planning, and that exclusivity introduces a barrier to possible contributors.

[pawamoy]

Thanks for joining, and for your feedback @LukeCarrier 🙂
We're chatting right now, hopefully we'll have good news very soon 🙂

[Guts]

Hi there,

I'm getting late here and I feel like I've missed a train or two!

I'm interested even though my investment in the Mkdocs ecosystem is on my spare time and I'd rather not promise you a huge commitment.

On a broader level, I'm a firm believer in community dynamics in open source projects, and I've often had the impression that the open source aspect of Mkdocs remained very (too?) closely linked to one or a few people at most, without any real community dynamic. I have a similar feeling about the Material for Mkdocs project, but it's likely more understandable since it's the @squidfunk's project and there's a business model with a company behind it (even an individual one).

Maybe it's not yet time to bring out the heavy artillery of open source project management (PSC, etc.), but why start with a private discussion space and not a public one?

[pawamoy]

This is being addressed, and all the discussions that already happened in the private space will probably be moved into the public discussions of the mkdocs repository 🙂 I'll check with @tomchristie if we can do that tomorrow.

[squidfunk]

I feel that it might be good to explain something here ☺️

I'd love to put Material for MkDocs on more shoulders and I've started to build a team lately to pull that off that I financially compensate. The fact that Material for MkDocs is more or less managed by me for the last 8 years stems from its original purpose and its target group – it started as a theme built for non-web-devs, and that's still the main demographic. Most users have no front end skills, so can't really help out on anything in that domain, and the project has grown to be quite complex, so there're almost no low hanging fruits there. Frontend-savvy users use something like Docusaurus or Vitepress anyway, as they want more control, which is understandable. Additionally, they have a JS stack on their system anyway, which is an absolute no-go for many of our users, especially from within enterprises.

Another problem might be our use of a harder to learn paradigm called "Reactive Programming" (implemented in RxJS) to model the UI, as we cannot just use a predefined web framework like React or Svelte to strap on business logic on static HTML.

We have only lately begun to add plugins and business logic to the mix, and I expect more helping hands here in the near future as adoption grows. My goal is definitely to build a welcoming and inviting community, and I think we will eventually be successful with that. Making MkDocs more modular and inviting to third-party extension and plugin authors would definitely help.

[pawamoy]

All discussions were moved into the main, public repository 🙂

[pawamoy]

Hello again everyone,

@tomchristie and I moved forward and decided to transfer all discussions that were created in the private space mentioned here into the public, main repository 🙂 Use the category filters on the left (Maintenance, Meta, etc.) to find some of them.

The possible next step is to create a team for all users and authors who expressed interest here. This team would give them a bit more privileges (triaging, approving PRs? to be defined yet with core maintainers).

[Andre601]

It would be nice if those could receive a designated label for easier finding, as I myself was not in the private repository for a while now, making it more difficult to recognize posts from there compared to other posts made here.

[pawamoy]

That's a good suggestion. However there weren't a lot of discussions. They would be the most recent ones in the General and Ideas categories, as well as all the ones in Maintenance and Meta. And since all discussions are in the same place now, there's nothing distinguishing them from previous ones.