Reformulate the goals and audience for this course

[brson]

The goals for this project still aren't clear, or they are not what they should be, or the content of the course so far doesn't support the goals.

The readme contains statements about both, but they don't seem to be clear, or they are not what some people expect. Especially, the goal

convey knowledge relevant to real-world Rust programming, including inherited wisdom and best practices; to look "under the hood" and understand how and why Rust works like it does; to tour the world of Rust documentation and code such that the student can learn how to find answers to their questions on their own.

is too expansive, and doesn't seem to resonate. Reviewers seem to see the project progression and wonder why the lesson plan contains so many topics that don't directly contribute to the projects.

Furthermore, this goal of generally conveying practical knowledge is not PingCAP's goal, it is my own. By my understanding, PingCAP wants students to demonstrate specific knowledge that is relevant to working on TiKV.

I could use help on this since I have tried a few times to express the goals for the course, but questions continue to arise. Maybe if others could suggest clear statements about the goals and audience that would help me reorient.

cc @siddontang @nrc @sticnarf @overvenus @Hoverbear

[jonathan-s]

This is from a person from the outside looking in and who is following this repo with eagerness. The statement as described above does make sense and resonates with me. And if this course manages to accomplish the following.

such that the student can learn how to find answers to their questions on their own.

Then you've managed to accomplish a lot. That's far more valuable than only learning about the internals about TiKV. That said, be able to learn enough to be able to work on TiKV and the more general skill of being able to navigate around other Rust projects are not mutually exclusive. They are complementary.

Me, as an aspiring student of this course, would definitely prefer that the course not only focuses on specific knowledge relevant to TiKV. Perhaps that could be one of the objectives, but I hope that the other objective also could be to look "under the hood" and understand how and why Rust works like it does.

Perhaps it would be possible to delineate sections with more general purpose knowledge around Rust and if the student knows that they know about this, it would be possible to skip these sections. I still think it would be valuable if these existed.

[Hoverbear]

It's important to keep us distinct from the Rust book, for example.

Ultimately, we're not rewriting book. As my understanding follows, this is roughly the goal:

To train folks who already know how to program how to get into Rust and help them into building distributed systems with it. Teach them the tools and concepts they might need to apply, the guidance to learn them, and how to find new knowledge and tools.

[nrc]

Quoting README and #7 (comment), so as to keep discussion in one place:

This book is not for novice programmers, but it is for novice Rust programmers.

This course is for programmers with some knowledge of data structures, algorithms and I/O.

This course is for programmers who have begun learning Rust, but still need guidance in production Rust programming and best practices.

This is a course for active learners — it will lead you in the right direction, but the journey is yours.

Those who complete the course will have the knowledge and experience to begin to write high performance, reliable, networked applications in Rust.

This course, combined with Deep Dive TiKV is intended to be enough to enable programmers to meaningfully contribute to TiKV or any other Rust project. It is most specifically designed to teach those in the Chinese Rust community enough Rust to work on TiKV. The language used is intended to be simple so that those who read only a little English can follow.

[nrc]

I'm on vacation, so I don't have time to give this proper thought until next week. But, some initial things:

this goal of generally conveying practical knowledge

I think this is a great goal for a course, and is something that is generally missing from the wider Rust community (I have tried to encourage others to fill this gap and even tried myself to a small extent). However, as a formal training course offered by PingCAP, it feels a little nebulous.

I think the thing for me is that the course structure itself seems pretty formal (like a short university course), but this goal is fairly informal, something I'd expect from a seminar series or workshop at a conference, etc.

It would help me, I think, to consider some concrete user stories/use cases for the course, and try and derive a more general vision from that. Some possible user stories I can imagine are:

• interns starting at PingCAP: likely to have 1-3 years of formal computer science education, be competent but not experienced programmers in Java or Python, have no previous Rust experience, have no professional expereience, have been taught networking and databases, but have no practical experience. At the end of the course, they are expected to work in Rust but be closely mentored by an experienced programmer, and to be learning as well as working for some time.
• junior or senior-ish programmers starting at PingCAP with 3-10 years experience and 3-4 years formal education, competent to expert programmers in Java, Go, etc. Have a little Rust experience (e.g., worked through the book, but not written a serious project or used professionally). Knowledgeable about networking, but perhaps missing a low-level understanding and no experience in Rust. After the course should be able to contribute at PingCAP with relatively hands-off mentoring and only a short ramping up period.
• Rust enthusiasts (e.g., at RustCon Asia or something similar). 0-10 years professional experience an 0-3 years formal education. Competent Rust programmers,. but not used in professionally. Some knowledge of networking/databases. After the course, they are interested enough to apply to PingCAP in the near future, and have a more polished and less 'amateur' approach to using Rust.

Perhaps something completely different?

[nrc]

I also want to think about what PingCAP get out of the course: name recognition, a wider pool of applicants, better trained employees/interns, more/better contributors, more clients, downstream users, etc.?

[brson]

Thank you for the enthusiastic feedback @jonathan-s, and the goal statement @Hoverbear , and the user stories @nrc.

Me, as an aspiring student of this course, would definitely prefer that the course not only focuses on specific knowledge relevant to TiKV.

I also want this to be attractive to a general audience. We can preserve that property.

To train folks who already know how to program how to get into Rust and help them into building distributed systems with it. Teach them the tools and concepts they might need to apply, the guidance to learn them, and how to find new knowledge and tools.

Yeah this is a lot more focused than mine.

In my original conception the course had a ton of digressions into interesting subjects that follow from the projects but aren't necessary to complete the projects. Maybe we can still try to "look under the hood" but with subjects that are relevant to the projects. e.g. I had planned a deep dive into rust's formatting infrastructure (thinking that project 1 would have a good bit of formatting); maybe a dive through tokio down to mio would be more helpful, and provide a lot of futures context.

@nrc thanks for the user stories. Very helpful. It reminds me that our audience are already experienced programmers, so we don't need to water down the projects to newbie level, something that I worry about as we reduce the scope for the first iteration.

The value I think we can provide is to quickly bootstrap newcomers into understanding "the Rust way". Some of the ways we can do that are to lead them to

• best in class crates for doing everyday things
• Rust idioms
• resources
• tricky language features
• understanding the rust ecosystem
• async
• regions
• (todo)

I have largely focused on the first 3 of those. I don't even have plans to do anything substantial with borrowing, but that is typically the big thing newcomers want help with.

Maybe after completing a first draft of the projects we can think about augmenting it with more difficult subject matter. Maybe extending beyond the 5th project.

I probably have more thoughts but it's late right now. Thanks again for the help.

[nrc]

To iterate on

To train folks who already know how to program how to get into Rust and help them into building distributed systems with it. Teach them the tools and concepts they might need to apply, the guidance to learn them, and how to find new knowledge and tools.

This sounds like we are assuming the audience:

• has some programming experience (e.g., new grad or self-taught with some professional experience), possibly more
• has little or no Rust experience
• has no dist systems or DB expereience

Is that correct?

In which case, it sounds like we are proposing a course which covers both beginner and intermediate Rust plus distributed systems, the latter both for their own sake and as a context in which to teach the Rust stuff.

So, dist systems is a huge area, do we want to:

• teach a broad and relatively complete course,
• cover the core principles,
• cover the parts of the domain most relevant to TiKV in depth,
• give students a 'taste' of the topic,
• or cover some other subset in depth?

Do we also want to cover DB implementation topics? Should we assume the student already knows some of that? Or at least how to use an SQL and/or KV DB? (Some stuff like query planning seems not relevant to this course, but LSM trees (for example) are sort-of relevant, even though they are not at all core dist systems stuff).

How much time are we expecting this course to take? (As a comparison, at university introducing a new programming language would usually be a one semester course, and a full treatment of dist systems one or two semesters, where a semester is ~10 weeks and each week has something like 3 hours of lectures one or two hours of labs, and an expectation of 5-10 hours of self-directed study).

Maybe we can still try to "look under the hood" but with subjects that are relevant to the projects

I think this is a nice thing to do, but I think it would be good to make explicit these are digressions from the core material and not mandatory. They might take the form of an into paragraph, a few suggested readings, a couple of exercises, and a small project, rather than providing the material ourselves.

thanks for the user stories. Very helpful

I think it would be good to iterate on those and perhaps add more in order to keep focus.

Some of the ways we can do that are to lead them to

I think these are great topics! I worry a little bit about mixing this intermediate stuff with the beginner stuff. Something about context switching between two different kinds of thinking feels a bit difficult to me, but I don't have anything specific to suggest

It's important to keep us distinct from the Rust book, for example.

I agree with this, but I want to dig deeper into it because I don't like defining a thing in terms of the things that it isn't. I would like to think about our relationship with the various Rust books which are available - are they a prerequisite or a resource or required textbooks? How do we avoid duplicating work between us and the book. From a student's perspective, what do we offer that the books don't - obviously we have the dist systems stuff and intermediate level topics, but if that is all, then we should be targeting intermediate Rust programmers and completely skip the beginner material.

[brson]

Is that correct?

Yes.

So, dist systems is a huge area, do we want to:

* teach a broad and relatively complete course,

* cover the core principles,

* cover the parts of the domain most relevant to TiKV in depth,

* give students a 'taste' of the topic,

* or cover some other subset in depth?

This course is not teaching distributed systems beyond that it uses client-server networking. There is a separate course for distributed systems in Rust.

Do we also want to cover DB implementation topics? Should we assume the student already knows some of that? Or at least how to use an SQL and/or KV DB? (Some stuff like query planning seems not relevant to this course, but LSM trees (for example) are sort-of relevant, even though they are not at all core dist systems stuff).

I think we should assume they have used SQL or some database. Key-value is a simple concept so I don't think previous use is a requirement.

How much time are we expecting this course to take?

I am thinking 1 week per project. The MVP is 5 projects, and @siddontang expects the next batch of students to complete it in a month. Ultimately, I can see the course easily expanding to 10 projects.

For experienced programmers with some Rust experience 1 week part time per project should be plenty. @sticnarf seems to knock them out in a few days.

I think this is a nice thing to do, but I think it would be good to make explicit these are digressions from the core material and not mandatory.

Agreed. Right now there is no under-the-hood content and my current revision of README.md removes mention. I am though feeling tempted to write an aside about the history of Rust error handling (the project currently uses failure and I want to demure and say like "well, failure may not actually be a best practice". I'm intending to remove failure from the course in the future. This "aside" section is titled "Aside: ...".

Generally, I am now down with really focusing the scope.

they a prerequisite or a resource or required textbooks?

Reading the book is a prereq.

skip the beginner material.

If you are considering someone who has finished the Rust book and written some rust intermediate then I agree. Project 1 (tooling and project setup) may be too basic for most students as written, though it does include content about clippy and rustfmt, with encouragement to go read those documents to understand what they are capable of, and I want it to include CI setup.

@nrc Right now I am intending to focus the audience down to PingCAP's primary audience: students and recent grads with an undergraduate CS education looking to learn systems programming and begin their careers. So something like your

interns starting at PingCAP: likely to have 1-3 years of formal computer science education, be competent but not experienced programmers in Java or Python, have no previous Rust experience, have no professional expereience, have been taught networking and databases, but have no practical experience. At the end of the course, they are expected to work in Rust but be closely mentored by an experienced programmer, and to be learning as well as working for some time.

I'll include an explicit list of topics that they should have some experience with, though not necessarily in Rust, including network programming, parallel programming, and asynchronous programming.

I have also had a discussion recently with a CS educator (Kevin from cryptape), and he described to me the difference between top-down and bottom-up teaching. And what I've got right now is top-down, where we say "we're going to build this grand palace", then jump into it, integrating all the concepts at once; where bottom up teaches concepts individually, then maybe teaches a few of them together, then works on integration. Top-down is tough for students without a strong background coming in.

So the course already has some accommodation for this problem by requiring "background readings" for each project, but I think that needs to beefed up. For now I am planning to extract these to "Building Blocks" sections in between each project, with the same importance as the project sections. And also add links to existing online coding problems on those individual topics. Ideally in the long-term those building blocks sections would be custom-written for the course, with the external links supplementary.

I'm overhauling the README and will link it here when done.

[brson]

Here's the overhauled readme and lesson plan: #162

cc @knwang this thread is discussing some of the issues we talked about last night about audience and prerequisites. The last two or three comments sum things up fine. Just if you are curious.