Render more readable macro matcher tokens in rustdoc #92908

dtolnay · 2022-01-14T21:16:59Z

Follow-up to #92334.

This PR lifts some of the token rendering logic from https://github.com/dtolnay/prettyplease into rustdoc so that even the matchers for which a source code snippet is not available (because they are macro-generated, or any other reason) follow some baseline good assumptions about where the tokens in the macro matcher are appropriate to space.

The below screenshots show an example of the difference using one of the gnarliest macros I could find. Some things to notice:

In the before, notice how a couple places break in between $(....)↵*, which is just about the worst possible place that it could break.
In the before, the lines that wrapped are weirdly indented by 1 space of indentation relative to column 0. In the after, we use the typical way of block indenting in Rust syntax which is put the open/close delimiters on their own line and indent their contents by 4 spaces relative to the previous line (so 8 spaces relative to column 0, because the matcher itself is indented by 4 relative to the macro_rules header).
In the after, macro_rules metavariables like $tokens:tt are kept together, which is how just about everybody writing Rust today writes them.

Before

After

r? @camelid

src/test/rustdoc/macro-generated-macro.rs

src/librustdoc/clean/utils.rs

camelid · 2022-01-14T21:55:56Z

src/librustdoc/clean/utils.rs

+        "as" | "box" | "break" | "const" | "continue" | "crate" | "else" | "enum" | "extern"
+        | "for" | "if" | "impl" | "in" | "let" | "loop" | "macro" | "match" | "mod" | "move"
+        | "mut" | "ref" | "return" | "static" | "struct" | "trait" | "type" | "unsafe" | "use"
+        | "where" | "while" | "yield" => true,
+        _ => false,


This seems really brittle. Is there some way we can call a function in rustc to get this information? Symbol::is_used_keyword_conditional looks promising.

rustc_span has functions for evaluating whether a Symbol is a reserved word, or is a keyword on a particular edition.

This logic isn't that though. It is quite specific to rendering spacing in stringified token streams. I would not expect to find a function for that in rustc, because here in rustdoc is the first place that cares about nicely stringifying token streams in a way that human users of the language are expected to regularly encounter.

I pushed a commit that inverts the list though, since among keywords there are fewer that do not need a space than need a space. Let me know you like that better.

camelid · 2022-01-15T01:18:13Z

I'll take another look at this soon. Thanks for the very clear and helpful comments. ❤️

cc @GuillaumeGomez in case you want to review as well

src/librustdoc/clean/render_macro_matchers.rs

GuillaumeGomez · 2022-01-15T16:05:47Z

Could you add another "more complete" HTML test please?

dtolnay · 2022-01-17T18:42:43Z

Let me do that after #92914 has landed and I've rebased.

…omez htmldocck: Add support for `/text()` in `@snapshot` This allows just testing the text, in cases where the HTML tags don't matter. See rust-lang#92908 (comment) for an example of when this would be useful. r? `@GuillaumeGomez`

dtolnay · 2022-01-18T20:27:45Z

Rebased on #92914 and added a more exhaustive test in 039a058.

GuillaumeGomez · 2022-01-19T10:20:40Z

Looks good to me, thanks! Let's wait for @camelid approval as well.

camelid · 2022-01-25T20:02:38Z

I feel a little uncomfortable about having so much custom code for a fairly niche use case (macro-generated macros), but I won't block it. I guess we could always delete this code if we had to since it's not part of our stability guarantee.

dtolnay · 2022-01-25T20:16:57Z

I don't agree that this is niche -- doesn't this affect all standard library macros according to #86208 / #92334 (review)? That seems front and center for a large fraction of the macros that users of Rust are going to be looking at.

camelid · 2022-01-25T20:19:34Z

I thought your previous PR (#92334) already fixed most cases?

dtolnay · 2022-01-25T20:23:25Z

Your review comment on that PR says "it won't fix the rendering of macros in the standard library since the lack of sources there triggered the switch to pretty-printing in the first place". The screenshot above is from https://doc.rust-lang.org/nightly/std/macro.write.html so it confirms that the source-based rendering does not kick in for the standard macros. Maybe that can be solved a different way but for now #92334 by itself does not fix it.

camelid · 2022-01-25T20:25:53Z

Oops, it looks like I should've listened to past me ;)

I think this change is reasonable then. Sorry for the confusion!

GuillaumeGomez · 2022-01-30T12:55:15Z

Thanks for working on this!

@bors: r+

bors · 2022-01-30T12:55:16Z

📌 Commit 039a058 has been approved by GuillaumeGomez

Rollup of 5 pull requests Successful merges: - rust-lang#92887 (Bootstrap compiler update) - rust-lang#92908 (Render more readable macro matcher tokens in rustdoc) - rust-lang#93183 (rustdoc: mobile nav fixes) - rust-lang#93192 (Add VS 2022 into error message) - rust-lang#93475 (Add test to ensure that theme is applied correctly when going back in history) Failed merges: r? `@ghost` `@rustbot` modify labels: rollup

rust-highfive assigned camelid Jan 14, 2022

rustbot added T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. labels Jan 14, 2022

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jan 14, 2022

camelid reviewed Jan 14, 2022

View reviewed changes

src/test/rustdoc/macro-generated-macro.rs Outdated Show resolved Hide resolved

camelid reviewed Jan 14, 2022

View reviewed changes

camelid added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 14, 2022

camelid mentioned this pull request Jan 15, 2022

htmldocck: Add support for /text() in @snapshot #92914

Merged

camelid added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jan 15, 2022

dtolnay commented Jan 15, 2022

View reviewed changes

src/librustdoc/clean/render_macro_matchers.rs Outdated Show resolved Hide resolved

src/librustdoc/clean/render_macro_matchers.rs Outdated Show resolved Hide resolved

src/librustdoc/clean/render_macro_matchers.rs Outdated Show resolved Hide resolved

dtolnay added 8 commits January 18, 2022 11:04

Extend rustdoc macro-generated-macro test

d950c1b

Render more readable macro matchers in rustdoc

0904614

Move render_macro_matcher to own module

81b1e32

Move test of macro-generated macro with linebreak to snapshot

aff5296

Eliminate string comparison from rustdoc keyword spacing logic

c70c646

Invert the keyword list to list only *no* space before delim

a6c9d8f

Convert macro-generated-macro snapshot to /text() using PR 92914

f562b24

Add a more complete test of rustdoc macro_rules matcher rendering

039a058

dtolnay force-pushed the rustdoc branch from 292b05e to 039a058 Compare January 18, 2022 20:26

GuillaumeGomez approved these changes Jan 19, 2022

View reviewed changes

camelid removed their assignment Jan 25, 2022

camelid assigned GuillaumeGomez Jan 25, 2022

dtolnay removed the T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. label Jan 27, 2022

GuillaumeGomez approved these changes Jan 30, 2022

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 30, 2022

ehuss mentioned this pull request Jan 30, 2022

Rollup of 5 pull requests #93482

Merged

bors merged commit ba01337 into rust-lang:master Jan 30, 2022

rustbot added this to the 1.60.0 milestone Jan 30, 2022

dtolnay deleted the rustdoc branch January 31, 2022 08:04

dtolnay added the A-pretty Area: Pretty printing. label Dec 28, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Render more readable macro matcher tokens in rustdoc #92908

Render more readable macro matcher tokens in rustdoc #92908

dtolnay commented Jan 14, 2022

camelid Jan 14, 2022

dtolnay Jan 14, 2022

camelid commented Jan 15, 2022 •

edited

GuillaumeGomez commented Jan 15, 2022

dtolnay commented Jan 17, 2022

dtolnay commented Jan 18, 2022

GuillaumeGomez commented Jan 19, 2022

camelid commented Jan 25, 2022 •

edited

dtolnay commented Jan 25, 2022

camelid commented Jan 25, 2022

dtolnay commented Jan 25, 2022

camelid commented Jan 25, 2022 •

edited

GuillaumeGomez commented Jan 30, 2022

bors commented Jan 30, 2022

Render more readable macro matcher tokens in rustdoc #92908

Render more readable macro matcher tokens in rustdoc #92908

Conversation

dtolnay commented Jan 14, 2022

Before

After

camelid Jan 14, 2022

Choose a reason for hiding this comment

dtolnay Jan 14, 2022

Choose a reason for hiding this comment

camelid commented Jan 15, 2022 • edited

GuillaumeGomez commented Jan 15, 2022

dtolnay commented Jan 17, 2022

dtolnay commented Jan 18, 2022

GuillaumeGomez commented Jan 19, 2022

camelid commented Jan 25, 2022 • edited

dtolnay commented Jan 25, 2022

camelid commented Jan 25, 2022

dtolnay commented Jan 25, 2022

camelid commented Jan 25, 2022 • edited

GuillaumeGomez commented Jan 30, 2022

bors commented Jan 30, 2022

camelid commented Jan 15, 2022 •

edited

camelid commented Jan 25, 2022 •

edited

camelid commented Jan 25, 2022 •

edited