Add documentation for meta variable expressions #1485

c410-f3r · 2024-03-23T15:27:49Z

jdahlstrom · 2024-04-07T12:14:20Z

src/macros-by-example.md

@@ -195,6 +195,220 @@ compiler knows how to expand them properly:
    not have the same number. This requirement applies to every layer of nested
    repetitions.

+## Meta variable expressions


"Meta variable" should be "metavariable" for consistency with everything else. Ditto for all other appearances of the phrase.

jdahlstrom · 2024-04-07T12:15:48Z

src/macros-by-example.md

@@ -195,6 +195,220 @@ compiler knows how to expand them properly:
    not have the same number. This requirement applies to every layer of nested
    repetitions.

+## Meta variable expressions
+
+Give access to additional metadata about meta variables that otherwise would be difficult or even impossible to get through elements such as `count` or `index`.


elements such as count or index.

The wording implies that count and index are "elements" through which it would be difficult or impossible to get metadata, but presumably that's not the intention.

jdahlstrom · 2024-04-07T12:21:27Z

src/macros-by-example.md

+}
+```
+
+### length(depth)


Bikeshedding, but should consider changing the name to "len". Length is abbreviated to "len" everywhere in std; there is only a single, unstable, API that uses "length".

@jdahlstrom you may want to bring that up on the implementation PR, currently under FCP proposal rust-lang/rust#122808

@tgross35 Indeed, done!

ehuss · 2024-05-17T00:49:21Z

@c410-f3r Can you update this from the changes in rust-lang/rust#124987?

c410-f3r · 2024-05-17T11:58:32Z

Updated

tgross35

Added some suggestions for wording that I think is a bit easier to understand. I think this section could really get its own page since "Macros By Example" is already huge, and metavariable expressions will likely have more entries in the future.

Also, most of this md doc is wrapped to 80 chars, probably good to keep that consistent rather than switching wrapping styles

tgross35 · 2024-05-22T00:42:15Z

src/macros-by-example.md

+
+### count($ident, depth)
+
+Expands to an unsuffixed integer literal representing the number of times a ***metavariable*** repeats in total.


Suggested change

Expands to an unsuffixed integer literal representing the number of times a ***metavariable*** repeats in total.

Expands to an unsuffixed integer literal representing the number of times a

***metavariable*** repeats in total. That is, this will be the number of times

that a metavariable gets replaced in expanded code.

I think it is good to clarify what "repeats" means here

tgross35 · 2024-05-22T00:47:10Z

src/macros-by-example.md

@@ -196,6 +196,220 @@ compiler knows how to expand them properly:
    not have the same number. This requirement applies to every layer of nested
    repetitions.

+## Metavariable expressions
+
+Gives access to additional information about metavariables that otherwise would be difficult or even impossible to obtain manually.


Suggested change

Gives access to additional information about metavariables that otherwise would be difficult or even impossible to obtain manually.

Metavariable expressions are function-like operators that are used to access "meta"

information about how macros get matched and expanded, such as how many items

get matched in a repetition group. This information is typically difficult or even

impossible to otherwise obtain.

Metavariable expressions live within a special braced syntax `${...}`. There are a few

different operators available:

tgross35 · 2024-05-22T00:50:04Z

src/macros-by-example.md

+
+Gives access to additional information about metavariables that otherwise would be difficult or even impossible to obtain manually.
+
+### count($ident, depth)


Suggested change

### count($ident, depth)

### `count($ident, depth=0)`

Indicating the default is probably good

tgross35 · 2024-05-22T00:52:03Z

src/macros-by-example.md

+
+Expands to an unsuffixed integer literal representing the number of times a ***metavariable*** repeats in total.
+
+The output of `count` depends on where it is placed as well the provided index. If no index is provided, then it will always start at the innermost level.


Suggested change

The output of `count` depends on where it is placed as well the provided index. If no index is provided, then it will always start at the innermost level.

I would give a simple example first and mention depth after

I think this shouldn't be removed, rather it should be expanded to fully describe the exact algorithm used to determine the result. I assume it's something like this but I don't know the right terms for all the concepts involved (and I only experimented with depth=0).

In general, it seems to be something like

let's say $ident occurs under N repetition groups in the matcher

and now we use ${count($ident, D)} under M repetition groups in the transcriber

then we must have M+D < N

and we are counting how often the repetition group D up from $ident in the matcher occurs inside the current iteration of the M outer repetitions in the transcriber. IOW, for the case where D=0 (IMO the only one we should stabilize for now), we are counting how many times $ident would expand if we surrounded $ident in enough repetition groups to make it legal to occur at this point.

tgross35 · 2024-05-22T01:38:02Z

src/macros-by-example.md

+macro_rules! no_repetition0 {
+    ( $( $a:ident: $( $b:literal ),* );+ ) => {
+        [${count($b)}]
+    };
+}
+
+macro_rules! no_repetition1 {
+    ( $( $a:ident: $( $b:literal ),* );+ ) => {
+        [${count($b, 1)}]
+    };
+}
+
+macro_rules! outermost {
+    ( $( $a:ident: $( $b:literal ),* );+ ) => {
+        [$( ${ignore($a)} ${count($b)}, )+]
+    };
+}
+
+fn main() {
+    // 1 2 3 4 5 = 5 elements
+    assert_eq!(no_repetition0!(a: 1, 2, 3; b: 4, 5), [5]);
+
+    // a b = 2 elements
+    assert_eq!(no_repetition1!(a: 1, 2, 3; b: 4, 5), [2]);
+
+    // 1 2 3 = 3 elements
+    // 4 5 = 2 elements
+    assert_eq!(outermost!(a: 1, 2, 3; b: 4, 5), [3, 2]);
+}


Suggested change

macro_rules! no_repetition0 {

( $( $a:ident: $( $b:literal ),* );+ ) => {

[${count($b)}]

};

}

macro_rules! no_repetition1 {

( $( $a:ident: $( $b:literal ),* );+ ) => {

[${count($b, 1)}]

};

}

macro_rules! outermost {

( $( $a:ident: $( $b:literal ),* );+ ) => {

[$( ${ignore($a)} ${count($b)}, )+]

};

}

fn main() {

// 1 2 3 4 5 = 5 elements

assert_eq!(no_repetition0!(a: 1, 2, 3; b: 4, 5), [5]);

// a b = 2 elements

assert_eq!(no_repetition1!(a: 1, 2, 3; b: 4, 5), [2]);

// 1 2 3 = 3 elements

// 4 5 = 2 elements

assert_eq!(outermost!(a: 1, 2, 3; b: 4, 5), [3, 2]);

}

macro_rules! count_value {

( $( $name:ident: $( $value:literal ),* );+ ) => {

// Count the total number of times that the (innermost) group

// containing `$value` gets matched.

${count($value)}

// This is the same as `${count($value, 0)}`

};

}

macro_rules! count_name1 {

( $( $name:ident: $( $value:literal ),* );+ ) => {

// This is one way to get the number of times that the group

// containing `$name` gets matched. Alternatively...

${count($name)}

};

}

macro_rules! count_name2 {

( $( $name:ident: $( $value:literal ),* );+ ) => {

// ...`$value` can be used again with a depth specifier of 1,

// indicating that repetitions of the 1st parent group of

// `$value` should be counted, rather than `$value`'s innermost group.

//

// `1` is the maximum value here since `$value`'s group has a single

// parent.

${count($value, 1)}

};

}

macro_rules! count_value_nested {

( $( $name:ident: $( $value:literal ),* );+ ) => {

[ $(

// using `count` within a repetition group will return the number

// of times `$value` is matched _within that group_.

${count($value)},

)+ ]

};

}

fn main() {

// all instances of `$value` counted: count(1, 2, 3, 4, 5) = 5

assert_eq!(count_value!(a: 1, 2, 3; b: 4, 5), 5);

// count(1, 2, 3, ... 11) = 11

assert_eq!(

count_value!(a: 1, 2, 3; b: 4, 5; c: 6, 7; d: 8, 9, 10, 11),

11

);

// `$value` is never matched; count() = 0

assert_eq!(count_value!(a:), 0);

// count(a, b) = 2 matches

assert_eq!(count_name1!(a: 1, 2, 3; b: 4, 5), 2);

// count(a, b, c, d) = 4 matches

assert_eq!(

count_name1!(a: 1, 2, 3; b: 4, 5; c: 6, 7; d: 8, 9, 10, 11),

4

);

// count(a) = 1 match

assert_eq!(count_name1!(a:), 1);

// These have the same results as the above

assert_eq!(count_name2!(a: 1, 2, 3; b: 4, 5), 2);

assert_eq!(

count_name2!(a: 1, 2, 3; b: 4, 5; c: 6, 7; d: 8, 9, 10, 11),

4

);

assert_eq!(count_name2!(a:), 1);

// first match: count(1, 2, 3) = 3. second match: count(4, 5) = 2.

assert_eq!(count_value_nested!(a: 1, 2, 3; b: 4, 5), [3, 2]);

// first match: count(1, 2, 3) = 3. second match: count(4, 5) = 2.

// third match: count(6, 7) = 4. fourth match: count(8, 9, 10, 11) = 4.

assert_eq!(

count_value_nested!(a: 1, 2, 3; b: 4, 5; c: 6, 7; d: 8, 9, 10, 11),

[3, 2, 2, 4]

);

// `$value` is never matched; count() = 0

assert_eq!(count_value_nested!(a:), [0]);

}

Added guiding comments

Change $a and $b matcher names, which I found confusing since they don't align with the a and b identifiers within macro usage blocks.

ignore isn't needed here

Add some more examples

using count within a repetition group will return the number>
of times $value is matched within that group.

This is a key part of the spec. It should be in the reference text above, not just in the examples.

tgross35 · 2024-05-22T01:54:50Z

src/macros-by-example.md

+}
+```
+
+Contrary to `count`, `len` must be placed inside a repetition.


Suggested change

Contrary to `count`, `len` must be placed inside a repetition.

Unlike `count`, `len` must be placed inside a repetition.

tgross35 · 2024-05-22T01:55:40Z

src/macros-by-example.md

+}
+```
+
+If repetitions are nested, then the optional depth parameter can be used to limit the number of nested repetitions that are counted.


Suggested change

If repetitions are nested, then the optional depth parameter can be used to limit the number of nested repetitions that are counted.

If repetitions are nested, then the optional depth parameter can be used to

count the repetitions of a parent repetition group.

tgross35 · 2024-05-22T01:56:12Z

src/macros-by-example.md

+}
+```
+
+If repetitions are nested, then the optional depth parameter can be used to limit the number of nested repetitions that are counted.


Suggested change

If repetitions are nested, then the optional depth parameter can be used to limit the number of nested repetitions that are counted.

If repetitions are nested, then the optional depth parameter can be used to

count the repetitions of a parent repetition group.

tgross35 · 2024-05-22T01:58:09Z

src/macros-by-example.md

+
+The `ignore` metavariable acts as if the ident was used for the purposes of repetition, but expands to nothing.
+
+### index(depth)


Suggested change

### index(depth)

### `index(depth=0)`

tgross35 · 2024-05-22T01:58:22Z

src/macros-by-example.md

+}
+```
+
+### len(depth)


Suggested change

### len(depth)

### `len(depth=0)`

RalfJung · 2024-05-30T06:19:35Z

src/macros-by-example.md

+}
+```
+
+`count` can not be placed inside the innermost repetition.


Would be good to have an example of that; I don't know what is meant here.

Is this even about the innermost repetition? In ( $( $name:ident: $( $value:literal ),* );+ ), can I do $( ${count($name) }? That is not the innermost repetition, as value is even-more-inner -- but I am trying to count name inside a repetition for name which will always return 1.

jdahlstrom reviewed Apr 7, 2024

View reviewed changes

tgross35 mentioned this pull request Apr 10, 2024

Tracking Issue for Hasher::{write_str, write_length_prefix} (feature(hasher_prefixfree_extras)) rust-lang/rust#96762

Open

3 tasks

ehuss added the S-waiting-on-stabilization Waiting for a stabilization PR to be merged in the main Rust repository label Apr 27, 2024

Add documentation for meta variable expressions

327c2f8

c410-f3r force-pushed the again-again-again branch from a85c9f7 to 327c2f8 Compare May 17, 2024 11:57

tgross35 reviewed May 22, 2024

View reviewed changes

RalfJung reviewed May 30, 2024

View reviewed changes

RalfJung mentioned this pull request May 30, 2024

Stabilize count, ignore, index, and length in Rust 1.80 rust-lang/rust#122808

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add documentation for meta variable expressions #1485

Add documentation for meta variable expressions #1485

c410-f3r commented Mar 23, 2024

jdahlstrom Apr 7, 2024

jdahlstrom Apr 7, 2024

jdahlstrom Apr 7, 2024 •

edited

tgross35 Apr 10, 2024 •

edited

jdahlstrom Apr 10, 2024

ehuss commented May 17, 2024

c410-f3r commented May 17, 2024

tgross35 left a comment

tgross35 May 22, 2024

tgross35 May 22, 2024

tgross35 May 22, 2024

tgross35 May 22, 2024

RalfJung May 30, 2024

RalfJung May 30, 2024 •

edited

tgross35 May 22, 2024

RalfJung May 30, 2024

tgross35 May 22, 2024

tgross35 May 22, 2024

tgross35 May 22, 2024

tgross35 May 22, 2024

tgross35 May 22, 2024

RalfJung May 30, 2024


		### count($ident, depth)

		Expands to an unsuffixed integer literal representing the number of times a *metavariable* repeats in total.

-Gives access to additional information about metavariables that otherwise would be difficult or even impossible to obtain manually.
+Metavariable expressions are function-like operators that are used to access "meta"
+information about how macros get matched and expanded, such as how many items
+get matched in a repetition group. This information is typically difficult or even
+impossible to otherwise obtain.
+Metavariable expressions live within a special braced syntax `${...}`. There are a few
+different operators available:


		Gives access to additional information about metavariables that otherwise would be difficult or even impossible to obtain manually.

		### count($ident, depth)


		Expands to an unsuffixed integer literal representing the number of times a *metavariable* repeats in total.

		The output of `count` depends on where it is placed as well the provided index. If no index is provided, then it will always start at the innermost level.

-macro_rules! no_repetition0 {
-    ( $( $a:ident: $( $b:literal ),* );+ ) => {
-        [${count($b)}]
-    };
-}
-macro_rules! no_repetition1 {
-    ( $( $a:ident: $( $b:literal ),* );+ ) => {
-        [${count($b, 1)}]
-    };
-}
-macro_rules! outermost {
-    ( $( $a:ident: $( $b:literal ),* );+ ) => {
-        [$( ${ignore($a)} ${count($b)}, )+]
-    };
-}
-fn main() {
-    // 1 2 3 4 5 = 5 elements
-    assert_eq!(no_repetition0!(a: 1, 2, 3; b: 4, 5), [5]);
-    // a b = 2 elements
-    assert_eq!(no_repetition1!(a: 1, 2, 3; b: 4, 5), [2]);
-    // 1 2 3 = 3 elements
-    // 4 5 = 2 elements
-    assert_eq!(outermost!(a: 1, 2, 3; b: 4, 5), [3, 2]);
-}
+macro_rules! count_value {
+    ( $( $name:ident: $( $value:literal ),* );+ ) => {
+        // Count the total number of times that the (innermost) group
+        // containing `$value` gets matched.
+        ${count($value)}
+        // This is the same as `${count($value, 0)}`
+    };
+}
+macro_rules! count_name1 {
+    ( $( $name:ident: $( $value:literal ),* );+ ) => {
+        // This is one way to get the number of times that the group
+        // containing `$name` gets matched. Alternatively...
+        ${count($name)}
+    };
+}
+macro_rules! count_name2 {
+    ( $( $name:ident: $( $value:literal ),* );+ ) => {
+        // ...`$value` can be used again with a depth specifier of 1,
+        // indicating that repetitions of the 1st parent group of
+        // `$value` should be counted, rather than `$value`'s innermost group.
+        //
+        // `1` is the maximum value here since `$value`'s group has a single
+        // parent.
+        ${count($value, 1)}
+    };
+}
+macro_rules! count_value_nested {
+    ( $( $name:ident: $( $value:literal ),* );+ ) => {
+        [ $(
+            // using `count` within a repetition group will return the number
+            // of times `$value` is matched _within that group_.
+            ${count($value)},
+        )+ ]
+    };
+}
+fn main() {
+    // all instances of `$value` counted: count(1, 2, 3, 4, 5) = 5
+    assert_eq!(count_value!(a: 1, 2, 3; b: 4, 5), 5);
+    // count(1, 2, 3, ... 11) = 11
+    assert_eq!(
+        count_value!(a: 1, 2, 3; b: 4, 5; c: 6, 7; d: 8, 9, 10, 11),
+    );
+    // `$value` is never matched; count() = 0
+    assert_eq!(count_value!(a:), 0);
+    // count(a, b) = 2 matches
+    assert_eq!(count_name1!(a: 1, 2, 3; b: 4, 5), 2);
+    // count(a, b, c, d) = 4 matches
+    assert_eq!(
+        count_name1!(a: 1, 2, 3; b: 4, 5; c: 6, 7; d: 8, 9, 10, 11),
+    );
+    // count(a) = 1 match
+    assert_eq!(count_name1!(a:), 1);
+    // These have the same results as the above
+    assert_eq!(count_name2!(a: 1, 2, 3; b: 4, 5), 2);
+    assert_eq!(
+        count_name2!(a: 1, 2, 3; b: 4, 5; c: 6, 7; d: 8, 9, 10, 11),
+    );
+    assert_eq!(count_name2!(a:), 1);
+    // first match: count(1, 2, 3) = 3. second match: count(4, 5) = 2.
+    assert_eq!(count_value_nested!(a: 1, 2, 3; b: 4, 5), [3, 2]);
+    // first match: count(1, 2, 3) = 3. second match: count(4, 5) = 2.
+    // third match: count(6, 7) = 4. fourth match: count(8, 9, 10, 11) = 4.
+    assert_eq!(
+        count_value_nested!(a: 1, 2, 3; b: 4, 5; c: 6, 7; d: 8, 9, 10, 11),
+        [3, 2, 2, 4]
+    );
+    // `$value` is never matched; count() = 0
+    assert_eq!(count_value_nested!(a:), [0]);
+}

	Contrary to `count`, `len` must be placed inside a repetition.
	Unlike `count`, `len` must be placed inside a repetition.

	If repetitions are nested, then the optional depth parameter can be used to limit the number of nested repetitions that are counted.
	If repetitions are nested, then the optional depth parameter can be used to
	count the repetitions of a parent repetition group.


		The `ignore` metavariable acts as if the ident was used for the purposes of repetition, but expands to nothing.

		### index(depth)

Add documentation for meta variable expressions #1485

Are you sure you want to change the base?

Add documentation for meta variable expressions #1485

Conversation

c410-f3r commented Mar 23, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jdahlstrom Apr 7, 2024 • edited

Choose a reason for hiding this comment

tgross35 Apr 10, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ehuss commented May 17, 2024

c410-f3r commented May 17, 2024

tgross35 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RalfJung May 30, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jdahlstrom Apr 7, 2024 •

edited

tgross35 Apr 10, 2024 •

edited

RalfJung May 30, 2024 •

edited