Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use Ident::new_raw to quote raw identifiers #225

Merged
merged 1 commit into from Jun 20, 2022
Merged

Use Ident::new_raw to quote raw identifiers #225

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion Cargo.toml
View file
Open in desktop
Expand Up @@ -14,7 +14,7 @@ autobenches = false
rust-version = "1.31"

[dependencies]
proc-macro2 = { version = "1.0.36", default-features = false }
proc-macro2 = { version = "1.0.40", default-features = false }

[dev-dependencies]
rustversion = "1.0"
Expand Down
50 changes: 6 additions & 44 deletions src/runtime.rs
View file
Open in desktop
Expand Up @@ -205,27 +205,11 @@ fn respan_token_tree(mut token: TokenTree, span: Span) -> TokenTree {
}

pub fn push_ident(tokens: &mut TokenStream, s: &str) {
// Optimization over `mk_ident`, as `s` is guaranteed to be a valid ident.
//
// FIXME: When `Ident::new_raw` becomes stable, this method should be
// updated to call it when available.
if s.starts_with("r#") {
parse(tokens, s);
} else {
tokens.append(Ident::new(s, Span::call_site()));
}
tokens.append(mk_ident(s, None));
}

pub fn push_ident_spanned(tokens: &mut TokenStream, span: Span, s: &str) {
// Optimization over `mk_ident`, as `s` is guaranteed to be a valid ident.
//
// FIXME: When `Ident::new_raw` becomes stable, this method should be
// updated to call it when available.
if s.starts_with("r#") {
parse_spanned(tokens, span, s);
} else {
tokens.append(Ident::new(s, span));
}
tokens.append(mk_ident(s, Some(span)));
}

pub fn push_lifetime(tokens: &mut TokenStream, lifetime: &str) {
Expand Down Expand Up @@ -392,36 +376,14 @@ pub fn push_underscore_spanned(tokens: &mut TokenStream, span: Span) {

// Helper method for constructing identifiers from the `format_ident!` macro,
// handling `r#` prefixes.
//
// Directly parsing the input string may produce a valid identifier,
// although the input string was invalid, due to ignored characters such as
// whitespace and comments. Instead, we always create a non-raw identifier
// to validate that the string is OK, and only parse again if needed.
pub fn mk_ident(id: &str, span: Option<Span>) -> Ident {
let span = span.unwrap_or_else(Span::call_site);

let is_raw = id.starts_with("r#");
let unraw = Ident::new(if is_raw { &id[2..] } else { id }, span);
if !is_raw {
return unraw;
}

// At this point, the identifier is raw, and the unraw-ed version of it was
// successfully converted into an identifier. Try to produce a valid raw
// identifier by running the `TokenStream` parser, and unwrapping the first
// token as an `Ident`.
//
// FIXME: When `Ident::new_raw` becomes stable, this method should be
// updated to call it when available.
if let Ok(ts) = id.parse::<TokenStream>() {
let mut iter = ts.into_iter();
if let (Some(TokenTree::Ident(mut id)), None) = (iter.next(), iter.next()) {
id.set_span(span);
return id;
}
if id.starts_with("r#") {
Ident::new_raw(&id[2..], span)
} else {
Ident::new(id, span)
}

panic!("not allowed as a raw identifier: `{}`", id);
}

// Adapts from `IdentFragment` to `fmt::Display` for use by the `format_ident!`
Expand Down