Skip to content
This repository has been archived by the owner on Sep 8, 2023. It is now read-only.

Commit

Permalink
Make lexing of class-like identifiers more consistent in C++ lexer (r…
Browse files Browse the repository at this point in the history
…ouge-ruby#1495)

Rouge's C++ lexer currently tokenises identifiers that come after the
class keyword with the `Name::Class` token and identifiers that come
after the `struct` keyword with the generic `Name` token. This is
despite the fact that the only difference between a struct and a class
in C++ is the default access specifier. Similarly, when used in a
template definition, a typename is equivalent to a class but Rouge 
treats the identifiers that follow the `typename` and `class` keywords 
differently.

This commit addresses this problem. It treats class and struct as
equivalent in all cases while treating typename as equivalent when used
in a template definition.
  • Loading branch information
pyrmont authored and mattt committed May 21, 2020
1 parent 6fd4024 commit cda2e76
Show file tree
Hide file tree
Showing 2 changed files with 28 additions and 5 deletions.
17 changes: 12 additions & 5 deletions lib/rouge/lexers/cpp.rb
Expand Up @@ -22,10 +22,10 @@ class Cpp < C

def self.keywords
@keywords ||= super + Set.new(%w(
asm auto catch const_cast delete dynamic_cast explicit export
friend mutable namespace new operator private protected public
reinterpret_cast restrict size_of static_cast template this throw
throws typeid typename using virtual final override
asm auto catch const_cast delete dynamic_cast explicit export friend
mutable namespace new operator private protected public
reinterpret_cast restrict size_of static_cast this throw throws
typeid typename using virtual final override

alignas alignof constexpr decltype noexcept static_assert
thread_local try
Expand Down Expand Up @@ -57,7 +57,8 @@ def self.reserved
dq = /\d('?\d)*/

prepend :statements do
rule %r/class\b/, Keyword, :classname
rule %r/(class|struct)\b/, Keyword, :classname
rule %r/template\b/, Keyword, :template
rule %r/\d+(\.\d+)?(?:h|(?:min)|s|(?:ms)|(?:us)|(?:ns))/, Num::Other
rule %r((#{dq}[.]#{dq}?|[.]#{dq})(e[+-]?#{dq}[lu]*)?)i, Num::Float
rule %r(#{dq}e[+-]?#{dq}[lu]*)i, Num::Float
Expand All @@ -77,6 +78,12 @@ def self.reserved
rule %r/[.]{3}/, Operator
mixin :whitespace
end

state :template do
rule %r/>/, Punctuation, :pop!
rule %r/typename\b/, Keyword, :classname
mixin :root
end
end
end
end
16 changes: 16 additions & 0 deletions spec/visual/samples/cpp
Expand Up @@ -78,6 +78,22 @@ double distance = [](double x, double y, double xx, double yy) -> double {
return abs(x-xx) + abs(y-yy);
};

// templates
namespace N
{
template<class T>
class Y // template definition
{
void mf() { }
};
}
template class N::Y<char*>;
template void N::Y<double>::mf();
template struct Z<double>;
template<typename T> concept C1 = sizeof(T) != sizeof(int);
template<C1 T> struct S1 { };
template<C1 T> using Ptr = T*;

// variadic template
template<class F, class... Args>
void forward_args(F f, Args... args) {
Expand Down

0 comments on commit cda2e76

Please sign in to comment.