Commits on Jun 2, 2021

1. Fix CFamilyLexer preprocessor tokenization errors …

   CFamilyLexer failed to tokenize preprocessor macros when they were
   preceded by line break surrounded by spaces. This was the case because
   prerpocessor regex rule expected to start at the beginning of the line,
   but the space regex rule matched also the whitespace after the line
   break. Now the space rule has been refined not to match the line break.
   Because of this, the preprocessor regex rule correctly matches
   prerpocessor tokens even when they are preceded by white spaces, at the
   cost of adding some more tokens in the token stream in some cases. This
   change preserves the behavior of invalid preprocessor usage failing to
   tokenize.

   

   henkkuli committed Jun 2, 2021
   Configuration menu

   • View commit details
   • Copy full SHA for 30ecbb8
   • Browse repository at this point

   Copy the full SHA

   30ecbb8 View commit details

   Browse the repository at this point in the history