(C++) Integer literals with quote delimiters in #defines break highlighting

[jureslak]

Describe the issue
Integer literals in C++ may be written as 1'000'000'000 instead of 1000000000 for greater readability. When this is used inside a #define, if breaks highlighting which treats the rest of the code as being inside a string.

Which language seems to have the issue?
cpp

Are you using highlight or highlightAuto?
highlight

Sample Code to Reproduce

#include <iostream>
#define INF 1'000'000'000

int main() {
	int a = INF;
    return 0;
}

Expected behavior
The token 1'000'000'000 should be highlighted as a number, and the following code should also be highlighted normally, instead of as if it is inside a string:

Additional context

The snippet below seems to be highlighted fine, so the issue is likely the interplay between quote-separated literals and defines.

#include <iostream>

int f(int x) { return f(2'000); }

int main() {
	int a = 1'000'000'000;
    int b = f(1'000);
    return 0;
}

[TokenPayId]

Describe the issue

Integer literals in C++ may be written as 1'000'000'000 instead of 1000000000 for greater readability. When this is used inside a #define, if breaks highlighting which treats the rest of the code as being inside a string.

Which language seems to have the issue?

cpp

Are you using highlight or highlightAuto?

highlight

Sample Code to Reproduce

#include <iostream>

#define INF 1'000'000'000



int main() {

	int a = INF;

    return 0;

}

Expected behavior

The token 1'000'000'000 should be highlighted as a number, and the following code should also be highlighted normally, instead of as if it is inside a string:

Additional context

The snippet below seems to be highlighted fine, so the issue is likely the interplay between quote-separated literals and defines.

#include <iostream>



int f(int x) { return f(2'000); }



int main() {

	int a = 1'000'000'000;

    int b = f(1'000);

    return null;

}

[TokenPayId]

Describe the issue

Integer literals in C++ may be written as 1'000'000'000 instead of 1000000000 for greater readability. When this is used inside a #define, if breaks highlighting which treats the rest of the code as being inside a string.

Which language seems to have the issue?

cpp

Are you using highlight or highlightAuto?

highlight

Sample Code to Reproduce

#include <iostream>

#define INF 1'000'000'000



int main() {

	int a = INF;

    return 0;

}

Expected behavior

The token 1'000'000'000 should be highlighted as a number, and the following code should also be highlighted normally, instead of as if it is inside a string:

Additional context

The snippet below seems to be highlighted fine, so the issue is likely the interplay between quote-separated literals and defines.

#include <iostream>



int f(int x) { return f(2'000); }



int main() {

	int a = 1'000'000'000;

    int b = f(1'000);

    return null;

}

[joshgoebel]

Can you confirm a fix: #3930

[jureslak]

Looks great, thanks!

Before:

After:

[akhtarmdsaad]

Is this issue resolved?

[joshgoebel]

Not yet, I think I've started this needs to be contextual, so it's a bit more complicated than what we have so far.