Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Futhark lexer #1691

Merged
merged 1 commit into from
Feb 14, 2021
Merged

Add Futhark lexer #1691

merged 1 commit into from
Feb 14, 2021

Conversation

Munksgaard
Copy link
Contributor

The purpose of this PR is to add support for the Futhark programming language.

I've tried to follow the guide on https://pygments.org/docs/lexerdevelopment/, but please let me know if there's something else I need to do.

@Munksgaard
Copy link
Contributor Author

Oh my, perhaps I overdid it by adding so many examples...

@Munksgaard Munksgaard force-pushed the futhark branch 2 times, most recently from 869b85c to 6b46c36 Compare January 20, 2021 17:43
@Anteru
Copy link
Collaborator

Anteru commented Jan 21, 2021

Hi, yes, I think that's a bit much on the example side. Could you please try to condense it to 1-2 files at most? Please keep in mind that all code you submit here must be yours or BSD compatible, and I do get nervous when I see large amounts of code getting submitted :)

@Munksgaard
Copy link
Contributor Author

Munksgaard commented Jan 21, 2021

Hi @Anteru, thanks for your comment. I have added just a single example, that I've written myself for the occasion. Hopefully, this is more to your liking :)

Copy link
Member

@birkenfeld birkenfeld left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update! Unfortunately quite a few of the tokens generated by the examplefile are Errors :)

(The current way examplefiles are handled is pretty new, which is why this wasn't caught by the test. We'll get a fix whipped up for that.)

@Munksgaard
Copy link
Contributor Author

Thanks for the update! Unfortunately quite a few of the tokens generated by the examplefile are Errors :)

Ah, yes, I can see that. I didn't know that it needed to classify everything correctly. I'll try to find some time to address this.

@Munksgaard Munksgaard force-pushed the futhark branch 3 times, most recently from 14a9329 to 0f2caf6 Compare February 5, 2021 10:21
@Munksgaard
Copy link
Contributor Author

@birkenfeld I've tried to update my lexer a bit, but now I'm getting an error in the linter: overlap in character class. I don't know what that means, perhaps you can help me understand?

@birkenfeld
Copy link
Member

@Munksgaard it means that you have a single character multiple times in a character class. One way to do that is simply something like [aabc], the other way is what seems to have happened here:

(r'[+-/%=!><|&*^][+-/%=!><|&*^.]*', Operator),

The +-/ is a range from + to / which includes +,-./. - is special in character classes unless it is escaped with \ or put at the beginning or end of the class. So [-+... instead of [+-... will fix it.

@Munksgaard
Copy link
Contributor Author

@Munksgaard it means that you have a single character multiple times in a character class. One way to do that is simply something like [aabc], the other way is what seems to have happened here:

(r'[+-/%=!><|&*^][+-/%=!><|&*^.]*', Operator),

The +-/ is a range from + to / which includes +,-./. - is special in character classes unless it is escaped with \ or put at the beginning or end of the class. So [-+... instead of [+-... will fix it.

Thank you, I've tried that now :)

@Anteru Anteru self-assigned this Feb 12, 2021
@Anteru Anteru added this to the 2.8 milestone Feb 12, 2021
@Anteru Anteru added the changelog-update Items which need to get mentioned in the changelog label Feb 14, 2021
@Anteru
Copy link
Collaborator

Anteru commented Feb 14, 2021

Merged, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants