Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add procfile lexer #1808

Merged
merged 10 commits into from Jun 20, 2021
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion AUTHORS
Expand Up @@ -32,7 +32,7 @@ Other contributors, listed alphabetically, are:
* Sébastien Bigaret -- QVT Operational lexer
* Jarrett Billingsley -- MiniD lexer
* Adam Blinkinsop -- Haskell, Redcode lexers
* Stéphane Blondon -- SGF and Sieve lexers
* Stéphane Blondon -- Procfile, SGF and Sieve lexers
* Frits van Bommel -- assembler lexers
* Pierre Bourdon -- bugfixes
* Martijn Braam -- Kernel log lexer, BARE lexer
Expand Down
1 change: 1 addition & 0 deletions doc/languages.rst
Expand Up @@ -263,6 +263,7 @@ Other markup
* `OMG IDL <https://www.omg.org/spec/IDL/About-IDL/>`_
* `PEG <https://bford.info/packrat/>`_
* POV-Ray scenes
* `Procfile <https://devcenter.heroku.com/articles/procfile#procfile-format>`_
* `PromQL <https://prometheus.io/docs/prometheus/latest/querying/basics/>`_
* `Puppet <https://puppet.com/>`_
* QML
Expand Down
1 change: 1 addition & 0 deletions pygments/lexers/_mapping.py
Expand Up @@ -355,6 +355,7 @@
'PowerShellLexer': ('pygments.lexers.shell', 'PowerShell', ('powershell', 'posh', 'ps1', 'psm1'), ('*.ps1', '*.psm1'), ('text/x-powershell',)),
'PowerShellSessionLexer': ('pygments.lexers.shell', 'PowerShell Session', ('ps1con',), (), ()),
'PraatLexer': ('pygments.lexers.praat', 'Praat', ('praat',), ('*.praat', '*.proc', '*.psc'), ()),
'ProcfileLexer': ('pygments.lexers.procfile', 'Procfile', ('procfile',), ('Procfile',), ()),
'PrologLexer': ('pygments.lexers.prolog', 'Prolog', ('prolog',), ('*.ecl', '*.prolog', '*.pro', '*.pl'), ('text/x-prolog',)),
'PromQLLexer': ('pygments.lexers.promql', 'PromQL', ('promql',), ('*.promql',), ()),
'PropertiesLexer': ('pygments.lexers.configs', 'Properties', ('properties', 'jproperties'), ('*.properties',), ('text/x-java-properties',)),
Expand Down
40 changes: 40 additions & 0 deletions pygments/lexers/procfile.py
@@ -0,0 +1,40 @@
"""
pygments.lexers.procfile
~~~~~~~~~~~~~~~~~~~~~~~~

Lexer for Procfile file format.

:copyright: Copyright 2006-2021 by the Pygments team, see AUTHORS.
:license: BSD, see LICENSE for details.
"""

from pygments.lexer import RegexLexer, bygroups
from pygments.token import Name, Number, String, Text, Punctuation

__all__ = ["ProcfileLexer"]


class ProcfileLexer(RegexLexer):
"""
Lexer for Procfile file format.

The format is used to run processus on Heroku or is used by Foreman or
Anteru marked this conversation as resolved.
Show resolved Hide resolved
Honcho tools.
For more information about the definition of the format, see:
https://devcenter.heroku.com/articles/procfile#procfile-format
"""
Anteru marked this conversation as resolved.
Show resolved Hide resolved
name = 'Procfile'
aliases = ['procfile']
filenames = ['Procfile']

tokens = {
'root': [
(r'^([a-z]+)(:)', bygroups(Name.Label, Punctuation)),
(r'"[^"]*"', String),
(r"'[^']*'", String),
(r'[0-9.]+', Number),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will match ..9.. as a number -- is that intentional? Could you please add a number into your Procfile example?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a basic number in Procfile file and removed the dot from the regular expression so it detects Number.Integer. I think it's good enough because I doubt someone use float number in commands. It is ok for you?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not great but I guess it'll do for now. I assume people will try to highlight valid Procfiles with it :)

(r'\$[a-zA-Z_][\w]*', Name.Variable),
(r'(\w+)(=)(\w+)', bygroups(Name.Variable, Punctuation, String)),
(r'([\w\s\-\./])', Text),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you want a + in here. Right now this matches each individual letter as a single token as you can see in the output, and I doubt that's what you're after. Additionally, this matches whitespace -- I'd suggest a separate rule for whitespace (and using the right token type for whitespace.) In that case you need to match whitespace first because you have a dot in this rule (which I hope is intentional :) )

Copy link
Contributor Author

@sblondon sblondon May 22, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I modified it: there are now 2 patterns. One for Whitespace and one for Text. These two patterns end with +.
Is it good for you?

],
}
3 changes: 3 additions & 0 deletions tests/examplefiles/procfile/Procfile
@@ -0,0 +1,3 @@
db: runzeo -C zeoserver.config
web: bundle exec rails server -p $PORT
stuff: env FLASK_ENV=development run --stuff 'foobar'
90 changes: 90 additions & 0 deletions tests/examplefiles/procfile/Procfile.output

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

40 changes: 40 additions & 0 deletions tests/test_procfile.py
@@ -0,0 +1,40 @@
# -*- coding: utf-8 -*-
"""
Basic ProcfileLexer Test
~~~~~~~~~~~~~~~~~~~~

:copyright: Copyright 2006-2020 by the Pygments team, see AUTHORS.
:license: BSD, see LICENSE for details.
"""

import pytest

from pygments.token import Name, Punctuation, Text
from pygments.lexers.procfile import ProcfileLexer


@pytest.fixture(scope='module')
def lexer():
yield ProcfileLexer()


def test_basic_line(lexer):
text = 'task: executable --options'

tokens = lexer.get_tokens(text)

for index, token in enumerate(tokens):
if index == 0:
assert token == (Name.Label, 'task')
elif index == 1:
assert token == (Punctuation, ':')
else:
assert token[0] == Text


def test_environment_variable(lexer):
text = '$XDG_SESSION_PATH'

token = list(lexer.get_tokens(text))[0]

assert token == (Name.Variable, text)