Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix comment parsing in Console lexer #1379

Merged
merged 4 commits into from Dec 23, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
37 changes: 37 additions & 0 deletions lib/rouge/lexers/console.rb
Expand Up @@ -3,6 +3,38 @@

module Rouge
module Lexers
# The {ConsoleLexer} class is intended to lex content that represents the
# text that would display in a console/terminal. As distinct from the
# {Shell} lexer, {ConsoleLexer} will try to parse out the prompt from each
# line before passing the remainder of the line to the language lexer for
# the shell (by default, the {Shell} lexer).
#
# The {ConsoleLexer} class accepts four options:
# 1. **lang**: the shell language to lex (default: `shell`);
# 2. **output**: the output language (default: `plaintext?token=Generic.Output`);
# 3. **prompt**: comma-separated list of strings that indicate the end of a
# prompt (default: `$,#,>,;`);
# 4. **comments**: whether to enable comments.
#
# The comments option, if enabled, will lex lines that begin with a `#` as a
# comment. Please note that this option will only work if the prompt is
# either not manually specified or, if manually specified, does not include
# the `#` character.
#
# Most Markdown lexers that recognise GitHub-Flavored Markdown syntax, will
# pass the language string to Rouge as written in the original document.
# This allows an end user to pass options to {ConsoleLexer} by passing them
# as CGI-style parameters as in the example below.
#
# @example
# <pre>Here's some regular text.
#
# ```console?comments=true
# # This is a comment
# $ cp foo bar
# ```
#
# Some more regular text.</pre>
class ConsoleLexer < Lexer
tag 'console'
aliases 'terminal', 'shell_session', 'shell-session'
Expand Down Expand Up @@ -31,6 +63,8 @@ def prompt_regex
def end_chars
@end_chars ||= if @prompt.any?
@prompt.reject { |c| c.empty? }
elsif allow_comments?
%w($ > ;)
else
%w($ # > ;)
end
Expand Down Expand Up @@ -100,6 +134,9 @@ def stream_tokens(input, &output)
def process_line(input, &output)
input.scan(line_regex)

# As a nicety, support the use of elisions in input text. A user can
# write a line with only `<...>` or one or more `.` characters and
# Rouge will treat it as a comment.
if input[0] =~ /\A\s*(?:<[.]+>|[.]+)\s*\z/
puts "console: matched snip #{input[0].inspect}" if @debug
output_lexer.reset!
Expand Down
45 changes: 45 additions & 0 deletions spec/lexers/console_spec.rb
@@ -0,0 +1,45 @@
# -*- coding: utf-8 -*- #
# frozen_string_literal: true

describe Rouge::Lexers::ConsoleLexer do
let(:subject) { Rouge::Lexers::ConsoleLexer.new }
let(:klass) { Rouge::Lexers::ConsoleLexer }

include Support::Lexing

it 'parses a basic prompt' do
assert_tokens_equal '$ foo',
['Generic.Prompt', '$'],
['Text.Whitespace', ' '],
['Text', 'foo']
end

it 'parses a custom prompt' do
subject_with_options = klass.new({ prompt: '%' })
assert_tokens_equal '% foo', subject_with_options,
['Generic.Prompt', '%'],
['Text.Whitespace', ' '],
['Text', 'foo']
end

it 'parses single-line comments' do
subject_with_options = klass.new({ comments: true })
assert_tokens_equal '# this is a comment', subject_with_options,
['Comment', '# this is a comment']
end

it 'ignores single-line comments' do
assert_tokens_equal '# this is not a comment',
['Generic.Prompt', '#'],
['Text.Whitespace', ' '],
['Text', 'this is not a comment']
end

describe 'guessing' do
include Support::Guessing

it 'guesses by filename' do
assert_guess :filename => 'foo.cap'
end
end
end