Add support for labels to Kotlin lexer

[goodhoko]

Hello,
I'm trying to add support for labels to the Kotlin lexer. I managed to get it work for the label definition. Eg.

loop@ for (i in 1..100) {

but can't make it work for the rule for label reference. Eg.

if (...) break@loop 

I suspect the look behind group is the cause but I'm not really sure. I'd like to ask for help on this. .)

For reference how labels are used in Kotlin see the Break and Continue Labels.

[pyrmont]

@goodhoko Sorry about the confusion. This should have worked but unfortunately didn't because of a longstanding bug in the StringScanner library that Rouge uses. As a result of the bug, lookbehinds don't work :( That bug has been fixed but enabling the fix is opt-in and we haven't enabled that yet (see #1491 for more details).

In any event, the simplest fix is to match the terms in the lookbehind earlier in the state. That's what I've done with the latest commit. Let me know how it looks on your end.

[goodhoko]

@pyrmont thanks for the link it all makes sense now. Also thanks for your commit. It's completely ok from my side so we can merge, possibly squashing the commits? .)

By the way I noticed that there's quite a lot of look behind groups in rouge lexers.

# not even accounting for negative look behind
goodhoko:~/Workspace/rouge/lib/rouge/lexers$ grep -rn '(?<=' .
./scala.rb:30:      idrest = %r(#{letter}(?:#{letter}|#{digits})*(?:(?<=_)#{op}+)?)x
./julia.rb:224:        rule %r/(?<=[.\w)\]])\'+/, Operator
./javascript.rb:148:        rule %r((?<=\n)(?=\s|/|<!--)), Text, :expr_start
./actionscript.rb:114:        rule %r((?<=\n)(?=\s|/|<!--)), Text, :expr_start
./viml.rb:36:        rule %r/(?<=\s)"[^-:.%#=*].*?$/, Comment
./elixir.rb:30:             (?<=[\s])!+|&(&&?|(?!\d))|\|\||\^|\*|\+|\-|/|
./awk.rb:106:        rule %r((?<=\n)(?=\s|/)), Text, :expr_start
./bsl.rb:12:      KEYWORDS = /(?<=[^\wа-яё]|^)(?:
./bsl.rb:30:      BUILTINS = /(?<=[^\wа-яё]|^)(?:
./bsl.rb:67:        rule %r/(?<=[^\wа-яё]|^)\&.*$/, Keyword::Declaration
./bsl.rb:69:        rule %r/(?<=[^\wа-яё]|^)\#.*$/, Keyword::Declaration
./bsl.rb:78:        rule %r/(?<=[^\wа-яё]|^)\|((?!\"\").)*?(\"|$)/, Literal::String
./perl.rb:98:        rule %r(((?<==~)|(?<=\())\s*/(\\\\|\\/|[^/])*/[msixpodualngc]*),
./erlang.rb:80:        rule %r{(?:^|(?<=:))(#{atom_re})(\s*)(\()} do

Does this mean they all don't work?

[goodhoko]

Uh, maybe wait a little, I'll also add some tests for the labels to the lexer spec.

[goodhoko]

@pyrmont OK, LGTM. Should be squashed to have feature and tests in one commit. 👍

// I come from GitLab where squashing is a single click before merging. Let me know if I need to do this manually here.

[pyrmont]

@goodhoko I think that yes, those rules probably don't match precisely as intended. That said, in most cases, I'd expect it results in syntactically incorrect code lexing without problems rather than syntactically correct code having errors. Since syntax correctness is a non-goal of Rouge that's not the biggest problem (although it is something to consider when we move to the improved StringScanner).

In any event, thank you for adding the tests. These all look good—will merge in now (our merging strategy is squash and merge when we accept PRs so nothing for you to do!).

[pyrmont]

@goodhoko OK, merged in now. Thanks for the PR! This will be part of the next release of Rouge. That's scheduled for Tuesday 14 April which, depending on where you are when you read this, is either today or tomorrow 🎉