Remove . as a operator in Python lexer

[ajnisbet]

Fixes #1374

[pyrmont]

Thanks for submitting this, @ajnisbet. This does look like an error in the way Pygments and Rouge approach things. Will have a review over the next few days and see!

[pyrmont]

@ajnisbet I made a change as removing the line that tokenised . as an Operator caused all dot-notation syntax to break.

My suggestion is that we add . to the list of punctuation:

rouge/lib/rouge/lexers/python.rb

Line 79 in a2d870a

rule %r/[\[\]{}:(),;.]/, Punctuation

In themes that render Puncutation tokens in the same colour as Name tokens, the two will be indistinguishable, but in themes where punctuation is highlighted in a different colour, that will continue to work.

Let me know your thoughts.

[ajnisbet]

Yeah that seems like a good solution!

[pyrmont]

@jneen I don't imagine you're a big Python user but do you have an opinion?

I'm hesitant to break ranks with Pygments (see the relevant line in their codebase)—especially given it is a Python project—but tokenising . as Operator does seem to me to be incorrect given it's not an operator.

[pyrmont]

@ajnisbet Sorry it took so long to address this. This update will be part of the next version of Rouge which is scheduled for release on Tuesday 14 April. Thanks for the contribution!