(elixir) interpolation, sigils and / functions

[eksperimental]

Update 11/2019: As far as I know #2257 leaves only item 3 open.

• 1) Interpolation: It cannot deal properly with interpolation.
• 2) Sigils : It cannot deal with these ~S sigil. ~s is fine though.
• 3) Function with a slash ( "Kernel./" ): this one takes /: as the beggining of a regular expression (Closed via fix(elixir) Support function names with a slash #2406)

---

We are adding hightlightjs to https://github.com/elixir-lang/ex_doc/
and when I ran it against the test file for pygments.
In addition to #707, I have run into three new issues:

1. Interpolation: It cannot deal properly with interpolation.

Every single line bellow, has an issues, it takes it as a comment whatever comes after the #

~R'this + i\s "a" regex too'
~w(hello #{ ["has" <> "123", '\c\d', "\123 interpol" | []] } world)s
~W(hello #{no "123" \c\d \123 interpol} world)s
~s{Escapes terminators \{ and \}, but no {balancing} # outside of sigil here }
~S"No escapes \s\t\n and no #{interpolation}"

2. Sigils : It cannot deal with these ~S sigil. ~s is fine though.

defmodule Long.Module.Name do
  @doc ~S'''
  No #{interpolation} of any kind.
  \000 \x{ff}

  \n #{\x{ff}}
  '''
  def func(a, b \\ []), do: :ok

  @doc ~S"""
  No #{interpolation} of any kind.
  \000 \x{ff}

  \n #{\x{ff}}
  """
  def func(a, b \\ []), do: :ok
end

3. Function with a slash ( "Kernel./" ): this one takes /: as the beggining of a regular expression

import Kernel, except: [spawn: 1, +: 2, /: 2, Unless: 2]

You can see this live,

• Original test doc page, with the problems: http://eksperimental.github.io/issues/highlightjs/Highlight.Elixir.html#pygments/0

• Page with the problematic lines removed: http://eksperimental.github.io/issues/highlightjs/Highlight.Elixir+.html#pygments/0

  @tonini @knewter

[Sannis]

@eksperimental to fix this I'm going to remove constant mode from Elixir definition. In fact it works bad here and in constructions like IO.puts "string" and also we will remove such extra modes in 9.0.

[eksperimental]

@Sannis thanks a lot. please do as you consider convenient. We will adapt our CSS files accordingly.

[Sannis]

@eksperimental, am I right that sigils can be also included into string interpolations? It looks like this will be too CPU heavy to do so :-(

[Sannis]

@eksperimental, I fixed some cases and added more regression tests.
Unfortunately, we do not distinguish s and S sigils with different interpolation rules for now.
Also adding more sigils formats results in many variants in the modes definitions and increase highlighting and autohighlight time.

[eksperimental]

that's great @Sannis
So one question. we don't support interpolation inside the ~S sigil, right?
if so, what happens when it's matched, does it break the code? or just ifgnores is and highlights it as regular text?
thank you for looking after this issue.
Elixir 1.1 is coming out soon, and a new version of ExDoc which uses this, so even if the next version of hightlight.js in not out, we will try to ship it outselves.

[Sannis]

I'll post highlighting example to the pull request thread today.

We currently do releases each 6 weeks, so next one will be only in October.

[isagalaev]

Elixir 1.1 is coming out soon, and a new version of ExDoc which uses this, so even if the next version of hightlight.js in not out, we will try to ship it outselves.

Here's the building docs, just in case: http://highlightjs.readthedocs.org/en/latest/building-testing.html

[eksperimental]

@isagalaev excellent! thanks a lot for all the help

[Sannis]

@eksperimental can you please post an example highlighting for ~s{Escapes terminators \{ and \}, but no {balancing} # outside of sigil here } that you expect? Am I right that whole this fragment is sigil?

[eksperimental]

@Sannis nope, it is not, starting from # is a comment

This is how it gets interpreted by Elixir interactive shell

iex(1)> ~s{Escapes terminators \{ and \}, but no {balancing} # outside of sigil here }
"Escapes terminators { and }, but no {balancing"

so the { in {balancing is not balanced

[eksperimental]

hi @Sannis is there anything else that you need from me? Or anything we can do to help you sort this out?

[umamaistempo]

About the issue 3, the same thing happens when using simplified anonymous function. I (wrongly) created an issue about this on Ex_doc repo: elixir-lang/ex_doc#486

(&fun!/1 is interpreted as a regex)

[isagalaev]

Won't make it to 9.0…

[joshgoebel]

@eksperimental Any chance you can test PR #2257? I think it fixes points 1 and 2.

[joshgoebel]

@eksperimental Any chance you could have a look at #2257 and see what you think?

[joshgoebel]

Oh sorry I already pinged you 3 days ago, lol... I guess think of it was a reminder. :)

[eksperimental]

I there a live version where I can test this?

[joshgoebel]

No, but I could probably build you a demo/devtool zip if it would truly be helpful.

[eksperimental]

No, but I could probably build you a demo/devtool zip if it would truly be helpful.

if you could. I'm out of touch with this lib (and the whole JS ecosystem), and it's been 4 years since I opened up this issue. I 'm doing it out of appreciation becuase someone took the time to take care of this after all these years.
If you think it would be helpful for your PR for me to review it, please build it so. thanks

[joshgoebel]

@eksperimental Sounds like you're busy - and I haven't gotten around to this either. I'm just going to run with this as it. It passes all the original examples provided. If there are additional edge cases we can just let people file them as they see them.

Thanks for the offer to help though!

[eksperimental]

Cool. Thank you!