Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support escaping space in string literals using \s #20110

Open
unkarjedy opened this issue Apr 5, 2024 · 9 comments
Open

support escaping space in string literals using \s #20110

unkarjedy opened this issue Apr 5, 2024 · 9 comments

Comments

@unkarjedy
Copy link
Contributor

Java multiline blocks supports \s escape sequence.

https://docs.oracle.com/en/java/javase/21/text-blocks/index.html

The \s escape sequence simple translates to space (\040, ASCII character 32, white space.) Since escape sequences don't get translated until after incident space stripping, \s can act as fence to prevent the stripping of trailing white space. Using \s at the end of each line in the following example, guarantees that each line is exactly six characters long.

String colors = """
   red  \s
   green\s
   blue \s
   """;

Scala doesn't support \s escape sequence.
Scala 2 compiler crashes
image

Scala 3 compiler says

invalid escape '\s' not one of [\b, \t, \n, \f, \r, \, ", ', \uxxxx] at index 4 in "aaa \s bbb". Use \ for literal .

Given that \s is not reserved, the compiler could support by simply replacing with \s
At first glance, this should be trivial to fix.
This wouldn't work in non-interpolated multiline string literals and in raw string literals because they don't support escape sequences.
Still, it could still be a nice addition.
image

@unkarjedy
Copy link
Contributor Author

@som-snytt
Copy link
Contributor

Scala 2 compiler does not crash.

escapes.scala:3: error: invalid escape '\s' not one of [\b, \t, \n, \f, \r, \\, \", \', \uxxxx] at index 4 in "aaa \s bbb". Use \\ for literal \.
  s"""aaa \s bbb"""
          ^
1 error

@unkarjedy
Copy link
Contributor Author

unkarjedy commented Apr 6, 2024

@som-snytt

It looks like it depends on the code.
I use 2.13.13.
With this example, the compiler indeed produces the proper error:

object Main {
  def main(args: Array[String]): Unit = {
    println(s"""aaa \s bbb""")
  }
}
invalid escape '\s' not one of [\b, \t, \n, \f, \r, \\, \", \', \uxxxx] at index 4 in "aaa \s bbb". Use \\ for literal \.
    println(s"""aaa \s bbb""")

But it fails with this code:

object Main extends App {
  println(s"""aaa \s bbb""")
}
java.lang.IndexOutOfBoundsException: 57
scala.reflect.internal.util.BatchSourceFile.offsetToLine(SourceFile.scala:213)
scala.tools.xsbt.DelegatingReporter$.makePosition$1(DelegatingReporter.scala:137)
scala.tools.xsbt.DelegatingReporter$.convert(DelegatingReporter.scala:190)
scala.tools.xsbt.DelegatingReporter.doReport(DelegatingReporter.scala:225)
scala.reflect.internal.Reporter.filteredInfo(Reporting.scala:126)
scala.reflect.internal.Reporter.error(Reporting.scala:121)
scala.tools.nsc.Reporting$PerRunReporting.error(Reporting.scala:377)
scala.tools.nsc.typechecker.Contexts$ImmediateReporter.error(Contexts.scala:1875)
scala.tools.nsc.typechecker.Contexts$ContextReporter.errorAndDumpIfDebug(Contexts.scala:1752)
scala.tools.nsc.typechecker.Contexts$ContextReporter.issue(Contexts.scala:1741)
scala.tools.nsc.typechecker.Contexts$Context.issue(Contexts.scala:817)
scala.tools.nsc.typechecker.Typers$Typer.$anonfun$typed1$26(Typers.scala:5151)
scala.tools.nsc.typechecker.Typers$Typer.$anonfun$typed1$26$adapted(Typers.scala:5151)
scala.collection.immutable.List.foreach(List.scala:334)
scala.tools.nsc.typechecker.Typers$Typer.onError$2(Typers.scala:5151)
scala.tools.nsc.typechecker.Typers$Typer.tryTypedApply$1(Typers.scala:5158)
scala.tools.nsc.typechecker.Typers$Typer.normalTypedApply$1(Typers.scala:5244)
scala.tools.nsc.typechecker.Typers$Typer.typedApply$1(Typers.scala:5257)
scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:6193)
scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:6249)
scala.tools.nsc.typechecker.Typers$Typer.typedStat$1(Typers.scala:6327)
scala.tools.nsc.typechecker.Typers$Typer.$anonfun$typedStats$9(Typers.scala:3529)
scala.tools.nsc.typechecker.Typers$Typer.typedStats(Typers.scala:3529)
scala.tools.nsc.typechecker.Typers$Typer.typedTemplate(Typers.scala:2134)
scala.tools.nsc.typechecker.Typers$Typer.typedModuleDef(Typers.scala:2010)
scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:6157)
scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:6249)
scala.tools.nsc.typechecker.Typers$Typer.typedStat$1(Typers.scala:6327)
scala.tools.nsc.typechecker.Typers$Typer.$anonfun$typedStats$9(Typers.scala:3529)
scala.tools.nsc.typechecker.Typers$Typer.typedStats(Typers.scala:3529)
scala.tools.nsc.typechecker.Typers$Typer.typedPackageDef$1(Typers.scala:5836)
scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:6159)
scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:6249)
scala.tools.nsc.typechecker.Analyzer$typerFactory$TyperPhase.apply(Analyzer.scala:125)
scala.tools.nsc.Global$GlobalPhase.applyPhase(Global.scala:481)
scala.tools.nsc.typechecker.Analyzer$typerFactory$TyperPhase.run(Analyzer.scala:112)
scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1549)
scala.tools.nsc.Global$Run.compileUnits(Global.scala:1533)
scala.tools.nsc.Global$Run.compileSources(Global.scala:1525)
scala.tools.nsc.Global$Run.compileFiles(Global.scala:1638)
scala.tools.xsbt.CachedCompiler0.run(CompilerBridge.scala:176)
scala.tools.xsbt.CachedCompiler0.run(CompilerBridge.scala:139)
scala.tools.xsbt.CompilerBridge.run(CompilerBridge.scala:43)
sbt.internal.inc.AnalyzingCompiler.compile(AnalyzingCompiler.scala:91)
sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$7(MixedAnalyzingCompiler.scala:193)
scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
sbt.internal.inc.MixedAnalyzingCompiler.timed(MixedAnalyzingCompiler.scala:248)
sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$4(MixedAnalyzingCompiler.scala:183)
sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$4$adapted(MixedAnalyzingCompiler.scala:163)
sbt.internal.inc.JarUtils$.withPreviousJar(JarUtils.scala:239)
sbt.internal.inc.MixedAnalyzingCompiler.compileScala$1(MixedAnalyzingCompiler.scala:163)
sbt.internal.inc.MixedAnalyzingCompiler.compile(MixedAnalyzingCompiler.scala:211)
sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1(IncrementalCompilerImpl.scala:534)
sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1$adapted(IncrementalCompilerImpl.scala:534)
sbt.internal.inc.Incremental$.$anonfun$apply$5(Incremental.scala:180)
sbt.internal.inc.Incremental$.$anonfun$apply$5$adapted(Incremental.scala:178)
sbt.internal.inc.Incremental$$anon$2.run(Incremental.scala:464)
sbt.internal.inc.IncrementalCommon$CycleState.next(IncrementalCommon.scala:116)
sbt.internal.inc.IncrementalCommon$$anon$1.next(IncrementalCommon.scala:56)
sbt.internal.inc.IncrementalCommon$$anon$1.next(IncrementalCommon.scala:52)
sbt.internal.inc.IncrementalCommon.cycle(IncrementalCommon.scala:263)
sbt.internal.inc.Incremental$.$anonfun$incrementalCompile$8(Incremental.scala:419)
sbt.internal.inc.Incremental$.withClassfileManager(Incremental.scala:506)
sbt.internal.inc.Incremental$.incrementalCompile(Incremental.scala:406)
sbt.internal.inc.Incremental$.apply(Incremental.scala:172)
sbt.internal.inc.IncrementalCompilerImpl.compileInternal(IncrementalCompilerImpl.scala:534)
sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileIncrementally$1(IncrementalCompilerImpl.scala:488)
sbt.internal.inc.IncrementalCompilerImpl.handleCompilationError(IncrementalCompilerImpl.scala:332)
sbt.internal.inc.IncrementalCompilerImpl.compileIncrementally(IncrementalCompilerImpl.scala:425)
sbt.internal.inc.IncrementalCompilerImpl.compile(IncrementalCompilerImpl.scala:137)
org.jetbrains.jps.incremental.scala.local.SbtCompiler.$anonfun$doCompile$3(SbtCompiler.scala:87)
scala.util.Try$.apply(Try.scala:210)
org.jetbrains.jps.incremental.scala.local.SbtCompiler.doCompile(SbtCompiler.scala:85)
org.jetbrains.jps.incremental.scala.local.SbtCompiler.compile(SbtCompiler.scala:17)
org.jetbrains.jps.incremental.scala.local.LocalServer.doCompile(LocalServer.scala:50)
org.jetbrains.jps.incremental.scala.local.LocalServer.compile(LocalServer.scala:28)
org.jetbrains.jps.incremental.scala.remote.Main$.compileLogic(Main.scala:210)
org.jetbrains.jps.incremental.scala.remote.Main$.$anonfun$handleCommand$1(Main.scala:193)
org.jetbrains.jps.incremental.scala.remote.Main$.decorated$1(Main.scala:180)
org.jetbrains.jps.incremental.scala.remote.Main$.handleCommand(Main.scala:190)
org.jetbrains.jps.incremental.scala.remote.Main$.serverLogic(Main.scala:163)
org.jetbrains.jps.incremental.scala.remote.Main$.nailMain(Main.scala:103)
org.jetbrains.jps.incremental.scala.remote.Main.nailMain(Main.scala)
jdk.internal.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.base/java.lang.reflect.Method.invoke(Method.java:568)
com.facebook.nailgun.NGSession.runImpl(NGSession.java:312)
com.facebook.nailgun.NGSession.run(NGSession.java:198)
          

Also, JFTR with scala compiler 2.12.19 and 2.11.12 don't produce any error.
The program crashes at runtime

[error] scala.StringContext$InvalidEscapeException: invalid escape '\s' not one of [\b, \t, \n, \f, \r, \\, \", \'] at index 4 in "aaa \s bbb". Use \\ for literal \.
image

@som-snytt
Copy link
Contributor

@unkarjedy you have other actors to blame in your stack

org.jetbrains.jps.incremental.scala.local.SbtCompiler.doCompile(SbtCompiler.scala:85)

I think this is why it's important to keep a "plain scalac" tool. I'm using sdkman currently, and if scala-cli supplants it in that role, then various simple bugs may be due to "other tooling". This is not contingent on any particular tool; anything adjacent to scalac adds points of failure. The bug or impedance mismatch could be anywhere.

I can attest that just the plain REPL introduces distortions. Edit: I was actually thinking of scala-cli supplanting scala runner.

In this case, I wonder what magic is performed on App files.

➜  scalac -version
Scala compiler version 2.13.13 -- Copyright 2002-2024, LAMP/EPFL and Lightbend, Inc.
➜  cat escapes.scala

class Escapes {
  s"""aaa \s bbb"""
}
object MoreEscapes extends App {
  s"""aaa \s bbb"""
}
➜  scalac -d /tmp/sandbox escapes.scala
escapes.scala:3: error: invalid escape '\s' not one of [\b, \t, \n, \f, \r, \\, \", \', \uxxxx] at index 4 in "aaa \s bbb". Use \\ for literal \.
  s"""aaa \s bbb"""
          ^
escapes.scala:6: error: invalid escape '\s' not one of [\b, \t, \n, \f, \r, \\, \", \', \uxxxx] at index 4 in "aaa \s bbb". Use \\ for literal \.
  s"""aaa \s bbb"""
          ^
2 errors

@som-snytt
Copy link
Contributor

Also wish to voice my support for \s.

@unkarjedy
Copy link
Contributor Author

unkarjedy commented Apr 7, 2024

@som-snytt

I think this is why it's important to keep a "plain scalac" tool. I'm using sdkman currently, and if scala-cli supplants it in that role, then various simple bugs may be due to "other tooling". This is not contingent on any particular tool; anything adjacent to scalac adds points of failure. The bug or impedance mismatch could be anywhere.

In practice, frequently when I say "in Scala X" or "scala compiler", I don't mean scalac application, I mean the entire pipeline. It can include scalac, compiler/sbt/zinc interfaces, sometimes REPL.
From my POV, it's what matters the most given that almost all Scala users won't use plain scalac manually.
Though I agree that "in compiler" phrasing can confuse core compiler developers, I will try to be more clear next time.


you have other actors to blame in your stack
org.jetbrains.jps.incremental.scala.local.SbtCompiler.doCompile(SbtCompiler.scala:85)

I should have mentioned that I got the same results in SBT terminal.

sbt:untitled68> reload
[info] welcome to sbt 1.9.9 (Eclipse Adoptium Java 11.0.20)
[info] loading settings for project global-plugins from idea.sbt ...
[info] loading global plugins from /Users/dmitrii.naumenko/.sbt/1.0/plugins
[info] loading project definition from /Users/dmitrii.naumenko/Desktop/dev/debug/untitled68/project
[info] loading settings for project root from build.sbt ...
[info] set current project to untitled68 (in build file:/Users/dmitrii.naumenko/Desktop/dev/debug/untitled68/)
sbt:untitled68> clean ; cleanFiles
[success] Total time: 0 s, completed 7 Apr 2024, 06:54:59
[success] Total time: 0 s, completed 7 Apr 2024, 06:54:59
sbt:untitled68> show scalaVersion
[info] 2.13.13
sbt:untitled68> compile
[info] compiling 1 Scala source to /Users/dmitrii.naumenko/Desktop/dev/debug/untitled68/target/scala-2.13/classes ...
[error] ## Exception when compiling 1 sources to /Users/dmitrii.naumenko/Desktop/dev/debug/untitled68/target/scala-2.13/classes
[error] java.lang.IndexOutOfBoundsException: 57
[error] scala.reflect.internal.util.BatchSourceFile.offsetToLine(SourceFile.scala:213)
[error] scala.tools.xsbt.DelegatingReporter$.makePosition$1(DelegatingReporter.scala:137)
...
[error] sbt.internal.inc.IncrementalCompilerImpl.compile(IncrementalCompilerImpl.scala:137)
[error] sbt.Defaults$.compileIncrementalTaskImpl(Defaults.scala:2371)
...
[error]
[error] stack trace is suppressed; run last Compile / compileIncremental for the full output
[error] (Compile / compileIncremental) java.lang.IndexOutOfBoundsException: 57
[error] Total time: 1 s, completed 7 Apr 2024, 06:55:10

Also, it turns out it's specific to App.
The same error will occur in this code:

class Main {
  s"""aaa \s bbb"""
}

Also scala-cli can crash as well when I compile command
Here is an example:

cat Main.scala
class Main {
  s"""aaa \s bbb"""
}
scala-cli --version
Scala CLI version: 1.0.6
Scala version (default): 3.3.1
Your Scala CLI version is outdated. The newest version is 1.2.1
It is recommended that you update Scala CLI through the same tool or method you used for its initial installation for avoiding the creation of outdated duplicates.
scala-cli compile --scala 2.13.13 Main.scala
Compiling project (Scala 2.13.13, JVM (11))
Error compiling project (Scala 2.13.13, JVM (11))
Error: Unexpected error when compiling tmp_e8caf7c7ca-7c19f4e7f9: '36'
Compilation failed

Notice error Unexpected error when compiling tmp_e8caf7c7ca-7c19f4e7f9: '36'
I couldn't figure out how to get more stack trace. Appending -v -v -v didn't help.
But from the magic number 36 I see that it's the same issue because in SBT/IDEA I see the same magic number java.lang.IndexOutOfBoundsException: 36

Interestingly, when not using compile it doesn't crash:

cat Main.scala
class Main {
  s"""aaa \s bbb"""
}
scala-cli Main.scala
Compiling project (Scala 3.3.1, JVM (11))
[error] ./Main.scala:2:11
[error] invalid escape '\s' not one of [\b, \t, \n, \f, \r, \\, \", \', \uxxxx] at index 4 in "aaa \s bbb". Use \\ for literal \.
[error]   s"""aaa \s bbb"""
[error]           ^
Error compiling project (Scala 3.3.1, JVM (11))
Compilation failed

Also, if you replace s"""aaa \s bbb""" with s"""a \s b"` it works fine in scala-cli (but not in sbt or IntelliJ)

cat Main.scala
class Main {
  s"""a \s b"""
}
scala-cli compile --scala 2.13.13 Main.scala
Compiling project (Scala 2.13.13, JVM (11))
[error] ./Main.scala:2:9
[error] invalid escape '\s' not one of [\b, \t, \n, \f, \r, \\, \", \', \uxxxx] at index 2 in "a \s b". Use \\ for literal \.
Error compiling project (Scala 2.13.13, JVM (11))
Compilation failed

UPD
The issue can be "fixed" by appending 3 blank lines in the end.
The length of the file will be >36 chars then and there will be no index offset exception

cat Main.scala
class Main {
  s"""aaa \s bbb"""
}



scala-cli compile --scala 2.13.13 Main.scala
Compiling project (Scala 2.13.13, JVM (11))
[error] ./Main.scala:2:11
[error] invalid escape '\s' not one of [\b, \t, \n, \f, \r, \\, \", \', \uxxxx] at index 4 in "aaa \s bbb". Use \\ for literal \.
Error compiling project (Scala 2.13.13, JVM (11))
Compilation failed

So turns out the main difference in the example with the main method is that it has some extra characters in the end.
You can "break" it by placing everything on a single line:
class Main {def main(args: Array[String]): Unit = {s"""aaa \s bbb"""}}
java.lang.IndexOutOfBoundsException: 72

If you use compiler-based highlighting, you can see in the editor that a wrong range is reported:
image

@som-snytt
Copy link
Contributor

Sorry to comment further instead of on a scala 2 ticket, but another clue is that it doesn't happen under --server false

scala-cli compile --scala 2.13.13 --verbose --server=false short.scala

Bloop wasn't starting for me with scala-cli 1.1.3, but I fetched 1.2.1 and could reproduce your issue with

Downloading compilation server 1.5.16-sc-1

I joked once that we no sooner got rid of fsc compilation daemon than it came back in another form.

@unkarjedy
Copy link
Contributor Author

If you use compiler-based highlighting, you can see in the editor that a wrong range is reported:

In the end, even though plain scalac doesn't crash itself, it's still the one to blame as it seems.
I moved the details here: scala/bug#12981 (comment)
Don't want to steal the thread of the original report even more.

@mbovel
Copy link
Member

mbovel commented Apr 8, 2024

That might be appropriate for a Spree?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants