Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow \" in single-double-quoted string interpolations #8830

Merged
merged 2 commits into from Mar 31, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
28 changes: 15 additions & 13 deletions spec/01-lexical-syntax.md
Expand Up @@ -463,7 +463,7 @@ arbitrary, except that it may contain three or more consecutive quote characters
only at the very end. Characters
must not necessarily be printable; newlines or other
control characters are also permitted. [Escape sequences](#escape-sequences) are
not processed, except for Unicode escapes.
not processed, except for Unicode escapes (this is deprecated since 2.13.2).

> ```scala
> """the present string
Expand Down Expand Up @@ -503,8 +503,9 @@ not processed, except for Unicode escapes.
#### Interpolated string

```ebnf
interpolatedString ::= alphaid ‘"’ {printableChar \ (‘"’ | ‘$’) | escape} ‘"’
| alphaid ‘"""’ {[‘"’] [‘"’] char \ (‘"’ | ‘$’) | escape} {‘"’} ‘"""’
interpolatedString ::= alphaid ‘"’ {[‘\’] interpolatedStringPart | ‘\\’ | ‘\"’} ‘"’
| alphaid ‘"""’ {[‘"’] [‘"’] char \ (‘"’ | ‘$’) | escape} {‘"’} ‘"""’
interpolatedStringPart ::= printableChar \ (‘"’ | ‘$’ | ‘\’) | escape
escape ::= ‘$$’
| ‘$"’
| ‘$’ id
Expand All @@ -514,23 +515,24 @@ alphaid ::= upper idrest

```

Interpolated string consist of an identifier starting with a letter immediately
An interpolated string consists of an identifier starting with a letter immediately
followed by a string literal. There may be no whitespace characters or comments
between the leading identifier and the opening quote ‘”’ of the string.
The string literal in a interpolated string can be standard (single quote)
between the leading identifier and the opening quote `"` of the string.
The string literal in an interpolated string can be standard (single quote)
or multi-line (triple quote).

Inside a interpolated string none of the usual escape characters are interpreted
(except for unicode escapes) no matter whether the string literal is normal
(enclosed in single quotes) or multi-line (enclosed in triple quotes).
Instead, there are three new forms of dollar sign escape.
Inside an interpolated string none of the usual escape characters are interpreted
no matter whether the string literal is normal (enclosed in single quotes) or
multi-line (enclosed in triple quotes). Note that the sequence `\"` does not
close a normal string literal (enclosed in single quotes).

There are three forms of dollar sign escape.
The most general form encloses an expression in `${` and `}`, i.e. `${expr}`.
The expression enclosed in the braces that follow the leading `$` character is of
syntactical category BlockExpr. Hence, it can contain multiple statements,
and newlines are significant. Single ‘$’-signs are not permitted in isolation
in a interpolated string. A single ‘$’-sign can still be obtained by doubling the ‘$’
character: ‘$$’. A single ‘"’-sign in a single quoted interpolation would end the
interpolation. A single ‘"’-sign can be obtained by the sequence ‘\$"’.
in an interpolated string. A single ‘$’-sign can still be obtained by doubling the ‘$’
character: ‘$$’. A single ‘"’-sign can be obtained by the sequence ‘\$"’.

The simpler form consists of a ‘$’-sign followed by an identifier starting with
a letter and followed only by letters, digits, and underscore characters,
Expand Down
6 changes: 4 additions & 2 deletions spec/13-syntax-summary.md
Expand Up @@ -60,9 +60,11 @@ stringElement ::= charNoDoubleQuoteOrNewline
| escapeSeq
multiLineChars ::= {[‘"’] [‘"’] charNoDoubleQuote} {‘"’}

interpolatedString
::= alphaid ‘"’ {printableChar \ (‘"’ | ‘\$’) | escape} ‘"’
interpolatedString
::= alphaid ‘"’ {[‘\’] interpolatedStringPart | ‘\\’ | ‘\"’} ‘"’
| alphaid ‘"""’ {[‘"’] [‘"’] char \ (‘"’ | ‘\$’) | escape} {‘"’} ‘"""’
interpolatedStringPart
::= printableChar \ (‘"’ | ‘$’ | ‘\’) | escape
escape ::= ‘\$\$’
| ‘\$"’
| ‘\$’ id
Expand Down
30 changes: 23 additions & 7 deletions src/compiler/scala/tools/nsc/ast/parser/Scanners.scala
Expand Up @@ -854,7 +854,12 @@ trait Scanners extends ScannersCommon {
} else unclosedStringLit()
}

private def unclosedStringLit(): Unit = syntaxError("unclosed string literal")
private def unclosedStringLit(seenEscapedQuoteInInterpolation: Boolean = false): Unit = {
val note =
if (seenEscapedQuoteInInterpolation) "; note that `\\\"` no longer closes single-quoted interpolated string literals since 2.13.6, you can use a triple-quoted string instead"
lrytz marked this conversation as resolved.
Show resolved Hide resolved
else ""
syntaxError(s"unclosed string literal$note")
}

private def replaceUnicodeEscapesInTriple(): Unit =
if(strVal != null) {
Expand Down Expand Up @@ -890,7 +895,8 @@ trait Scanners extends ScannersCommon {
}
}

@tailrec private def getStringPart(multiLine: Boolean): Unit = {
// for interpolated strings
@tailrec private def getStringPart(multiLine: Boolean, seenEscapedQuote: Boolean = false): Unit = {
def finishStringPart() = {
setStrVal()
token = STRINGPART
Expand All @@ -904,18 +910,27 @@ trait Scanners extends ScannersCommon {
setStrVal()
token = STRINGLIT
} else
getStringPart(multiLine)
getStringPart(multiLine, seenEscapedQuote)
} else {
nextChar()
setStrVal()
token = STRINGLIT
}
} else if (ch == '\\' && !multiLine) {
putChar(ch)
nextRawChar()
val q = ch == '"'
if (q || ch == '\\') {
putChar(ch)
nextRawChar()
}
getStringPart(multiLine, seenEscapedQuote || q)
} else if (ch == '$') {
nextRawChar()
if (ch == '$' || ch == '"') {
putChar(ch)
nextRawChar()
getStringPart(multiLine)
getStringPart(multiLine, seenEscapedQuote)
} else if (ch == '{') {
finishStringPart()
nextRawChar()
Expand Down Expand Up @@ -946,13 +961,14 @@ trait Scanners extends ScannersCommon {
if (isUnclosedLiteral) {
if (multiLine)
incompleteInputError("unclosed multi-line string literal")
else
unclosedStringLit()
else {
unclosedStringLit(seenEscapedQuote)
}
lrytz marked this conversation as resolved.
Show resolved Hide resolved
}
else {
putChar(ch)
nextRawChar()
getStringPart(multiLine)
getStringPart(multiLine, seenEscapedQuote)
}
}
}
Expand Down
4 changes: 4 additions & 0 deletions test/files/neg/t6476.check
@@ -0,0 +1,4 @@
t6476.scala:8: error: unclosed string literal; note that `\"` no longer closes single-quoted interpolated string literals since 2.13.6, you can use a triple-quoted string instead
mimi"\"
^
1 error
9 changes: 9 additions & 0 deletions test/files/neg/t6476.scala
@@ -0,0 +1,9 @@
// only the last one doesn't parse
class C {
mimi"""\ """
mimi"""\\"""
mimi"""\"""
mimi"\ "
mimi"\\"
mimi"\"
}
7 changes: 7 additions & 0 deletions test/files/neg/t6476b.check
@@ -0,0 +1,7 @@
t6476b.scala:2: error: invalid escape at terminal index 0 in "\". Use \\ for literal \.
val sa = s"""\"""
^
t6476b.scala:4: error: invalid escape '\ ' not one of [\b, \t, \n, \f, \r, \\, \", \', \uxxxx] at index 0 in "\ ". Use \\ for literal \.
val sc = s"""\ """
^
2 errors
8 changes: 8 additions & 0 deletions test/files/neg/t6476b.scala
@@ -0,0 +1,8 @@
class C {
val sa = s"""\"""
val sb = s"""\\"""
val sc = s"""\ """
val ra = raw"""\"""
val rb = raw"""\\"""
val rc = raw"""\ """
}
4 changes: 2 additions & 2 deletions test/files/neg/t8266-invalid-interp.check
@@ -1,6 +1,6 @@
t8266-invalid-interp.scala:4: error: Trailing '\' escapes nothing.
f"a\",
^
f"""a\""",
^
t8266-invalid-interp.scala:5: error: invalid escape '\x' not one of [\b, \t, \n, \f, \r, \\, \", \', \uxxxx] at index 1 in "a\xc". Use \\ for literal \.
f"a\xc",
^
Expand Down
2 changes: 1 addition & 1 deletion test/files/neg/t8266-invalid-interp.scala
@@ -1,7 +1,7 @@

trait X {
def f = Seq(
f"a\",
f"""a\""",
f"a\xc",
// following could suggest \u000b for vertical tab, similar for \a alert
f"a\vc"
Expand Down
2 changes: 1 addition & 1 deletion test/files/pos/t11966.scala
Expand Up @@ -3,5 +3,5 @@
object Test {
val original = """\/ \/ /\"""
val minimal = """\1234\"""
val alternative = raw"\1234\"
val alternative = raw"""\1234\"""
}
12 changes: 12 additions & 0 deletions test/files/run/interpolation-repl.check
@@ -0,0 +1,12 @@

scala> raw"\""
val res0: String = \"

scala> raw"\" // this used to be a comment, but after scala/pull#8830 it's part of the string! "
val res1: String = "\" // this used to be a comment, but after scala/pull#8830 it's part of the string! "

scala> raw"\" // this used to compile, now it's unclosed
^
error: unclosed string literal; note that `\"` no longer closes single-quoted interpolated string literals since 2.13.6, you can use a triple-quoted string instead

scala> :quit
9 changes: 9 additions & 0 deletions test/files/run/interpolation-repl.scala
@@ -0,0 +1,9 @@
import scala.tools.partest.ReplTest

object Test extends ReplTest {
def code = """
raw"\""
raw"\" // this used to be a comment, but after scala/pull#8830 it's part of the string! "
raw"\" // this used to compile, now it's unclosed
"""
}
13 changes: 13 additions & 0 deletions test/files/run/t6476.check
@@ -0,0 +1,13 @@
"Hello", Alice
"Hello", Alice
"Hello", Alice
"Hello", Alice
\"Hello\", Alice
\"Hello\", Alice
\TILT\
\TILT\
\\TILT\\
\TILT\
\TILT\
\\TILT\\
\TILT\
23 changes: 23 additions & 0 deletions test/files/run/t6476.scala
@@ -0,0 +1,23 @@
object Test {
def main(args: Array[String]): Unit = {
val person = "Alice"
println(s"\"Hello\", $person")
println(s"""\"Hello\", $person""")

println(f"\"Hello\", $person")
println(f"""\"Hello\", $person""")

println(raw"\"Hello\", $person")
println(raw"""\"Hello\", $person""")

println(s"\\TILT\\")
println(f"\\TILT\\")
println(raw"\\TILT\\")

println(s"""\\TILT\\""")
println(f"""\\TILT\\""")
println(raw"""\\TILT\\""")

println(raw"""\TILT\""")
}
}