Logging strings with surrogate code points crashes deno

[rotu]

Logging a string with an unpaired surrogate unit (console.log("\uDE00")) causes Deno to crash. Similarly, when an object contains such a string, logging it also causes a crash (console.log({x:"\uDE00"}))

Note that alert and Deno.core.print do not crash in this situation

RUST_BACKTRACE=1 deno 
Deno 1.14.1
exit using ctrl+d or close()
> console.log("\uDE00")
�
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error("lone leading surrogate in hex escape", line: 1, column: 100)', /Users/ry/src/deno/core/inspector.rs:718:52
stack backtrace:
   0: _rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::result::unwrap_failed
   3: core::result::Result<T,E>::unwrap
   4: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
   5: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
   6: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
   7: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
   8: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
   9: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
  10: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
  11: deno::tools::repl::run::{{closure}}
  12: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
  13: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
  14: deno::main
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

[rotu]

Possibly related is that tab-complete does not work for such binary strings.

e.g. if I have const a="a"; const b="\uDE00", then tab completing a.ind tab autocompletes to a.indexOf but b.ind does not.

[littledivy]

Related serde_json issue: serde-rs/json#495

[lucacasonato]

My tenative plan to fix this:

1. Allow lone surrogates to be deserialized into ByteBufs by serde (Deserialize lone surrogates into byte bufs serde-rs/json#828)
2. Change our inspector implementation to deserialize into a serde_json::value::RawValue rather than a serde_json::Value
3. Add a LossyString type which has a serde::Deserialize implementation that replaces lone surrogate pairs with replacement characters using APIs from encoding_rs
4. Use LossyString to deserialize inspector messages for the REPL

Let's see how it goes.

[lucacasonato]

All upstream changes have landed in serde and are released now. This is blocked on me putting in the work to hook this all up correctly now.

[marvinhagemeister]

Tried to reproduce this with Deno 1.42.4 and I'm unable to. This is issue seems to be resolved 🎉