Unquoted object keys should support non-ASCII characters

[aseemk]

ES5 allows it. References:

• http://es5.github.com/#x7.6
• https://developer.mozilla.org/en/Core_JavaScript_1.5_Guide/Core_Language_Features#Variables
• http://docstore.mik.ua/orelly/webprog/jscript/ch02_07.htm

Potentially some helpful resources:

• http://stackoverflow.com/questions/147824/javascript-how-to-find-whether-a-particular-string-has-unicode-characters-esp
• http://stackoverflow.com/questions/280712/javascript-unicode

[EliSnow]

Other references:
http://mathiasbynens.be/notes/javascript-properties
http://mothereff.in/js-properties (linked from above article)

[jordanbtucker]

Are there any ideas on how to support Unicode and their escape sequences in identifiers? What are the options, and what are the pros and cons of those options?

It seems the biggest hurdle is that there is no standard way to determine what Unicode category a code point belongs to. For example, an identifier can start with $, _, or any character in the five Letter categories or in the Number, Letter category. That's 17196 different characters, all spread across the Unicode code point list.

Here are some ideas on perhaps how to solve this:

• Require a dependency that provides support for testing Unicode categories.
• Bake in support for it yourself. I've already done some prototyping and benchmarking.
• Consume Unicode escape sequences and forego testing them against appropriate Unicode classes, willfully violating the ES5 spec.
• Something very obvious I haven't thought of yet.

I think it's worth discussing the costs and benefits of implementing this support. What do you think?

[aseemk]

Great work investigating this!

If you've already prototyped and profiled implementing this directly, I'd say that's a very appealing option. No dependency means this continues to be easily portable (e.g. in the browser).

👍

[rlidwka]

Full unicode testing could take as much space as the rest of the library. :)

But I think it's worth it. You could generate this using esprima tools and just hardcode it into the library, since it's unlikely to be changed, or just take this one.

PS: dependencies aren't an issue, because you can always use browserify.

[aseemk]

Nice @rlidwka, thanks!

I agree re: browserify (we use and love it at FiftyThree), but I think there's still something to be said for having a simple drop-in file for developers who aren't knowledgeable about these tools. Maybe it's not worth worrying about that audience in this case though. We could also always provide a browserify-built distribution, either in the repo or via GitHub releases/downloads.

[ArcticLampyrid]

Consume Unicode escape sequences and forego testing them against appropriate Unicode classes, willfully violating the ES5 spec.

Maybe this is enough.

[jordanbtucker]

Fixed in 35269da.