Allow unicode (Fixes #279)

[iansw246]

Fixes #279 and other issues relating to invalid characters in identifiers. It will cover the functionality of #538

This allows any type name that is valid in ES5 by removing invalid characters. Valid characters were determined according to https://mathiasbynens.be/notes/javascript-identifiers.

It uses lodash.template, regenerate, and unicode-11.0.0 to generate the regex in scripts/generate-identifier-regex.ts

It uses code from this gist https://gist.github.com/mathiasbynens/6334847

There is no given license, so I'm not sure if we can use that.

If we can target ES6 or newer, this code can be significantly simplified with Unicode-aware regex.

[bcherny]

Thanks for the contribution, and sorry for the delay!

I'm supportive of this change, but would love to see a simpler implementation. Can we just use https://www.npmjs.com/package/identifierfy (or a similar external library) instead?

cc @mathiasbynens, in case you have a recommendation for the right NPM module to use for coercing strings to valid JS/TS identifiers.

[iansw246]

Would the Unicode-aware regex in the latest commit work? That doesn't require any other packages nor the regex-generation scripts.

[iansw246]

I tried using identifierfy but faced some issues when toSafeString was called on the output of toSafeString. Also, it removes more than the minimum in some cases. Ultimately, using identifierfy should work with a few changes to calls of toSafeString

[bcherny]

I tried using identifierfy but faced some issues when toSafeString was called on the output of toSafeString. Also, it removes more than the minimum in some cases. Ultimately, using identifierfy should work with a few changes to calls of toSafeString

Great -- want to try doing this?

[iansw246]

I've gotten Identifierfy to work

I've had to remove several calls to toSafeString() to prevent double conversion. This should be OK since the parser converts standaloneNames with generateName()