[WIP] Initial Handlebars/Glimmer support

[azz]

Experimenting with HTML/Handlebars/Mustache/Glimmer support for #1882

Picked the glimmer parser as I thought it would have the widest range of support.

I've ran into a few issues so far:

• Some handlebars features ({{> foo}}, {{../foo}}) are not supported. source
• <!doctype> doesn't parse.

The formatting is also super ugly at the moment because I figured the best way to go would be re-use some of the JSX formatting logic.

I also moved the postcss printer to a separate file.

[SimenB]

.moustache as well

[azz]

Or .mustache?

[azz]

I'll go with /mo?ustache/ :p

[SimenB]

hah, indeed!

[azz]

Not sure what's going on with the build on Node.js v4.

FAIL  tests/html_glimmer/jsfmt.spec.js
  ● Test suite failed to run
    SyntaxError: Unexpected token =
      
      at ScriptTransformer._transformAndBuildScript (node_modules/jest-runtime/build/ScriptTransformer.js:289:17)
      at Object.<anonymous> (node_modules/@glimmer/syntax/dist/commonjs/index.js:4:16)
      at parse (src/parser-glimmer.js:6:25)

src/parser-glimmer.js:6:25 is:

  const glimmerSyntax = require("@glimmer/syntax");

[azz]

Misleading error. Tracked it down to glimmer's source, turns out they're building JS without node v4 support:

node_modules/@glimmer/syntax/dist/commonjs/lib/parser.js:33:

     constructor(source, options = {}) {

[vjeux]

Omg, this is so exciting!

[vjeux]

Cc @mixonic, @wycats

[azz]

Not having much luck with patching @glimmer/syntax, will need to clone it and do a custom build tomorrow.

[azz]

@vjeux I disabled the new tests in case you want to merge this.

[vjeux]

Is glimmer parser a fully compliant html5 parser?

[vjeux]

Looks like it's not using an html5 compliant parser and doesn't intend to be: https://github.com/tildeio/simple-html-tokenizer

That makes me uneasy to include inside of prettier as people are likely to throw all sorts of crazy code at it.

[wycats]

simple-html-tokenizer is intended to be a compliant tokenizer for HTML that could be found inside of an HTML body, not including scripts. It doesn't always error-correct (in cases where the HTML spec specifies that a particular character is an error, we sometimes choose to throw an exception rather than follow the error correction behavior).

We think this is a good trade-off of size and simplicity to compliance for the use case of HTML templates. We also have found that Glimmer and Ember users really appreciate early errors for invalid HTML, which can be confusing when read later on.

[wycats]

Is your concern about applying prettier to Handlebars templates that are emitting HTML but less strict than the Glimmer parser?

[vjeux]

I'd love for prettier to support regular html, vue files, angular ones, glimmer templates, mustache, handlebar... I don't have a lot of context around the different ways each of them parse code, but if we can find a single parser that is able to parse them in a way that if we print them, they have the same semantics, that would be awesome. I'm not sure if one exists though.

[vjeux]

It looks like glimmer parser is simple-html-tokenizer for the html parsing and then uses handlebars itself to compile the templates: import { parse } from "./handlebars/compiler/base";.

Would it be possible to use a real html5 parser like https://github.com/inikulin/parse5 which is used by angular2 and polymer to parse the html and then use handlebars parser for the templates part?

[vjeux]

Vue uses a small handrolled html parser: https://github.com/vuejs/vue/blob/dev/src/compiler/parser/html-parser.js

[azz]

Looking into parse5, it supports two AST formats:

default: TreeAdapter
Default tree format for parse5.

htmlparser2: TreeAdapter
Quite popular htmlparser2 tree format (e.g. used by cheerio and jsdom).

Any preferences?

[azz]

Angular also appears to use a handmade parser.

https://github.com/angular/angular/blob/master/packages/compiler/src/ml_parser/html_parser.ts

[wycats]

@vjeux not quite. Stuff like this needs to be parsed with an integrated parser:

<div title="{{t "frontpage.title"}}">

Otherwise, the HTML parser will think that the second quotation mark (after t) is closing the title attribute. What we do instead is parse as handlebars first, then tokenize the leftover content using the HTML tokenizer library above.

[vjeux]

Thanks for the insight! @wycats do you know if glimmer parser is able to parse all the angular 2 and vue files as well?

[aboyton]

I know it's a lot to ask, but any idea if glimmer can also parse underscore/lodash templates?

[vjeux]

While we figure out which parser to use, would you mind shipping the part that splits css in its own file? This is good and should help the folks trying to integrate graphql: https://twitter.com/stubailo/status/871584723524505600

[azz]

Done. #1969

[vjeux]

Are you going to keep working on this PR or should you open a new one?

[azz]

I'll look at this today.

[azz]

Gah, didn't get a chance to look further into this today. Got #2108 merged though.

Since we've recently merged a separate HTML parser, I'm thinking it might be wise to stabilise that before merging in another parser.

[vjeux]

I'm going to close it. Feel free to re-open it when you'll be working on it again.