Use `content-tag` instead of `ember-template-imports` #655

NullVoxPopuli · 2023-11-30T19:12:28Z

ember-template-imports @ v4 removed all the APIs that glint was relying on for working with <template>-tags

Resolves: #609
Resolves: #706
Resolves: #694
~~Blocked by: embroider-build/content-tag#19~~
~~Blocked by: embroider-build/content-tag#31~~

Unblocks: NullVoxPopuli/polaris-starter#11
Supersedes: #615

Extractions:

Allows:

folks to <pm> add ember-template-imports in their apps and not have to downgrade to v3 if they want to use Glint
better syntax handling
better incorrect syntax handling

NullVoxPopuli · 2023-12-18T22:48:21Z

packages/environment-ember-template-imports/-private/environment/preprocess.ts


+    // TODO: is there a way to convert bytes to chars?
+    //       I think maybe all of this code needs to live in content-tag,
+    //       and give us the option to generate this sort of structure
    templateLocations.push({


This is all likely wrong, as the original code used character-indices

Is this still likely wrong?

yeah, char positions are expected, but I only have bytes here, so whenever there is a multi-byte character, where the errors show up may be offset

This is how it is currently done in eslint parser:
https://github.com/NullVoxPopuli/ember-eslint-parser/blob/d1b8796436670e496f4a21de8b1a9d9219038b66/src/parser/transforms.js#L25

You need also the original string.

ah ok, I'll try that

pushed!

this is looking excellent!

NullVoxPopuli · 2023-12-18T22:51:08Z

yarn.lock

@@ -6461,7 +6466,7 @@ ember-template-imports@^1.1.1:
    ember-cli-version-checker "^5.1.2"


above here, more reason to have glint not depend on content-tag:

=> Found "ember-template-imports@1.1.1" info Reasons this module exists - "_project_#@glint#environment-glimmerx#@glimmerx#component" depends on it - Hoisted from "_project_#@glint#environment-glimmerx#@glimmerx#component#ember-template-imports"

NullVoxPopuli · 2023-12-18T22:51:23Z

yarn.lock

@@ -6461,7 +6466,7 @@ ember-template-imports@^1.1.1:
    ember-cli-version-checker "^5.1.2"
    validate-peer-dependencies "^1.1.0"

-ember-template-imports@^3.0.0, ember-template-imports@^3.1.1:
+ember-template-imports@^3.1.1:


This'll soon be removed from ember-template lint as well:

=> Found "ember-template-lint#ember-template-imports@3.4.0" info This module exists because "_project_#ts-ember-app#ember-template-lint" depends on it. info Disk size without dependencies: "232KB"

NullVoxPopuli · 2023-12-18T22:56:52Z

I'm marking as ready for review, as I think this is about at the point, where I want to make sure I'm not doing anything crazy.

There is still a failing test: https://github.com/typed-ember/glint/actions/runs/7254626609/job/19763687113?pr=655#step:6:107
which... I have no idea how this code affects the reporting of untyped {{component}} usage, since in the "debug extension" thing, the 4 use cases all error correctly (as expected).

cc @dfreeman @wagenet @gitKrystan

Some things left for this PR tho:

push more work in to content-tag, per @ef4's comment here: For linting, we'd need to be able to emit a full AST, including nodes for the template embroider-build/content-tag#39 (comment)
put content-tag in ember-template-lint
ideally, when this PR is done, we'll see ember-template-imports removed from the lockfile entirely (tho, we'd also have to remove GlimmerX for that.... )

NullVoxPopuli · 2023-12-18T22:58:10Z

packages/core/src/transform/template/rewrite-module.ts

+  } catch (e) {
+    if (typeof e === 'object' && e !== null) {
+      // Parse Errors from the rust parser
+      if ('source_code' in e) {


It may be useful for content-tag to emit this error as a JSON object or something (maybe a property on the already emitted error?) so I don't have to parse the error-string here.

NullVoxPopuli · 2023-12-18T22:59:23Z

packages/environment-ember-template-imports/-private/environment/preprocess.ts

+//
+//    https://github.com/embroider-build/content-tag/blob/v1.2.2-content-tag/package.json#L13-L21
+//
+// @ts-expect-error see above


even though this ts-expect-error is here, we still get the correct type for Preprocessor and p.parse below

Interesting. I wonder why I don't get this in the prettier plugin.

is it using 1.2.2 / same tsconfig settings from glint?

NullVoxPopuli · 2023-12-18T22:59:40Z

packages/environment-ember-template-imports/-private/environment/preprocess.ts

+  let sourceOffsetBytes = 0;
+  let deltaBytes = 0;
+
+  // @ts-expect-error TS couldn't find @types/node, which are specified


idk why this couldn't be found

NullVoxPopuli · 2023-12-18T23:00:22Z

packages/environment-ember-template-imports/-private/environment/preprocess.ts

-    let endTagLength = template.end[0].length;
-    let startTagOffset = start.index ?? -1;
-    let endTagOffset = end.index ?? -1;
+    let startTagLengthBytes = template.startRange.end - template.startRange.start;


I added Bytes to all the variables here as a suffix to keep the units straight.

See also: embroider-build/content-tag#45

NullVoxPopuli · 2023-12-18T23:02:04Z

packages/environment-ember-template-imports/-private/environment/preprocess.ts

+     * SEE: https://github.com/embroider-build/content-tag/issues/39#issuecomment-1832443310
+     */
+    let prefixingSegment = sourceBuffer.slice(sourceOffsetBytes, startTagOffsetBytes);
+    segments.push(prefixingSegment.toString());


is there a way to not do this segments stuff?

@dfreeman since content-tag can transform to valid JS, could we just parse that instead?

I believe this is what @patricklx may have run in to in the past as well while iterating on the eslint parser, we need a pre-transform to post-transform index mapping for all AST nodes

NullVoxPopuli · 2023-12-18T23:06:54Z

packages/environment-ember-template-imports/-private/environment/preprocess.ts

+
+  // @ts-expect-error TS couldn't find @types/node, which are specified
+  // in the root package.json
+  let sourceBuffer = Buffer.from(source);


See https://runkit.com/nullvoxpopuli/content-tag-byte-vs-char-offsets for a brief overview of how Buffer solves the byte-index vs char-index issue.

(As well as embroider-build/content-tag#45 )

lifeart · 2024-01-23T12:54:56Z

I like this changes! We definitely need to push it forward and land it! Works just fine inside custom glint env https://github.com/lifeart/glimmer-next/tree/master/glint-environment-gxt

wagenet

I think this is ok. I still don't have 100% context.

wagenet · 2024-03-08T23:33:22Z

packages/environment-ember-template-imports/-private/environment/preprocess.ts


-    if (startTagOffset === -1 || endTagOffset === -1) continue;
+    // if (startTagOffset === -1 || endTagOffset === -1) continue;


Should this be removed?

yup, can do

wagenet · 2024-03-08T23:34:15Z

packages/environment-ember-template-imports/-private/environment/preprocess.ts


+    // TODO: is there a way to convert bytes to chars?
+    //       I think maybe all of this code needs to live in content-tag,
+    //       and give us the option to generate this sort of structure
    templateLocations.push({


Is this still likely wrong?

`ember-template-imports`. Resolves: typed-ember#609 Resolves: typed-ember#706 Resolves: typed-ember#694

…onfig.json

NullVoxPopuli · 2024-03-11T16:18:20Z

tsconfig.compileroptions.json

@@ -11,6 +11,6 @@
    "composite": true,
    "declaration": true,
    "sourceMap": true,
-    "types": []
+    "types": ["@types/node"]


this change was needed so that the types for Buffer existed

NullVoxPopuli · 2024-03-11T16:18:59Z

tsconfig.json

-    { "path": "packages/vscode" },
-    { "path": "packages/scripts" },
-    { "path": "test-packages/test-utils" }
+    { "path": "./packages/core" },


in one of the commits, it's explained that this change was needed so that you can ctrl+click to go to each of these tsconfigs.

It's kinda silly that tsc works with both path-formats but vscode only works with leading ./

NullVoxPopuli force-pushed the content-tag-take-2 branch from 9de2b1b to 83f87c7 Compare December 18, 2023 22:46

NullVoxPopuli commented Dec 18, 2023

View reviewed changes

NullVoxPopuli marked this pull request as ready for review December 18, 2023 22:53

NullVoxPopuli commented Dec 18, 2023

View reviewed changes

NullVoxPopuli mentioned this pull request Dec 21, 2023

Move ember-template-imports to dependencies instead of peerDependencies #660

Merged

lifeart mentioned this pull request Jan 22, 2024

Custom Glint environment lifeart/glimmer-next#49

Closed

1 task

NullVoxPopuli mentioned this pull request Jan 22, 2024

GJS parsed as JSX #694

Closed

lifeart mentioned this pull request Jan 23, 2024

use content-tag in glint env lifeart/glimmer-next#52

Merged

NullVoxPopuli mentioned this pull request Feb 14, 2024

Add transform callback to process that is overridable (with the default value being what our transform does today) embroider-build/content-tag#70

Open

NullVoxPopuli force-pushed the content-tag-take-2 branch from 83f87c7 to 781ad32 Compare March 8, 2024 18:55

wagenet approved these changes Mar 8, 2024

View reviewed changes

chancancode mentioned this pull request Mar 9, 2024

Parse error on {{! mustache comment in .gts #706

Closed

NullVoxPopuli added 4 commits March 11, 2024 12:04

Use content-tag to parse <template> instead of

5bdb53e

`ember-template-imports`. Resolves: typed-ember#609 Resolves: typed-ember#706 Resolves: typed-ember#694

Fix the ability to control-click references to hop to a project's tsc…

ad98cbe

…onfig.json

Add @types/node to the available types for local dev

24d0f8a

Fix byte-offset vs char offset mismatch

8223b1a

NullVoxPopuli force-pushed the content-tag-take-2 branch from cb694bf to 8223b1a Compare March 11, 2024 16:05

NullVoxPopuli commented Mar 11, 2024

View reviewed changes

chancancode mentioned this pull request Mar 12, 2024

${ in templates breaks Glint #707

Open

wagenet merged commit 8d9e274 into typed-ember:main Mar 12, 2024
4 checks passed

patricklx mentioned this pull request Mar 15, 2024

[Bug] Required semicolons incorrectly removed gitKrystan/prettier-plugin-ember-template-tag#218

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use `content-tag` instead of `ember-template-imports` #655

Use `content-tag` instead of `ember-template-imports` #655

NullVoxPopuli commented Nov 30, 2023 •

edited

NullVoxPopuli Dec 18, 2023

wagenet Mar 8, 2024

NullVoxPopuli Mar 11, 2024 •

edited

patricklx Mar 11, 2024 •

edited

NullVoxPopuli Mar 11, 2024

NullVoxPopuli Mar 11, 2024

NullVoxPopuli Dec 18, 2023

NullVoxPopuli Dec 18, 2023

NullVoxPopuli commented Dec 18, 2023

NullVoxPopuli Dec 18, 2023 •

edited

NullVoxPopuli Dec 18, 2023

gitKrystan Dec 19, 2023

NullVoxPopuli Dec 19, 2023

NullVoxPopuli Dec 18, 2023

NullVoxPopuli Dec 18, 2023

NullVoxPopuli Dec 18, 2023 •

edited

NullVoxPopuli Dec 18, 2023

lifeart commented Jan 23, 2024

wagenet left a comment

wagenet Mar 8, 2024

NullVoxPopuli Mar 11, 2024

wagenet Mar 8, 2024

NullVoxPopuli Mar 11, 2024

NullVoxPopuli Mar 11, 2024

		@@ -6461,7 +6466,7 @@ ember-template-imports@^1.1.1:
		ember-cli-version-checker "^5.1.2"


		if (startTagOffset === -1 \|\| endTagOffset === -1) continue;
		// if (startTagOffset === -1 \|\| endTagOffset === -1) continue;

Use content-tag instead of ember-template-imports #655

Use content-tag instead of ember-template-imports #655

Conversation

NullVoxPopuli commented Nov 30, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NullVoxPopuli Mar 11, 2024 • edited

Choose a reason for hiding this comment

patricklx Mar 11, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NullVoxPopuli commented Dec 18, 2023

NullVoxPopuli Dec 18, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NullVoxPopuli Dec 18, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lifeart commented Jan 23, 2024

wagenet left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Use `content-tag` instead of `ember-template-imports` #655

Use `content-tag` instead of `ember-template-imports` #655

NullVoxPopuli commented Nov 30, 2023 •

edited

NullVoxPopuli Mar 11, 2024 •

edited

patricklx Mar 11, 2024 •

edited

NullVoxPopuli Dec 18, 2023 •

edited

NullVoxPopuli Dec 18, 2023 •

edited