Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use extensible object syntax? #18

Closed
jonathantneal opened this issue Jan 13, 2022 · 68 comments
Closed

Use extensible object syntax? #18

jonathantneal opened this issue Jan 13, 2022 · 68 comments

Comments

@jonathantneal
Copy link

Would you be open to using an extensible key/value format, to future-proof reflection in case it's needed for other use cases, even though none are implemented now?

import JoyStyle from "./joy.css" as { type: "css", media: "(width > 640px)" }
@justinfagnani
Copy link

justinfagnani commented Jan 13, 2022

I think the media example is really compelling! Whether it fits into import reflection or not I think the use case should be strongly considered within the realm of imports in general, and it's probably a good idea to consider how it intersects with reflection and assertions and whether we want potentially three related import features: assertions, reflection, and something else (attributes?).

As CSS module scripts are a way for a JS module to depend directly on CSS, it somewhat takes the place of <link> but embedded in the module graph rather than in the top-level HTML. However imports are missing some features of <link> notably the media attribute.

Adding a media attribute would enable a form of conditional imports that close the gap with <link>. I'm not sure where else conditional static imports might have been discussed, but I could see something like this:

import * as desktopStyles from './desktop.css' assert { type: 'css'} with { media: '(width > 640px)' };
import * as mobileStyles from './mobile.css' assert { type: 'css'} with { media: '(width <= 640px)' };

if (desktopStyles !== undefined) {
  document.adoptesStyleSheets.push(desktopStyles.default);
}
if (mobileStyles !== undefined) {
  document.adoptesStyleSheets.push(mobileStyles.default);
}

@jonathantneal
Copy link
Author

jonathantneal commented Jan 14, 2022

Other potential examples in some distant future might include:

import boldOpenSans from './OpenSans-Bold.woff' assert { type: 'font' } with { weight: 700 }

await boldOpenSans.load()
import bgm from './background-music.flac' assert { type: 'audio' } with { loop: true }

bgm.addEventListener('canplaythrough', () => {
  bgm.play()
})
import myWorker from './my-worker.js' with { type: 'worker' }

myWorker.postMessage({ cmd: 'start', msg: 'Hi' })

@devongovett
Copy link

devongovett commented Feb 21, 2022

I think it would be great if a more extensible syntax were used rather than requiring a parser change each time a new attribute is introduced. There are a lot of potential use cases for import attributes, and I don't think it always necessarily makes sense for each one to go through the TC39 process since they won't be implemented by all engines (e.g. some attributes might not make sense in browsers). I raised this back on the import assertions proposal as well, and a number of good use cases came up there: tc39/proposal-import-attributes#99.

For example, one thing that would be cool is support for preload/prefetch attributes. For example, something like this:

import foo from "./foo.js" with { load: "prefetch" };

// some time later...
const exports = await foo.get()
// or maybe
const exports = await import(foo);

Somewhat related: #16.

@guybedford
Copy link
Collaborator

@devongovett I wonder if a usecase like preload might not be better suited to a dedicated import.preload function in due course over attributes. Are there specific reasons you'd want attributes over a dynamic mechanism here?

@devongovett
Copy link

Yeah might also be useful for some cases, e.g. on-demand preloading when a user hovers over a link.

I think one benefit of attributes is that they are much more easily statically analyzed. This not only benefits build tools, but could potentially benefit browsers and other runtimes as well. A quick pass could be done to determine what to preload without parsing and evaluating the entire module.

Btw, this was actually @littledan's idea, and there was some prior discussion about it here.

Another potential use case is lazy loading, i.e. subsuming https://github.com/tc39/proposal-defer-import-eval.

@ljharb
Copy link
Member

ljharb commented Feb 22, 2022

import.preload(specifier) would be just as statically analyzeable as import(specifier), as well as lintable (if you want to restrict it to a static specifier), no?

@devongovett
Copy link

devongovett commented Feb 22, 2022

Yeah, but you have to evaluate the code to know when to preload, whereas a declarative import statement allows immediate preloading without fully parsing or running any code. Browser engineers could tell us if that's useful, but seems like it could be (esp if it supported non-JS resources, i.e. subsuming asset references - #16).

I also like the idea that it could be done the same way either as an import statement or a dynamic import.

import foo from "./foo.js" with { load: "prefetch" };

import('./foo.js', {with: {load: 'prefetch' }});

@ljharb
Copy link
Member

ljharb commented Feb 22, 2022

Given that import.preload would be syntactic, it would show up on a first parse (which engines have to do anyways to hoist import statements), so it doesn't seem like there'd be a need to run any code if the argument is a static string.

@guybedford
Copy link
Collaborator

<link rel=modulepreload> is the early hint for browsers, and is preferable to discovered preloads (which involves latency) for the static case.

@devongovett
Copy link

Sure, for initial page loads but would be nice to have a way to preload things for later loaded things other than insert one of those into the document manually.

Anyway, this is all kinda off topic. I am mainly interested in having a syntax that is forward compatible with future additions, as well as extensible for build tools to innovate in this space without waiting for TC39.

@guybedford
Copy link
Collaborator

Sure, for initial page loads but would be nice to have a way to preload things for later loaded things other than insert one of those into the document manually.

This is a great discussion and one I would love to have further.

Anyway, this is all kinda off topic. I am mainly interested in having a syntax that is forward compatible with future additions, as well as extensible for build tools to innovate in this space without waiting for TC39.

I don't think it's at all off-topic, we should discuss features in the context of use cases. The more use case discussions the better.

@jonathantneal
Copy link
Author

jonathantneal commented Feb 23, 2022

I don't think it's at all off-topic, we should discuss features in the context of use cases. The more use case discussions the better.

I appreciate the optimism, and what you write is true, but it’s also incomplete. For me, a single extensible API is important because 1. I’m concerned TC39 folks might not understand the needs of source file tooling where I want to innovate, and because 2. a single extensible API can be delivered in a more timely manner than several divergent APIs.

Concern 1

I’m not super encouraged to lean into a conversation about use cases if I’m concerned it won’t go anywhere.
I’m not super encouraged to lean into a conversation about uses cases if I’m concerned delegates and advocates misunderstand my environment, anyway.

“Content type on the local FS can only be derived from file extensions.”
https://twitter.com/lcasdev/status/1495743689930457095

”Do you not have file extensions?”
https://twitter.com/justinfagnani/status/1481446609128865796

Meanwhile...

”... the only change i do now is .js > open all files as > javascript > jsx. that’s all”
https://twitter.com/dan_abramov/status/1488956873390923780

“In Next.js, a page is a React Component exported from a .js, .jsx, .ts, or .tsx file”
https://nextjs.org/docs/basic-features/pages

Concern 2

I see a not insignificant number of features that would require different keywords to accomplish what we want from one.


Still, here’s another 'transform' use case I was reviewing, borrowing Justin’s syntax:

import images from './profile.jpg' as 'image-set' with { size: [ 320, 480, 640 ], type: [ 'avif', 'webp' ] }

@ggoodman
Copy link

I think that there seem to be two related but distinct use-cases that are floating around in this discussion. I believe that they may receive better attention if they were explicitly called out.

  1. Use cases that have semantic meaning for engines at runtime (ie: media queries, workers and prefetching).
  2. Use cases that relate to passing intent from code authors to the wider tooling ecosystem and that should not have semantic meaning at runtime (ie: font weights, image optimizations, etc).

In the spirit of the types-as-comments spec, perhaps it would be worthwhile to think about some explicit namespace or mechanism for code authors to pass expressive intent to the tooling ecosystem. If that was a first-class concept then we might find ourselves avoiding situations where semantic concepts like import specifiers or assertions are being used in unintended ways to achieve important and necessary goals.

@littledan
Copy link
Member

littledan commented Jun 2, 2022

I agree that work should go towards a general, object-based syntax rather than a single string. Many other use cases have been discussed already, in the import assertions repo, when folks were arguing that it shouldn't be limited to assertions. On the other hand, I am unconvinced that the functionality exposed in this proposal is all that high priority. Rather than as for the keyword, I would prefer something which implies that the module will be changed, e.g. with.

@littledan
Copy link
Member

littledan commented Jun 9, 2022

After seeing the presentation at TC39 and talking more with @guybedford , I'm convinced that this proposal meets an important use case; I've striked out part of my comment above.

While I would prefer a general syntax exposing other parameters, but there's a legitimate concern that this would be blocked in committee, as @ljharb previously did. I don't think it's worth stopping this proposal in its tracks for such a broader generalization, though the generalization would be my preferred outcome. Overall, I agree with @jridgewell 's comment that we probably made an error in import assertions using the "assert" keyword and should've been more general in the first place (however, this preference contradicts the arguments that @ljharb and @devsnek previously made, and does not have TC39 consensus; in any case, it's too late to make changes in that proposal).

If we don't go with a generalized key-value pair, I'd suggest that we use some other syntax besides as, for all the reasons that as was rejected in the import assertions case. If we're going for something specific, let's be fully specific with a keyword indicating the change in mode of importing the module. So, for example, import reflective and import asset.

For now, I'd like to focus on working out the details of this proposal, especially how it relates to module blocks, compartments, module fragments, Wasm/ESM integration, Wasm components, etc. It's good to have this syntax debate running in the background, but let's not get too bogged down on it; I am confident that we'll be able to find some syntax or other which is agreeable, and we have a lot of other details to work out.

@guybedford
Copy link
Collaborator

Thanks @littledan for clarifying here, we are open to using an alternative syntax to as and would like to explore these options further, in a way that can work with and leave the door open for evaluator attributes syntax. From our discussion I'm also confident we can find a suitable approach.

Agreed the primary proposal details to work out right now though are how a user-exposed JS module record gets specified between these proposals, and what other cross-cutting concerns apply. It definitely makes sense to continue to focus on that for now.

@devongovett
Copy link

I'm disappointed to see the syntax has been changed to yet again use a static keyword in yet another position rather than an extensible syntax. Just like I raised for import assertions (tc39/proposal-import-attributes#99), the syntax is not symmetric between dynamic and static imports. Dynamic imports use an extensible object syntax, whereas static imports do not. From the readme:

import asset x from "<specifier>";
await import("<specifier>", { reflect: "asset" });

If the import assertions proposal had used an extensible object syntax as raised then and in this issue, this proposal wouldn't even have been necessary. We could simply do this:

import x from '<specifier>' with { reflect: 'asset' };

This syntax would allow engines to add new attributes where it makes sense for them. Again, it's already the case with the second parameter to dynamic import, just not static import, and I really don't understand why. How long are we going to keep adding attributes to the language one by one? Why does each attribute need to be in a different part of the syntax?

Lots of tools and developers want to use import attributes of some kind, for purposes beyond just the ones specified. Many use cases are covered here and in the other issue linked above. Some tools have already started abusing import assertions for this, which is bad. In my strong opinion, this needs to be solved once and for all, and not by adding new attributes one by one every few years. Progress is too slow this way, and it doesn't leave enough room for tools and engines to innovate in their respective domains.

@ljharb
Copy link
Member

ljharb commented Jul 19, 2022

@devongovett tools that have abused import assertions for this sort of thing are likely violating the spec; while the spec has no enforcement power, it's very important that intentions be explicitly conveyed. "of some kind" is frighteningly vague, and I'd love to hear more concrete use cases if the existing proposals don't address them.

@devongovett
Copy link

Here are two examples I saw recently where it happened. Not sure if either of them ended up shipping because it was called out, but still. The demand for more extensibility is there.

@ljharb
Copy link
Member

ljharb commented Jul 19, 2022

Further discussion on both threads shows that neither of those tools were brazen enough to blatantly violate the spec - it remains a critical gate for the ecosystem.

@jridgewell
Copy link
Member

The spec restriction that forbids import assertions from altering the interpretation of the module is completely arbitrary and should be gleefully ignored. Allow the bundler ecosystem to experiment with the syntax space to better support their customizability.

@devongovett
Copy link

@ljharb What is your opposition to extensibility at a syntax level? People have been raising this for years, and every time it just goes nowhere. Clear demand and use cases have been documented and discussed. Every time it's raised someone punts anything that isn't their exact problem to some other future proposal. When are we gonna solve this? What is the actual technical reason why we can't solve it once and for all rather than blocking all future features for module loading and evaluation on TC39 adding yet another new syntax?

The semantics of specific keys in an object syntax could be specified just as they are today for dynamic import. I see no difference between import asset x from '...' and import x from '...' with { reflect: 'asset' } from a semantic point of view, but the latter allows additional keys, as well as new values for the reflect key to be added without changing the parser. How is this not better?

@Ayc0
Copy link

Ayc0 commented Jul 20, 2022

I feel like having a relaxed syntax for import statements could really help the community as a whole:
A lot of different bundlers have different and custom syntaxes to be able to apply custom process pipelines to files:

Even though all of those don't have actual meaning during the runtime, having an extensible spec could allow for all the bundlers to give the possibility to follow a similar syntax and reduce the gap between all bundlers (and also allow to make bundlers feel like more align with the spec)

I opened in the past an issue to add the possibility in parcel to support this (see parcel-bundler/parcel#7648) but @devongovett rejected this idea as it is out of scope (syntax assertions aren't supposed to be used for transformations).

@ggoodman
Copy link

The semantics of specific keys in an object syntax could be specified just as they are today for dynamic import. I see no difference between import asset x from '...' and import x from '...' with { reflect: 'asset' } from a semantic point of view, but the latter allows additional keys, as well as new values for the reflect key to be added without changing the parser.

If the concern is about future collisions between user-land metadata keys and keys having semantic meaning, then let's get ahead of the problem. Give us a metadata key or key prefix that is reserved for non-semantic purposes.

Tooling could then freely strip this syntax during optimization passes but runtimes would still be able to consume it as-is.

Reserved prefix:

import Article, { metadata } from './path/to/Article.mdx' with { 'x-loaders': ['mdx'] };

Would be symmetric to dynamic import:

const { default: Article, metadata } = await import('./path/to/Article.mdx', { 'x-loaders': ['mdx'] });

Reserved key:

import Article, { metadata } from './path/to/Article.mdx' with { extra: { loaders: ['mdx'] } };

Would also be symmetric to dynamic import:

const { default: Article, metadata } = await import('./path/to/Article.mdx', { extra: { loaders: ['mdx'] } });

@ljharb
Copy link
Member

ljharb commented Jul 20, 2022

@jridgewell it's the entire reason the feature is allowed to exist. If such things are "gleefully ignored", then that will just ensure that future dangerous features, including this one, never advance. It's also disheartening for a TC39 delegate to be publicly advocating willfully violating the spec, and comes across as very bad-faith behavior.

Things can experiment with syntax all they want - it just makes them noncompliant.

@devongovett You can think it's better all you want, but unrestrained syntax experimentation is a net detriment for the ecosystem. We all benefit from interoperability, and that means that semantics have to be tightly specified and not wildly host-defined.

Tools can do whatever they want within specifiers already - that's the space for innovation. You don't need permanent, never-removable, expensive-to-implement syntax to test things out.

@Pauan
Copy link

Pauan commented Jul 20, 2022

You can think it's better all you want, but unrestrained syntax experimentation is a net detriment for the ecosystem. We all benefit from interoperability, and that means that semantics have to be tightly specified and not wildly host-defined.

That's incorrect, the entire motivation behind import.meta is to be a random grab-bag of metadata which can be host or bundler defined. Having extensible metadata is a good and useful thing, as has been repeatedly proven both in the JS ecosystem and in other language's ecosystems.

@ljharb
Copy link
Member

ljharb commented Jul 20, 2022

@Pauan that's for metadata for a module author, which is quite distinct from a module importer, by design.

@Pauan
Copy link

Pauan commented Jul 20, 2022

@ljharb And? That doesn't make any difference, it's useful in both cases, you're trying to make a distinction which doesn't exist. People have provided multiple use cases for extensible properties on imports.

And in both the case of import.meta and import x from '...' with { foo: 'bar' } the behavior is specified by the bundler or host. So all of your arguments apply equally to import.meta.

@ljharb
Copy link
Member

ljharb commented Jul 20, 2022

It makes a huge difference. A module shouldn't behave differently based on who is importing it - metadata should come FROM a module or be provided to it by a host, not be passed to it by a consumer.

@devongovett
Copy link

You can think it's better all you want, but unrestrained syntax experimentation is a net detriment for the ecosystem.

That is the entire point of making it extensible - so that we don't have to change the syntax in order to experiment or add future standard features. As stated before, the goal of interoperability is not hindered by an extensible syntax. Semantics can be specified based on keys within an object, just as they are for dynamic import already. Why do you think static imports are different?

@Pauan
Copy link

Pauan commented Jul 22, 2022

@ljharb You're simply incorrect. Go read the thread I linked to.

The cache is a part of the ES6 spec, and it is specified based on an (importingModule, specifier) tuple. Which means that it's possible for the same specifier to return a completely different module.

Which is why import { currentScript, url } from "js:context"; was discussed as a possible alternative to import.meta.url, because the "js:context" module can be different every time it is imported.

And if desired, it would be possible to spec it so that the cache is based on an (importingModule, specifier, metadata) tuple, so that way modules with different metadata can resolve to different modules.

ES6 modules have never been cached only based on their specifier, it has always been possible to have completely different modules with the same specifier. Your intuition is simply incorrect.

Even in NodeJS the same specifier can result in completely different modules being imported. For example import foo from "foo"; can import different modules (because there can be multiple versions of the package foo).

@ljharb
Copy link
Member

ljharb commented Jul 22, 2022

That is a fair point. None the less, you can’t import the same specifier twice in the same file and get different modules, except in cases of pathological server behavior with a remote module.

@RoyalIcing
Copy link

RoyalIcing commented Jul 26, 2022

Hi all, I just want to bring up a concern I have with media queries in particular. How do people interpret this code snippet?

import JoyStyle from "./joy.css" as { type: "css", media: "(width > 640px)" }

There’s a number of interpretations I see:

  • A: Only import this if the viewport at the time of initial load is greater than 640 pixels.
  • B: Dynamically evaluate the media query of "(width > 640px)" whenever the viewport changes (say if the user changes the width of their window). Does mean the entire module is evaluated again, because the following logic might have changed?
  • Something else?

I’m curious what @jonathantneal was intending.

From @justinfagnani example here it seems like A is the interpretation:

import * as desktopStyles from './desktop.css' assert { type: 'css'} with { media: '(width > 640px)' };
import * as mobileStyles from './mobile.css' assert { type: 'css'} with { media: '(width <= 640px)' };

if (desktopStyles !== undefined) {
  document.adoptesStyleSheets.push(desktopStyles.default);
}
if (mobileStyles !== undefined) {
  document.adoptesStyleSheets.push(mobileStyles.default);
}

The problem is that this is evaluating it only at the initial load time is that the window might be resized (say on a laptop or tablet), and now the other styles should be applied but they since they were initially undefined the page doesn’t work as expected. Which means the user would have to reload the page.

<link> tags today don’t have this problem, they are each independent and don’t affect each other directly. Which means they can later load if say the viewport changes in size.

@guybedford
Copy link
Collaborator

guybedford commented Jul 26, 2022

You've hit the nail on the head that media queries are closer to conditional loading than a loading attribute.

The recommended pattern today would probably be the following:

await import(matchMedia('(width > 640px)') ? './desktop.css' : './mobile.css', { assert: { 'type': 'css' });

Moving this from a dynamic import into a static import you'd then effectively want the static import to reify the media query into something like a conditional export mechanism.

There are a couple of ways to do conditional exports statically:

  1. At the resolution level - the import map, or load-time dynamic import map generation could handle this in the import map itself.
  2. Statically at the resolution level - having import maps somehow support conditional branches which can be enabled or disabled, something like Node.js package exports conditions.
  3. At the actual syntax level - designing a conditional import syntax at imports like your example

Note that a conditional import system that nulls its imports on predicate results basically means a DSL or out-of-band configuration system. Hard-coding media queries is fine, but it's difficult to see how that generalizes easily at all given we don't have many other similar DSLs on the web,

It's probably worth addressing the merits of the first dynamic import example, and (1) above as well before considering such an option.

@RoyalIcing
Copy link

Yes I agree we should be more eager to use dynamic imports — especially if what we are doing is actually dynamic.

With your example, how would you handle reevaluating the media query when the viewport changed? This is still only “evaluate the media query at initialisation”:

await import(matchMedia('(width > 640px)') ? './desktop.css' : './mobile.css', { assert: { 'type': 'css' } });

@justinfagnani
Copy link

The problems with dynamic import are the poor behavior of top-level await wrt to loading, and the likely incompatibility with prescanners.

TLA use will block importing modules from evaluating, which will delay their similar TLA+dynamic-imports, and so-on up to the root of the module graph.

Prescanners are very unlikely to understand the conditional logic and so won't be able to preload any resources. Being a built-in feature would enable preloading.

That is a very good question about what's expected to happen when media queries change. Answering that would be necessary, and some re-evaluation capability would be needed to match HTML.

@ggoodman
Copy link

It seems important to question which part of the code's lifecycle is targeted by the proposed syntax. It does seem like there are some compelling ways the intent could be encoded in today's primitives.

But what I find interesting is that it is very difficult to execute on that intent for the average dev. Doing so involves a depth of understanding that few people in the world hold. However, the proposed syntax and it's intent could be transformed by tooling into the right primitives if such syntax were available.

This is where I believe that the power of an extensible syntax lies, especially when it has no runtime semantics.

@ljharb
Copy link
Member

ljharb commented Jul 26, 2022

We already have infinitely extensible syntax with no runtime semantics - comments. Adding syntax that does nothing at runtime is adding a lot of cost for decidedly non universal value.

@devongovett
Copy link

Still no one has answered why it's an object literal for dynamic import (in which unknown keys have no runtime semantics) but not import statements. Why can't the syntax be the same? Import assertions already define such a syntax, so clearly it's possible.

@ljharb
Copy link
Member

ljharb commented Jul 27, 2022

@devongovett because it can’t be any other way for dynamic imports (because JavaScript), and there was opposition to having to type the extra boilerplate every time for the syntax case.

@devongovett
Copy link

I'm concerned that every time there's a new usecase for an import attribute, a new bespoke syntax will need to be invented making the import statement more and more complex, and slowing down the standards process. IMO a syntax more similar to import assertions would keep reflection in import statements both more consistent with dynamic import and with import assertions, and also more extensible for future use cases (which might not make sense for TC39 to standardize, but perhaps another standards body).

@Jack-Works
Copy link
Member

Jack-Works commented Jul 27, 2022

We already have infinitely extensible syntax with no runtime semantics - comments. Adding syntax that does nothing at runtime is adding a lot of cost for decidedly non universal value.

Out of Topic: That's how I think of the Type Comment proposal

@jridgewell
Copy link
Member

We already have infinitely extensible syntax with no runtime semantics - comments. Adding syntax that does nothing at runtime is adding a lot of cost for decidedly non universal value.

Out of Topic: That's how I think of the Type Comment proposal

By this logic, we can call JS a complete language. Who needs pipelines, pattern matching, or import reflection? We can just add comments to the appropriate places and let the build steps magic everything together.

@ljharb
Copy link
Member

ljharb commented Jul 27, 2022

@devongovett yes, that is the exact idea. Adding something to the language is a permanent decision that can never be unmade; “time to add” is, in the long run, a negligible cost compared to permanent mistakes. Moving too fast breaks things, which slows down everything that comes after it. That’s the job of JS language stewards imo - to minimize mistakes, not to accelerate experimentation (which requires no standards process anyways)

@ljharb
Copy link
Member

ljharb commented Jul 27, 2022

@jridgewell i don’t think your slippery slope there is accurate. What may be an accurate logical extension is, that anything without runtime semantics can and perhaps should be achieved without any alterations to the language. Anything with runtime semantics, however, is unrelated to that logic.

@devongovett
Copy link

Not everything can or should be standardized by TC39. What if browsers want a feature that doesn't make sense in other runtimes? Or the reverse? Is JS going to standardize a feature of a single runtime? How about features of a bundler? I don't think the JS language should be playing gatekeeper.

@ggoodman
Copy link

We already have infinitely extensible syntax with no runtime semantics - comments. Adding syntax that does nothing at runtime is adding a lot of cost for decidedly non universal value.

Adding something to the language is a permanent decision that can never be unmade; “time to add” is, in the long run, a negligible cost compared to permanent mistakes. Moving too fast breaks things, which slows down everything that comes after it. That’s the job of JS language stewards imo - to minimize mistakes, not to accelerate experimentation (which requires no standards process anyways)

This comes across as dismissing community feedback because we're perceived not to fully appreciate the problem or problem space.

Is this a forum for discussion that is legitimately open to non-TC39 members of the community?

That first comment is such a facile argument; it doesn't contribute to the discussion and is unnecessarily dismissive. There is a major undertaking right now for exactly the sort of non-semantic syntax 'that can be accomplished in comments' before this very same group and with significant momentum.

I will bow out of this discussion.

@devongovett
Copy link

As requested in the tools meeting today, here are some potential use cases for import attributes. Tried to group them into some kind of semantic categories.

// this proposal
import mod from './foo.wasm' with { type: 'module' };

// more reflections
import img from './foo.png' with { type: 'url' };
import img from './foo.png' with { type: 'arraybuffer' };
import img from './foo.png' with { type: 'dataURL' };
import img from './foo.png' with { type: 'arraybuffer' };
import worker from './worker.js' with { type: 'worker' };
import sw from './worker.js' with { type: 'service-worker' };
import svg from './foo.svg' with { type: 'dom' };
import svg from './foo.svg' with { type: 'image' };

// tc39/proposal-defer-import-eval
import {x} from "y" with { lazyInit: true };

// attributes to apply to a constructed object
import stylesheet from './foo.css' with { layer: 'utilities' }; // css cascade layers
import stylesheet from './foo.css' with { media: 'print' };
import audio from './file.mp3' with { loop: true };

// preloading (possibly related to lazyInit?)
import x from './foo.js' with { preload: true };
let mod = await x.load();

// more assertions
import x from 'y' with { assert: { integrity: '...' }};
import x from 'y' with { assert: { referrerPolicy: '...' }};

// transforms that bundlers/tools could implement
import img from './foo.png' with { width: 500 };
import img from './foo.png' with { convertTo: 'jpeg' };

I'm sure there are more as well (please leave a comment!), but hopefully this demonstrates the need for an extensible syntax.

@ljharb
Copy link
Member

ljharb commented Jul 28, 2022

@devongovett thanks! i'd love some more elaboration on what these do (in prose), as well as what they would be expected to do in browsers and node.

@guybedford
Copy link
Collaborator

@devongovett the examples really do help a lot to try to think more concretely about the requirements here.

We also went through a lot of these cases when exploring the possibilities for import reflection, so I can share some feedback based on what came out of that process (mixed in with my own opinions of course!), see comments below:

  • import mod from './foo.wasm' with { type: 'module' }; This is explicitly a non-goal for the Wasm ESM integration. The content type determines the format, and that is fully set at this point. To be very clear, there will most likely never be a custom module type for Wasm, as it will work in the ESM integration without any custom attributes.
  • import img from './foo.png' with { type: 'url' };: The way to do this is via import.meta.resolve('./foo.png'). This use case seems ideally is most likely captured by the asset references proposal as well.
  • import img from './foo.png' with { type: 'arraybuffer' };, import img from './foo.png' with { type: 'dataURL' };: These are effectively decorated fetches, similar to fetch(import.meta.resolve('./foo.png')). If the benefit of moving these to syntax is being able to standardize on a static data method that tooling can inline etc I'm all for that. But that does require full standardization here I think to draw the benefits over direct fetching. Having this discussion as part of the asset references proposal seems to make sense to me. Similar concepts have also come up in the context of data imports for Wasm and there was even some mention of hope that alignment might work out for an asset type to support data segment imports somehow.
  • import {x} from "y" with { lazyInit: true };: This is being actively worked through. It may well align with import reflection yet.
  • import x from './foo.js' with { preload: true };: Lazy initialization would load all modules but not execute them. I think preload is identical from that perspective, so likely is the same?
  • import worker from './worker.js' with { type: 'worker' };, import sw from './worker.js' with { type: 'service-worker' };: Worker reflections were shot down by the Chrome team when I brought this up. I don't know the exact reasons, but I believe efforts are best focused on blank workers for now here per the hoped worker.addModule APIs. In turn import reflection will be designed to be compatible with this for transferring modules to workers so I think between those we should hopefully capture similar use cases.
  • import x from 'y' with { assert: { integrity: '...' }};, import x from 'y' with { assert: { referrerPolicy: '...' }};: These assertions could certainly be specified, it would be very interesting to see further discussion form on new assertions, that's something that hasn't happened enough yet.
  • import svg from './foo.svg' with { type: 'dom' };: Can this be implemented like JSON and CSS modules as a content-type-based module type?
  • import svg from './foo.svg' with { type: 'image' };: I'm not sure I understand what the expectation is for this one in the browser?
  • import stylesheet from './foo.css' with { media: 'print' };: The media query discussion above was very interesting. This proposal suffers from similar concerns as was discussed - how to deal with nullability and changes?
  • import stylesheet from './foo.css' with { layer: 'utilities' };: What is the expectation here for utilities?
  • import audio from './file.mp3' with { loop: true };: I'm not sure I understand what this one would do.
  • import img from './foo.png' with { width: 500 };, import img from './foo.png' with { convertTo: 'jpeg' };: I think you're going to have a hard time getting this specified anywhere? It seems these are the only ones arguing for a custom space for bundlers to have their own semi-standard meta import system?

@nayeemrmn
Copy link

To me it would make a lot of sense if module references didn't have evaluator attributes but asset references did:

import module x from "./foo.js";
// x is a module-block-like
await import(x);
import asset y from "./bar.png" with { type: "image" };
hostUseImage(y);

There is a need to separate these two because tc39 could have a lot to say about x and would have very little to say about y.

i.e. I think module references are a special case where most of what they do can be understood at the vanilla level, they should preload deps as static imports would and be dynamically importable. Whereas arbitrary host-managed evaluator attributes and all of their use cases would be much better suited to asset references, which already export arbitrary host-defined values.

In other words, as soon as a module reference includes host-specified evaluator attributes to make it yield a host-specified value, it becomes an asset instead.

Rather than lamenting module vs asset not being selected by an extensible scheme, we should accept module as a necessary carve-out for importable module references, and let asset be the extensible category.

@devongovett
Copy link

Ok, I'll try to go through them with some more explanation and also try to answer @guybedford's questions along the way.

  • import mod from './foo.wasm' with { type: 'module' };
    This was meant to be the exact same thing as the current proposed syntax import module mod from './foo.wasm'. I used type as a key here rather than reflect because I think it's more understandable to the average dev, but it's the same. Confusion with import assertions may be an issue though. Again, the name isn't really important for the example.

  • import img from './foo.png' with { type: 'url' };
    I was thinking this would supersede the asset references proposal. Perhaps type: 'asset' would be better. Idea would be to return a constructed URL object. new URL or import.meta.resolve are also fine syntaxes, but more dynamic. Having a static syntax could also be useful potentially. Many bundlers support importing assets as resolved urls like this today because it's pretty ergonomic.

  • import img from './foo.png' with { type: 'arraybuffer' }; (and similar for dataURL, or other formats)
    This would return the data for the image as an ArrayBuffer. Yes, you can do this with fetch, but doing so with an import statement has some benefits. It's statically analyzable, which is good for tools. It's also good for browsers, which could start downloading these resources much earlier (i.e. in a prescanning phase).

  • import worker from './worker.js' with { type: 'worker' }; (and service worker)
    This would return a constructed Worker object. Again, the benefits are that it's static, and that the worker can be downloaded in parallel with the rest of the module graph, without waiting for JS code execution to run the Worker constructor. Sure, you could use link preloads, but this is nice and ergonomic. Also relative urls in the Worker constructor are resolved relative to the page, not the current module, so usually you have to use import.meta.url to resolve the url manually as well.

  • import svg from './foo.svg' with { type: 'dom' };
    This would return a parsed SVGElement DOM instance for an SVG file, which could be inserted into the document. Same benefits as above. Could be implemented with import assertions like CSS/WASM modules too, but if there are multiple possible reflections for an SVG (e.g. an Image object, as described below), then there might also need to be a way to switch between them.

  • import svg from './foo.svg' with { type: 'image' };
    I was thinking this would return an Image object. Same benefits: early loading, static dependencies, relative resolution.

  • import {x} from "y" with { lazyInit: true };
    See https://github.com/tc39/proposal-defer-import-eval

  • import stylesheet from './foo.css' with { layer: 'utilities' };
    Idea here would be to be able to apply attributes to a reflected object, in this case a CSSStyleSheet. In CSS, the @import rule supports applying a cascade layer to an imported stylesheet, so why not JS as well? Could be useful when importing CSS stylesheets to have control over the cascade without making a copy of a CSS file wrapped in a @layer rule.

  • import stylesheet from './foo.css' with { media: 'print' };
    Similar to above, could apply a media query just like you can with @import in CSS. The discussion about what to do when the media query doesn't match on initial load but does later is a good one. I guess I would expect it to behave the same way @import and <link> with a media query do: it would still be downloaded (at low priority), just not applied. https://medium.com/@tomayac/why-browsers-download-stylesheets-with-non-matching-media-queries-eb61b91b85a2

  • import audio from './file.mp3' with { loop: true };
    Same idea: applying an attribute to an imported Audio object. 🤷

  • import img from './foo.png' with { width: 500 };, import img from './foo.png' with { convertTo: 'jpeg' };
    Goal here was to show that with an extensible syntax, tools could support custom transforms using the same syntax. I would not expect these to be standardized. Tools could strip these before shipping compiled JS to clients.

To be clear, my goal with these examples is to show that there are a wide range of potential use cases for import reflections and attributes. Some of these could eventually be standardized, either by TC39 or by other standards bodies (e.g. as a part of the CSS module script spec). Some of them could be implemented in tools and stripped before reaching runtimes. Making the syntax extensible now will avoid needing to specify new keywords or syntax later for each possible reflection or attribute. This is important not just for tools, but also for runtimes.

@ljharb
Copy link
Member

ljharb commented Jul 28, 2022

What would importing an image do outside of a browser environment?

The language also needs to ensure it stays universal for environments beyond browsers and node, like IoT devices and other chipset environments, and the mental and implementation cost of having syntax (as opposed to API) that works in some environments but not others can be quite high.

@devongovett
Copy link

I think the host runtime should "register" the attributes that it supports with the JS engine, and unknown ones should be rejected. This would allow browsers to have different supported attributes from other environments, and would match the behavior of import assertions.

@Pauan
Copy link

Pauan commented Jul 28, 2022

@ljharb The language also needs to ensure it stays universal for environments beyond browsers and node, like IoT devices and other chipset environments, and the mental and implementation cost of having syntax (as opposed to API) that works in some environments but not others can be quite high.

ES6 module resolving and loading is already environment-specific. import.meta is already environment-specific. There's nothing wrong with having a feature which varies between different environments. And having extensible syntax makes things better for different environments, because each environment can define its own custom properties.

If extensible syntax doesn't exist, then every environment must either invent its own syntax for environment-specific things, or it must create a new TC39 proposal, which isn't always desirable or possible. And if the feature is environment-specific, then it's unlikely to be accepted as a TC39 proposal. Extensible syntax fixes all of those problems.

@jridgewell
Copy link
Member

Rollup just landed rollup/rollup#4646, which adds support for arbitrary assertions and custom processing of those assertions by plugins.

The bundler ecosystem continues to land on the same solution, and I think it's because it's extensible.

@nayeemrmn
Copy link

With import assertions having transitioned to import attributes (slides) and import reflections transitioning to import phases (decoupling from attributes in the process), I think this issue is addressed.

The use cases brought up are relevant to and satisfied by import attributes now. You can see them used with their extensible syntax in conjunction with phases in one of the slides: https://docs.google.com/presentation/d/1Abdr54Iflz_4sah2_yX2qS3K09qDJGV84qIZ6pHAqIk/edit#slide=id.g216c73c7b74_0_35

(Or at least unless anyone thinks the phase should use an extensible syntax?)

@Jack-Works
Copy link
Member

at least allow booleans plz. only allowing string properties is annoying when I try to use them.

@lucacasonato
Copy link
Member

This has been resolved, as this proposal is specifically dealing with only import phases.

As import attributes is now stage 3, please direct any feedback regarding the import attributes options bag to the TC39 Discourse.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests