Output ES Modules

[dandv]

I'm targeting esnext in tsconfig, and trying to import { prop } from '@typegoose/typegoose'; fails with

SyntaxError: The requested module '@typegoose/typegoose' does not provide an export named 'prop'

Same issue that graphql-tools had.

[hasezoey]

sorry, but when doing that, it might affect older versions of node and might affect the code of the users

[dandv]

sorry, but when doing that, it might affect older versions of node and might affect the code of the users

I doubt it. Much bigger projects, with many more users, like graphql-tools which I linked above, have done it.

Also, graphql-modules agreed to implement ES Modules.

Anyway, there's an easy way to prevent any incompatibilities: release a new major version announcing the addition of ES Modules. That way, npm won't automatically update the user's typegoose package beyond the current version.

[hasezoey]

i have looked more into it, i would like to emit ES-Modules, but they are only included in node 13 (without flag) and from what i know are still somewhat experimental and the next version 7.0 will remove nodejs 8 support (node 10 will be the lowest supported then), so for the near future this will probably not be happening - sorry

PS: i will look into this again when the time comes

[dandv]

Maybe the release of Node 14 is a good reason to look into this again. I ended up using a nasty workaround to get typegoose to play with "modules": "esnext", which apparently breaks TypeScript reflection and autocompletion.

import * as typegoose from '@typegoose/typegoose';
import { DocumentType, Ref } from '@typegoose/typegoose';  // these are types
const { index, prop, arrayProp, getModelForClass, defaultClasses } = typegoose['default'] || typegoose;

One of the nice features available only with ES Modules is top-level await.

[101arrowz]

This actually brings up a bigger question; since Typegoose is targeted solely towards TypeScript users, why not remove the build step entirely and export the raw .ts files to NPM? The user then gets to decide whether to use CommonJS or ES Modules for the final output.

[hasezoey]

@101arrowz you cant really import and compile ts files installed from an provider (like npm), at least i dont know and simple way

[dandv]

This may work for other reasons (Gatsby uses TypeScript) but here's an example of an NPM package that only ships .ts files: https://github.com/epilande/gatsby-theme-auth0/blob/master/package.json

[andreialecu]

I think simply adding a bunch of .mjs files to the npm repository can't hurt anything. Also see: https://nodejs.org/api/esm.html#esm_conditional_exports

Everything should stay backwards compatible even if modules were supported.
https://nodejs.org/api/esm.html#esm_approach_1_use_an_es_module_wrapper

[hasezoey]

i tried to add esm and test it, but i couldnt get it to work, while also including cjs
branch: https://github.com/typegoose/typegoose/tree/feature/esm

run npm run build and it will output /lib as cjs and /lib/esm as mjs

i loaded it into my demo project
with
tsconfig

{
  "compilerOptions": {
        "target": "es6",
        "module": "ES6",
        "allowSyntheticDefaultImports": true
  }
}

and package

{
  "type": "module"
}

and code

// NodeJS: 14.7.0
// MongoDB: 4.2-bionic (Docker)
import { modelOptions, prop, getModelForClass } from "@typegoose/typegoose/lib/esm";
import mongoose from "mongoose"; // mongoose@5.9.26 @types/mongoose@5.7.34

@modelOptions({ schemaOptions: { timestamps: true, strict: "throw", collection: "user" } })
class User {
  @prop({ required: true, index: true })
  public userId!: number;
}
const UserModel = getModelForClass(User);

(async () => {
  await mongoose.connect(`mongodb://localhost:27017/`, { useNewUrlParser: true, dbName: "esm", useCreateIndex: true, useUnifiedTopology: true });



  await mongoose.disconnect();
})();

(typegoose-testing/esm)

and got error Error [ERR_MODULE_NOT_FOUND]: Cannot find module '/home/hasezoey/Downloads/typegoose-testing/node_modules/@typegoose/typegoose/lib/esm/logSettings' imported from /home/hasezoey/Downloads/typegoose-testing/node_modules/@typegoose/typegoose/lib/esm/typegoose.js
then tried to replace

import { logger } from './logSettings';
// with
import { logger } from 'logSettings';

and the next module came up - but i cant ship it this was and i didnt find an typescript option to export it the correct way

if someone can fix this and make an pr, i would merge it

[dandv]

I've just published a dual package using conditional exports. It can be imported by name from ESM TypeScript code ("module": "esnext").

https://github.com/dandv/local-iso-dt.

Here is the diff.

[hasezoey]

this issue might be worth reconsidering (i tried some times before) with typescript 4.5 because from then on it allows to use ESM way more conveniently

especially the section package.json Exports, Imports, and Self-Referencing, which i had problems with before

[AndyClausen]

I solved a lot of issues with esm imports by setting this node flag: --experimental-specifier-resolution=node - maybe this is useful to you as well.

[waynehaffenden]

Any updates on when this is gonna get merged and published?

[hasezoey]

Any updates on when this is gonna get merged and published?

no, no updates, last time i tried i didnt get typescript to cooperate, but will try again when upgrading to typescript 4.5 (see #644)

[WillsterJohnson]

I cloned this repo and made no changes other than

-"module": "commonjs",
+"module": "esnext",

(I didn't update any packages.)
Then ran

npm run build:tests
npm test

and got the output

Test Suites: 24 passed, 24 total
Tests:       255 passed, 255 total
Snapshots:   67 passed, 67 total
Time:        40.246 s
Ran all test suites.

I also ran

npm run build

which completed successfully and with no errors.

I then updated typescript to the latest release with

npm i -D typescript

and ran the same commands again, I got the same successful results.

Obviously there are nuances to this as mentioned in #644, but it does seem well within reach.

---

Currently at work, we've been pretty much backed into a corner (by updates to services we can't control) where the only options are to get hold of an ESM version of typegoose, or rewrite everything in mongoose or plain mongoDB.
We could fork typegoose and maintain our own ESM version util this one upgrades to ESM, but none of us particularly want to maintain a package this large.

Is v10 going to be released any time soon?
ESM is now a non-negotiable necessity for us, and for anyone using a similar enough setup to ours.

[RyannGalea]

I noticed this ORM on Twitter the other day, I have not spent any time looking deep into it yet but it's compatible with MongoDB & might be an avenue to look into, just thought I'd share

https://github.com/mikro-orm

[hasezoey]

@willster277

Is v10 going to be released any time soon?

currently no major features are planned, so no

which completed successfully and with no errors.

like i said earlier, this is how far i also got, but i got problems providing both at the same time, which is what i would like to do before completely migrating

ESM is now a non-negotiable necessity for us, and for anyone using a similar enough setup to ours.

is ESM really required and not working with CJS? also, even if typegoose moves, mongoose and mongodb (from what i know) have not moved yet and also do not provide any ESM

---

Update: i also forgot to mention the other reason of why not completely switching to ESM:
the way i would like to keep the package (or all packages i maintain) is to support all Active NodeJS Versions (Current, LTS, Active Maintenance), and NodeJS 12 still requires a flag for ESM support (i didnt know nodejs 12.22 stabilized ESM and not just 14.x - so just ignore this statement)

also there is that mongoose supports nodejs 12 (and will probably until to at least the next major version, which will probably take a long time) and i would like to typegoose to support all nodejs versions mongoose does

[101arrowz]

like i said earlier, this is how far i also got, but i got problems providing both at the same time, which is what i would like to do before completely migrating

I've done this many times before, you just need to specify multiple fields in your package.json. See this package.json for an example: to support bundlers and Node, use module and exports.*.import for ESM and main or exports.*.require for require. The browser, unpkg, jsdelivr are only for browsers.

You could just build the codebase twice, once with ESM and once with CJS, and provide both exports.

[hasezoey]

You could just build the codebase twice, once with ESM and once with CJS, and provide both exports.

that is what i wanted to do

I've done this many times before, you just need to specify multiple fields in your package.json

i have also got as far as that in my tries, but it somehow never worked, either it was only providing the default, or it was providing nothing so i have up and wanted to try again later

Note: i was also trying at the same type to have the typings be correct and provided (i didnt care if i needed 1 or 2 typing files, but typescript never wanted to do it), so with typescript 4.5, the types should also work better and compile should also work better, i will re-investigate this when upgrading the typescript version next time i do it

[hasezoey]

i have updated the feature/esm branch, this time it compiled without problems, though i still have problems importing it - maybe someone can help out (this will not be included until this can be resolved and documented)

Current Problems:

• i cannot import ESM modules into a ESM project, error src/test.ts:3:10 - error TS2305: Module '"@typegoose/typegoose"' has no exported member 'prop' (typescript error, not runtime)
• i cannot import the CJS modules into a CJS project, error src/test.ts:3:28 - error TS2305: Module '"@typegoose/typegoose"' has no exported member 'prop'. (typescript error, not runtime)

project for esm testing: https://github.com/typegoose/typegoose-testing/tree/test_esm
project for cjs testing: https://github.com/typegoose/typegoose-testing/tree/master

to replace typegoose:

• compile typegoose with yarn run build
• run rm -rf /typegoose-testing/node_modules/@ŧypegoose/typegoose
• run cp -r /typegoose/lib /typegoose-testing/node_modules/@ŧypegoose/typegoose/*
  Note: replace /(root) with the path to the mentioned projects

[WillsterJohnson]

Edit: this is separate to the current problems with compiling this lib to ESM, those problems need to be sorted before publishing. This is more for when publishing ESM is ready, but deprecating CJS is not.

---

Until you're ready to fully switch to ESM, a possibility might be to have an @esm tag?

The build process might look like this (bash);

# get version from package.json (run `npm version [type]` beforehand if needed)
PKG_VER=$(cat package.json | grep version | head -1 | awk -F: '{ print $2 }' | sed 's/[",]//g')

# build as normal
npx tsc -p tsconfig.build.json
npm publish . --access=public

# build ESM
npx tsc -p tsconfig.build.json --module esnext # or whichever ESM
npm version --no-git-tag-version $PKG_VER-esm # different publish needs different version
npm publish. --access=public --tag esm # availiable at @typegoose/typegoose@esm

# go back to $PKG_VER from $PKG_VER-esm
npm version --no-git-tag-version $PKG_VER

Basically;

1. build & publish exactly as currently done
2. set version to have -esm suffix
3. build to ESM & publish to @esm tag
4. remove version -esm suffix

This way anyone who doesn't want ESM can continue as normal, because for them nothing has changed. Anyone who does want ESM simply installs the @esm tag, and can use ESM.

---

This is taken from a publish script I have for one of my own packages, which runs the following code;

# prebuild checks
npm run install
npm run prettier:check || exit 1
npm run test || exit 1
# get version
PKG_VER=$(cat package.json | grep version | head -1 | awk -F: '{ print $2 }' | sed 's/[",]//g')
# build & publish @latest (ESM)
npm run tsc:esm # package.json script for building ESM
npm publish . --access=public
# build & publish @esm (ESM)
npm version --no-git-tag-version $PKG_VER-esm
npm publish . --access=public --tag esm
# build & publish @cjs (CJS)
npm run tsc:cjs # package.json script for building CJS
npm version --no-git-tag-version $PKG_VER-cjs
npm publish . --access=public --tag cjs
# revert version & bump patch
npm version --no-git-tag-version $PKG_VER
npm version patch

As far as I've been able to test, there are no issues.

[hasezoey]

Until you're ready to fully switch to ESM, a possibility might be to have an @esm tag?

i am not sure yet how it will be done, so i will first wait until i can even get ESM builds to work

The build process might look like this (bash);

our build process is not just some bash / sh script, it uses semantic-release, though i dont think it will be much of a problem once actually going to do it

---

as a final note, typegoose will probably not switch fully to ESM until there are any benefits or mongoose switches to ESM (which will very likely not happen anytime soon)

[AndyClausen]

Dude, every time you comment you're notifying like 20 people.

It's great you want to help, but could you open a PR draft linked to this issue and comment there instead? Not everyone here is as invested in this issue.

[hasezoey]

i have updated the feature/esm branch again, this time almost everything just works, but is currently blocked by:

• globalSetup, globalTeardown and testEnvironment does not recognize moduleNameMappers config jestjs/jest#11644

[nhhockeyplayer]

Wish I could stay on but Im dropping my typegoose models for graphql models

new things are emerging like fauna modeling natural relationships with models.

Good luck with this seems like 90% there but 2 yrs and waiting too long

if typegoose nails this anytime soon days/weeks I might look back but Im beginning my transition

graphql though is trending in the field

high end commercial apps/mobile-apps are done in angular which compels @nrwl/nx and they mandate everything be ESM

thanks for all the hard work... loved typegoose and managed to goto production on version 5/6

• cheers

[smolinari]

@nhhockeyplayer - GraphQL has absolutely nothing to do with which database you have storing what data. In fact, one of the problems GraphQL solves is the need for a Gateway to be able to collect data from a plethora of different data sources. In other words, "GraphQL Models" are about the data sent back and forth via the API. The goal of these models is to form the data shapes needed by the UI, which is another one of its strengths. This in turn means absolutely nothing about the data models needed to make business decisions or to persist state for your app(s) (i.e. the data in databases). I believe, if you continue with the mental model you've expressed above about GraphQL, you'll run into a lot of issues. Just trying to hopefully avoid that for you.

Fauna is an interesting NoSQL database for sure, but it is a DBaaS. That means, to use it, you have to take what they offer. You can't build your own database server with it. That can mean two things, 1. you'll have to pay a good penny to use it professionally. 2. you'll more than likely have network latentencies to contend with, depending on where you place your app. Maybe neither are a problem for you. Just sayin.

I didn't check this, but I'd bet you can have different package module resolution scenarios between your apps with Nx. I'd be a really piss-poor monorepo manager if couldn't. If it doesn't, I can suggest using Rush. It can. I know, because I have this going with it right now.

At any rate, I hope you find success no matter what you end up doing.

Scott