npm: webpack doesn't like the package

[bjornharrtell]

Regressed after 3.17.4. Error message with 3.18.0:

Module not found: Error: Package path ./lib/mjs/geojson is not exported from package /tmp/broccoli-1848999uiU6yCVIbUjp/cache-615-webpack_bundler_ember_auto_import_webpack/node_modules/flatgeobuf (see exports field in /tmp/broccoli-1848999uiU6yCVIbUjp/cache-615-webpack_bundler_ember_auto_import_webpack/node_modules/flatgeobuf/package.json)

[bjornharrtell]

I think I figured out that this is due to two issues in TypeScript:

• Compiled JavaScript import is missing file extension microsoft/TypeScript#40878
• A compiler option to enforce file extensions. microsoft/TypeScript#42813

I will have to revert the package to be a plain non-module npm package. :(

Update: While the above still seem potentially problematic I'm no longer convinced it is causing the particular issue I have with consuming this package with ember-auto-import / webpack 5.

[bjornharrtell]

How annoying, webpack 4 can consume the package without issues...

[bjornharrtell]

Related issue embroider-build/ember-auto-import#457.

[RandomFractals]

so, what's the latest stable version of your lib should we use while you resolve this?

I also get cjs/geojson error after updating to your latest v3.19.0 you published to npm.

Tried v3.18.5. This is the error I get in vscode when trying to activate geo data viewer that uses your library for fgb data files loading:

...

ok. v3.17.4 seems to work. I'll use that version for now.

[bjornharrtell]

Sorry to hear that @RandomFractals. It seems to be a mess but I have no good fix yet.

[ef4]

I tried to provide more detail in embroider-build/ember-auto-import#457 (comment)

[bjornharrtell]

So I think I understand the problem now but the solution seems hacky (working around typescript issues mentioned at #141 (comment))

[bjornharrtell]

Node 12+ TypeScript module support postponed :(

• Concerns with TypeScript 4.5's Node 12+ ESM Support microsoft/TypeScript#46452

[bjornharrtell]

Note to self: I believe that the desired situation here is to have all compiled files in /lib/cjs have file ending .cjs instead of .js. Perhaps a hack can be found to do this in the build step...

Update: Not enough.. when using cjs extension it seems all require names must be fully qualified with file extensions...

[bjornharrtell]

Ok I've decided to drop cjs entirely because:

1. All attempts made both cjs and mjs use problematic
2. https://nodejs.org/api/packages.html#writing-dual-packages-while-avoiding-or-minimizing-hazards sums up the dangers of it

So as of 3.21.2 the flatgeobuf package is module only and it works with webpack without issues for me again.

[iwpnd]

After spending 3 frustrating days to get flatgeobuf running in my typescript project I can fully understand why you'd resign and drop cjs. I still cannot fully grasp the whole concept of ESM vs CJS, and how something like this can even be considered when your entire package ecosystem has to follow for an interruption free experience. It is what it is, I guess.

But however that leaves people to only have mapbox/geobuf for cjs, which is missing some very cool features that you provide with flatgeobuf.

[bjornharrtell]

Sorry to hear that. While the TS/JS ecosystem still have its issues it's been pretty smooth since Node 16 and with build systems like vite so I'm curious what kind of setup that was so difficult?

Also, I'm not sure who have to use cjs these days? Even in older node versions I've been able to use https://www.npmjs.com/package/esm relatively easy.

[iwpnd]

I'm not excluding the possibility that I'm a dumbass here, and maybe it's only a configuration issue.

I have a typescript monorepo with this tsconfig:

{
  "compilerOptions": {
    "allowJs": false,
    "allowSyntheticDefaultImports": true,
    "types": ["node", "jest"],
    "baseUrl": ".",
    "declaration": true,
    "declarationMap": true,
    "composite": true,
    "esModuleInterop": true,
    "experimentalDecorators": true,
    "lib": ["es2019", "dom"],
    "module": "commonjs",
    "moduleResolution": "node",
    "noImplicitAny": true,
    "noUnusedLocals": true,
    "rootDir": ".",
    "sourceMap": true,
    "strictNullChecks": true,
    "target": "es2019",
    "paths": {
      "@iwpnd/*": ["packages/*/src"]
    }
  },
  "include": ["packages/*/src"],
  "exclude": [
    "node_modules",
    "dist",
    "packages/*/node_modules",
    "packages/*/dist",
    "packages/*/src/jest",
    "packages/**/*.test.ts",
    "packages/**/*.spec.ts"
  ],
  "files": [],
  "references": []
}

and a separate package that has this little class here using flatgeobuf

// flatgeobuffer.ts
import { FeatureCollection } from '@vpriem/geojson';
import { geojson } from 'flatgeobuf';

export interface Rect {
    minX: number;
    minY: number;
    maxX: number;
    maxY: number;
}

export class Flatgeobuffer {
    static serialize(collection: FeatureCollection): Uint8Array {
        const buf = geojson.serialize(collection);
        return buf;
    }

    static deserialze(buf: Uint8Array, bbox?: Rect): FeatureCollection {
        const collection = geojson.deserialize(buf, bbox);
        return collection as FeatureCollection;
    }
}

I can compile this without an issue. So far so good and pretty straight forward.

"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.Flatgeobuffer = void 0;
const flatgeobuf_1 = require("flatgeobuf");
class Flatgeobuffer {
    static serialize(collection) {
        const buf = flatgeobuf_1.geojson.serialize(collection);
        return buf;
    }
    static deserialze(buf, bbox) {
        const collection = flatgeobuf_1.geojson.deserialize(buf, bbox);
        return collection;
    }
}
exports.Flatgeobuffer = Flatgeobuffer;
//# sourceMappingURL=flatgeobuffer.js.map

If I run ts-jest against a small test suite importing the Flatgeobuffer class I'm greeted with:

Jest encountered an unexpected token

Jest failed to parse a file. This happens e.g. when your code or its dependencies use non-standard JavaScript syntax, or when Jest is not configured to support such syntax.

Details:

.../iwpnd-monorepo/node_modules/flatgeobuf/lib/mjs/flatgeobuf.js:1

({"Object.<anonymous>":function(module,exports,require,__dirname,__filename,jest){import * as generic_1 from './generic.js';

Now this is were my odyssee really starts. There's guides on ESM support with ts-jest. Different configs for jest, tsconfig. But no matter what I do, the issue is that the compiled javascript is trying to require() flatgeobuf. The proposition to use await import() however does not work for commonjs.

So my conclusion is (with the limited knowledge I have on the commonjs/esm topic), that I can only use flatgeobuf with type: module packages.

[bjornharrtell]

Mm, yes more or less it is least amount of pain to use type module but it can be worked around if you really want. In either case you should not use commonjs as module target for TS, because that makes the assumption that everything is in commonjs including dependencies. Instead you should let TS compile to es modules and if you don't want type module for your package you need to transpile the whole result into commonjs or use a shim loader like https://www.npmjs.com/package/esm in the thing consuming your package.

NOTE: TS has commonjs as module target because of historical reasons. It will likely switch to es as default "soon".

[bjornharrtell]

More simply put, IMO it is a good idea to target Node 16 and type module packages going forward. Staying in CJS land will only become more painful than it already is.

[iwpnd]

You’re probably right. 🥲