Add support for TextEncoder and TextDecoder

[chyzwar]

TextEncoder and TextDecoder are added as globals in node11.
Maybe it is possible to use these for jsdom?

https://developer.mozilla.org/en-US/docs/Web/API/TextDecoder
https://developer.mozilla.org/en-US/docs/Web/API/TextDecoder

https://nodejs.org/api/globals.html#globals_textdecoder
https://nodejs.org/api/globals.html#globals_textencoder

I made a change in jest to support these in jest-environment-node. jestjs/jest#8022 I can make simmilar change in jsdom but I am not sure were it should be done.

[chyzwar]

@domenic can you advice where this can be added in jsdom ?

[JamesHenry]

There are a lot of upvotes and linked issues to this one (which is the correct etiquette), but given this has not received any explicit interaction for over a year, I hope nobody minds if I give it a jolt! ⚡

I am currently running a patched custom environment in jest to work around this not being in jsdom itself:

const Environment = require('jest-environment-jsdom');

module.exports = class CustomTestEnvironment extends Environment {
  async setup() {
    await super.setup();
    if (typeof this.global.TextEncoder === 'undefined') {
      const { TextEncoder, TextDecoder } = require('util');
      this.global.TextEncoder = TextEncoder;
      this.global.TextDecoder = TextDecoder;
    }
  }
};

[juharris]

@JamesHenry looks a lot like the example on Stackoverflow which also shows the react-scripts command to use that custom env: npx react-scripts test --env=./test/custom-test-env.js https://stackoverflow.com/a/57713960/1226799

[amedwardson]

I would also appreciate support for TextDecoder and TextEncoder. Didn't have much joy with @JamesHenry 's custom test environment solution but adding this to the top of my test files worked:

    const { TextEncoder, TextDecoder } = require('util');
    global.TextEncoder = TextEncoder;
    global.TextDecoder = TextDecoder;

[cpiber]

Note: A breaking change in whatwg-url caused a breaking change in jsdom as well, before TextEncode and TextDecoder were polyfilled from util, they aren't anymore now. Still works in 16.7.0.

[bobsilverberg]

Note: A breaking change in whatwg-url caused a breaking change in jsdom as well, before TextEncode and TextDecoder were polyfilled from util, they aren't anymore now. Still works in 16.7.0.

Is there any plan to fix this in whatwg-url? I don't see any issues about it in the repo.

[domenic]

jsdom and whatwg-url recently created new major versions which require the latest Node.js releases. There is no plan to stop requiring the latest Node.js releases in the new major versions. You can use the old unsupported versions of jsdom or whatwg-url if you want to continue using non-supported Node.js releases.

[bobsilverberg]

jsdom and whatwg-url recently created new major versions which require the latest Node.js releases. There is no plan to stop requiring the latest Node.js releases in the new major versions. You can use the old unsupported versions of jsdom or whatwg-url if you want to continue using non-supported Node.js releases.

I'm not sure if this was in reply to my question above, but I am on Node 14 and am getting an error about TextEncoder is not defined when importing jsdom, so it doesn't seem to have to do with a particularly old version. The changelog for jsdom 17 says "Node v12 is now the minimum supported version."

[domenic]

You need the supported LTS version of Node 14, which at this time is Node 14.17.5.

[bobsilverberg]

You need the supported LTS version of Node 14, which at this time is Node 14.17.5.

I am testing with Node 14.17.5 locally and am still getting that error. I have seen the workaround where one adds code to one's Jest config, but this was working fine with jsdom 16, and is not working now with 17, and ideally we wouldn't have to add that code to our config. The comment at #2524 (comment) makes it sound like this is a breaking change that was introduced in whatwg-url.

FWIW, I get the same error with Node 16.7.0

[domenic]

Fascinating. If you can post a small repro example in a new issue following the issue template that would be helpful. (In particular not using Jest or anything similar.)

[domenic]

FWIW I cannot reproduce; jsdom 16.7.0 did not have TextEncoder/TextDecoder exposed on its windows: https://runkit.com/domenicdenicola/611d6c290366f4001b2745f2

[cpiber]

I also use Node 14.7.5.

Here is a minimal example: https://github.com/cpiber/jest-demo

I can use jsdom just fine in normal code and in jest with test environment node, but in jest test environment jsdom I get the following error:

I should mention that jest itself has no problems, test environment jsdom works normally, I just can't import/require inside a test. It seems to be a problem when the jsdom environment is active, so it's probably not a problem with jsdom directly.

[ethaizone]

I understand why this issue is open for so long and it isn't treated as bug due it's about point of view.

If we saw from Jsdom side then global variable should be Node.js environment but if we saw from end user like Jest then global is window that provided from Jsdom itself.

https://github.com/jestjs/jest/blob/a30a52fbdd35db6ca7cfc2f03efc03153da18e99/packages/jest-environment-jsdom/src/index.ts#L70-L77

Problem is when Jsdom loaded under Jest then it mean global is overrided by Jsdom already. I'm not sure if maintainer aware about how Jsdom use in Jest. That cause many people come to this issue again and again.

then here is question.

Is https://github.com/jsdom/whatwg-url create to run under Node.js or browser?

I don't know actual answer from maintainer but if it's me, it need to run under Node.js for sure.

but even answer is like that then it doesn't mean we should add those TextEncoder and TextDecoder into jest-environment-jsdom as how this package know which variable in global that Jsdom need. If in future this happen with other global variable then we will see another issue similar like this again.

Right now I don't have actual answer that this should be fix in which repo. I just post this to make everyone aware situation that we faced because we might have some miss communication.

---

then here is just maybe good news. I have other solution that you might interest. As I said above that because jest-environment-jsdom make change of global to be window from Jsdom then we can do other solution like this.

• Use testEnvironment: 'node', so we don't use jest-environment-jsdom. This is important as above explanation.
• Create jest setup file and enable Jsdom manually and inject required variable that you might need.

I edited after I saw document from @domenic https://github.com/jsdom/jsdom/wiki/Don't-stuff-jsdom-globals-onto-the-Node-global.

Adding code to initial Jsdom in https://jestjs.io/docs/configuration#globalsetup-string or beforeEach in each test case

import * as jsdom from 'jsdom'

const dom = new jsdom.JSDOM('<!DOCTYPE html><head/><body></body>', {
  url: 'http://someexample.com/',
  referrer: 'https://example.com/',
  contentType: 'text/html',
  userAgent: 'Jest',
  includeNodeLocations: true,
  storageQuota: 10000000,
  pretendToBeVisual: true,
})
// Access value that you need with these.
// dom.window as unknown as Window & typeof globalThis
// dom.window.document
// dom.window.navigator
// dom.window.localStorage

With this you can use window/document or whatever in your unit test. This is one way that we can make Jsdom can run under Node.js environment normally and we don't need to worry if we need to add some hack on it or not.

[domenic]

See https://github.com/jsdom/jsdom/wiki/Don't-stuff-jsdom-globals-onto-the-Node-global

[ethaizone]

See https://github.com/jsdom/jsdom/wiki/Don't-stuff-jsdom-globals-onto-the-Node-global

@domenic Thank you for pointing this out then I think only option is adding it in https://jestjs.io/docs/configuration#globalsetup-string or beforeEach in each test file.

I hope someone can be in middle between this lib and maintainer of jest-environment-jsdom package because now I saw what jest-environment-jsdom is antipattern actually as this document described. It's main issue why people keep come to this topic.

I updated above comment.

[pdfabbro]

If anyone is compiling their tests to ES5 and using typescript this is how I resolved the issue(s):

import $JSDOMEnvironment, {
  TestEnvironment as $TestEnvironment,
} from 'jest-environment-jsdom'
import { TextDecoder, TextEncoder } from 'util'

/**
 * This patched JSDOMEnvironment serves as a proxy for the default JSDOMEnvironment from Jest.
 * It patches the global objects TextEncoder, TextDecoder, and Uint8Array
 * which are missing, or improperly implemented (Uint8Array is a node Buffer) in the JSDOM environment.
 *
 * A proxy pattern is used due to ES5 transpilation limitations, as direct
 * class extension isn't possible in ES5. The class delegates other methods
 * and properties to the original JSDOMEnvironment.
 *
 * We use this wrapper for full compatibility with browser global
 * objects in our Jest testing environment.
 */
class JSDOMEnvironment {
  private environment: $JSDOMEnvironment

  constructor(...args: ConstructorParameters<typeof $JSDOMEnvironment>) {
    this.environment = new $JSDOMEnvironment(...args)
    this.environment.global.TextEncoder = TextEncoder
    this.environment.global.TextDecoder =
      TextDecoder as typeof global.TextDecoder
    this.environment.global.Uint8Array = Uint8Array

    return new Proxy(this, {
      get: (target, prop, receiver) => {
        if (Reflect.has(target, prop)) {
          return Reflect.get(target, prop, receiver)
        } else {
          return Reflect.get(target.environment, prop, receiver)
        }
      },
      set: (target, prop, value) => {
        if (Reflect.has(target, prop)) {
          return Reflect.set(target, prop, value)
        } else {
          return Reflect.set(target.environment, prop, value)
        }
      },
    })
  }
}

const TestEnvironment =
  $TestEnvironment === $JSDOMEnvironment ? JSDOMEnvironment : $TestEnvironment

export default JSDOMEnvironment
export { TestEnvironment }

@dmmulroy thank-you so much, this worked for me!!! (with a slight variation - I had to change the top import to just import $JSDOMEnvironment from "jest-environment-jsdom"; and the export to just export default JSDOMEnvironment;)

[luchillo17]

If anyone is compiling their tests to ES5 and using typescript this is how I resolved the issue(s):

import $JSDOMEnvironment, {
  TestEnvironment as $TestEnvironment,
} from 'jest-environment-jsdom'
import { TextDecoder, TextEncoder } from 'util'

/**
 * This patched JSDOMEnvironment serves as a proxy for the default JSDOMEnvironment from Jest.
 * It patches the global objects TextEncoder, TextDecoder, and Uint8Array
 * which are missing, or improperly implemented (Uint8Array is a node Buffer) in the JSDOM environment.
 *
 * A proxy pattern is used due to ES5 transpilation limitations, as direct
 * class extension isn't possible in ES5. The class delegates other methods
 * and properties to the original JSDOMEnvironment.
 *
 * We use this wrapper for full compatibility with browser global
 * objects in our Jest testing environment.
 */
class JSDOMEnvironment {
  private environment: $JSDOMEnvironment

  constructor(...args: ConstructorParameters<typeof $JSDOMEnvironment>) {
    this.environment = new $JSDOMEnvironment(...args)
    this.environment.global.TextEncoder = TextEncoder
    this.environment.global.TextDecoder =
      TextDecoder as typeof global.TextDecoder
    this.environment.global.Uint8Array = Uint8Array

    return new Proxy(this, {
      get: (target, prop, receiver) => {
        if (Reflect.has(target, prop)) {
          return Reflect.get(target, prop, receiver)
        } else {
          return Reflect.get(target.environment, prop, receiver)
        }
      },
      set: (target, prop, value) => {
        if (Reflect.has(target, prop)) {
          return Reflect.set(target, prop, value)
        } else {
          return Reflect.set(target.environment, prop, value)
        }
      },
    })
  }
}

const TestEnvironment =
  $TestEnvironment === $JSDOMEnvironment ? JSDOMEnvironment : $TestEnvironment

export default JSDOMEnvironment
export { TestEnvironment }

@dmmulroy thank-you so much, this worked for me!!! (with a slight variation - I had to change the top import and bottom export to just import $JSDOMEnvironment from "jest-environment-jsdom"; and the export to just export default JSDOMEnvironment;)

This solution requires a lot of code, I am in a TS environment, and if I remember correctly I just did this in the jest setup file:

    const { TextEncoder, TextDecoder } import 'util';
    global.TextEncoder = TextEncoder;
    global.TextDecoder = TextDecoder;

[pdfabbro]

    const { TextEncoder, TextDecoder } import 'util';
    global.TextEncoder = TextEncoder;
    global.TextDecoder = TextDecoder;

@luchillo17 That doesn't work for me and neither does

const { TextEncoder, TextDecoder } = require("util");
global.TextEncoder = TextEncoder;
global.TextDecoder = TextDecoder;

[luchillo17]

@pdfabbro Where are you importing it? maybe I remember it wrong, for example, the solutions Backstage.io maintainers came up with are all over the place, I believe I fixed it globally then the maintainers were paranoid and instead decided to move the fix to each plugin that required it, in an inconsistent way might I add:

https://github.com/search?q=repo%3Abackstage%2Fbackstage%20TextEncoder&type=code

If you look there the setupTests.ts file repeated a few times if you tell GH to show similar files, and it uses the require import version, meanwhile DryRunContext.test.tsx#L20 uses the import version.

[skondrashov]

From a user's perspective, it feels like jsdom is responsible for setting up the environment in such a way that it resembles a browser, with all of the browser globals provided for me. I've read through the comments here and in some other places, but I don't really understand the developer perspective here. Over at msw (which I'm trying to use with jest, bringing me here), it sounds like this is considered to be a jest problem, but it makes perfect sense to me that jest would completely strip the default node environment down and allow jsdom to set up the new environment. Why isn't this a jsdom issue? TextDecoder appears to be available in my browser, just like document, which jsdom provides for me, what's the difference?

[luchillo17]

From a user's perspective, it feels like jsdom is responsible for setting up the environment in such a way that it resembles a browser, with all of the browser globals provided for me. I've read through the comments here and in some other places, but I don't really understand the developer perspective here. Over at msw (which I'm trying to use with jest, bringing me here), it sounds like this is considered to be a jest problem, but it makes perfect sense to me that jest would completely strip the default node environment down and allow jsdom to set up the new environment. Why isn't this a jsdom issue? TextDecoder appears to be available in my browser, just like document, which jsdom provides for me, what's the difference?

It is a JSDom issue, basically, the browser has TextEncoder and TextDecoder, I can only assume JSDom doesn't provide it by default because it is not used by most projects, thus reducing the size of the test environment in those cases, and then for us that do use them we need to add it to the environment, this whole thread is basically a bunch of different ways to do so.

[domenic]

It is a jsdom issue. Jsdom doesnt provide it because nobody has implemented TextEncoder/TextDecoder in jsdom. Just like nobody has implemented fetch(), or the dialog element, or many many other features. If you want a feature added to jsdom, working on a PR that passes all the tests is the correct way of doing so. That is very hard for many web platform features, which is why browsers employ hundreds of full-time employees.