innerText implementation #1245

vsemozhetbyt · 2015-09-25T00:20:17Z

jsdom is a great tool for web scraping. However the textContent is a very inconvenient way to get readable text for html2text conversion.

There is a wonderful article about usefulness of negligible innerText in many cases:

http://perfectionkills.com/the-poor-misunderstood-innerText/

The author suggests getSelection().toString() as a very slow workaround, but getSelection is not implemented in the jsdom yet.

Could you consider an implementing of the innerText in the jsdom? The author has done a great exploration about it, he has even added a simple spec at the end.

The text was updated successfully, but these errors were encountered:

vsemozhetbyt · 2015-09-25T11:23:40Z

And what a pity that rangy Selection and innerText library is not compatible with jsdom: timdown/rangy#348

domenic · 2015-09-25T11:48:08Z

So, innerText is not standard, and not implemented in at least one major engine (Firefox). Without a standard, I don't think we should implement it.

Edit by @TimothyGu (April 24, 2023): this comment was accurate when it was written (in 2015!) but is no longer accurate, since a spec has been created. We welcome any contribution to implement that feature in jsdom.

Sebmaster · 2015-10-09T13:06:49Z

Looks like there's some movement in this whole thing with a draft spec here. See also all the references. There are no issues on the repo though, so I wonder how complete it already is / how quick progress will be.

vsemozhetbyt · 2016-01-25T09:49:29Z

Firefox has implemented: https://bugzilla.mozilla.org/show_bug.cgi?id=264412

WHATWG semms to approve: whatwg/compat#5 (comment)

inikulin · 2016-01-25T09:59:15Z

From the spec it's seems like we can't implement innerText properly without basic layout support.

domenic · 2016-01-25T16:07:41Z

Yeah, this is not really going to be implementable in jsdom anyway, without a lot of infrastructure work... nobody get their hopes up :(.

vsemozhetbyt · 2016-01-30T13:16:43Z

As to layout support requirement: rocallahan/innerText-spec#2

…erText` usage to `textContent` based on [this discussion](jsdom/jsdom#1245). Added tests for many evaluators.

vsemozhetbyt · 2016-08-27T19:16:47Z

Is there any plan to implement it because of WHATWG adoption?

domenic · 2016-08-27T19:17:37Z

Yeah... Although the spec requires a lot of stuff jsdom doesn't have, around CSS boxes :(. Not sure what to do.

vsemozhetbyt · 2016-08-27T19:20:16Z

Is there any lib for this to plug along with jsdom?

snuggs · 2016-08-29T19:22:39Z

@domenic care to drop some knowledge on why this is such an infrastructure overhaul? We thought the 800lb gorilla in the room would leave lo-key. But looks like it's not going anywhere. As you know have been wrapping my head around the innards of jsdom. Where would be a great place in the repo to start reviewing code to a jsdom newb?

Thanks in advance 🙏 /cc @vsemozhetbyt

dmethvin · 2016-08-29T19:40:07Z

The primary issue is the fact that innerText leans on the layout engine for guidance, and jsdom has no layout engine. See https://html.spec.whatwg.org/multipage/dom.html#the-innertext-idl-attribute and
http://perfectionkills.com/the-poor-misunderstood-innerText/ . From the second link:

Notice how innerText almost precisely represents exactly how text appears on the page. textContent, on the other hand, does something strange — it ignores newlines created by
and around styled-as-block elements ( in this case). But it preserves spaces as they are defined in the markup.

vsemozhetbyt · 2017-04-26T12:00:25Z

Still out of scope and no workaround?

coreh · 2017-05-24T23:49:59Z

Apparently the spec says:

If this element is not being rendered, or if the user agent is a non-CSS user agent, [emphasis added] then return the same value as the textContent IDL attribute on this element.

I think a workaround would be then to simply return textContent.

domenic · 2017-05-25T00:42:18Z

We implement enough CSS that I don't think that applies. We just don't implement the layout parts...

Suzii · 2018-01-24T20:02:15Z

Hi guys, any news on this one?

Bnaya · 2018-01-25T00:32:59Z

Just use headless chrome :)

Janpot · 2018-08-05T12:39:58Z

@domenic from that spec that @coreh mentioned:
https://html.spec.whatwg.org/multipage/dom.html#the-innertext-idl-attribute

If this element is not being rendered, or if the user agent is a non-CSS user agent, then return the same value as the textContent IDL attribute on this element.

https://html.spec.whatwg.org/multipage/rendering.html#being-rendered

An element is being rendered if it has any associated CSS layout boxes, SVG layout boxes, or some equivalent in other styling languages.

If jsdom doesn't implement the layout parts, doesn't that mean "not being rendered" applies?

andrewcartwright1 · 2023-11-05T09:10:17Z

Not the nicest of way's to do it but I had some cases where sometimes HTMLElement would work and sometimes it wouldnt.. I couldnt figure out why, so i changed it to be even more hacky. I mocked it using jest spies.

innerText.ts

import sanitizeHtml from 'sanitize-html';

let spyGet;
let spySet;

export const createSpies = () => {
  Object.defineProperty(Object.prototype, 'innerText', {
    get: () => undefined,
    set: () => undefined,
    configurable: true
  });

  // eslint-disable-next-line @typescript-eslint/ban-types
  spyGet = jest.spyOn(Object.prototype, 'innerText' as keyof Object, 'get');
  spyGet.mockImplementation(function () {
    if (this.textContent === undefined) {
      return undefined;
    }

    return sanitizeHtml(this.textContent, {
      allowedTags: [],
      allowedAttributes: {}
    })
      .split('\\n')
      .filter((text: string) => text && !text.match(/^\\s+$/))
      .map((text: string) => text.trim())
      .join('\\n');
  });

  // eslint-disable-next-line @typescript-eslint/ban-types
  spySet = jest.spyOn(Object.prototype, 'innerText' as keyof Object, 'set');
  spySet.mockImplementation(function (value: unknown) {
    this.textContent = value;
  });
};

export const resetSpies = () => {
  spyGet?.mockRestore();
  spySet?.mockRestore();
};

Then you can call it using createSpies and resetSpies in your jest tests.

You can use it without sanitize-html, I've used that to strip out any html

macgyver · 2023-11-05T12:09:15Z

can we close this ticket? people seem pretty capable of / content to roll their own solutions

…

On Sun, Nov 5, 2023 at 04:10 Andrew Cartwright ***@***.***> wrote: Not the nicest of way's to do it but I had some cases where sometimes HTMLElement would work and sometimes it wouldnt.. I couldnt figure out why, so i changed it to be even more hacky. I mocked it using jest spies. innerText.ts let spyGet; let spySet; export const createSpies = () => { Object.defineProperty(Object.prototype, 'innerText', { get: () => {}, set: () => {}, configurable: true }); // eslint-disable-next-line @typescript-eslint/ban-types spyGet = jest.spyOn(Object.prototype, 'innerText' as keyof Object, 'get'); spyGet.mockImplementation(function () { return this.textContent .split('\\n') .filter((text: string) => text && !text.match(/^\\s+$/)) .map((text: string) => text.trim()) .join('\\n'); }); // eslint-disable-next-line @typescript-eslint/ban-types spySet = jest.spyOn(Object.prototype, 'innerText' as keyof Object, 'set'); spySet.mockImplementation(function (value) { this.textContent = value; }); }; export const resetSpies = () => { spyGet?.mockRestore(); spySet?.mockRestore(); }; Then you can call it using createSpies and resetSpies in your jest tests. — Reply to this email directly, view it on GitHub <#1245 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAAJC54FMGXSOUUM3SQAZLYC5JYJAVCNFSM4BQODFC2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCNZZGM3DQMBSGMZQ> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

DanKaplanSES · 2023-11-30T02:15:02Z

Can this please be documented as a limitation in the README? Speaking of which, is there a link that lists all the JSDOM limitations? If so, can this link be added to the README? If it's already there, it seems like it should be found on the "Unimplemented parts" section.

EDIT: Here's my attempt at documenting this: #3631

See jsdom#1245 for context.

DanKaplanSES · 2023-11-30T02:46:18Z

It could at least fail with an exception instead of quietly returning undefined.

@AntonKhorev that makes sense indeed, but it would be a breaking change. Some people may not have innerText explicitly in their tests, but dependencies may use it internally.

In my opinion, that's even scarier to think about. It's hard enough to troubleshoot when it's in your own project, imagine how hard it would be when you're degrees of separation away from the code that caused the problem.

I'm not arguing for a breaking change, but maybe:

Document it
Provide some configuration that warns developers when it's depended on
Provide some configuration that throws an error when it's depended on
Use "the next best thing" as a fake response, such as textContent.

IMO, any one of these would improve the current situation.

The issue addressed in PR #3351 unfortunately used a `contentEditable` value that is not compatible with Firefox [1] This revision implements the first strategy explored by the aforementioned PR by removing the full format from the contenteditable span when we stop the edition. Testing required to define some properties as JSdom does not properly support the implementation of innerText (see [2]) [1]: https://developer.mozilla.org/en-US/docs/Web/HTML/Global_attributes/contenteditable#browser_compatibility [2]: jsdom/jsdom#1245 Task: 3754944

The issue addressed in PR #3351 unfortunately used a `contentEditable` value that is not compatible with Firefox [1] This revision implements the first strategy explored by the aforementioned PR by removing the full format from the contenteditable span when we stop the edition. Testing required to define some properties as JSdom does not properly support the implementation of innerText (see [2]) [1]: https://developer.mozilla.org/en-US/docs/Web/HTML/Global_attributes/contenteditable#browser_compatibility [2]: jsdom/jsdom#1245 closes #3894 Task: 3754944 Signed-off-by: Lucas Lefèvre (lul) <lul@odoo.com>

The issue addressed in PR #3351 unfortunately used a `contentEditable` value that is not compatible with Firefox [1] This revision implements the first strategy explored by the aforementioned PR by removing the full format from the contenteditable span when we stop the edition. Testing required to define some properties as JSdom does not properly support the implementation of innerText (see [2]) [1]: https://developer.mozilla.org/en-US/docs/Web/HTML/Global_attributes/contenteditable#browser_compatibility [2]: jsdom/jsdom#1245 Task: 3754944 X-original-commit: a11b5dc

The issue addressed in PR #3351 unfortunately used a `contentEditable` value that is not compatible with Firefox [1] This revision implements the first strategy explored by the aforementioned PR by removing the full format from the contenteditable span when we stop the edition. Testing required to define some properties as JSdom does not properly support the implementation of innerText (see [2]) [1]: https://developer.mozilla.org/en-US/docs/Web/HTML/Global_attributes/contenteditable#browser_compatibility [2]: jsdom/jsdom#1245 closes #3915 Task: 3754944 X-original-commit: a11b5dc Signed-off-by: Lucas Lefèvre (lul) <lul@odoo.com> Signed-off-by: Rémi Rahir (rar) <rar@odoo.com>

The issue addressed in PR #3351 unfortunately used a `contentEditable` value that is not compatible with Firefox [1] This revision implements the first strategy explored by the aforementioned PR by removing the full format from the contenteditable span when we stop the edition. Testing required to define some properties as JSdom does not properly support the implementation of innerText (see [2]) [1]: https://developer.mozilla.org/en-US/docs/Web/HTML/Global_attributes/contenteditable#browser_compatibility [2]: jsdom/jsdom#1245 Task: 3754944 X-original-commit: 91d7a19

The issue addressed in PR #3351 unfortunately used a `contentEditable` value that is not compatible with Firefox [1] This revision implements the first strategy explored by the aforementioned PR by removing the full format from the contenteditable span when we stop the edition. Testing required to define some properties as JSdom does not properly support the implementation of innerText (see [2]) [1]: https://developer.mozilla.org/en-US/docs/Web/HTML/Global_attributes/contenteditable#browser_compatibility [2]: jsdom/jsdom#1245 closes #3919 Task: 3754944 X-original-commit: 91d7a19 Signed-off-by: Lucas Lefèvre (lul) <lul@odoo.com> Signed-off-by: Rémi Rahir (rar) <rar@odoo.com>

The issue addressed in PR #3351 unfortunately used a `contentEditable` value that is not compatible with Firefox [1] This revision implements the first strategy explored by the aforementioned PR by removing the full format from the contenteditable span when we stop the edition. Testing required to define some properties as JSdom does not properly support the implementation of innerText (see [2]) [1]: https://developer.mozilla.org/en-US/docs/Web/HTML/Global_attributes/contenteditable#browser_compatibility [2]: jsdom/jsdom#1245 Task: 3754944 X-original-commit: ed7d3fa

The issue addressed in PR #3351 unfortunately used a `contentEditable` value that is not compatible with Firefox [1] This revision implements the first strategy explored by the aforementioned PR by removing the full format from the contenteditable span when we stop the edition. Testing required to define some properties as JSdom does not properly support the implementation of innerText (see [2]) [1]: https://developer.mozilla.org/en-US/docs/Web/HTML/Global_attributes/contenteditable#browser_compatibility [2]: jsdom/jsdom#1245 closes #3922 Task: 3754944 X-original-commit: ed7d3fa Signed-off-by: Lucas Lefèvre (lul) <lul@odoo.com> Signed-off-by: Rémi Rahir (rar) <rar@odoo.com>

The issue addressed in PR #3351 unfortunately used a `contentEditable` value that is not compatible with Firefox [1] This revision implements the first strategy explored by the aforementioned PR by removing the full format from the contenteditable span when we stop the edition. Testing required to define some properties as JSdom does not properly support the implementation of innerText (see [2]) [1]: https://developer.mozilla.org/en-US/docs/Web/HTML/Global_attributes/contenteditable#browser_compatibility [2]: jsdom/jsdom#1245 closes #3921 Task: 3754944 X-original-commit: ed7d3fa Signed-off-by: Lucas Lefèvre (lul) <lul@odoo.com> Signed-off-by: Rémi Rahir (rar) <rar@odoo.com>

The issue addressed in PR #3351 unfortunately used a `contentEditable` value that is not compatible with Firefox [1] This revision implements the first strategy explored by the aforementioned PR by removing the full format from the contenteditable span when we stop the edition. Testing required to define some properties as JSdom does not properly support the implementation of innerText (see [2]) [1]: https://developer.mozilla.org/en-US/docs/Web/HTML/Global_attributes/contenteditable#browser_compatibility [2]: jsdom/jsdom#1245 closes #3920 Task: 3754944 X-original-commit: ed7d3fa Signed-off-by: Lucas Lefèvre (lul) <lul@odoo.com> Signed-off-by: Rémi Rahir (rar) <rar@odoo.com>

vsemozhetbyt mentioned this issue Sep 25, 2015

rangy is not compatible with jsdom timdown/rangy#348

Open

r4j4h added a commit to r4j4h/jasmine-phantom-utils that referenced this issue Jun 17, 2016

Added istanbul/jsdom for code coverage and leaner tests. Changed `inn…

42d373b

…erText` usage to `textContent` based on [this discussion](jsdom/jsdom#1245). Added tests for many evaluators.

domenic added the feature label Jul 2, 2016

domenic mentioned this issue Sep 8, 2016

textContent vs. innerText #1596

Closed

domenic mentioned this issue Mar 29, 2017

jsdom does not support .innerText #1793

Closed

Zirro mentioned this issue Mar 30, 2017

Not able to get rendered DOM markup #1794

Closed

thetrav mentioned this issue May 22, 2017

innerHTML not supported by jsdom for server side rendering chartist-js/chartist#955

Open

tinesoft mentioned this issue Jan 10, 2018

Fixing lib.component.spec.ts failed on JEST test tinesoft/generator-ngx-library#139

Merged

eddyerburgh mentioned this issue Mar 31, 2018

setData() doesn't set data correctly vuejs/vue-test-utils#495

Closed

truongsinh added a commit to truongsinh/Semantic-UI-React that referenced this issue Apr 10, 2018

jsdom/jsdom#1245 replace .innerText with .textContent

7cef260

tidoust mentioned this issue May 12, 2018

Browser executable-less preprocessor w3c/respec#1469

Open

Zirro added the layout Blocked on implementing a layout engine label May 27, 2018

glenfannin-okta mentioned this issue Oct 27, 2023

OKTA-656562 karma to jest okta/okta-signin-widget#3418

Merged

5 tasks

DanKaplanSES added a commit to DanKaplanSES/jsdom that referenced this issue Nov 30, 2023

Document the innerText limitation

4ffb82e

See jsdom#1245 for context.

DanKaplanSES mentioned this issue Nov 30, 2023

Document the innerText limitation #3631

Open

StephenCooper mentioned this issue Jan 23, 2024

Update aggrid to v31.0.2 breaks all test - Aggrid render issue ? ag-grid/ag-grid#7465

Closed

mcous mentioned this issue Jan 24, 2024

Unable to test a non-bound change in the DOM testing-library/svelte-testing-library#119

Closed

ps100000 mentioned this issue Feb 25, 2024

Front kernsanierung phase2 user event reporter schmidti2002/Softwarepraktikum#105

Merged

rrahir mentioned this issue Mar 25, 2024

[FIX] BottomBarSheet: Rename a sheet with style content on Firefox odoo/o-spreadsheet#3894

Closed

14 tasks

fw-bot mentioned this issue Mar 25, 2024

[FW][FIX] BottomBarSheet: Rename a sheet with style content on Firefox odoo/o-spreadsheet#3915

Closed

14 tasks

fw-bot mentioned this issue Mar 25, 2024

[FW][FIX] BottomBarSheet: Rename a sheet with style content on Firefox odoo/o-spreadsheet#3919

Closed

14 tasks

fw-bot mentioned this issue Mar 25, 2024

[FW][FIX] BottomBarSheet: Rename a sheet with style content on Firefox odoo/o-spreadsheet#3920

Closed

14 tasks

fw-bot mentioned this issue Mar 25, 2024

[FW][FIX] BottomBarSheet: Rename a sheet with style content on Firefox odoo/o-spreadsheet#3921

Closed

14 tasks

fw-bot mentioned this issue Mar 25, 2024

[FW][FIX] BottomBarSheet: Rename a sheet with style content on Firefox odoo/o-spreadsheet#3922

Closed

14 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

innerText implementation #1245

innerText implementation #1245

vsemozhetbyt commented Sep 25, 2015

vsemozhetbyt commented Sep 25, 2015

domenic commented Sep 25, 2015 •

edited by TimothyGu

Sebmaster commented Oct 9, 2015

vsemozhetbyt commented Jan 25, 2016

inikulin commented Jan 25, 2016

domenic commented Jan 25, 2016

vsemozhetbyt commented Jan 30, 2016

vsemozhetbyt commented Aug 27, 2016

domenic commented Aug 27, 2016

vsemozhetbyt commented Aug 27, 2016

snuggs commented Aug 29, 2016

dmethvin commented Aug 29, 2016

vsemozhetbyt commented Apr 26, 2017

coreh commented May 24, 2017

domenic commented May 25, 2017

Suzii commented Jan 24, 2018

Bnaya commented Jan 25, 2018

Janpot commented Aug 5, 2018 •

edited

andrewcartwright1 commented Nov 5, 2023 •

edited

macgyver commented Nov 5, 2023 via email

DanKaplanSES commented Nov 30, 2023 •

edited

DanKaplanSES commented Nov 30, 2023 •

edited

innerText implementation #1245

innerText implementation #1245

Comments

vsemozhetbyt commented Sep 25, 2015

vsemozhetbyt commented Sep 25, 2015

domenic commented Sep 25, 2015 • edited by TimothyGu

Sebmaster commented Oct 9, 2015

vsemozhetbyt commented Jan 25, 2016

inikulin commented Jan 25, 2016

domenic commented Jan 25, 2016

vsemozhetbyt commented Jan 30, 2016

vsemozhetbyt commented Aug 27, 2016

domenic commented Aug 27, 2016

vsemozhetbyt commented Aug 27, 2016

snuggs commented Aug 29, 2016

dmethvin commented Aug 29, 2016

vsemozhetbyt commented Apr 26, 2017

coreh commented May 24, 2017

domenic commented May 25, 2017

Suzii commented Jan 24, 2018

Bnaya commented Jan 25, 2018

Janpot commented Aug 5, 2018 • edited

andrewcartwright1 commented Nov 5, 2023 • edited

macgyver commented Nov 5, 2023 via email

DanKaplanSES commented Nov 30, 2023 • edited

DanKaplanSES commented Nov 30, 2023 • edited

domenic commented Sep 25, 2015 •

edited by TimothyGu

Janpot commented Aug 5, 2018 •

edited

andrewcartwright1 commented Nov 5, 2023 •

edited

DanKaplanSES commented Nov 30, 2023 •

edited

DanKaplanSES commented Nov 30, 2023 •

edited