Added aria-live to commands to improve screen reader accessibility. #15048

m158261 · 2023-08-30T15:03:50Z

This task implements an aria-live region into jupyterlab to improve screen reader accessibility when running commands either via the command palette or keyboard shortcut (i.e. Ctrl + B - close/open left sidebar).

Here is the aria-live documentation...

https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA/ARIA_Live_Regions

Expected Functionality

This functionality provides customisable text to be announced by a screen reader whenever a command is run either via the command palette or via a keyboard shortcut. This will provide an audible cue to screen reader users when a command has successfully been run.

References

#9399
#6573

Code changes

The changes to application-extension get the aria-live region by it's ID then append text to the div. This text will be read out by a screen reader and provides an audible cue when a command is run.

CSS has been added to make the live region visually hidden while keeping it visible to the screen reader. Documentation on the pattern can be found here and here.

User-facing changes

Users that are using a screen reader will now get a prompt when they use a keyboard shortcut or access a command via the command palette. No visual changes otherwise.

jupyterlab-probot · 2023-08-30T15:04:26Z

Thanks for making a pull request to jupyterlab!
To try out this branch on binder, follow this link:

welcome · 2023-08-30T15:07:20Z

Thanks for submitting your first pull request! You are awesome! 🤗

If you haven't done so already, check out Jupyter's Code of Conduct. Also, please make sure you followed the pull request template, as this will help us review your contribution more quickly.

You can meet the other Jovyans by joining our Discourse forum. There is also a intro thread there where you can stop by and say Hi! 👋

Welcome to the Jupyter community! 🎉

krassowski · 2023-08-30T17:24:29Z

packages/application-extension/src/index.tsx

@@ -203,6 +203,7 @@ const mainCommands: JupyterFrontEndPlugin<void> = {

    commands.addCommand(CommandIDs.close, {
      label: () => trans.__('Close Tab'),
+      commandDialog: () => trans.__('Close Tab'),


Why would we want to have a new property for this? I think we should reuse label and caption. This is be related to which shows and discusses how we can customize/caption for different uses Allow a custom label for menu items, overriding the command name lumino#570

If a new property is really needed (for now I am not convinced), let's think of a new which explains the intent better. the command dialog is very unclear for me. Why add command? What is dialog?

Hi @krassowski, thanks for the feedback. I've modified this locally to use label in a similar way to the dialog. Alas the screen reader is not reading the content of label. I've tried combinations of aira-live and setting the innerHTML. I have also tried using an aria-label then focusing on the element when the command is executed.

I wonder if a different approach is needed. Do you have any suggestions?

I would defer to @gabalafou here who I believe has more experience with screen readers.

Hey @gabalafou, do you have any suggestions on how to proceed with this? Thanks in advance.

if can you share some rendered html from your local demo i might be able to help you. i'd need to see the structure of the dom elements you are trying to get announcements from.

Hey @tonyfast, thanks for helping out. I have attached a screenshot of the label div with the aria-live region. I am setting the innerHTML of this div when the open/close left sidebar command (ctrl + B) is run. As the text is changing in the aria-live region I would expect the screen reader (MS Narrator) to announce the text but it's not. It announces the keyboard shortcut instead.

Any feedback or suggestions would be much appreciated.

isabela-pf · 2023-09-22T21:04:12Z

Hello and thanks for the tag! Based on reading this, I'm going to try and summarize what I think you are saying the user experience should be to make sure I understand.

User is in JupyterLab and using a screen reader.
User activates the command palette via menu item, sidebar option (if configured for that), or via keyboard shortcut.
User focus is changed to the filter/search input box at the top of the command palette.
The screen reader announces this new focus by reading the default text filling that input box ("Search" if I recall correctly).
User has been informed of where they are in the interface and what actions they can take from there.

Does this sound like what you intend? Or are there details I am missing?

m158261 · 2023-09-25T09:53:48Z

Hello and thanks for the tag! Based on reading this, I'm going to try and summarize what I think you are saying the user experience should be to make sure I understand.

User is in JupyterLab and using a screen reader.

User activates the command palette via menu item, sidebar option (if configured for that), or via keyboard shortcut.

User focus is changed to the filter/search input box at the top of the command palette.

The screen reader announces this new focus by reading the default text filling that input box ("Search" if I recall correctly).

User has been informed of where they are in the interface and what actions they can take from there.

Does this sound like what you intend? Or are there details I am missing?

Hi Isabela, thanks for commenting. That's almost the intended functionality.

User is in JupyterLab using a screen reader.
User activates the command palette via menu item, sidebar option (if configured for that), or via keyboard shortcut.
User focus is changed to the filter/search input box at the top of the command palette.
The screen reader announces this new focus by reading the default text filling that input box ("Search" if I recall correctly).
The User activates a command.
The screen reader informs the user that the command has been executed.

The alternative is...

User is in JupyterLab using a screen reader.
User activates a command using a keyboard shortcut i.e. opening the sidebar
The screen reader announces the command has been executed.

isabela-pf · 2023-09-26T19:06:05Z

That mostly makes sense to me experience-wise. To be clear, I currently have no way of validating that this PR makes these experience changes. For anyone reviewing this that can validate it, that's what I would personally be looking for.

The only question I still have is about the meaning of

the command has been executed.

Because we're talking about the command palette, I think you mean that there is feedback when (after opening the command palette) a command from the command palette has been executed. So there is no change in telling users once they've entered the command palette, only a change when a user selects a command to execute.

I did think we were just talking about opening the command palette at all, so that's my misunderstanding. Is the idea here that when a user selects a command, they do not know when that command has completed execution? So this feedback could come immediately or even minutes or hours later depending on what command is run?

m158261 · 2023-09-27T15:28:11Z

That mostly makes sense to me experience-wise. To be clear, I currently have no way of validating that this PR makes these experience changes. For anyone reviewing this that can validate it, that's what I would personally be looking for.

The only question I still have is about the meaning of

the command has been executed.

Because we're talking about the command palette, I think you mean that there is feedback when (after opening the command palette) a command from the command palette has been executed. So there is no change in telling users once they've entered the command palette, only a change when a user selects a command to execute.

I did think we were just talking about opening the command palette at all, so that's my misunderstanding. Is the idea here that when a user selects a command, they do not know when that command has completed execution? So this feedback could come immediately or even minutes or hours later depending on what command is run?

Yeah, thats right. There is only a change when a user runs a command. When a command is run either from the command palette or via a keyboard shortcut, configurable text about that command will be announced by the screen reader. This should run immediately.

m158261 · 2023-11-08T15:48:56Z

Hey @tonyfast, I have updated my implementation by moving the live region to the tab bar. This now appends text to the region when a command is run which is announced by the screen reader. Please let my know your thoughts and what the future direction of aria-live in jupyterlab should be.

Thanks in advance!

tonyfast · 2023-11-08T17:50:02Z

are these changes merged into https://github.com/t03857785/jupyterlab/tree/demo?

m158261 · 2023-11-09T09:11:59Z

are these changes merged into https://github.com/t03857785/jupyterlab/tree/demo?

Not at the moment. I'll let you know when they are.

tonyfast · 2023-11-09T16:47:51Z

thanks. since there is a lumino dependency i need to test both on that branch built together.

tonyfast · 2023-12-05T18:43:58Z

Hey @tonyfast, thanks for the feedback. I have had to change the implementation as appending the live region to the tabbar didn't work. There are multiple tabbars rendered which means multiple live regions with the same ID.

be aware that there are cases where you will have to have multiple live regions. the nbconvert-a11y implementation i shared with you has them in a few places. in my example, i'm using dialog elements that when active make all content outside the dialog inert. when aria live elements are inert they are NOT announced to the screen reader. in the case of dialogs, i need to carry around multiple activity logs so that aria live is announced. each new dialog i open has an activity log at the bottom so announcements can make it to the screen reader. the changes propagate to all of the activity logs on the page when i announce something. we'll need to be aware of these conditions if we use proper dialog elements .

I've moved the region into JupyterLab and appended it to the dock panel. This means the Lumino PR is no longer necessary to test this. I have also appended the text to be announced rather than setting the innerHTML so it keeps the record of actions.

this makes sense. i think there might be future cases where users or developers might want to explore the activity log.

i'll rebuild a lite demo without the lumino pr. https://readthedocs.org/projects/jupyak/builds/22759565/

Do you think I should add a timestamp like in your implementation to make it more like a log?

i don't know if it needs to be there, but i thought this feature sounded like a user facing log so i borrowed design principles from system log. i think there is a future when some visits the activity log to undo past changes at specific times. i see time as a door into more visual features, audibly i don't feel like the time in important for the aria live announcement.

tonyfast · 2023-12-06T19:13:46Z

the jupyterlite build has been updated to reflect these changes https://jupyak--16.org.readthedocs.build/en/16/_static/work/lite/lab/index.html (this page must be opened in a private/incognito browser).

can confirm that #commands-aria-live exists

tonyfast

thanks for suggesting that test to run on my side. that really helped.

this work is so exciting to me for reasons:

it works with Orca! it feels great. super 😎 stuff.
i'm really excited we're thinking about aria-live it will add some must to the assistive experience and make it feel like a native screen reader application
we can use a11y prs deathbeds/jupyak#16 to incrementally update your changes into a working jupyter lite ui that i can test with a screen reader. we've wanted this experience for years and it has arrived and feels good.

i tested open and closing the sidebar with Ctrl+B and i can confirm that the i am hearing the aria-live announcements. the announcements come in an active tense, but they are announced after the action completes. i made two suggestions that i think apply generally to these log messages. if a log happens before an event it should use the active phrasing you've applied and if the after happens after the action then use a past tense.

i can't confirm this at the moment. however, i bet if we run the IBM Equal Access checker will raise Sensory Characteristics Needs Review. this is because of the use of left and right. this intention of the sensory characteristic is:

Instructions provided for understanding and operating content do not rely solely on sensory characteristics of components such as shape, size, visual location, orientation, or sound.

there should be a way to test actual content in one of these jupyak lite builds. i'll think about that, but in the mean time its hand done.

packages/application-extension/src/index.tsx

Co-authored-by: Tony Fast <tony.fast@gmail.com>

m158261 · 2023-12-07T16:53:15Z

thanks for suggesting that test to run on my side. that really helped.

this work is so exciting to me for reasons:

it works with Orca! it feels great. super 😎 stuff.

i'm really excited we're thinking about aria-live it will add some must to the assistive experience and make it feel like a native screen reader application

we can use a11y prs deathbeds/jupyak#16 to incrementally update your changes into a working jupyter lite ui that i can test with a screen reader. we've wanted this experience for years and it has arrived and feels good.

That's excellent. Really glad it's working.

i tested open and closing the sidebar with Ctrl+B and i can confirm that the i am hearing the aria-live announcements. the announcements come in an active tense, but they are announced after the action completes. i made two suggestions that i think apply generally to these log messages. if a log happens before an event it should use the active phrasing you've applied and if the after happens after the action then use a past tense.

Ok, I'll review the announcements and make sure they're in the correct tense.

i can't confirm this at the moment. however, i bet if we run the IBM Equal Access checker will raise Sensory Characteristics Needs Review. this is because of the use of left and right. this intention of the sensory characteristic is:

Instructions provided for understanding and operating content do not rely solely on sensory characteristics of components such as shape, size, visual location, orientation, or sound.

That's a good point. I'll see what I can do. I used left/right to describe sidebars as that's what they're named in the command palette.

there should be a way to test actual content in one of these jupyak lite builds. i'll think about that, but in the mean time its hand done.

tonyfast · 2023-12-07T19:04:52Z

That's a good point. I'll see what I can do. I used left/right to describe sidebars as that's what they're named in the command palette.

yea i figured that would be a bigger issue with the design system. i don't know what is better phrasing either. "primary" "secondary" doesnt make sense because its really subjective where icons in sidebars are.

m158261 · 2023-12-21T15:25:39Z

Hey @krassowski, I'm happy for this to be re-reviewed. Please let me know what you think.

Thank you @tonyfast for your input with this.

tonyfast · 2023-12-22T17:06:31Z

This is some great work @m158261 Thanks for braving new territory. it is necessary to start experimenting with aria live because of all the state changes that can happen in the application. I hope this feature can be highlighted in the release notes as a step towards more assistive experiences; it is a significant step beyond wcag. We will have to be cautious about regressions in this feature since it can’t be tested automatically. I hope we can document it rigorously to avoid that.

krassowski

My review echoes what @tonyfast has pointed previously:

the announcements come in an active tense, but they are announced after the action completes

I think that every string added needs to be reviewed to ensure that it accurately reflects that an action has been completed (or not).

packages/application-extension/src/index.tsx

krassowski · 2024-02-01T10:52:42Z

packages/application/style/core.css

+
+.jp-ContentPanel-ariaLive {
+  width: 0;
+  overflow: hidden;


I do not follow this one. Was the intent to use visibility: hidden?

An aria-live region must be visible on the DOM for the screen reader to pick up changes to it. This code makes the div visually hidden while visible to the screen reader.

Relevant Stack Overflow Page

Ok, in this case, can you also add height: 0; and a comment to explain the intent?

here is a recommended pattern for visually hidden elements https://www.tpgi.com/the-anatomy-of-visually-hidden/

cf https://www.a11yproject.com/posts/how-to-hide-content/

might be worth having sr-only somewhere in lab or lumino. then this would be .jp-ContentPanel-arialive.sr-only

Thanks @tonyfast, I agree we should follow what you linked (with a reference in a comment).

might be worth having sr-only somewhere in lab or lumino

+1. I would suggest to name it jp-mod-sr-only to avoid conflicts with any third party styles if they have it slightly different, but for the purposes of getting this PR in I would suggest that having a more generic class could be postponed (as this would become a public API we need to think more where to put it, document it etc).

Thanks @tonyfast for that documentation. I've added it to the PR description.

Shall I leave the name as .jp-ContentPanel-arialive. for now and address sr-only in a future PR?

packages/application-extension/src/index.tsx

m158261 · 2024-02-02T16:44:48Z

Hey @krassowski, thank you for the feedback. I have gone through the announcements and made sure they reflect the tense of the action. Most of the announcements come after the action has completed.

I have addressed the other comments too. Please let me know what you think.

Many thanks!

m158261 · 2024-03-28T15:04:46Z

I am unable to continue working on this PR after March 28, 2024. If further refinements are identified, I welcome any future contributors to pick up and progress the work.

Updated branch to sync with jupyterlab

1240588

github-actions bot added the pkg:application label Aug 30, 2023

github-actions bot assigned m158261 Aug 30, 2023

m158261 mentioned this pull request Aug 30, 2023

Added aria-live to the tab bar to improve screen reader accessibility. jupyterlab/lumino#627

Closed

krassowski reviewed Aug 30, 2023

View reviewed changes

krassowski added the tag:Accessibility label Aug 30, 2023

m158261 marked this pull request as draft September 19, 2023 16:02

m158261 and others added 6 commits October 2, 2023 13:37

Merge branch 'main' into command-dialog

ba3a777

Changed label to match command dialog functionality

fadeb33

Added code to update innerHTML of label div

ec821c7

Merge branch 'jupyterlab:main' into command-dialog

2525cd6

Merge branch 'jupyterlab:main' into command-dialog

d409e54

Tidied up branch

dd8a119

m158261 changed the title ~~Added a command dialog field to enhance command palette screen reader accessibility.~~ Added aria-live to command palette to improve screen reader accessibility. Nov 2, 2023

m158261 and others added 3 commits November 7, 2023 09:10

Updated show left sidebar aria live comments

6f1f08b

Merge branch 'jupyterlab:main' into command-dialog

e110b7e

Added custom aria-live messages to be read when a command is run

90bc58f

m158261 changed the title ~~Added aria-live to command palette to improve screen reader accessibility.~~ Added aria-live to commands to improve screen reader accessibility. Nov 8, 2023

tonyfast closed this Nov 9, 2023

tonyfast reopened this Nov 9, 2023

Moved live region to hbox panel

ac16633

github-actions bot added tag:CSS For general CSS related issues and pecadilloes Design System CSS labels Dec 6, 2023

Renamed ID and CSS class

f8f1402

m158261 marked this pull request as ready for review December 6, 2023 15:42

tonyfast suggested changes Dec 6, 2023

View reviewed changes

packages/application-extension/src/index.tsx Outdated Show resolved Hide resolved

packages/application-extension/src/index.tsx Outdated Show resolved Hide resolved

m158261 and others added 2 commits December 7, 2023 10:14

Update packages/application-extension/src/index.tsx

673ed61

Co-authored-by: Tony Fast <tony.fast@gmail.com>

Update packages/application-extension/src/index.tsx

2f0b4a7

Co-authored-by: Tony Fast <tony.fast@gmail.com>

m158261 mentioned this pull request Dec 7, 2023

JupyterLab/Lumino Accessibility Demo Branches t03857785/jupyterlab#12

Open

m158261 requested a review from krassowski January 8, 2024 17:39

krassowski added this to the 4.1.0 milestone Jan 25, 2024

krassowski requested changes Feb 1, 2024

View reviewed changes

m158261 added 2 commits February 2, 2024 16:33

Merge branch 'main' into command-dialog

84a11f0

Added const for live region ID and updated hints with correct tense

47518a9

Merge branch 'jupyterlab:main' into command-dialog

414d4e4

krassowski modified the milestones: 4.1.0, 4.2.0 Feb 5, 2024

m158261 added 2 commits February 5, 2024 15:06

Updated visually hidden css

a690668

Added documentation in comment about visually hidden pattern

306f0b5

krassowski mentioned this pull request Apr 10, 2024

New plugin (token) for aria-live #16145

Open

krassowski modified the milestones: 4.2.0, 4.3.0 May 7, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added aria-live to commands to improve screen reader accessibility. #15048

Added aria-live to commands to improve screen reader accessibility. #15048

m158261 commented Aug 30, 2023 •

edited

jupyterlab-probot bot commented Aug 30, 2023

welcome bot commented Aug 30, 2023

krassowski Aug 30, 2023

m158261 Aug 31, 2023

krassowski Aug 31, 2023

m158261 Sep 19, 2023

tonyfast Sep 28, 2023

m158261 Oct 3, 2023

isabela-pf commented Sep 22, 2023

m158261 commented Sep 25, 2023

isabela-pf commented Sep 26, 2023

m158261 commented Sep 27, 2023

m158261 commented Nov 8, 2023

tonyfast commented Nov 8, 2023

m158261 commented Nov 9, 2023

tonyfast commented Nov 9, 2023

tonyfast commented Dec 5, 2023

tonyfast commented Dec 6, 2023 •

edited

tonyfast left a comment

m158261 commented Dec 7, 2023

tonyfast commented Dec 7, 2023

m158261 commented Dec 21, 2023

tonyfast commented Dec 22, 2023

krassowski left a comment

krassowski Feb 1, 2024

m158261 Feb 2, 2024

krassowski Feb 3, 2024

tonyfast Feb 3, 2024

tonyfast Feb 3, 2024 •

edited

krassowski Feb 5, 2024

m158261 Feb 5, 2024

m158261 commented Feb 2, 2024

m158261 commented Mar 28, 2024

Added aria-live to commands to improve screen reader accessibility. #15048

Are you sure you want to change the base?

Added aria-live to commands to improve screen reader accessibility. #15048

Conversation

m158261 commented Aug 30, 2023 • edited

References

Code changes

User-facing changes

jupyterlab-probot bot commented Aug 30, 2023

welcome bot commented Aug 30, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

isabela-pf commented Sep 22, 2023

m158261 commented Sep 25, 2023

isabela-pf commented Sep 26, 2023

m158261 commented Sep 27, 2023

m158261 commented Nov 8, 2023

tonyfast commented Nov 8, 2023

m158261 commented Nov 9, 2023

tonyfast commented Nov 9, 2023

tonyfast commented Dec 5, 2023

tonyfast commented Dec 6, 2023 • edited

tonyfast left a comment

Choose a reason for hiding this comment

m158261 commented Dec 7, 2023

tonyfast commented Dec 7, 2023

m158261 commented Dec 21, 2023

tonyfast commented Dec 22, 2023

krassowski left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tonyfast Feb 3, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

m158261 commented Feb 2, 2024

m158261 commented Mar 28, 2024

m158261 commented Aug 30, 2023 •

edited

tonyfast commented Dec 6, 2023 •

edited

tonyfast Feb 3, 2024 •

edited