Mechanism to mark up "alternative" read order

[brennanyoung]

“Content order” is an important design characteristic, especially for accessibility (“a11y”).

It is intimately connected with ‘good semantics’.

WCAG 2.1 and 2.2 specifiy several success criteria which pertain to this characteristic:

1.3.1 Info and Relationships
1.3.2 Meaningful Sequence
2.4.3 Focus Order

Users of Assisitive Technologies (ATs) such as screen readers rely on authors arranging their content in such a way that the most meaningful “next” item appears subsequently in the DOM. Keyboard shortcuts might direct the reading cursor to the next heading or the next image, or just the next palpable element.
Similarly, the “tab sequence” is most reliable when it follows the order of operable elements in the DOM.

In most cases, this works very well. It is a modest constraint which is relatively easy to follow.

However, there are various classes of content which are not well-served by this approach. To generalize, this content is characterised by the potential for (or expectation of) multiple viable pathways, to the extent that the information, structure, and relationships conveyed through presentation are neither fully programmatically determined, nor readily reduced to flat text.

What is called for is a way for any 'node' (palpable element) to indicate a flow of meaning or continuation to any other palpable element(s), as its natural or most meaningful "next" item(s), regardless of the position of that/those elements in the DOM.

Some Concrete Use Cases

• Mathematical expressions and chemical formulae - where the most appropriate ‘reading’ might traverse the expression in multiple directions.
• Flow Charts - where content is related by connecting lines (which may themselves have labels) that often express branches, jumps and loops.
• Data Views - where the ‘story’ is expressed by how data points relate to the value of other data points (or other arbitrary limits and thresholds), irrespective of the sequence in which they are plotted (i.e. irrespective of DOM order).
• Filters and sorting mechanisms which permit subsets of large amounts of data to be approached according to distinct (but arbitrary) criteria.
• CSS flex-direction, grid, floats and absolute position - where content may be deliberately presented visually in a sequence that differs from DOM order.
• Multi-stave music notation

Ideal and Necessary Features

• The mechanism should be exposed to user agents’ accessibility APIs in a consistent and predictable way.
• The mechanism should complement/extend existing accessibility tree implementations.
• The mechanism should be available to all palpable content.
• It should be possible to ‘browse’ the alternative reading pathways, without switching to a fully operational interaction mode. This is especially relevant for assistive technologies that make a distinction between “browse mode” (aka. “document mode”) and “forms mode” (aka. “applications mode”).
• Looped content pathways should be supported. (A simple tree structure is not enough).
• Going ‘backwards’ should be supported, perhaps requiring some kind of history stack logic, that remembers which path was taken, even if multiple alternative paths might otherwise have been taken.
• Connecting lines/paths should permit (but not require) naming (i.e. “accessible names” for assistive technology).
• Nodes should permit rich text structures such as lists, headings etc. and not limited to flat/plain text. We have flowchart boxes that contain headings, bullet lists, hints in subscript, emphatic text and so on.
• Where there is more than one alternative 'next' node from a given node (for example a flowchart choice box) ATs are able to present these choices to their users using appropriate cues, and appropriate interactions.

Prior Art

• Anchors / hyperlinks. The primary abstraction of HTML is the link, especially including links to anchors within a resource. Links are well understood by authors and users, with solid support from assistive technology, and they are well-integrated into user agent browsing history, although as a downside, the distinction between links and buttons is routinely blurred by web content authors. A real advantage is that hyperlinks are primarily content. They do not put screen readers in “forms mode”, but instead can be consumed as part of a flow of text. They also support or require accessible names (typically the text node inside the element).
• A related semantic, the navigation role ( in HTML5) might be a viable model for grouping alternative read order pathways, but as a landmark role, it may generate noise or chatter on ATs, which may not be appropriate for lower level connections. Suggestion: a non-landmark version of ? i.e. something which is to <nav what
• Operable elements and roles such as button, so that we “hack” a browsable experience without using “browse mode”. This is a solution that works today, but breaks the tidy but perhaps rather dogmatic distinction between operable and non-operable content. Maybe we should officially “bless” the idea of using operable elements for browsing content which may have multiple ‘reading paths’?
• aria-flowto - is the part of ARIA which most closely approaches the use cases that this enquiry intends to serve. The existing spec is good, as far as it goes, but has received valid criticism mainly for what it omits. In practice, there are very few implementations (user agents or ATs) that do anything interesting with this attribute. That’s a shame, but it does mean that we do not have to worry about legacy implementations in the wild.
• aria-details - is a relatively recent addition to the ARIA spec, and has been suggested as a candidate for replacing aria-flowto. Some (including me) find it no better (or worse) at meeting the “alternative read order” use cases than the flawed aria-flowto attribute. Piggybacking on this attribute has the disadvantage of adding a different semantic meaning. Flowchart boxes are not each others’ “details”. A datapoint that relates to another datapoint is not a “details” relationship, it’s an alternative reading. Something quite different.
  SVG Connectors - is an intriguing proposed extension to the SVG spec, which defines ways for elements to be connected. It’s very old, and didn’t go anywhere, but there are some good ideas in the proposal, not least the promise of connecting elements visually using declarative means, rather than pixel-perfect positioning.- - -

After a deep-dive discussion, the ARIA community agreed not to deprecate aria-flowto until we have a viable replacement which better serves the use cases mentioned above. We agreed that the best way to approach this was via a WICG proposal, because any solution might potentially become part of in HTML, rather than merely part of ARIA. I hope this Proposal captures the essence of the discussion, now we just have to hash it out and find something that can work.

Questions, opinions and discussion are very welcome.