Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows Automation tree breaking change #8679

Open
Tobiidave opened this issue Jan 12, 2024 · 9 comments · May be fixed by #8712 or #8715
Open

Windows Automation tree breaking change #8679

Tobiidave opened this issue Jan 12, 2024 · 9 comments · May be fixed by #8712 or #8715
Assignees
Labels
🚧 work in progress Investigate Requires further investigation by the WPF team.

Comments

@Tobiidave
Copy link

Tobiidave commented Jan 12, 2024

Description

Dotnet 8 renders the controls in the Windows automation tree differently compared to earlier dotnet versions.
Buttons in an itemscontrol can't be found anymore, breaking the existing automation.

Using the "Accessabilitiy Insights for Windows" shows a different graph compared to dotnet 6/7 (see below)

Reproduction Steps

WpfApp3.zip

Expected behavior

With <TargetFramework>net7.0-windows</TargetFramework> , the automation tree looks like
pane "Desktop 1"
window "MainWindow"
title bar ''
button "TopButton"
button "ItemButton1"

Actual behavior

With <TargetFramework>net8.0-windows</TargetFramework>, the automation tree looks like

pane "Desktop 1"
window "MainWindow"
title bar ''
button "TopButton"
list view #note that here was the "ItemButton1" - now it is not present anymore!

Regression?

Yes

Known Workarounds

No response

Impact

The UI Tests for our applications breaks; need workarounds

Configuration

Microsoft.WindowsDesktop.App 8.0.1
Microsoft.NETCore.App 8.0.1
windows 10.19045.3803
x64

Vs2022

Other information

No response

@lindexi lindexi added the Investigate Requires further investigation by the WPF team. label Jan 12, 2024
@lindexi
Copy link
Contributor

lindexi commented Jan 12, 2024

@Tobiidave Maybe this behavior be changed by #6862

I'm not sure.

@Tobiidave
Copy link
Author

I don't know how the 6862 affected this. But the problem exists in both 8.0.0 and 8.0.1

@psmulovics
Copy link

I do not think you can expect automation trees to stay intact between major version upgrades - I would consider it business-as-usual these getting broken on major upgrade.

@Tobiidave
Copy link
Author

Tobiidave commented Jan 15, 2024

@psmulovics In this case the WPF controls change type and are not usable anymore, I edited and tried to emphasize this by a small comment. It is not only a tree re-ordering issue, but elements that were clickable can not be found anymore! Ofc, any suggested workarounds are welcome! We have paused dotnet8 migration due to this, the automation tech is central to our development.

@lindexi
Copy link
Contributor

lindexi commented Jan 19, 2024

We have paused dotnet8 migration due to this

@Tobiidave Sorry hear that. I still haven't found the exact change code for this issues.

@jimm98y
Copy link

jimm98y commented Jan 20, 2024

I've created a test app and I've tried to restore the original behavior simply by setting this in OnStartup:

System.AppContext.SetSwitch("Switch.System.Windows.Controls.ItemsControlDoesNotSupportAutomation", true);

Unfortunately, it looks like it's too late and the value is already set and cached. So I had to use reflection instead:

var accessibilitySwitches = Type.GetType("System.Windows.AccessibilitySwitches, WindowsBase, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089");
var field = accessibilitySwitches.GetField("_ItemsControlDoesNotSupportAutomation", System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Static | BindingFlags.DeclaredOnly);
field.SetValue(null, 1);

This seems to fix the issue and restore the original pre-NET8 behavior. To test it, I used the following piece of code in the test app from this bug report:

public static void VerifyMainWindowAccessibility()
{
    AutomationElement mainWindow = AutomationElement.RootElement.FindFirst(TreeScope.Children, new PropertyCondition(AutomationElement.NameProperty, "MainWindow"));
    Debug.WriteLine(DumpUIATree(mainWindow));
}

private static string DumpUIATree(AutomationElement element)
{
    var s = element.Current.Name + " : " + element.Current.ControlType.ProgrammaticName;
    DumpChildrenRecursively(element, 1, ref s);
    return s;
}

private static List<AutomationElement> GetChildNodes(AutomationElement automationElement)
{
    var children = new List<AutomationElement>();
    TreeWalker walker = TreeWalker.ControlViewWalker;
    AutomationElement child = walker.GetFirstChild(automationElement);
    while (child != null)
    {
        children.Add(child);
        child = walker.GetNextSibling(child);
    }
    return children;
}

private static void DumpChildrenRecursively(AutomationElement node, int level, ref string s)
{
    var children = GetChildNodes(node);
    foreach (AutomationElement child in children)
    {
        if (child != null)
        {
            for (int i = 0; i < level; i++)
                s += "-";
            s += " " + child.Current.Name + " : " + child.Current.ControlType?.ProgrammaticName + "\r\n";
            DumpChildrenRecursively(child, level + 1, ref s);
        }
    }
}

NET8 before the fix:

- TopButton : ControlType.Button
-- TopButton : ControlType.Text
-  : ControlType.List
-- ItemButton1 : ControlType.DataItem
--- ItemButton1 : ControlType.Text

NET8 after the fix:

- TopButton : ControlType.Button
-- TopButton : ControlType.Text
- ItemButton1 : ControlType.Button
-- ItemButton1 : ControlType.Text

NET6 behavior:

- TopButton : ControlType.Button
-- TopButton : ControlType.Text
- ItemButton1 : ControlType.Button
-- ItemButton1 : ControlType.Text

@walterlv walterlv linked a pull request Jan 22, 2024 that will close this issue
@Kuldeep-MS Kuldeep-MS self-assigned this Jan 23, 2024
@Kuldeep-MS
Copy link
Contributor

I am investigating the issue from my end.

@Tobiidave
Copy link
Author

Good work guys - but be sure to test this with the ms app "Accessibilities for Windows" to make sure it looks the same.
It shows larger differences than what I see from your inspection using the TreeWalker-code above!

@walterlv
Copy link
Contributor

I've tested this code below to set the AppContext switch and find it worked.

public class Program
{
    [STAThread]
    static void Main()
    {
        AppContext.SetSwitch("Switch.System.Windows.Controls.ItemsControlDoesNotSupportAutomation", true);

        var app = new App();
        app.InitializeComponent();
        app.Run();
    }
}

This means that if it is late on startup, it can be set before the WPF initialization which reflection is not needed. @jimm98y

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🚧 work in progress Investigate Requires further investigation by the WPF team.
Projects
None yet
6 participants