GD-322: Add option for automatic discovery of tests

[dreed-sd]

The used GdUnit4 version:
4.2.1

The used Godot version:
4.2 stable

OS including version:
Mac M1 / OSX

Describe the bug
The GdUnit panel shows no tests when first opening the project. If I click 'run overall tests', it successfully finds them all and runs them. I am not actually sure whether this is a bug, because maybe it isn't intended to do that? If not I'd call this a feature request. The 'run overall tests' button isn't even enabled by default in the config, so for a while when setting this up I thought GdUnit was just not working because I could not find a way to run any tests (and I am using VSCode, it seems 4.2 does not yet have a VSCode plugin, so I couldn't even run a specific test from code).

Steps to Reproduce

1. Start a new project with GdUnit4 installed and enabled
2. Create a new test
3. -> The GdUnit panel does not show your test
4. -> Even if you restart, it shows nothing
5. Enable the 'run overall' button and click it, successfully runs all tests, and they now appear in the panel.

[MikeSchulze]

Hi @dreed-sd, yes this is intended.
Test are not discovered by default like in .NET explorer in VSC.

I can just add a new option to enable real-time test discovering.
I will start a vote in the GdUnit4 discord channel if should be enabled by default.

The 'run overall tests' button isn't even enabled by default in the config, so for a while when setting this up I thought GdUnit was just not working because I could not find a way to run any tests

You do not always have to execute all tests, but can simply use the context menu, the FileSystemInspector or directly the script editor to run one or more test cases.

[dreed-sd]

We use VSCode with Godot, and when it's setup to do that you actually can't even view scripts in the built-in editor. If there's a way to get to your context menus with that in place, I don't know what it is. If there's a way to do it it's not obvious (to be fair I'm still fairly new to Godot). That might be fine if the tests were runnable in VSCode, but if I'm not mistaken that isn't available in 4.2 yet? All that seems to mean I have no way to run any tests without enabling the run overall button. If the issue is the performance of discovering all the tests, then I suggest at least starting off in a "click here to discover tests" state. The current behavior just looks like there are no tests, which makes it look broken.

[MikeSchulze]

Just to understand, you are using the GdUnit4 c# API in vscode?
I am working on a test adapter to GdUnit4 that can be used with .NET explorer.

[dreed-sd]

No we are just using GdScript with VSCode

[MikeSchulze]

No we are just using GdScript with VSCode

Ok, so you are using GDScript in VS Code, the upcoming .NET explorer will only recognize C# tests.
It is actually not planned to support GDScript in VS Code, but you can write your own task and use the GdUnit4 cmd tool to run a specific test suite.
Sorry, my understanding was that the automatic test detection on Godot editor is missing and that you are working with C# + test API on VS Code.

[dreed-sd]

Oh, that's disappointing to hear that there won't be GDScript support in VSCode? Are we crazy for using GdScript with VSCode? That I would have thought is a common configuration. We are new to the engine, maybe it isn't the case. I found GdUnit by looking for unit test frameworks that have a VSCode plugin. It doesn't though, if you're on the latest, and it won't, if you're not using C#... I suggest updating the docs to be clear about that.

[MikeSchulze]

Well, primarily, I would see the Godot Editor for GDScripts. VS/Studio/Code should rather be used for C# scripts.
The Godot-plugin GdUnit4 supports both GDScripts and c# test execution.
The upcoming test explorer for VS will therefore only recognize and execute C# tests.
However, there is nothing to be said against creating a feature request to also support GDScript in VS, but this would currently be of very low priority as it is rather one of the exceptions to process GDScripts via VS.

[dreed-sd]

Ok thanks. We use VSCode because it's a superior code editor. The built-in Editor is great, but VSCode is just a much better experience.

[MikeSchulze]

Moved to milestone v4.3.0 it will be a bigger change set and needs a UI redesign.

[MikeSchulze]

Ok thanks. We use VSCode because it's a superior code editor. The built-in Editor is great, but VSCode is just a much better experience.

if you want to support execution of GDScript unit test on VSC please create a future request here, but this will have a very low priority. I need to focus on getting the C# API in good shape.