Question: Pass/Fail criteria checked in Python Script

[bryankerr1986]

What if I want to a Python Script controlling the Simulation.

For example, what if I want to compare results of two .txt files produced by TB in python and use that knowledge to determine if the testbench passed. Can I do this?

Or what if I want to use Python to control the run of the testbench. I.E. Loop through the same testbench two times with different parameters. Can I do this?

[LarsAsplund]

Yes, this is possible. If you read the latest blog https://vunit.github.io/blog/2023_08_26_vhdl_configurations.html you will find information about VUnit configurations which is the tool to run the same test with different generics/parameters. If you look at the documentation for add_config you will also find an option to have a post_check Python function executed after the test exits simulation. That function can compare the text files. If the function returns False, the test will fail.

[bryankerr1986]

Thank you, this was very helpful. I was able to accomplish both of these tasks with ease using VUnit.

Follow up question...
If I changed testbench parameters 30 times to run 30 different senarios, then I now have 30 separate testbenches under 'python3 run.py --list'. Is it possible to log a comment about each scenario in the python run script so when I type 'python3 run.py --list', then all 30 scenarios will be printed with a short description of what the testbench does? If this is possible, is it also possible to save this in html, or some other, format?

[LarsAsplund]

What I usually do is to insert the parameter(s) in the name of the configuration. Something like:

for parameter in range(30):
    testbench.add_config(name=f"parameter={parameter}", generics=dict(parameter=parameter))

You can also use the attributes parameter to add_config. For example attributes={".description", "This is a description for my configuration"}. See also https://vunit.github.io/py/ui.html#attributes

The attributes for all tests can be exported with the --export-json option. That JSON file can then be parsed to build a HTML page for example.