Disable RuntimeAndVersion

[jnyrup]

As appending RuntimeAndVersion for received is now on by default, is there a way to disable it again?

[SimonCropp]

out of curiosity. why?

[jnyrup]

I have a test suite that verifies images and upon failure it additionally takes verified and received and creates a visual diff which highlights the changed pixels to ease visual inspection of the differences.

While I could easy change that code to look for DotNet6_0.received I'd like not to have to update that when I update the the TFM of the test project.

[SimonCropp]

I have a test suite that verifies images and upon failure it additionally takes verified and received and creates a visual diff which highlights the changed pixels to ease visual inspection of the differences.

but that is an OOTB feature of Verify. wht do u have custom code that does this?

[jnyrup]

By OOTB are you then referring to Verify.ImageSharp.Compare?
I did not write this part of our code myself, but as I recall we did start out using Verify.ImageSharp.Compare to generate the diff, but wanted another kind of visual diff.

For this we use ImageSharp and loads in verified and received from disk, write the diff to disk and attach all three images to the test with TestContext.AddResultFile.

[jnyrup]

Found a workaround for us using AsyncLocal<T> so we could use OnVerifyMismatch together with the per test method TestContext.

[TestClass]
public class TestClass : VerifyBase
{
    private static readonly AsyncLocal<TestContext> s_testContext = new();

    [ClassInitialize]
    public static void ClassInitialize(TestContext context)
    {
        VerifierSettings.OnVerifyMismatch((filePair, _) =>
        {
            var verified = filePair.VerifiedPath;
            var received = filePair.ReceivedPath;
            var diff = GenerateDiff(verified, received);

            var testContext = s_testContext.Value;
            testContext.AddResultFile(verified);
            testContext.AddResultFile(received);
            testContext.AddResultFile(diff);

            return Task.CompletedTask;
        });
    }

    [TestInitialize]
    public void TestInitialize() => s_testContext.Value = TestContext;

    private static string GenerateDiff(string verified, string received)
    {
        var diff = verified.Replace(".verified.png", ".diff.png");
        // ...
        return diff;
    }
}

[kseitz10]

This surprised me today and I had to spend some time hunting down why I was getting test failures for contracts that should not have test failures. I too would also prefer to turn it off because the project I work on doesn't need to be built for different platforms or runtimes.

Changing defaults is potentially a breaking change and should be better documented in the release notes and/or made obvious through commit messages (instead of just a . as a message... c'mon...)

[SimonCropp]

@kseitz10

This surprised me today and I had to spend some time hunting down why I was getting test failures for contracts that should not have test failures.

changing the temp received file name should not have caused test failures. hence i did not consider it a breaking change

Changing defaults is potentially a breaking change and should be better documented in the release notes and/or made obvious through commit messages (instead of just a . as a message... c'mon...)

please remember this is a free project. and i am doing the best i can in my free time. ... c'mon..

[kseitz10]

changing the temp received file name should not have caused test failures

Fair, I might be conflating two different issues. I need to figure out whether VerifierSettings.OnFirstVerify's behavior has changed and if that has anything to do with this. We copy the received file over to the verified file's location on first verify, which is what broke while I simultaneously observed the new names showing up.

this is a free project

Oh, I know, and your efforts are appreciated -- but that doesn't change my opinion :) If I ultimately find something odd here that needs fixin', I'll try to get a PR out though.

[SimonCropp]

what are u using OnFirstVerify for?

[kseitz10]

We were automating the acceptance/approval of the first received contract file instead of having to accept it manually. The latest beta resolves the issue for us.

[SimonCropp]

can u try 18.0.0-beta.8

[jnyrup]

I can confirm that 18.0.0-beta.8 restores the previous received file path.
Thanks ❤️

[SimonCropp]

it now detects if a project is multi-targeting