Add support for more languages

[metareven]

One of the main features of gherkin is the support for writing scenarios and features in your own language. It seems like this project only supports steps written in english.

Please add support for more languages.
A list of translations for the different keywords can be found here:
https://github.com/cucumber/cucumber/blob/master/gherkin/gherkin-languages.json

[AndreSteenveld]

I've run in to this issue as well and started looking in to it. Test case

After debugging for a bit I found a few things; When setting a breakpoint here you can see the file is parsed and returns what seems to be a perfectly fine GherkinDocument. It now seems its only a matter of making sure the keywords are interpreted correctly.

But if for what ever reason the parsing fails it does throw an error there detailing the problem but it never seems to surface anywhere to inform the developer there is a problem with a specific feature file. I ran in to this as Voorbeeld doesn't seem to be a good alias for Scenario even though the documentation does suggest that.

Will continue on this.

[AndreSteenveld]

After debugging some more I found the following snippet:

// StepMethodInfo in <>/source/Xunit.Gherkin.Quick/CoreModel/StepMethod/StepMethodInfo.cs
public ScenarioStepPattern GetMatchingPattern(Step gherkinScenarioStep)
{
    var gherkinStepText = gherkinScenarioStep.Text.Trim();

    foreach (var pattern in ScenarioStepPatterns)
    {
        if (!pattern.Kind.Matches(gherkinScenarioStep.Keyword.Trim()))
            continue;

        var match = Regex.Match(gherkinStepText, pattern.RegexPattern);
        if (!match.Success || !match.Value.Equals(gherkinStepText))
            continue;

        return pattern;
    }

    return null;
}

In the dutch test the value of gherkinScenarioStep.Keyword.Trim() is "Stel". When running pattern.Kind.Matches(...) it doesn't match anything in the enum as the Keyword is dutch and not "Given". For tonight I'm going to leave it here, I think I'm slowing getting there. The key is to make sure that we map the correct steps to the correct method infos.

[AndreSteenveld]

So after a little digging it seems the most reasonable thing to do is to start with updating the version of Gherkin used. A more recent version of version of the parser also exposes the step types, using this info it wouldn't be necessary to guess the type of step based on the (translated) keyword.

[ttutisani]

The problem with that approach is that Gherkin has breaking changes, so if you try to upgrade, that opens up a whole new can of worms. Maybe I'm mistaken, but that's what I remember from the last time I tried to upgrade.

[AndreSteenveld]

I just found that out... 😅. But moving Xunit.Gherkin.Quick to .netstandard2.0 and updating to Gherkin v23.0.1 only gives me 11 errors initially. (missing classes, missing overloads, etc) So that seems pretty reasonable to just plow through. From the looks a lot of the Scenario* classes seemed to be renamed or consolidated.

I'm having a little troubling finding a good change log for Gherkin, would you by chance know where to find one?

[ttutisani]

No, I don't know where to find that. In fact, I reached out to them with the request to document the new AST (abstract syntax tree, i.e., the new object model), but they haven't done it yet I assume.

[AndreSteenveld]

Ah, I think I found the commit with the relevant changes! the commit. It seems to rename ScenarioOutline to StepsContainer and also adds the examples. Anyways that is enough for tonight!

[AndreSteenveld]

I just did some brute force renaming to get the project to compile again and run the unit tests. And I have to say, the results are not half bad. This is obviously the easy 80 percent of this endeavour. 😃 (relevant commit)

[AndreSteenveld]

After fixing the background unit test I'm left with 5 integration tests which don't work:

Stel Als Dan :: Valideer de volgorde van de stappen still fails but that is as expected because the executor still looks at the keyword and not the step type.

Math with Infinity :: Infinity math with examples fails but I think this one fails intentionally to validate tests can actually fail in a meaningful way.

And Adding numbers to 5 :: Add various numbers to 5, AddTwoNumbers :: Add two numbers with examples and Placeholders Feature :: Placeholders in DataTables are replaced which are all scenario outlines and fail with similar error along the lines of: System.InvalidOperationException : Cannot match any method with step ``Then the DataTables MealExtra column should contain <Fruit>``. Scenario ``Placeholders in DataTables are replaced``.. These are the ones I'm currently look at and which are giving me the most trouble.

For example setting a breakpoint here(Xunit.Gherkin.Quick.Feature#ScenarioOutline(...)) never seems to trigger and there is a note in the UML diagram:

So my question here is, how fresh is this code? Am I looking at the right place?

[ttutisani]

Two things:

No test should fail, otherwise, the build will fail, which will not let me merge the PR. So, all tests must pass. Please download the original version of the code and verify, that there should not be any failures.

You are looking at an image, which is outdated (sorry). Here is the up-to-date diagram of the same topic:

[AndreSteenveld]

After giving this a little rest I continued today, I had a hunch that maybe XUnit was just confused on what tests to run so cleaning the entire thing and giving it another spin the ScenarioOutline on Feature is executed properly (subsequently triggering all the scenarios, which "all just work"). Although Now I'm puzzled because it seems that everything works with the exception of the aggregation of everything:

So I think I might have messed up the lookups, but I'm not completely sure how.

[AndreSteenveld]

HA! Checking out ttutisani/Xunit.Gherkin.Quick and just running that yields less tests (225 vs 230) and accounting that my fork contains one extra test (dutch) that leaves four more tests unaccounted for. Running all the tests we get the following result:

It seems that in my fork somehow the scenario outlines are discovered as regular scenarios, which are then executed but fail. So this is not a issue with the execution of the tests but the classification/discover of them.

[AndreSteenveld]

Yay, I fixed the scenario outline issue. It isn't super elegant but for now this does the job well. (commit) The only two things left now are handling the not implemented tests and feature files in other languages.

[AndreSteenveld]

So looking at the the "Math with infinity" it obviously fails because it is a scenario without examples (the examples table is @ignoreed). So like with languages the fix here should be trying to figure out what kind of scenario (outline or regular) it is, fixing that should fix the language related issues as well.

I'm initially going to do this without changing too much to the discovery and distinction between scenarios and outlines but looking at what Gherkin is doing it does seem like a good idea to fold these in to one.

[ttutisani]

Well, Gherkin's design is not exemplary (with all due respect). The idea behind the Xunit.Gherkin.Quick's design is that it should reflect the concepts that exist within the BDD context. If we combine Scenario Outline and Scenario, my initial concern is that it will add abstraction rather than clarify what is really going on. There must be strong design reason (not Gherkin's approach) to go with one or another pattern.

[AndreSteenveld]

YAY, I got mu dutch test working! It is also funny that with the exception of the "Math with infinities" all tests in the ProjectConsumer project pass and quite a significant number of unit tests fail. The solution itself is also not the most polished for now... It is still a work in progress. As for any performance issues (due to the iteration over bunch of arrays), I'm not seeing much of a impact. (relevant commit)

[AndreSteenveld]

Done! Everything passes, I want to add a few more tests in the project and translate these to emjoi-lang so not to favor any specific langauge. Maybe do some more clean up here and there.

I'm going to open a PR.

[AndreSteenveld]

I have tested a local build of this package on the project where I intend to use it in and ran in to some weirdness with other languages. So although all tests pass now I am going to translate every feature file in Xunit.Gherkin.Quick.ProjectConsumer to emoji. Simply to validate that everything works. I am not completely sure what is going on but it seems that not all components of a feature get parsed correctly.

A pattern I found to do make the same Feature class work for multiple files is by creating a child class which is then decorated with the [FeatureFile(...)] attribute, something along the lines of:

public partial class Letter : Feature {
    
    [FeatureFile("A.feature")] public class A : Letter { }
    [FeatureFile("B.feature")] public class B : Letter { }

    //
    // Given, When. Then methods go here
    //
}

[AndreSteenveld]

Dang it! So I translated all the tests, created a subclass for all of them and they all pass 😭. I thought I had something with the missing functionality but it just worked (or was a typo from my side).

[AndreSteenveld]

I found the issue was! And it didn't have anything to do with the workings of Xunit.Gherkin.Quick but with how NuGet works and I packaged it locally. So to test if this solved the issue of having dutch test for the project I'm currently working on I just ran dotnet pack --configuration Release and told NuGet there was a local feed in the bin directory.

When looking at it through the NuGet browser in visual studio it does say it has the correct dependencies but when installing it it just picks up the public package from NuGet.org. (That was because they are both version 4.2.0, I think) Upping the version number made sure the correct (local) package was installed. Which also, just worked 😄

I'm going to take a look at appveyor later, there is quite a section on how to automagically bump the version number when creating a release build.