Reusable steps across features?

[deyanp]

Can you confirm that currently it is not possible to define reusable steps in a separate implementation file, to which several features can be mapped?

I see that there is an inheritance-based approach chosen, however it requires all steps to be in 1 file, and at a first glance does not allow to have several features "sharing" them ...

[ttutisani]

Hi @deyanp , I don't exactly understand the question. What do you mean exactly by the reusable steps? Please elaborate and I will confirm or explain.

Also, you said that the inheritance-based approach requires all steps to be in one file, I didn't understand that part too.

Sorry, I may be slow to understand the context which you probably have at hand.

Examples or code blocks would be helpful.

[deyanp]

Thanks for the quick reply, and you are right, I did not explain the issue well. Let me try:

1. I am using F# with Xunit.Gherkin.Quick
2. Imagine you have 2 Feature files with 1 Scenario in each. The scenarios are different, however there is 1 Given (or When, or Then) which is the same across the 2 scenarios in the 2 different files
3. I want to implement this scenario only once, and have Xunit.Gherkin.Quick find it in the assembly, wherever it is defined.
4. In order to have a shared step I have to create a base classe, have both feature classes inherit from it (see SharedSteps.fs, AddTwoNumbers class)
5. The base class/type must be in a single file in F#. It is possible in F# to extend it in a different file (see https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/type-extensions#optional-type-extensions), but if I do that (see SharedSteps2.fs; AddTwoNumbersRoot which is an extension to the SharedSteps.fs, AddTwoNumbersRoot class) then the step is not found when I run dotnet test

See a sample repo at https://github.com/deyanp/XunitGherkinTests, and specifically the 2nd commit to it

[ttutisani]

Honestly, I have no idea how F#'s inheritance works. In C#, the shared steps would be placed in a base class, and there is no limitation to put the base class in a separate file. That would result in having the same shared steps as part of the derived classes. I was always assuming that the inheritance would be the same across all .NET languages.

So a couple of questions:

• AddTwoNumbers derives from AddTwoNumbersRoot which has the Given step in it. I assume that the given step in the AddTwoNumbersRoot is correct and exactly copy-pasted from the commented code block which I see in AddTwoNumbers. So, is that Given step found by the test runner or not?
• SharedSteps2.fs also has AddTwoNumbersRoot defined in it. Is that a partial class then? (in C#, when you split the class across two files, it's called "partial").
• What exact error message do you receive when you try to run the tests? You should use dotnet test command for it.

[deyanp]

F# inheritance works the same as in C#. What I don't want to have is 1 single base class with 100 steps inside, which are shared then across 20 features/child classes. I would rather have steps grouped into several (base) classes, and reused across the 20 feature (classes).

To your questions:

• Yes, the Given is found in the base class

• Yes, this is called Optional Type Extensions in F# (see https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/type-extensions#optional-type-extensions), which means is that you can extend a class with instance methods in another file/project, but the client should reference both namespaces, otherwise the extension is not found.

• see below - Basically the runner does not find the type extension in SharedSteps2.fs, which means that the only way to have shared steps between multiple scenarios/features is to put them in SharedSteps.fs, which means a single huge base class ...

PS C:\Users\Deyan\TryProjects\XUnitTests> dotnet test
Test run for C:\Users\Deyan\TryProjects\XUnitTests\bin\Debug\netcoreapp2.2\XUnitTests.dll(.NETCoreApp,Version=v2.2)
Microsoft (R) Test Execution Command Line Tool Version 16.2.0-preview-20190606-02
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...

[xUnit.net 00:00:00.59]     Scenario(scenarioName: "Add three numbers") [FAIL]

[xUnit.net 00:00:00.60]     Scenario(scenarioName: "Add two numbers") [FAIL]

  X Scenario(scenarioName: "Add three numbers") [20ms]
  Error Message:
   System.InvalidOperationException : Cannot match any method with step `And I chose 15 as second number`. Scenario `Add three numbers`.
  Stack Trace:
     at System.Linq.Enumerable.SelectArrayIterator`2.ToList()
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
--- End of stack trace from previous location where exception was thrown ---

  X Scenario(scenarioName: "Add two numbers") [2ms]
  Error Message:
   System.InvalidOperationException : Cannot match any method with step `And I chose 15 as second number`. Scenario `Add two numbers`.
  Stack Trace:
     at System.Linq.Enumerable.SelectArrayIterator`2.ToList()
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
--- End of stack trace from previous location where exception was thrown ---


Test Run Failed.
Total tests: 3
     Passed: 1
     Failed: 2
 Total time: 1,2777 Seconds
PS C:\Users\Deyan\TryProjects\XUnitTests> 

[ttutisani]

Okay, I understand more now. Thanks for explaining the split between files.

I looked into the optional type extensions documentation and that looks like a way to define extension methods. However, it's an extension method and not the type itself. In other words, extension methods and partial classes are two different things. In C# for example, there is an extension method and there is partial class too. Partial class is a developer's convenience to split the class into pieces. Extension method is a way to attach a method to the class which you have not defined. This is an important difference - partial class is compiled into a single class, while extension is attached after compilation so it is a separate class (that's why it's useful when you want to attach a method to the class that you did not define - taken from the documentation). Although the syntax of F# extension looks similar to the C#'s partial, those are two different things.

I also tried to find a way to define partial classes in F#, and it seems there is no way. It's a limitation of the F# language and compiler, unfortunately. So, no, you cannot split your base class across files in F#. Again, this is the F# programming language limitation and has nothing to do with the framework. Base class is one class in .NET world in general, it's only a question of whether your programming language can compile a single base class from several files. After compilation it's a single class no matter the language.

[deyanp]

My question was not so much about splitting F# classes/partials, but rather the following: You do have some logic in the library to find the implementation of all steps, so would it be possible to change that logic to search in the whole assembly, instead of a single feature class (hierarchy)?

[ttutisani]

No, currently there is no way to split the steps across several feature classes.

How many shared steps do you expect across your derived feature classes? And how many derived classes will you have with the same shared step?

[deyanp]

I am starting a new project, so no statistics yet, but on my previous project (using SpecFlow) we had about 10 000 scenarios, with about 5-10 per feature ... There were hundreds of reusable steps, and some of the steps were even "scoped" to certain features only, as there were several steps matching the same regex attribute.

[ttutisani]

I see. This sounds like a good candidate for the implementation of the shared steps. I will keep the issue open and will work on it in the order of priority. This may take a couple of months.

If you would like to contribute, or fork and implement locally for your use, feel free to do so.

Sorry about the inconvenience. You are the first consumer who has such use case which is worth the implementation.

[ttutisani]

Meanwhile, I wanted to suggest one more thing, maybe it helps: you can also chain the classes, so have a base of base of base. Each base adds several shared steps, and all features derive from the most specific base. Obviously, this is a trade-off and not ideal because you will need to split methods into hierarchy and be careful from which base you want to derive. Just a thought.

[deyanp]

I fully understand, time is a constraint for everybody. Thank you nevertheless for the quick interaction on the issues!
Once the new project settles a bit I will try to find time to look at your sources.

P.S. Xunit.Gherkin.Quick is a really interesting project!

[ttutisani]

Sure, thanks!

I have also an additional question, which helps with implementing.

If there are shared steps, how will it know where to look for? Currently, the feature class specifies the feature file path. I want to avoid scanning the entire library to find the match. Any other thoughts?

[deyanp]

You mean scanning the whole assembly would be slow, right.
I can only refer you to SpecFlow's ScopeAttribute: https://github.com/techtalk/SpecFlow/wiki/Scoped-bindings - you can put it on the steps and scope them to certain features etc, but this is from step->feature. From feature->steps I don't see a way, besides tagging with attributes the text in the *.feature file itself ... Maybe there is a way to cache the assembly metadata, or retrieve it in a relatively fast way?

[deyanp]

But consider also this: https://cucumber.io/docs/guides/anti-patterns/#feature-coupled-step-definitions

[ttutisani]

I kind of disagree with their definition of anti-pattern. Sharing steps across scenarios seems more anti-pattern than keeping them separate. That is because (especially with the domain's specific focus in mind) each domain may understand the step differently. Sharing is not good in general in domain oriented architectures such as DDD or MDD.

That said, the more I think about it, the more I understand that sharing steps would be maybe confusing, unless carefully and properly implemented. Needs to be thought through before jumping to implementation. Maybe something will become clear over time.

P.S.: I am a believer that my framework should help design better systems, that's why I want to be careful. Shared steps is a dangerous field.

[deyanp]

Well, this is not the most straightforward topic, but

1. Simple logic would be search for step with the attribute in the whole assembly, if less or more than 1 found -> error
2. Better logic would be - if more than 1 found, have a simple algorithm about locality - the closest step to the feature will be chosen.

[ttutisani]

That introduces indirect, implicit, convention based design, which means guesswork and more issues open by confused developers. I want to avoid any "magic" and rather make things explicit, like what [FeatureFIle] attribute does.

[deyanp]

I am against magic myself, however I am also trying to be pragmatic.
The general question here is at what level do you want to reuse, and how fast do you want to be when writing new scenarios using the some domain objects (having different feature sets for different bounded contexts is anyway given).

Option 1): You want to be very fast, and even assemble new scenarios without writing a single line of code.
Option 2): You are OK to write the steps new every time, and just delegate the execution to "Helper" or "Driver" classes behind (push the reuse down 1 layer). Your steps implementations could even end up in single lines, which will turn them in a simple mapping layer (actually this is the workaround I am having in mind, not against it)

I have observed that people are happy with option 1 .. but I am not saying option 2 is a nogo, it will involve more effort (incl. maintenance) in the long run.

The problem with option 1 is that people may start fighting on the Single True Step Definition (as English statement).
The problem with option 2 is that different English statements will map to the same logic below, and it may get really difficult managing a big test suite. Other frameworks allow for years mapping of different regexes to the same step impl. method/function though.

My personal opinion is that going only with the FeatureFile (= Option 2) may work for very small test suites (and of course demo projects), however for any mid-size or big test suite you will start wondering how to quickly construct new scenarios without writing any code.

[glassenbury]

I'll add my opinion on this. After years of using specflow with c#, I have found global steps can easily get out of control and duplication of similar functional gets created. This is hard to police when in a team of people writing tests. With xunit.gherkin.quick I have used 3 level inheritance type steps with a base steps for whole project, then an 'area' base steps then a feature steps. The steps files are normally just a few lines per step that use other classes that are reusable. This makes it easy to debug as all steps are in 1-3 steps files.
We converted from a specflow set of tests and it was a good time to sort the steps out. There was a lot of global steps and code removing making the 1000 tests easier to maintain and debug as you look at one contained feature /steps at a time. We have another test project with 600 tests, which are backend tests. This is using one steps file, but there is only approximately 15 steps. All the data are in files of json.

[deyanp]

That was useful, @glassenbury!
I am still not convinced about the inheritance stuff (I am a great fan of "use composition over inheritance"), but I am thinking more and more of having 1-liner steps delegating everything to reusable Helpers/Drivers structured into as many classes I need. Of course, I have to repeat the same step mapping for different features, but it will be just a 1-liner, so not too bad ...

[alastair-todd]

There's always going to be basic stuff like "login as this persona", "make an authenticated request to this endpoint", "lookup xyz common data". But seems fine to have that in an abstract class

[ttutisani]

I wanted to briefly chime in and clarify that the code reuse across features is possible via two options: either via the base class or via dependency injection (because Xunit supports it). I was able to write up documentation covering both approaches here: https://github.com/ttutisani/Xunit.Gherkin.Quick/blob/master/docs/reuse-step-implementation-across-features.md

I hope this helps.