vscode cucumber-language-server: Quoted parameters in steps cause undefined-step in editor

[c92s]

👓 What did you see?

When using the official Cucumber VSCode extension with the Python behave package, some steps with string variables are underlined and marked as "Undefined step".

✅ What did you expect to see?

A (string) parameter in Gherkin is passed in quotation marks as such: Given pass "foobar". The respective behave implementation is done as such:

@step('pass "{string}"')
def step_impl(context, string):
    assert string == "bar", f"pass <{string}> is not <bar>"

The problem is, that the Cucumber extension cannot find the implementation, as the quotation marks "{string}" are interpreted as part of the keyword. When removing them in the decorator, the quotation marks from the Gherkin keyword are not escaped and passed as part of the string. Therefor, the following implementation leads to an error:

@step("user {string}")
def step_impl(context, string):
    assert string == "foo", f"user <{string}> is not <foo>"    # --> Error Msg:  Assertion Failed: user <"foo"> is not <foo>

📦 Which tool/library version are you using?

VSCode: 1.88.0
Cucumber Extension: v1.10.0

🔬 How could we reproduce it?

Create a gherkin features/test_step.feature file:

Feature: Some new feature

  Scenario: Some Scenario
    Given user "foo"
    Given pass "bar"

Create the python step definition features/steps/steps.py:

from behave import step


@step("user {string}")
def step_impl(context, string):
    assert string == "foo", f"user <{string}> is not <foo>"

@step('pass "{string}"')
def step_impl(context, string):
    assert string == "bar", f"pass <{string}> is not <bar>"

Lastly, run it with $ behave features/

$ behave features/ 
Feature: Some new feature # features/test_step.feature:1

  Scenario: Some Scenario  # features/test_step.feature:3
    Given user "foo"       # features/steps/test_step.py:4 0.000s
      Assertion Failed: user <"foo"> is not <foo>

    Given pass "bar"       # None


Failing scenarios:
  features/test_step.feature:3  Some Scenario

📚 Any additional context?

I see two possibilities on how to solve this:

• Either the Cucumber language-server is able to detected decorators with parameters in quotation marks (e.g. @step('pass "{string}"')
• behave is able to strip quotation marks in the passed parameter, so decorators like this are possible @step("user {string}")

I am not sure, however, which may be the ideal solution. Maybe you can give me some feedback. I already posted the issue in the official behave repository, but was forwarded to the cucumber language-server repo.

[kieran-ryan]

Hey @c92s, thanks for your interest in the project 👋

There are some nuances to be aware of when using the extension with Behave; which I will describe below; and will hopefully get you up and running with the extension.

1. Ensure the extension settings match the paths where your feature files (features/**/*.feature) and step definitions (features/**/*.py) are located; otherwise you will need to configure the extension in a .vscode/settings.json in your workspace
2. Integrate Cucumber Expressions into Behave (Optional) - which is the step matcher used within Cucumber and offers improved compatibility with the extension compared to using Behave's default parser matcher
   • Note: {string} is a reserved parameter type with Cucumber Expressions which also matches the quotations; so you wouldn't need to specify them if switching e.g. @step('pass "{string}"') in Behave would just be @step('pass {string}') with Cucumber Expressions
3. Use Behave's matchers, but with the following considerations:
   • Straight matches should work without issue, such as @step('a string pattern without any parameters')
   • Use positional arguments instead of named arguments - rather than using named parameters in patterns, such as @step('user {string}'); use positional arguments (empty curly braces) in which the order they are defined is important e.g. @step('user {}') - which will run the same.
   • Use Cucumber Expressions named parameters in your patterns e.g. use {word} to match a single word and use that name in your step definition declaration e.g. def step_impl(context, word):; or other keywords similarly such as {int} for integers, {double} for floats, {string} for strings within quotations; etc.
   • Use Behave's regular expressions matcher - some support is provided for patterns written using regular expressions

Beyond detecting step definitions, the extension will detect steps in your feature files and provided completions for them when writing other specifications; which should work without issue.

The output window can provide greater insight into what the extension is doing and what matches it detected:

Mentioning your original issue in case others encounter so they can reference this answer: behave/behave#1173.

[c92s]

Thanks for your answer. I did the following according to your guidance.

1. The feature & step file path matching works: I can automatically create undefined steps from the feature file or jump right to the implementation.
2. behave-cucumber-matcher was not installed, as you said, it should work with the default {string} parameters as well.
3. I tested positional arguments without specifying the type, this works out of the box!

However, I am still wondering, on which side (behave or cucumber) the error resides, when using {string} / "{string}" in the @step() annotation.

Feature: Some new feature

  Scenario: Some Scenario
    Given user "foo"           # implementation gets recognized
    Given pass "bar"           # underlined with "undefined step"
    Given item baz             # implementation gets recognized

from behave import step

# {string} contains the quotation marks, this messes with the python 
# implementation. However, the cucumber language server recognizes this
# function.
@step("user {string}")
def step_impl(context, string):
    assert string == "foo", f"user <{string}> is not <foo>"

# This python implemenation works, however the cucumber language server does
# not recognize this function
@step('pass "{string}"')
def step_impl(context, string):
    assert string == "bar", f"pass <{string}> is not <bar>"

# This works -->
@step('item {}')
def step_given(context, string):
    assert string == "baz", f"item <{string}> is not <baz>"

The language server output contains the following

[Info  - 3:26:48 PM] * Found 1 feature file(s) in ["src/test/**/*.feature","features/**/*.feature","tests/**/*.feature","*specs*/**/*.feature"]
[Info  - 3:26:48 PM] * Found 3 steps in those feature files
[Info  - 3:26:48 PM] * Found 1 glue file(s) in ["*specs*/**/*.cs","features/**/*.js","features/**/*.jsx","features/**/*.php","features/**/*.py","features/**/*.rs","features/**/*.rb","features/**/*.ts","features/**/*.tsx","features/**/*_test.go","src/test/**/*.java","tests/**/*.py","tests/**/*.rs"]
[Info  - 3:26:48 PM] * Found 0 parameter types in those glue files
[Info  - 3:26:48 PM] * Found 3 step definitions in those glue files
[Info  - 3:26:48 PM] * Built 4 suggestions for auto complete

In here, the lang-server identified the 3 step definitions....

[kieran-ryan]

The extension evaluates {string} as a Cucumber Expression - which matches text surrounded by either single (') or double (") quotes.

Matching with {string}

As such, the following step definition pattern...

@step("user {string}")
def step_impl(context, string):
    assert string == "foo", f"user <{string}> is not <foo>"

... would match the following test steps:

Given user "foo"
Given user 'foo'

Behave will pass everything within the scope of {string} to the function body. This includes the quotes themselves; which is not what you intend in your assert; although you could modify the string you are checking against to also include the quotes.

Matching with "{string}"

Surrounding {string} with quotes creates a mismatch where Behave would only take the content within scope of the string (and work with your assert as intended); though as Cucumber Expressions expect {string} to also include quotes and you have them surrounding that keyword, you would need to include an additional set of quotes for the extension to create a match (see below) - which then wouldn't work with your assert unless modified as mentioned above.

@step('pass "{string}"')
def step_impl(context, string):
    assert string == "bar", f"pass <{string}> is not <bar>"

...would match the following test steps:

Given pass ""foo""
Given pass "'foo'"

Thus your existing step Given pass "bar" would pass only bar to the step definition function through the string argument; and the extension would not recognise the step definition pattern as matching this step as it expects the quotes to be part of {string} rather than surrounding it.

Alternatives

Use the Cucumber Expressions word keyword

word matches a single word only. Thus, you can surround it with quotes, and only the string content will be passed to the step definition; rather than also passing through the double quotes. This will work correctly with both the extension and Behave.

@step('pass "{word}"')
def step_impl(context, word):
    assert string == "bar", f"pass <{string}> is not <bar>"

Use the match anything {}

In both Behave and Cucumber Expressions, {} will match anything passed to it. Thus you could surround it with quotes; and only content within the quotes would be passed to the function.

@step('pass "{}"')
def step_impl(context, word):
    assert string == "bar", f"pass <{string}> is not <bar>"

Use Behave Cucumber Matcher

You can integrate Cucumber Expressions into Behave, which align that syntax with Cucumber Expressions. Note, you are using Cucumber Expressions syntax (such as {string}), but to behave this is treated as any other keyword argument (such as {name}, {book}, etc.) rather than having any special significance or association with quotes. This is understandably a source of confusion for many.

Summary

Hope this explanation is helpful, let me know how you get on @c92s 👍

[c92s]

Hi @kieran-ryan thanks for the information.

I guess I will stick to the {} and {word} matching then, escaping with double quotes or removing quotes in the function are not that nice to use tbh.