Skip to content

1.0 upgrade guide

Jump to bottom
Peter Thomas edited this page Jul 25, 2023 · 111 revisions

To skip to what changes to expect, click here. This page is an extension of the 1.0.0 release notes, and please note that 1.0.1 has been released with bug fixes.

Binary artifacts (the ZIP release and stand-alone JAR files) are no longer offered via BinTray, they will only be downloadable from the GitHub releases page.

Improvements

Also see this article by Peter Quiel: 7 New Features in Karate Test Automation Version 1.0.

Support for all Java versions 8 and above

Achieved by moving to the Graal VM JS engine. This was a big risk we were facing when Oracle deprecated Nashorn which Karate was using extensively.

We have tested Karate on even Java 16 and it works ! Along with this migration, we were also able to clean up the code significantly - which means that Karate is well set to be around for a long, long time !

ECMAScript 2015 / ES6 - most notably the arrow operator

image
  • ES6 arrow functions
  • string interpolation within back-ticks
  • JS array behavior e.g. Array.map(), note that karate.map() is still an option

Improved error details for match failures

image
  • each level of the payload is reported as well-formed JSON (to make cutting and pasting easier)
  • JSON keys will be sorted so that the payloads "line up" as far as possible - which makes it easier to eyeball
  • the Json-Path (or XPath) of each "level" appears on the left, with a clear description of the mis-match

Improved error details for JavaScript failures

Yes we know. Especially for failures where karate-config.js was involved, the log messages could be woefully inadequate in some cases. That's a thing of the past now, with details of the JS source and feature names + line numbers.

Improved test-suite builder

image

the improvements to the Runner.builder() are subtle but you are likely to enjoy:

  • no more relying on System.setProperty()
  • set karate.env
  • set even the location of the karate-config.js
  • any existing reports (<build-dir>/karate-reports / configurable) will be backed up to not be over-written
  • and even set system properties per test-suite
  • the goal here is to allow you to run 2 instances of karate.env in parallel if you wish

Multi-Part request details will appear in the HTML report / logs

No more guesswork as to what that multi-part request was doing behind the scenes. MIME message names, Content-Type headers and payload sizes displayed for each "part".

Playwright Support

see: https://twitter.com/ptrthomas/status/1307678474627244032

Java API

This means you can choose to use Java instead of Gherkin, and still get the benefits of Karate's rich HTTP, JSON, Match (assertions) and Driver (browser automation) capabilities.

Note that you will sacrifice some of the test-automation domain capabilities of Karate, such as HTML reports with in-line logging, and parallel-execution. But this is an option that may appeal to passionate Java developers who would like to mix and match with other frameworks such as TestNG. Some areas need more work, such as being able to re-use Java tests as performance tests, so please contribute if you can.

We recommend that you look at jbang for some extreme scripting options. Did you know that jbang allows you to install scripts as local command-line applications ? It opens up some very interesting possibilities !

JS Functions for Data Driven Testing

see: https://twitter.com/KarateDSL/status/1343201222396821504

Distributed Testing

see: https://github.com/intuit/karate/wiki/Distributed-Testing

HTML Templating

see: https://twitter.com/KarateDSL/status/1338892932691070976 example: axe.feature

App Server (experimental)

see: https://twitter.com/ptrthomas/status/1338797269844312064

Retry Framework (experimental)

With some custom Java code you should be able to take any failed tests at the end of a Runner invocation and re-try them, and merge the results back into the final aggregated report. See example code.

Single Dependency

The karate-core Maven artifact contains everything you need, including the Apache HTTP Client which used to be a separate dependency. This simplifies certain use-cases, the one we are most excited about is being able to use jbang - and here are some examples of what is possible.

Note that Karate now bundles an HTTP Client, HTML templating, HTTP app-server, REST server, CLI executor and even web-browser automation into one mighty package - which opens up some very interesting automation possibilities.

Don't forget that Karate has additional dependencies for performance testing, mocking Java servlets and Windows desktop app automation as well.

Breaking Changes

Overall, it is quite likely that your API tests will run just like before, but see details of possible behavior changes.

Java Projects

  • only a single maven dependency is needed, karate-apache and karate-jersey do not exist any more
    • which means that you need only karate-junit5 (or karate-junit4, or just karate-core if you don't want JUnit at all, see single dependency)
    • so just remove the extra dependency from your pom.xml (typically karate-apache)
    • Karate will include the Apache HTTP Client by default, but we "maven shade" it so that you never run into library conflicts
  • HTML reports (and other artifacts) will be in target/karate-reports (or build/karate-reports for Gradle)
    • so if your CI was pointing to <build-dir>/surefire-reports, this has to be changed
  • the @KarateOptions annotation will be removed in the next version and has been marked as "deprecated"
    • so try to switch to the Runner (example below)
  • the Cucumber JSON and JUnit XML files are NOT output by default
    • use the builder methods on the Runner, there is also outputJunitXml(true) 
        Results results = Runner.path("classpath:demo")
                .outputCucumberJson(true)
                .tags("~@ignore").parallel(5);

JUnit 5

  • unlikely, but if you used the new Karate().feature('some.feature') builder, use Karate.run('some.feature') instead, since the class-level method feature() has been removed. Note that all the methods on the Runner builder (see example above) are now available, giving you full control over system-properties, karate.env, report output folder etc. For example:
    @Karate.Test
    Karate testSystemProperty() {
        return Karate.run("classpath:karate/tags.feature")
                .tags("@second").systemProperty("foo", "bar");
    }

Standalone JAR

  • HTML reports (and other artifacts) will be in target/karate-reports (not target/cucumber-reports or target/surefire-reports)
  • The cucumber-reporting library is no longer used (the HTML report is Karate's built-in one)
  • The Cucumber JSON and JUnit XML are not generated by default, use -f cucumber:json or -f junit:xml (or both at the same time, comma-delimited works)

Mocks

  • a.k.a. karate-netty - the Java API to start a mock has changed / see diff
  • the CLI to start a mock is unified with karate-core (there is no more karate-netty)
  • so this will work: java -jar karate.jar -m mock.feature
  • the --watch mode CLI option short-form is upper-case -W (not -w which now means "working directory")
  • you can even use the karate-core maven artifact via jbang, see this example
  • "Proxy Mode" or intercepting HTTPS does not work (but used to), refer open Armeria issue: https://github.com/line/armeria/issues/3168 - please contribute if you can !

Hooks

To see a sample (part of a larger example including Gradle, Gatling etc. by Kirk Slota) see this file: KarateHook.

Mock Servlet

  • karate-mock-servlet has been re-factored to use a new HttpClientFactory abstraction.

Karate UI

  • the (rarely used) driver.dialog "getter" API that gets you the text content of a browser-native pop-up has been renamed to driver.dialogText
  • if you implemented a custom Target, the method signature for start() and stop() takes a com.intuit.karate.core.ScenarioRuntime as the single argument instead of a Logger. This is future-proof and gives you more control, this diff will explain the details.

Other

  • if you were one of the rare few who implemented (or extended) a custom HttpClient - it has been refactored to be simpler, look at ArmeriaHttpClient as an example

Async

If you have async tests and are passing JS functions to Java listeners (e.g. message queues, gRPC) you need to use the new karate.toJava() API to "wrap" the function so that it is compatible with the JS engine (see image below)

image

And, the karate.listen() API has been removed, instead use the new listen keyword, and here is an example.

image

These changes are because of the new JS engine a) not allowing JS to be shared between two contexts and b) not allowing 2 threads to access the same context, refer: https://github.com/graalvm/graaljs/issues/59

Details and Examples

Breaking changes due to JS engine change

There are not too many, and it is highly likely that your tests will be fine.

Trailing semicolons in JS blocks not allowed

Before:

* def uuid = function(){ return java.util.UUID.randomUUID() + '' };

After:

* def uuid = function(){ return java.util.UUID.randomUUID() + '' }

JSON variables no longer mutable by called features

see diff

Scenario: called feature updates a nested element of 'foo' using the 'set' keyword
    * def foo = { key: 'value' }
    # improved in karate 1.0 - json cannot be mutated in called features
    * def result = call read('copy-called.feature')
    # so callers cannot mutate this context !
    * match foo == { key: 'value' }

print and karate.log() will fail if variables do not exist

For example if foo is not defined, this will not work:

* print 'value of foo:', foo

Note that you can do karate.get('foo', 'default') to specify a default value

Multi-Value params and headers

Use a JSON array, not just a comma-delimited list for header, param, form field etc.

before:

* param myParam = 'foo', 'bar'

after:

* param myParam = ['foo', 'bar']

Java API-s for Maps and Lists are no longer "visible" within JS blocks

    * def json = { a: 1, b: 2, c: 3 }
    * def map = karate.toBean(json, 'java.util.HashMap')
    # no longer possible
    * def count = map.size()
    * match count == 3

so array.size() will no longer work.

use karate.sizeOf(map) instead (recommended) while the JS way array.length should also work for an assert.

and, list.add() will not work, use array.push() instead.

    * def foo = []
    * foo.push('a')
    * match foo == ['a']

also, list.contains() will not work, use array.includes() instead.

    * def allowed = ['Music', 'Entertainment', 'Documentaries', 'Family']
    * def actual = ['Entertainment', 'Family']
    * match each actual == '#? allowed.includes(_)'

and a very rare case but in case you were trying to use a java.util.Properties() directly, that won't work any more:

    * def stream = karate.readAsStream(path)
    * def props = new java.util.Properties()
    * props.load(stream)

JS Engine is stricter about "truthy" values

here below, function(x){ return x } is not good enough

    * def isNotZero = function(x){ return x != 0 }
    * def result = karate.filter([0, 1, 2], isNotZero)
    * match result == [1, 2]

Java in JS

Objects originating from Java but when you are within a JS block will not "behave" like JSON. This applies only if you are calling Java code within a JS file (typically karate-config.js). The karate.toJson() API can be used as a solution.

Also refer: https://github.com/intuit/karate/issues/1469

before: (keep in mind this is a block of JS code)

  var JavaDemo = Java.type('com.myco.JavaDemo');
  var jd = new JavaDemo();
  var configMapFailure = jd.doWork("fromJS");
  configMapFailure['project'] = {name: "DEMO PROJECT"}

Even though jd.doWork() returns a java.util.Map, the last line above will NOT work.

after:

  var JavaDemo = Java.type('com.myco.JavaDemo');
  var jd = new JavaDemo();
  var configMapFailure = karate.toJson(jd.doWork("fromJS"));
  configMapFailure['project'] = {name: "DEMO PROJECT"}

JS to Java

The reverse of the above situation needs to be considered in the edge case where you have created an array in JS but need to pass it to Java code that expects a standard List. Use karate.toJava(), and here is an example:

image

Type Conversion

  • Most notably, integer results from math will not become doubles with decimal points, this used to be quite annoying !
  • new karate.fromString() to auto convert from JSON or XML strings within JS blocks
  • new karate.typeOf() to detect what "type" a given object is

see diff

JavaBean property short-cuts don't work in all cases

Before:

* def encoded = Base64.encoder.encodeToString('hello'.bytes)

After:

* def encoded = Base64.encoder.encodeToString('hello'.getBytes())

And if you are using a variable that is a string, you may have to call toString() on it first:

function fn(creds) {
     var temp = creds.username + ':' + creds.password;
     var Base64 = Java.type('java.util.Base64');
     var encoded = Base64.getEncoder().encodeToString(temp.toString().getBytes());
     return 'Basic ' + encoded;
}

One way to understand what's happening above is that when you are within "round brackets" the special handling for JavaBean naming conventions cannot be applied by the JS engine.

no more auto converting json or xml for "path" expressions

None of the examples here will work: compat.feature

  • convert strings to JSON before attempting to extract values
  • karate.get() needs the '$' prefix if attempting JsonPath
  • and attempting XPath needs the $ (or /) prefix as well

karate.info deprecated

use: karate.scenario or karate.feature instead.

karate.scenario now returns a rich JSON that contains all the info you would possibly need, here is a sample:

{
  "sectionIndex": 0,
  "stepResults": [
  ],
  "line": 4,
  "description": "",
  "durationMillis": 0.0,
  "failed": false,
  "tags": [
    "ignore"
  ],
  "executorName": "main",
  "name": "bar",
  "startTime": 1611595320114,
  "refId": "[1:4]",
  "endTime": 0,
  "exampleIndex": -1
}

notes:

  • an error property will be present if there was an error with a message. you can also use the failed boolean as a check.
  • if the exampleIndex is 0 or greater, it is a Scenario Outline example-row
  • and the sectionIndex is the index of the Scenario or Scenario Outline within the feature file

And if you need metadata about the current feature file: karate.feature returns a JSON like this:

{
  "fileName": "test.feature",
  "prefixedPath": "classpath:some/package/test.feature",
  "parentDir": "/some/path/some/package",
  "name": "foo",
  "description": ""
}

Get Started | Documentation | IntelliJ Plugin | VS Code Extension

Clone this wiki locally