karate-demo

Karate Demo

⚠️ Note that this is not the best example of a skeleton Java / Maven project, as it is designed to be part of the Karate code-base and act as a suite of regression tests. For a good "starter" project, please refer to the Karate Examples

This is a sample Spring Boot web-application that exposes some functionality as web-service end-points. And includes a set of Karate examples that test these services as well as demonstrate various Karate features and best-practices.

Example	Demonstrates
greeting.feature	Simple GET requests and multiple scenarios in a test.
headers.feature	Multiple examples of header management including dynamic setting of headers for each request using a JS file (classpath:headers.js). Also demonstrates handling of cookies, and path / query parameters. There are also examples of how to set up a re-usable *.feature file for per-request secure / auth headers after a sign-in. OAuth 2, OAuth 1 and JWT samples are also available.
sign-in.feature	HTML form POST example. Typically you use the response to get an authentication token that can be used to build headers for subsequent requests. This example also demonstrates getting past an end-point protected against CSRF.
cats.feature	Great example of embedded-expressions (or JSON templating).
kittens.feature	Reading a complex payload expected response from a file. You can do the same for request payloads as well.
read-files.feature	The above example reads a file with embedded expressions and this one reads normal JSON and XML for use in a match. Also a good example of using the set and table keywords to build JSON (or XML) payloads from scratch.
graphql.feature	GraphQL example showing how easy it is to prepare queries and deal with the highly dynamic and deeply nested JSON responses - by focusing only on the parts you are interested in
upload.feature	Multi-part file-upload example, as well as comparing the binary content of a download. Also shows how to assert for expected response headers. Plus an example of how to call custom Java code from Karate.
dogs.feature	How to easily use Java interop to make a JDBC / database call from a Karate test. Here is the utility-class code - which depends on Spring JDBC: DbUtils.java. The same approach can be used to mix things like Selenium or WebDriver into Karate.
cats-java.feature	Another example of how to call Java code showing how you can pass JSON data around.
schema.feature	Karate's simpler approach to schema-validation compared with an actual JSON-schema example taken from json-schema.org. If you really want to perform JSON-schema validation, this example shows you how you can easily do so using Java interop and the json-schema-validator, but should you really ?
first.feature	Along with second.feature shows how you can use tags to selectively run tests or groups of tests from the command-line.
soap.feature	Examples for both SOAP 1.1 and 1.2, also showing how you can read request payloads and even expected response XML from files. Note that examples of how to manipulate XML are linked from the main documentation - for example this one: xml.feature
dynamic-params.feature	Multiple examples of data-driven testing including using a Scenario Outline and Examples - so that you can compare approaches. Since the params keyword takes JSON (and keys with null values are ignored), you can easily script different permutations of query parameters. This example also uses a JavaScript function (to simplify a custom assertion), which is defined in a separate file.
outline.feature	The Karate enhanced Scenario Outline makes data-driven tests JSON-friendly.
dynamic-csv.feature	Using CSV for a data-driven dynamic scenario outline.
cat.feature	So you like Karate's data-driven and assertion capabilities ? Try Karate for unit testing Java code ! You can re-use the glue code in multiple tests.
call-feature.feature	How you can re-use a sequence of HTTP calls in a *.feature file from other test scripts. This is hugely useful for those common authentication or 'set up' flows that create users, etc. Refer to the main documentation on how you can pass parameters in and get data back from the 'called' script.
call-json-array.feature	This example loads JSON data from a file and uses it to call a *.feature file in a loop. This approach can enable very dynamic data-driven tests, since there are a variety of ways by which you can create the JSON data, for example by calling custom Java code.
call-table.feature	This is a great example of how Karate combines with Cucumber and JsonPath to give you an extremely readable data-driven test. Karate's table keyword is a super-elegant and readable way to create JSON arrays, perfect for setting up all manner of data-driven tests.
call-dynamic-json.feature	Shows how to dynamically create a JSON array and then use it to call a *.feature file in a loop. In this example, the JSON is created using a JavaScript function, but it can very well be the response from an HTTP call, the result of a JsonPath expression or even a List of HashMap-s acquired by calling Java. This test actually calls a second *.feature file in a loop to validate a 'get by id'. Using JsonPath and match each to validate all items within a JSON array is also demonstrated.
call-once.feature	Cucumber has a limitation where Background steps are re-run for every Scenario and even for every Examples row within a Scenario Outline. This is a problem when you have expensive and time-consuming HTTP calls in your 'set-up' routines. Fortunately you have an elegant work-around with Karate's callonce keyword.
polling.feature	Retry support is built-in to Karate, but you can also achieve this by combining JavaScript functions with a call to another *.feature file.
websocket.feature	How to write websocket tests, also see echo.feature.
JavaApiTest.java	If you need to call a Karate test from Java code you can do so using the Java API. Also see this.
main.feature	You can take Java interop and Karate test-doubles (mocks) to extremes. This particular test is described here.

Configuration and Best Practices

File	Demonstrates
karate-config.js	Shows how the demoBaseUrl property is injected into all the test scripts on startup. Notice how JavaScript allows you to perform simple conditional logic and string manipulation, while still being a 'devops-friendly' plain-text file. It is good practice to set the connectTimeout and readTimeout so that your tests 'fail fast' if servers don't respond. For advanced users - you can even run a 'global' init routine using karate.callSingle().
TestBase.java	This is specific to Spring Boot, but this code takes care of starting the embedded app-server and dynamically chooses a free port. The chosen port value is passed to the above config routine via a Java System.setProperty() call.
DemoTestParallel.java	Karate has a utility to run tests in parallel and this does not depend on JUnit, TestNG or even Maven. A Cucumber JSON report file would be generated for each feature executed. You can easily configure your CI with the location of these files so that you get proper test-reports after a build. This is the recommended way of running Karate as part of an automated build or CI pipeline. Here, the (optional) third-party cucumber-reporting library is being used (see details below).
pom.xml	Look out for how the maven-surefire-plugin can be configured to point to what is basically your 'test-suite'. You may not even need to do this if you follow the recommended naming conventions and folder structure, and then Maven defaults would work as you would expect. Note that this demo application has many dependencies that you will not need for a typical Karate project.

Gradle

Refer to the wiki page: Gradle.

Example Report

This is optional because Karate's native HTML reports should serve all your needs. But you can ask Karate to emit the "Cucumber JSON" report data format, which can be consumed by third-party utilities in the Cucumber ecosystem.

Since the maven-cucumber-reporting plugin has an issue where reports will not be generated if the build fails, we recommend that you directly use the cucumber-reporting library programmatically in combination with the Karate parallel runner. Here is how:

Maven Dependency

Add the net.masterthought:cucumber-reporting jar as a dependency in test scope

<dependency>
    <groupId>net.masterthought</groupId>
    <artifactId>cucumber-reporting</artifactId>
    <version>5.3.1</version>
    <scope>test</scope>
</dependency>

Log4j Config File

If you don't already have log4j (v2) in the mix, place this minimal config on the classpath as log4j2.properties (in the same folder as karate-config.js).

log4j.rootLogger = INFO, CONSOLE
log4j.appender.CONSOLE = org.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.layout = org.apache.log4j.PatternLayout

Generate Report

Refer to the code in the demo: DemoTestParallel.java, specifically the generateReport() method. Note that outputCucumberJson(true) is called on the Runner "builder".

And here is the output, which goes into target/cucumber-html-reports if you follow the above steps:

Code Coverage using Jacoco

In the pom.xml, code coverage using Jacoco is also demonstrated. Since this is set-up as a Maven profile, instrumentation and code-coverage reporting would be performed only when you use the coverage profile. Note that code-coverage data (binary) would be saved to this file: target/jacoco.exec.

So to run tests and perform code-coverage:

mvn clean test -Pcoverage

And the HTML reports would be output to target/site/jacoco/index.html.

As this demo example shows - if you are able to start your app-server and run Karate tests in the same JVM process, code-coverage reports for even HTTP integration tests will be very easy to generate.

Code Coverage for non-Java Projects

This has been demonstrated for JavaScript by Kirk Slota. You can find a working sample here: karate-istanbul - and you can read the discussion at Stack Overflow for more details.

You should be able to use the same approach for other platforms. Note that there are plenty of ways to start a Karate test via the command-line, such as the standalone JAR.