A highly flexible and configurable tool for testing interactions of SOA applications with web services (REST, SOAP, WSDL etc.) over HTTP(S) protocol. It is an actual HTTP server (stubby4j uses embedded Jetty) that allows stubbing of external systems with ease for integration, contract & behavior testing. Please refer to Key features for more information
It is a stub HTTP server after all, hence the "stubby". Also, in Australian slang "stubby" means beer bottle
- Quick start example
- Key features
- Why would a developer use stubby4j
- Why would a QA use stubby4j
- Building
- Third-party dependencies
- Adding stubby4j to your project
- Command-line switches
- Endpoint configuration HOWTO
- Performance optimization index
- The admin portal
- The stubs portal
- Programmatic API
- Change log
- Roadmap
- Authors
- Kudos
- See also
This section explains how to get stubby4j up and running using a very simple example "Hello, World", without building stubby4j from source locally using Gradle.
Minimum system requirements to run stubby4j archives hosted on Maven Central
- version >= 4.0.0: Oracle JRE v1.8.0_60
- version >= 3.0.0: Oracle JRE v1.7.0_76
- version = 2.0.22: Oracle JRE v1.7.0_04
- version < 2.0.22: Oracle JRE 1.6.0_65-b14-462
- Download the latest stubby4j version (the JAR archive).
- Create the following local YAML file:
- request:
method: GET
url: /hello-world
response:
status: 200
headers:
content-type: application/json
body: Hello World!
- Execute the downloaded stubby JAR using command
java -jar stubby4j-x.x.xx.jar -d <PATH_TO_YOUR_CREATED_LOCAL_YAML_FILE>
- Navigate to
http://localhost:8882/hello-world
to get the stubbed response "Hello World!" - Navigate to stubby4j admin portal at
http://localhost:8889/status
to see what has been stubbed & other useful data
That's it!
For more information and more complex examples, please dive into the rest of documentation, especially Endpoint configuration HOWTO
- Emulate external webservice in a SANDBOX for your application to consume over HTTP(S)
- HTTP request verification and HTTP response stubbing
- Regex support for dynamic matching on URI, query params, headers, POST payload (ie:.
mod_rewrite
in Apache) - Dynamic token replacement in stubbed response, by leveraging regex capturing groups as token values during HTTP request verification
- Record & Replay. The HTTP response is recorded on the first call, having the subsequent calls play back the recorded HTTP response, without actually connecting to the external server
- Dynamic flows. Multiple stubbed responses on the same stubbed URI to test multiple application flows
- Fault injection, where after X good responses on the same URI you get a bad one
- Serve binary files as stubbed response content (images, PDFs. etc.)
- Embed stubby4j to create a web service SANDBOX for your integration test suite
- Simulate responses from real server and don't care (or cannot) to go over the network
- Third party web service your application suppose to contract with is not ready yet
- Verify that your code makes HTTP requests with all the required parameters and/or headers
- Verify that your code correctly handles HTTP error codes
- You want to trigger response from the server based on the request parameters over HTTP or HTTPS
- Support for any of the available HTTP methods
- Simulate support for different types of HTTP Authorizations: Basic, Bearer Token & others
- Support for HTTP 30x redirects
- Provide canned answers in your contract/integration tests
- Enable delayed responses for performance and stability testing
- Avoid to spend time coding for the above requirements
- Concentrate on the task at hand
- Specifiable mock responses to simulate page conditions without real data.
- Ability to test polling mechanisms by stubbing a sequence of responses for the same URI
- Easily swappable data config files to run different data sets and responses.
- All-in-one stub server to handle mock data with less need to upkeep code for test generation
stubby4j is a multi-module Gradle project
Run gradle
command to:
- Clean
- Run unit, integration and functional tests without Cobertura
- Build (the generated JAR artifacts will be located under
<PROJECT_ROOT>/build/libs/
)
Run gradle cobertura
command to:
- Clean
- Generate Cobertura report under the
<PROJECT_ROOT>/main/build/reports/cobertura/
- javax.servlet-api-3.1.0.jar
- jetty-server-9.4.8.v20171121.jar
- jetty-servlets-9.4.8.v20171121.jar
- commons-cli-1.2.jar
- snakeyaml-1.17.jar
- jsonassert-1.3.0.jar
- xmlunit-1.6.jar
- json-20090211.jar
- slf4j-api:1.7.25.jar
The following are the stubby4j artifacts that are hosted on Maven Central:
stubby4j-x.x.x.jar
- anuber
JAR containing all the 3rd-party depsstubby4j-x.x.x-no-dependencies.jar
- askinny
JAR containing no 3rd-party dependencies at allstubby4j-x.x.x-no-jetty.jar
- anuber-ish
JAR containing all the 3rd-party deps except Jetty binariesstubby4j-x.x.x-sources.jar
stubby4j-x.x.x-javadoc.jar
compile("io.github.azagniotov:stubby4j:5.1.1")
or by adding a classifier
to the JAR name like no-dependencies
or no-jetty
, i.e.:
compile("io.github.azagniotov:stubby4j:5.1.1:no-jetty")
<dependency>
<groupId>io.github.azagniotov</groupId>
<artifactId>stubby4j</artifactId>
<version>5.1.1</version>
</dependency>
or by adding a classifier
to the JAR name like no-dependencies
or no-jetty
, i.e.:
<dependency>
<groupId>io.github.azagniotov</groupId>
<artifactId>stubby4j</artifactId>
<version>5.1.1</version>
<classifier>no-dependencies</classifier>
</dependency>
Run gradle install
command to:
- Install
stubby4j-5.1.2-SNAPSHOT*.jar
to local~/.m2/repository
- All the artifacts will be installed under
~/.m2/repository/{groupId}/{artifactId}/{version}/
, e.g.:~/.m2/repository/io/github/azagniotov/stubby4j/5.1.2-SNAPSHOT/
Now you can include locally installed stubby4j SNAPSHOT
artifacts in your project:
compile("io.github.azagniotov:stubby4j:5.1.2-SNAPSHOT")
or by adding a classifier
to the JAR name like no-dependencie
s or no-jetty
, i.e.:
compile("io.github.azagniotov:stubby4j:5.1.2-SNAPSHOT:no-jetty")
usage:
java -jar stubby4j-x.x.xx.jar [-a <arg>] [-d <arg>] [-da] [-ds]
[-h] [-k <arg>] [-l <arg>] [-m] [-o] [-p <arg>] [-s <arg>] [-t
<arg>] [-v] [-w]
-a,--admin <arg> Port for admin portal. Defaults to 8889.
-d,--data <arg> Data file to pre-load endpoints. Valid YAML
1.1 expected.
-da,--disable_admin_portal Does not start Admin portal
-ds,--disable_ssl Does not enable SSL connections
-h,--help This help text.
-k,--keystore <arg> Keystore file for custom TLS. By default TLS
is enabled using internal keystore.
-l,--location <arg> Hostname at which to bind stubby.
-m,--mute Mute console output.
-o,--debug Dumps raw HTTP request to the console (if
console is not muted!).
-p,--password <arg> Password for the provided keystore file.
-s,--stubs <arg> Port for stub portal. Defaults to 8882.
-t,--tls <arg> Port for TLS connection. Defaults to 7443.
-v,--version Prints out to console stubby version.
-w,--watch Periodically scans for changes in last
modification date of the main YAML and
referenced external files (if any). The flag
can accept an optional arg value which is
the watch scan time in milliseconds. If
milliseconds is not provided, the watch
scans every 100ms. If last modification date
changed since the last scan period, the stub
configuration is reloaded
This section explains the usage, intent and behavior of each property on the request
and response
objects.
Here is a fully-populated, unrealistic endpoint:
- description: Optional description shown in logs
request:
url: ^/your/awesome/endpoint$
method: POST
query:
exclamation: post requests can have query strings!
headers:
content-type: application/xml
post: >
<!xml blah="blah blah blah">
<envelope>
<unaryTag/>
</envelope>
file: tryMyFirst.xml
response:
status: 200
latency: 5000
headers:
content-type: application/xml
server: stubbedServer/4.2
body: >
<!xml blah="blah blah blah">
<responseXML>
<content></content>
</responseXML>
file: responseData.xml
- Description field which can be used to show optional descriptions in the logs
- Useful when you have a number of stubs loaded for the same endpoint and it starts to get confusing as to which is being matched
- description: Stub one
request:
url: ^/one$
method: GET
response:
status: 200
latency: 100
body: 'One!'
- description: Stub two
request:
url: ^/two$
method: GET
response:
status: 200
latency: 100
body: 'Two!'
- request:
url: ^/three$
method: GET
response:
status: 200
latency: 100
body: 'Three!'
This object is used to match an incoming request to stubby against the available endpoints that have been configured.
- is a full-fledged regular expression
- This is the only required property of an endpoint.
- signify the url after the base host and port (i.e. after
localhost:8882
). - must begin with
/
. - any query paramters are stripped (so don't include them, that's what
query
is for)./url?some=value&another=value
becomes/url
- no checking is done for URI-encoding compliance.
- If it's invalid, it won't ever trigger a match.
This is the simplest you can get:
- request:
url: /
A demonstration when not using regular expressions:
- request:
url: /some/resource/that/will/be/fully/matched
A demonstration using regular expressions:
- request:
url: ^/has/to/begin/with/this/
- request:
url: /has/to/end/with/this/$
- request:
url: ^/must/be/this/exactly/with/optional/trailing/slash/?$
- request:
url: ^/[a-z]{3}-[a-z]{3}/[0-9]{2}/[A-Z]{2}/[a-z0-9]+$
- defaults to
GET
. - case-insensitive.
- can be any of the following:
- HEAD
- GET
- POST
- PUT
- POST
- DELETE
- etc.
- request:
url: /anything
method: GET
- it can also be an array of values.
- request:
url: /anything
method: [GET, HEAD]
- request:
url: /anything
method:
- GET
- HEAD
- can be a full-fledged regular expression
- if not stubbed, stubby ignores query parameters on incoming request and will match only request URL
- stubby accommodates for HTTP requests that contain query string params with no values
- query params can be specified regardless of their order in incoming request. In other words - order agnostic
- query params can also be an array with double/single quoted/un-quoted elements:
attributes=["id","uuid"]
orattributes=[id,uuid]
. Please note no spaces between the CSV
- request:
method: GET
url: ^/with/parameters$
query:
type_name: user
client_id: id
client_secret: secret
random_id: "^sequence/-/\\d/"
session_id: "^user_\\d{32}_local"
attributes: '["id","uuid","created","lastUpdated","displayName","email","givenName","familyName"]'
- The following will match either of these:
/with/parameters?search=search+terms&filter=month
/with/parameters?filter=month&search=search+terms
- request:
url: ^/with/parameters$
query:
search: search terms
filter: month
- The following will match either of these:
/with/parameters?search&filter=month
/with/parameters?search=&filter=month
- request:
url: ^/with/parameters$
query:
search:
filter: month
- The following will match:
- From the browser:
http://localhost:8882/with/parameters?term=boo+and+foo
- From the browser:
http://localhost:8882/with/parameters?term=boo%2Band%2Bfoo
- From the browser:
http://localhost:8882/with/parameters?term=boo and foo
- From the browser:
http://localhost:8882/with/parameters?term=boo%20and%20foo
- From the code:
String request = "http://localhost:8882/with/parameters?term=boo+and+foo"
- From the code:
String request = "http://localhost:8882/with/parameters?term=boo%2Band%2Bfoo"
- From the code:
String request = "http://localhost:8882/with/parameters?term=boo and foo"
- From the code:
String request = "http://localhost:8882/with/parameters?term=boo%20and%20foo"
- From the browser:
- request:
url: ^/with/parameters$
query:
term: "boo and foo"
- The following will match:
- From the browser:
http://localhost:8882/with/parameters?term=['stalin+and+truman']
- From the browser:
http://localhost:8882/with/parameters?term=['stalin+and+++++++++++truman']
- From the browser:
http://localhost:8882/with/parameters?term=['stalin%2Band%2Btruman']
- From the browser:
http://localhost:8882/with/parameters?term=['stalin and truman']
- From the browser:
http://localhost:8882/with/parameters?term=['stalin%20and%20truman']
- From the code:
String request = "http://localhost:8882/with/parameters?term=%5B%27stalin%2Band%2Btruman%27%5D"
- From the code:
String request = "http://localhost:8882/with/parameters?term=%5B%27stalin+++++and+truman%27%5D"
- From the code:
String request = "http://localhost:8882/with/parameters?term=%5B%27stalin and truman%27%5D"
- From the code:
String request = "http://localhost:8882/with/parameters?term=%5B%27stalin%20and%20truman%27%5D"
- From the browser:
- request:
url: ^/with/parameters$
query:
term: "['stalin and truman']"
- Represents the body POST of incoming request, ie.: form data
- can be a full-fledged regular expression
- if not stubbed, any POSTed data on incoming request is ignored
- request:
url: ^/post/form/data$
post: name=John&email=john@example.com
- request:
method: [POST]
url: /uri/with/post/regex
post: "^[\\.,'a-zA-Z\\s+]*$"
- request:
url: ^/post/form/data$
post: "^this/is/\\d/post/body"
- request:
method: POST
url: /post-body-as-json
headers:
content-type: application/json
post: >
{"userId":"19","requestId":"(.*)","transactionDate":"(.*)","transactionTime":"(.*)"}
response:
headers:
content-type: application/json
status: 200
body: >
{"requestId": "<%post.1%>", "transactionDate": "<%post.2%>", "transactionTime": "<%post.3%>"}
- request:
method: POST
url: /post-body-as-json-2
headers:
content-type: application/json
post: >
{"objects": [{"key": "value"}, {"key": "value"}, {"key": {"key": "(.*)"}}]}
response:
headers:
content-type: application/json
status: 200
body: >
{"internalKey": "<%post.1%>"}
- holds a path to a local file (it can be an
absolute
orrelative
path to the main YAML specified in-d
or--data
). This property allows you to split up stubby data across multiple files instead of making one huge bloated main config YAML. For example, let's say you want to stub a big POST payload, so instead of dumping a lot of text under thepost
property, you could specify a local file with the payload using thefile
property:
- request:
method: POST
headers:
content-type: application/json
file: ../json/post.payload.json
- please note, if both
file
&post
properties are supplied, thefile
takes precedence & replacespost
with the contents from the provided file. - if
--watch
command-line argument was supplied during startup, then any modifications to the supplied local file infile
(e.g.file: ../json/post.payload.json
) will cause the whole configuration to be reloaded. - if the local file could not be loaded for whatever reason (ie.: not found), stubby falls back to
post
for matching. - please keep in mind:
SnakeYAML
library (used by stubby4j) parser ruins multi-line strings by not preserving system line breaks. Iffile
property is stubbed, the file content is loaded as-is, in other words - it does not go through SnakeYAML parser. Therefore it's better to load big POST content for request usingfile
property. Keep in mind, stubby4j stub server is dumb and does not use smart matching mechanism (i.e.: don't match line separators or don't match any white space characters) - whatever you stubbed, must be POSTed exactly for successful match. Alternatively you can consider using regular expression inpost
- request:
url: ^/match/against/file$
file: postedData.json
post: '{"fallback":"data"}'
postedData.json
{"fileContents":"match against this if the file is here"}
- if
postedData.json
doesn't exist on the filesystem when/match/against/file
is matched in incoming request, stubby will match post contents against{"fallback":"data"}
(frompost
) instead.
- can be a full-fledged regular expression
- if not stubbed, stubby ignores headers on incoming request and will match only request URL
- if stubbed, stubby will try to match only the supplied headers and will ignore other headers of incoming request. In other words, the incoming request must contain stubbed header values
- headers are case-insensitive during matching
- a hashmap of header/value pairs similar to
query
.
The following endpoint only accepts requests with application/json
post values:
- request:
url: /post/json
method: post
headers:
content-type: application/json
x-custom-header: "^this/is/\d/test"
x-custom-header-2: "^[a-z]{4}_\\d{32}_(local|remote)"
stubby supports regex stubbing for dynamic matching on the following properties:
url
, query
param values, header
name values, post
payloads & file
names & payloads.
Under the hood, stubby first attempts to compile the stubbed pattern into an instance of java.util.regex.Pattern
class using the Pattern.MULTILINE
flag. If the pattern compilation fails and PatternSyntaxException
exception is thrown, stubby compiles the stubbed pattern into an instance of java.util.regex.Pattern
class using the Pattern.LITERAL | Pattern.MULTILINE
flags.
Please note, before using regex patterns in stubs, first it is best to ensure that the desired regex pattern "works" outside of stubby. One of the safest (and easiest) ways to test the desired pattern would be to check if the following condition is met: Pattern.compile("YOUR_PATTERN").matcher("YOUR_TEST_STRING").matches() == true
.
The latter would ensure that the stubbed regex pattern actually works, also it is easier to debug a simple unit test case instead of trying to figure out why stub matching failed
- request:
url: ^/path/to/basic$
method: GET
headers:
# no "Basic" prefix nor explicit encoding in Base64 is required when stubbing,
# just plain username:password format. Stubby internally encodes the value in Base64
authorization-basic: "bob:password"
response:
headers:
Content-Type: application/json
status: 200
body: Your request with Basic was successfully authorized!
- request:
url: ^/path/to/bearer$
method: GET
headers:
# no "Bearer" prefix is required when stubbing, only the auth value.
# Stubby internally does not modify (encodes) the auth value
authorization-bearer: "YNZmIzI2Ts0Q=="
response:
headers:
Content-Type: application/json
status: 200
body: Your request with Bearer was successfully authorized!
- request:
url: ^/path/to/custom$
method: GET
headers:
# custom prefix name is required when stubbing, followed by space & auth value.
# Stubby internally does not modify (encodes) the auth value
authorization-custom: "CustomAuthorizationType YNZmIzI2Ts0Q=="
response:
headers:
Content-Type: application/json
status: 200
body: Your request with custom authorization type was successfully authorized!
Assuming a match has been made against the given request
object, data from response
is used to build the stubbed response back to the client.
- Can be a single response or a sequence of responses.
- When sequenced responses is configured, on each incoming request to the same URI, a subsequent response in the list will be sent to the client. The sequenced responses play in a cycle (loop). In other words: after the response sequence plays through, the cycle restarts on the next incoming request.
- request:
method: [GET,POST]
url: /invoice/123
response:
status: 201
headers:
content-type: application/json
body: OK
- request:
method: [GET]
url: /uri/with/sequenced/responses
response:
- status: 201
headers:
content-type: application/json
body: OK
- status: 201
headers:
content-stype: application/json
body: Still going strong!
- status: 500
headers:
content-type: application/json
body: OMG!!!
- request:
method: [GET]
url: /uri/with/sequenced/responses/infile
response:
- status: 201
headers:
content-type: application/json
file: ../json/sequenced.response.ok.json
- status: 201
headers:
content-stype: application/json
file: ../json/sequenced.response.goingstrong.json
- status: 500
headers:
content-type: application/json
file: ../json/sequenced.response.omfg.json
- request:
method: [GET]
url: /uri/with/single/sequenced/response
response:
- status: 201
headers:
content-stype: application/json
body: Still going strong!
- the HTTP status code of the response.
- integer or integer-like string.
- defaults to
200
.
- request:
url: ^/im/a/teapot$
method: POST
response:
status: 420
- contents of the response body
- defaults to an empty content body
- can be a URL (OAUTH is not supported) to record & replay. The HTTP response is recorded on the first call to stubbed
url
, having the subsequent calls play back the recorded HTTP response, without actually connecting to the external server
- request:
url: ^/give/me/a/smile$
response:
body: ':)'
- request:
url: ^/give/me/a/smile$
response:
status: 200
body: >
{"status": "hello world with single quote"}
headers:
content-type: application/json
- request:
method: GET
url: /atomfeed/1
response:
headers:
content-type: application/xml
status: 200
body: <?xml version="1.0" encoding="UTF-8"?><payment><paymentDetail><invoiceTypeLookupCode/></paymentDetail></payment>
- request:
url: /1.1/direct_messages.json
query:
since_id: 240136858829479935
count: 1
response:
headers:
content-type: application/json
body: https://api.twitter.com/1.1/direct_messages.json?since_id=240136858829479935&count=1
- similar to
request.file
, holds a path to a local file (it can be anabsolute
orrelative
path to the main YAML specified in-d
or--data
). This property allows you to split up stubby data across multiple files instead of making one huge bloated main config YAML. For example, let's say you want to render a large response body upon successful stub matching, so instead of dumping a lot of text under thebody
property, you could specify a local file with the response content using thefile
property (btw, thefile
can also refer to binary files):
response:
status: 200
headers:
content-type: application/json
file: ../json/response.json
- please note, if both
file
&body
properties are supplied, thefile
takes precedence & replacesbody
with the contents from the provided file - if
--watch
command-line argument was supplied during startup, then any modifications to the supplied local file infile
(e.g.file: ../json/response.json
) will cause the whole configuration to be reloaded. - if the file could not be loaded, stubby falls back to the value stubbed in
body
- if
body
was not stubbed, an empty string is returned by default - it can be ascii of binary file (PDF, images, etc.). Please keep in mind, that file is preloaded upon stubby4j startup and its content is kept as a byte array in memory. In other words, response files are not read from the disk on demand, but preloaded.
- request:
url: /
response:
file: extremelyLongJsonFile.json
- similar to
request.headers
except that these are sent back to the client. - by default, header
x-stubby-resource-id
containing resource ID is returned with each stubbed response. The ID is useful if the returned resource needs to be updated at run time by ID via Admin portal
- request:
url: ^/give/me/some/json$
response:
headers:
content-type: application/json
body: >
[{
"name":"John",
"email":"john@example.com"
},{
"name":"Jane",
"email":"jane@example.com"
}]
- time to wait, in milliseconds, before sending back the response
- good for testing timeouts, or slow connections
- request:
url: ^/hello/to/jupiter$
response:
latency: 800000
body: Hello, World!
During HTTP request verification, you can leverage regex capturing groups as token values for dynamic token replacement in stubbed response.
- request:
method: [GET]
url: ^/regex-fileserver/([a-z]+).html$
response:
status: 200
file: ../html/<% url.1 %>.html
- request:
method: [GET]
url: ^/account/(\d{5})/category/([a-zA-Z]+)
query:
date: "([a-zA-Z]+)"
headers:
custom-header: "[0-9]+"
response:
status: 200
body: Returned invoice number# <% url.1 %> in category '<% url.2 %>' on the date '<% query.date.1 %>', using header custom-header <% headers.custom-header.0 %>
The url
regex ^/account/(\d{5})/category/([a-zA-Z]+)
has two defined capturing groups: (\d{5})
and ([a-zA-Z]+)
, query
regex has one defined capturing group ([a-zA-Z]+)
. In other words, a manually defined capturing group has parenthesis around it.
Although, the headers
regex does not have capturing groups defined explicitly (no regex sections within parenthesis), its matched value is still accessible in a template (keep on reading!).
The tokens in response
body
follow the format of <%``PROPERTY_NAME``.``CAPTURING_GROUP_ID``%>
. If it is a token that should correspond to headers
or query
regex match, then the token structure would be as follows: <%``HEADERS_OR_QUERY``.``KEY_NAME``.``CAPTURING_GROUP_ID``%>
. Whitespace is allowed between the <%
& %>
and what's inside.
When giving tokens their ID based on the count of manually defined capturing groups within regex, you should start from 1
, not zero (zero reserved for token that holds full regex match) from left to right. So the leftmost capturing group would be 1
and the next one to the right of it would be 2
, etc.
In other words <% url.1 %>
and <% url.2 %>
tokens correspond to two capturing groups from the url
regex (\d{5})
and ([a-zA-Z]+)
, while <% query.date.1 %>
token corresponds to one capturing group ([a-zA-Z]+)
from the query
date
property regex.
In regex world, capturing groups can contain capturing sub-groups, as an example consider proposed url
regex: ^/resource/``(``([a-z]{3})``-``([0-9]{3})``)``$
. In the latter example, the regex has three groups - a parent group ([a-z]{3}-[0-9]{3})
and two sub-groups within: ([a-z]{3})
& ([0-9]{3})
.
When giving tokens their ID based on the count of capturing groups, you should start from 1
, not zero (zero reserved for token that holds full regex match) from left to right. If a group has sub-group within, you count the sub-group(s) first (also from left to right) before counting the next one to the right of the parent group.
In other words tokens <% url.1 %>
, <% url.2 %>
and <% url.3 %>
correspond to the three capturing groups from the url
regex (starting from left to right): ([a-z]{3}-[0-9]{3})
, ([a-z]{3})
and ([0-9]{3})
.
Tokens with ID zero can obtain full match value from the regex they reference. In other words, tokens with ID zero do not care whether regex has capturing groups defined or not. For example, token <% url.0 %>
will be replaced with the url
full regex match from ^/account/(\d{5})/category/([a-zA-Z]+)
. So if you want to access the url
full regex match, respectively you would use token <% url.0 %>
in your template.
Another example, would be the earlier case where headers
custom-header
property regex does not have capturing groups defined within. Which is fine, since the <% headers.custom-header.0 %>
token corresponds to the full regex match in the header
custom-header
property regex: [0-9]+
.
It is also worth to mention, that the full regex match value replacing token <% query.date.0 %>
, would be equal to the regex capturing group value replacing <% query.date.1 %>
. This is due to how the query
date
property regex is defined - the one and only capturing group in the query
date
regex, is also the full regex itself.
You can specify template with tokens in both body
as a string or using file
by specifying template as external local file. When template is specified as file
, the contents of local file from file
will be replaced.
Alternatively, you can also template the path to the file itself:
- request:
method: [GET]
url: ^/regex-fileserver/([a-z]+).html$
response:
status: 200
file: ../html/<% url.1 %>.html
When the request is recieved and the regex matches, the path to the file will get resolved and the file content will be served if it exists.
- request:
method: POST
url: /post-body-as-json
headers:
content-type: application/json
post: >
{"userId":"19","requestId":"(.*)","transactionDate":"(.*)","transactionTime":"(.*)"}
response:
headers:
content-type: application/json
status: 200
body: >
{"requestId": "<%post.1%>", "transactionDate": "<%post.2%>", "transactionTime": "<%post.3%>"}
Another example demonstrating the usage of tokens from the matched regex groups
After successful HTTP request verification, if your body
or contents of local file from file
contain tokens - the tokens will be replaced just before rendering HTTP response.
- Make sure that the regex you used in your stubby4j configuration actually does what it suppose to do. Validate that it works before using it in stubby4j
- Make sure that the regex has capturing groups for the parts of regex you want to capture as token values. In other words, make sure that you did not forget the parenthesis within your regex if your token IDs start from
1
- Make sure that you are using token ID zero, when wanting to use full regex match as the token value
- Make sure that the token names you used in your template are correct: check that property name is correct, capturing group IDs, token ID of the full match, the
<%
and%>
If body
of the stubbed response
contains a URL starting with http(s), stubby knows that it should record an HTTP response
from the provided URL (before rendering the stubbed response) and replay the recorded HTTP response on each subsequent call.
- request:
method: [GET]
url: /maps/api/geocode/json
query:
address: "1600%20Amphitheatre%20Parkway,%20Mountain%20View,%20CA"
sensor: false
response:
status: 200
headers:
content-type: application/json
body: http://maps.googleapis.com
Upon successful HTTP request verification, properties of stubbed request
(method
, url
, headers
, post
and query
) are used to construct
an HTTP request to the destination URL specified in body
of the stubbed response
.
In the above example, stubby will record HTTP response received after submitting an HTTP GET request to the url below:
http://maps.googleapis.com/maps/api/geocode/json?sensor=false&address=1600+Amphitheatre+Parkway,+Mountain+View,+CA
- Recorded HTTP response is not persistable, but kept in memory only. In other words, upon stubby shutdown the recording is lost
- Make sure to specify in
response
body
only the URL, without the path info. Path info should be specified inrequest
url
stubby4j uses a number of techniques to optimize evaluation of stubs
During parsing of stubs config, the request.url
, request.query
, request.headers
& request.post
(or request.file
)
values are checked for presence of regex. If one of the aforementioned properties is a stubbed regex, then a regex pattern
will be compiled & cached in memory. This way, the pattern(s) are compiled during config parsing, not stub evaluation.
On every incoming request, a local cache holding previously matched stubs is checked to see if there is a match for the
incoming request URI. If the incoming URI found in the cache, then the cached matched stub & the incoming request are
compared to each other to determine a complete equality based on the stubbed request
properties.
If a complete equality against the cached stub was not achieved, the incoming request is compared to all other stubs loaded in memory. If a full match was found, then that match will be cached using the incoming request URI as a key.
The admin portal is a RESTful(ish) endpoint running on localhost:8889
. Or wherever you described through stubby's command line args.
Submit POST
requests to localhost:8889
at runtime OR load a data-file (using non-optional -d
/ --data
flags) with the following structure for each endpoint:
description
: optional description shown in logsrequest
: describes the client's call to the servermethod
: GET/POST/PUT/DELETE/etc.url
: the URI regex string. GET parameters should also be included inline herequery
: a key/value map of query string parameters included with the request. Query param value can be regex.headers
: a key/value map of headers the server should respond to. Header value can be regex.post
: a string matching the textual body of the response. Post value can be regex.file
: if specified, returns the contents of the given file as the request post. If the file cannot be found at request time, post is used instead
response
: describes the server's response (or array of responses, refer to the examples) to the clientheaders
: a key/value map of headers the server should use in it's response.latency
: the time in milliseconds the server should wait before responding. Useful for testing timeouts and latencyfile
: if specified, returns the contents of the given file as the response body. If the file cannot be found at request time, body is used insteadbody
: the textual body of the server's response to the clientstatus
: the numerical HTTP status code (200 for OK, 404 for NOT FOUND, etc.)
- description: "this is a feature describing something"
request:
url: ^/path/to/something$
method: POST
headers:
authorization-basic: "bob:password"
x-custom-header: "^this/is/\d/test"
post: this is some post data in textual format
response:
headers:
Content-Type: application/json
latency: 1000
status: 200
body: Your request was successfully processed!
- request:
url: ^/path/to/bearer$
method: POST
headers:
authorization-bearer: "YNZmIzI2Ts0Q=="
post: this is some post data in textual format
response:
headers:
Content-Type: application/json
status: 200
body: Your request with Bearer was successfully authorized!
- request:
url: ^/path/to/anotherThing
query:
a: anything
b: more
custom: "^this/is/\d/test"
method: GET
headers:
Content-Type: application/json
post:
response:
headers:
Content-Type: application/json
Access-Control-Allow-Origin: "*"
status: 204
file: path/to/page.html
- request:
url: ^/path/to/thing$
method: POST
headers:
Content-Type: application/json
post: this is some post data in textual format
response:
headers:
Content-Type: application/json
status: 304
JSON is a subset of YAML 1.2, SnakeYAML (Third-party library used by stubby4j for YAML & JSON parsing) implements YAML 1.1 at the moment. It means that not all the JSON documents can be parsed. Just give it a go.
[
{
"description": "this is a feature describing something",
"request": {
"url": "^/path/to/something$",
"post": "this is some post data in textual format",
"headers": {
"authorization-basic": "bob:password" // for basic authorization DO NOT base64 encode when stubbing
},
"method": "POST"
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"latency": 1000,
"body": "Your request was successfully processed!"
}
},
{
"request": {
"url": "^/path/to/anotherThing",
"query": {
"a": "anything",
"b": "more"
},
"headers": {
"Content-Type": "application/json"
},
"method": "GET"
},
"response": {
"status": 204,
"headers": {
"Content-Type": "application/json",
"Access-Control-Allow-Origin": "*"
},
"file": "path/to/page.html"
}
},
{
"request": {
"url": "^/path/to/thing$",
"headers": {
"Content-Type": "application/json"
},
"post": "this is some post data in textual format",
"method": "POST"
},
"response": {
"status": 304,
"headers": {
"Content-Type": "application/json"
}
}
}
]
If you want to load more than one endpoint via file, use either a JSON array or YAML list (-) syntax. When creating or updating one stubbed request, the response will contain Location
in the header with the newly created resources' location
Performing a GET
request on localhost:8889
will return a YAML list of all currently saved responses. It will reply with 204 : No Content
if there are none saved.
Performing a GET
request on localhost:8889/<id>
will return the YAML object representing the response with the supplied id.
You can also view the currently configured endpoints by going to localhost:8889/status
If for some reason you do not want/cannot/not able to use --watch
flag when starting stubby4j (or cannot restart stubby),
you can submit GET
request to localhost:8889/refresh
(or load it in a browser) in order to refresh the stubbed data.
Perform PUT
requests in the same format as using POST
, only this time supply the id in the path. For instance, to update the response with id 4 you would PUT
to localhost:8889/4
.
Send a DELETE
request to localhost:8889/<id>
Send a DELETE
request to localhost:8889
Requests sent to any url at localhost:8882
(or wherever you told stubby to run) will search through the available endpoints and, if a match is found, respond with that endpoint's response
data
For a given endpoint, stubby only cares about matching the properties of the request that have been defined in the YAML. The exception to this rule is method
; if it is omitted it is defaulted to GET
.
For instance, the following will match any POST
request to the root url:
- request:
url: /
method: POST
response: {}
The request could have any headers and any post body it wants. It will match the above.
Pseudocode (StubRepository#matchStub):
if (<incoming request>.url found in <previous matched cache>) {
get <cached stubbed endpoint> from <previous matched cache> by <incoming request>.url
if (<cached stubbed endpoint> == <incoming request>) {
return <cached stubbed endpoint>
}
}
for each <stubbed endpoint> of stored endpoints {
for each <property> of <stubbed endpoint> {
if (<stubbed endpoint>.<property> != <incoming request>.<property>) {
next stubbed endpoint
}
}
store in <previous matched cache> the found <stubbed endpoint> by url
return <stubbed endpoint>
}
You can start-up and manage stubby4j with the help of StubbyClient
- Added ANSITerminal back. it is operational alongside the SLF4J for the cases when stubby is running as a standalone jar
- Upgraded from Jetty
9.4.6.v20170531
to9.4.8.v20171121
- Pull request #83 - ANSITerminal was replaced with SLF4J. It is up to the stuby4j consumer to choose their own logging implementation (https://github.com/asarkar)
- Pull request #77 - New field
description
for stubs/features (https://github.com/goughy000)
- Pull request #71 - Add endpoint to Delete all stubs (https://github.com/nningego)
- Pull request #74 - Adding parsing tokens capabilities to the file body (https://github.com/OtavioRMachado)
- Pull request #76 - Allow custom XML and JSON content types (https://github.com/goughy000)
- Upgraded from Jetty
9.4.0.v20161208
to9.4.6.v20170531
- 2017 release: a lot of internal maintenance such as code clean up, refactoring & improved test coverage
- Supporting additional 3xx redirect HTTP codes when rendering redirect response:
303
,307
&308
- Pull request #63 - Dynamic token replacement is also applied to stubbed response headers
- Upgraded from Jetty
9.3.13.v20161014
to9.4.0.v20161208
- Added dependency on https://github.com/azagniotov/collection-type-safe-converter
- 'Builder' sub-project got merged into the 'Main' sub-project
- Upgraded from Jetty
9.3.12.v20160915
to9.3.13.v20161014
- Shaved off stubby's start-up time due to parsing YAML config asynchronously
- Issue #61 - During record & play, the stubbed query params were sent with recording request instead of the actual request query params
- Optimized the stub matching algorithm by caching the previous matches StubRepository#matchStub
- Suppressed Jetty's default ErrorHandler with a custom JsonErrorHandler to send errors in JSON format
- Got rid off repackaged classes from Apache Commons in favor of Java 8 APIs
- Using Java NIO for file operations
- Log to terminal why a request fails to match https://github.com/soundcloud/stubby4j/commit/5901710efd31653a05804ebec62f67184c212832
- Square brackets were not escaped as literals for regular expression in Json POST [BUG]
- Pre-compiling & caching stubbed regex patterns upon parsing YAML stub configuration
- Issue #54 - Support for regular expression in Json POST
- Built using Java v1.8 (
1.8.0_60
) - Updated Gradle
build.gradle
to compile using Java v1.8 - Upgraded from Jetty
9.2.10.v20150310
to9.3.12.v20160915
- Updated Docker config
- Renamed project root package from
by.stub
toio.github.azagniotov.stubby4j
- Renamed Maven Central group ID from
by.stub
toio.github.azagniotov
- Issue #55 - When running in
--debug
, dumpingHttpServletRequest
parameters, would implicitly callServletRequest#getInputStream()
- Issue #56 - Requests with query parameters values containing white spaces
- Pull request #57 -
StubbyClient
starts Jetty with-m
to mute the console logger, but it wasn't actually muted
- Allow callers to wait for the StubbyClient to finish (Oliver Weiler, https://github.com/helpermethod)
- Serving response files from a local path constructed with regex tokens from the stubbed request (Radek Ostrowski, https://github.com/radek1st)
- Dumping more debug information to the console if
--debug
option is on, also for successfully matched requests - Added support for
PUT
andDELETE
methods inStubbyClient
class
- If POST'ed data type is
application/json
, the comparison of stubbed to posted data will be done using JSON entities with non-strict checking (content ordering wont matter, as long as it is the same) - If POST'ed data type is
application/xml
, the comparison of stubbed to posted data will be done using XML entities with non-strict checking (element & attribute ordering wont matter, as long as content is the same)
- Added support for custom authorization type header with the help of the new
header
propertyauthorization-custom
- Fixed issue #43 (Live refresh in response sequence only for first response)
- Added support for Bearer Token authorization with the help of the new
header
propertyauthorization-bearer
- Renamed existing
header
propertyauthorization
toauthorization-basic
- Some changes around the programmatic APIs in
StubbyClient
class due to the above changes - Respective changes in the current README due to the above changes
- Upgraded Jetty to v9.2.10.v20150310
- Added
--debug
option that dumps incoming raw HTTP request to the console - Added
--disable_admin_portal
option that does not configure Admin portal for stubby - Added
--disable_ssl
option that does not enable SSL for stubby - Added a new API to start stubby programmatically without specifying a YAML file
StubbyClient.startJettyYamless(...)
- Added a new
FaviconHandler
to handle requests forfavicon.ico
under the context root
- Built using Java v1.7.0_76
- Updated Jetty to v9.2.9.v20150224 (requires at least JRE v1.7.0_76: Issue with Java v1.7.0_04)
- Built using Java v1.7.0_04
- Cleaned up project Gradle configuration
- Updated Gradle configuration to be compatible with Gradle v2.2.1 & Gradle Nexus plugin v2.3
- Updated all (except Jetty) dependencies to their latest versions (as of May 10th, 2015)
- Added console outputs for record & play functionality
- Replacing all hardcoded
\n
with dynamically generated system line break characters
- Record&Play is now more intelligent: when stubbed
request
is matched, its stubbed properties (method
,url
,headers
,post
andquery
) are used to construct HTTP request to the recordable destination URL provided in stubbedresponse
body [ENHANCEMENT] - Added a workaround a limitation in SnakeYAML v1.13 used by stubby (it has limited JSON support, not all the JSON documents can be parsed) where it cannot parse escaped forward slashes in JSON [BUG]
- Refreshing Admin status page was changing sequenced response counter ID [BUG]
- Replaced hardcoded Unix new line character '\n' in YamlBuilderTest that caused the tests to fail on Windows [BUG]
- Admin status page now shows what is the next sequenced response in the sequence queue [ENHANCEMENT]
- Supporting HTTP requests with empty query params:
/uri?some_param=
or/uri?some_param
[ENHANCEMENT]
- When
--data
file was just a relative filename without parent directory, NPE was thrown when Admin portal status page was loaded [BUG] - Configured jetty GZIP handler for static resources and Stubs & Admin portals [ENHANCEMENT]
- Added resource hit stats section to Admin portal status page [ENHANCEMENT]
- Added a verification check in StubbyManager to make sure that Jetty has been started or shut down [ENHANCEMENT]
- Upgraded SnakeYAML dependency library to v1.13 [ENHANCEMENT]
- Admin status page: displaying YAML snippet for a given
request
orresponse
separately instead of pairing them up [ENHANCEMENT] - Admin status page: displaying metadata of loaded external files (if any) [ENHANCEMENT]
- Admin status page: displaying Ajax response in a JS popup, instead of injecting the Ajax response into HTML table [ENHANCEMENT]
- A bunch of cosmetic changes on Admin status page [COSMETICS]
- Displaying stubby JAR: version, its classpath location, built date, up-time, its input args and Java input args on status page [ENHANCEMENT]
- Displaying stubby heap/non-heap memory usage on status page [ENHANCEMENT]
- Added an option on admin status page to display YAML snippet for a given request & response pair [ENHANCEMENT]
- Making sure that 'x-stubby-resource-id' header is always the first item in the stubbed headers on status page [ENHANCEMENT]
- Changed colors of status page [COSMETICS]
- When creating template token names for
query
orheaders
regex matches, the name format to be followed should beheaders.key_name.ID
orquery.key_name.ID
[ENHANCEMENT]
- Whitespace was not allowed between the
<%
&%>
and what's inside when specifying template tokens for dynamic token replacement in stubbed response [BUG] - Regex matches were stored against incorrect token names for
query
andheaders
regexes [BUG] - Renamed command line arg
--ssl
to--tls
to reduce the confusion when having another command line arg that starts with letters
, like--stubs
[ENHANCEMENT] - Added command line arg
--version
that prints current stubby4j version to the console [ENHANCEMENT]
- Dynamic token replacement in stubbed response, by leveraging regex capturing groups as token values during HTTP request verification [FEATURE]
- Removed flag
--watch_sleep_time
. The--watch
flag can now accept an optional arg value which is the watch scan time in milliseconds. If milliseconds is not provided, the watch scans every 100ms [ENHANCEMENT] - Added additional API to start Jetty via StubbyClient by specifying an address to bind [ENHANCEMENT]
--watch
flag sleep time is now configurable via--watch_sleep_time
and defaults to100ms
if--watch_sleep_time
is not provided [ENHANCEMENT]- Added a
GET
endpoint on Admin portallocalhost:8889/refresh
for refreshing stubbed data [ENHANCEMENT]
- Record & Replay. The HTTP traffic is recorded on the first call to stubbed
uri
and subsequent calls will play back the recorded HTTP response, without actually connecting to the external server [FEATURE]
- Ensuring that Admin portal status page loads fast by not rendering stubbed response content which slows down page load. User can invoke Ajax request to fetch the desired response content as needed [ENHANCEMENT]
- Pre-setting header
x-stubby-resource-id
during YAML parse time, instead of on demand. This way resource IDs are viewable on Admin status page [ENHANCEMENT] - Making sure that header
x-stubby-resource-id
is recalculated accordingly after stubbed data in memory has changed (due to reset, or deletion etc.) [BUG] - Added date stamp to live reload success message [COSMETICS]
- Making sure that every stubbed response returned to the client contains its resource ID in the header
x-stubby-resource-id
. The latter is useful if the returned resource needs to be updated at run time by ID via Admin portal [FEATURE]
- Force regex matching only everywhere (url, query, post, headers, etc.) to avoid confusion and unexpected behaviour, with default fallback to simple full-string match (Michael England) [ENHANCEMENT]
- Live YAML scan now also check for modifications to external files referenced from main YAML [ENHANCEMENT]
- YAML parsing logic revisited [COSMETICS]
- Code cleanup [COSMETICS]
- Added ability to specify sequence of responses on the same URI using
file
(Prakash Kandavel) [ENHANCEMENT] - Minor code clean up [COSMETICS]
- Documentation update [COSMETICS]
- Making sure that operations starting up stubby and managing stubbed data are atomic [ENHANCEMENT]
- Typo in test was giving wrong indication that when
file
not set, stubbed response fallsback tobody
[BUG] - Eliminated implicit test to test dependencies in AdminPortalTest that was causing issues when running the tests under JDK 1.7 [BUG]
- Added convenience method in StubbyClient
updateStubbedData
[ENHANCEMENT]
- Stubbed request HTTP header names were not lower-cased at the time of match [BUG]
- Doing GET on Admin portal
/
will display all loaded stub data in a YAML format in the browser [FEATURE] - Doing GET on Admin portal
/<id>
will display loaded stub data matched by provided index in a YAML format in the browser [FEATURE] - Doing DELETE on Admin portal
/<id>
will delete stubbed request from the list of loaded stub requests using the index provided [FEATURE] - Doing PUT on Admin portal
/<id>
will update stubbed request in the list of loaded stub requests using the index provided [FEATURE] - When YAML is parsed, if
file
could not be loaded, the IOException is not thrown anymore. Instead a warning recorded in the terminal [ENHANCEMENT] - When could not match submitted HTTP request to stubbed requests, the not found message is much more descriptive [ENHANCEMENT]
- URI for registering new stub data programmatically via POST on Admin portal was changed from
/stubdata/new
to/
[COSMETICS] - URI for getting loaded stub data status was changed from
/ping
to/status
on Admin portal [COSMETICS] - Updated to SnakeYAML v1.12 [COSMETICS]
- Updated default response message when response content could not be loaded from
file
[ENHANCEMENT] - Documentation refinement [COSMETICS]
- Every
url
is treated as a regular expression now [ENHANCEMENT] - ANSI logging in the terminal was working only for HTTP requests with status 200 [BUG]
- Documentation refinement [COSMETICS]
- Mainly backend code improvements: A lot of refactoring for better code readability, expanding test coverage [COSMETICS]
- Added ability to specify sequence of stub responses for the same URI, that are sent to the client in the loop [FEATURE]
- Configuration scan was not enabled, even if the
--watch
command line argument was passed [BUG]
- Added ability to specify regex in stabbed URL for dynamic matching [FEATURE]
- A lot of minor fixes, refactorings and code cleaned up [COSMETICS]
- Documentation revisited and rewritten into a much clearer format [ENHANCEMENT]
- Just some changes around unit, integration and functional tests. Code cleanup [COSMETICS]
- stubby's admin page was generating broken hyper links if URL had single quotes [BUG]
- stubby is able to match URL when query string param was an array with elements within single quotes, ie:
attributes=['id','uuid']
[ENHANCEMENT]
- stubby's admin page was not able to display the contents of stubbed response/request
body
,post
orfile
[BUG] - stubby was not able to match URL when query string param was an array with quoted elements, ie:
attributes=["id","uuid","created","lastUpdated","displayName","email"]
[BUG]
- Making sure that stubby can serve binary files as well as ascii files, when response is loaded using the
file
property [ENHANCEMENT]
- Migrated the project from Maven to Gradle (thanks to Logan McGrath for his feedback and assistance). The project has now a multi-module setup [ENHANCEMENT]
- If
request.post
was left out of the configuration, stubby would ONLY match requests without a post body to it [BUG] - Fixing
See Also
section of readme [COSMETICS]
- Updated YAML example documentation [COSMETICS]
- Bug fix where command line options
mute
,debug
andwatch
were overlooked [BUG]
- Previous commit (
v1.0.53
) unintentionally broke use of embedded stubby [BUG]
- Add support for OAuth in Record & Replay feature
- Scenarios where multiple endpoints correlate with each other based on the scenario. Useful in e2e testing where system brought to a certain state (maybe?)
A number of people have contributed directly to stubby4j by writing documentation or developing software.
- Alexander Zagniotov azagniotov@gmail.com
- Eric Mrak enmrak@gmail.com
A number of people have contributed to stubby4j by reporting problems, suggesting improvements or submitting changes. Special thanks fly out to the following Ninjas for their help, support and feedback
- Isa Goksu
- Eric Mrak
- Oleksandr Berezianskyi
- Sankalp Saxena
- Simon Brunning
- Ed Hewell
- Kenny Lin
- Logan McGrath
- stubby4net: A .NET implementation of stubby
- stubby4node: A node.js implementation of stubby
Yes. See COPYRIGHT for details
MIT. See LICENSE for details