Assertions allow you to validate HTTP responses in your API
tests. For example, a basic assertion would ensure that a request
to assertible.com
returns a Status code 200
every time -- or
that the homepage contains the text Assertible
.
Assertions are checked every time a test is run. The result of each assertion, and the reason it passed or failed, can be found on the Test Result page. Assertions are also used to determine a test's state. If any of a test's assertions are failing, the test is in a failing state.
Sections
Add a new assertion
Assertible offers a selection of assertions that are commonly used to test HTTP services. For example, Assertible has assertions to validate HTML and JSON or check a webpage for broken links.
Each test configuration view has an Assertions tab. From there you can add, update, and delete any assertions for that test:
Types of assertions
Most assertions can be enabled with a single click. Some assertions offer additional configuration and customization. The available assertions are described below:
- Status code
- JSON path data
- JSON validation
- XPath XML data (XML/HTML)
- Response header
- Response time
- Response text
- HTML validation
- Link checker
Assert status code
Assert the desired HTTP status code of the test's request. The default
is 200
. This assertion is enabled by default for all new tests
and can be configured to any value.
Assert JSON data
Asserting JSON response data allows you to select parts of a JSON response and make assertions using JSON Path syntax; inspired by the original JSONPath implementation.
For example:
Sample JSON
{
"id": 1,
"org": {
"name": "Acme",
"founded": 2015
},
"store": {
"book": [
{ "category": "reference",
"author": "Nigel Rees",
"title": "Sayings of the Century",
"price": 8.95
},
{ "category": "fiction",
"author": "Evelyn Waugh",
"title": "Sword of Honour",
"price": 12.99
}
],
"bicycle": {
"color": "red",
"price": 19.95,
"parts": [
{
"id": "tire",
"psi": "100"
},
{
"id": "handlebars",
"style": "drop"
}
]
}
}
}
Given the sample JSON above, the following table shows the Assertible JSON Path data assertion configuration and result:
Property | Comparison | Target | Result |
---|---|---|---|
.id |
Equals |
1 |
PASS |
.id |
Equals |
2 |
FAIL |
.id |
Not equals |
2 |
PASS |
.id |
Greater than |
0 |
PASS |
.id |
Less than |
2 |
PASS |
.id |
Less than |
0 |
FAIL |
.store.bicycle.color |
Equals |
red |
PASS |
.store.book[0].author |
Equals |
"Nigel Rees" |
PASS |
.store.book[1].author |
Equals |
"Nigel Rees" |
FAIL |
.store.book[-1].author |
Equals |
"Evelyn Waugh" |
PASS |
.org.name |
Equals |
"Acme" |
PASS |
.store.book.length() |
Equals |
2 |
PASS |
.org.length() |
Equals |
2 |
PASS |
.length() |
Greater than |
2 |
PASS |
.org.name.length() |
Greater than |
3 |
PASS |
.org.name.length() |
Equals |
6 |
FAIL |
Note that if your JSON Path expression attempts to select a node that
does not exist, the null
value is not returned. In practice, this means
your assertion will fail (at the time of this writing, Assertible does
not support an "assert value exists" comparator but we're working on
it!). For example:
Property | Comparison | Target | Result |
---|---|---|---|
.something |
Equals |
null |
FAIL |
Assertible's JSON Path syntax does support wildcard path selections. However, only single node selections are valid at this time. See the JSON Path wildcard syntax docs for more detail:
Property | Comparison | Target | Result |
---|---|---|---|
.store.bike.parts[].psi |
Equals |
100 |
PASS |
.store.bike.parts[*].style |
Equals |
"drop" |
PASS |
.store.bike.parts[*].id |
Equals |
"red" |
FAIL |
.store.book[].category |
Equals |
"fiction" |
FAIL |
.store[].color |
Equals |
"red" |
PASS |
Note that .store[]
is a JSON object wildcard which returns an array
with all the top-level JSON objects at $.
.
Read more about the JSON Path syntax in the JSON Path wildcard syntax docs.
Assert JSON validation
Validate that JSON response bodies are valid JSON. Optionally, a JSON schema can be specified.
If no JSON schema is supplied, the JSON validation assertion will check that the HTTP response value is valid JSON. A top-level object, array, or scalar value will pass the test. Any un-parseable JSON errors or invalid values will fail the assertion.
JSON Schema Validation API
Assertible hosts a free JSON Schema Validation API
service. You can POST
your JSON Schema spec and JSON documents to validate and
debug validation errors.
The API is simple. Using this curl
command:
$ curl -XPOST "https://assertible.com/json" -d'{
"schema": {"type":"object"},
"json": {}
}'
The API will response with:
{"errors":[],"valid":true}
See the JSON Schema Validation API webpage for more details.
This API does not support external JSON Schema references. Please submit a message to us in the chat window or send an email if this is required for your testing purposes.
JSON schema
The JSON validation assertion also supports an optional JSON Schema Draft 4. When a JSON schema is supplied, the JSON validation assertion will validate the HTTP response data against the schema and report all validation errors.
There are some limitations with this assertion. A few json-schema features are not yet supported:
format
For example, the JSON string
"chris"
with a validation schema that usesformat
will ignore the formatting and validate thestring
:{ "type": "string", "format": "email" }
ECMA-262 regex
Some parts of ECMA-262 regular expressions are not supported:
[^]
: a validator with{"type":"string", "pattern": "[^]"}
will fail to compile due to being an invalid regex.
zero terminated floats
For example, a validator
{"type":"integer"}
will work on1.0
even though there is a trailing slash.
Speak up! Let us know if any of these unsupported features are valuable to your use-case. We are working on improving support for JSON schema but feel free to contact us.
Create JSON schema assertions for OpenAPI v3 responses
If you use OpenAPI v3, Assertible can automate the creation of JSON Schema
assertions for every operation that has a valid response with a corresponding
schema. Assertible checks the most commonly used response formats first. For
example, a 200 OK response with an application/json
media type will be
used first. If there is not a response defined with a successfuly status code,
the default response is used, assuming it includes a schema.
OpenAPI v3 JSON Schema compatibility support
OpenAPI uses an extended subset of JSON Schema to define data types in an OpenAPI definition. Assertible has support for the following OpenAPI schema keywords:
nullable
When
nullable: true
, assertible automatically translates any corresponding schema fields to the JSON Schema Draft 4.type: "string"
is converted totype: ["null", "string"]
type: [object,array]
is converted totype: [null, object, array]
If
enum
exists,null
is appended to the list ofenum
s.Here's some GitHub issues that exhibit some of the confusion around these fields:
Assert XML/HTML data
Asserting XML response data allows you to select parts of an XML or
HTML response using XPath and
make assertions on the selected value. For example, a key id
has a
value equal to 1
. The XML data assertions works well with both HTML
and XML data sources.
Sample XML XPath assertions
See more examples and information about the syntax below:
<?xml version="1.0" encoding="UTF-8"?>
<store>
<id>1</id>
<org>
<name>Acme</name>
<founded>2015</founded>
</org>
<bookstore>
<book category="cooking">
<title lang="en">Everyday Italian</title>
<author>Giada De Laurentiis</author>
<year>2005</year>
<price>30.00</price>
</book>
<book category="children">
<title lang="en">Harry Potter</title>
<author>J K. Rowling</author>
<year>2005</year>
<price>29.99</price>
</book>
<book category="web">
<title lang="en">XQuery Kick Start</title>
<author>James McGovern</author>
<author>Per Bothner</author>
<author>Kurt Cagle</author>
<author>James Linn</author>
<author>Vaidyanathan Nagarajan</author>
<year>2003</year>
<price>49.99</price>
</book>
<book category="web">
<title lang="en">Learning XML</title>
<author>Erik T. Ray</author>
<year>2003</year>
<price>39.95</price>
</book>
</bookstore>
</store>
1
,
not 0
.
Given the sample XML above, the following table shows the Assertible assertion configuration and result:
Property | Comparison | Target | Result |
---|---|---|---|
/store/id |
Equals |
<id>1</id> |
PASS |
/store/id/text() |
Equals |
1 |
PASS |
/store/id/text() |
Equals |
2 |
FAIL |
/store/id/text() |
Not equals |
2 |
PASS |
/store/bookstore/book[1]/author |
Equals |
<name>Giada De Laurentiis</name> |
PASS |
/store/bookstore/book[1]/author |
Equals |
Giada De Laurentiis |
FAIL |
/store/bookstore/book[1]/author/text() |
Equals |
Giada De Laurentiis |
PASS |
/store/bookstore/book[1] |
Contains |
Giada De Laurentiis |
PASS |
/store/bookstore/book[last()] |
Contains |
Learning XML |
PASS |
/store/bookstore/book[last()]/title |
Equals |
<title lang=\"en\">Learning XML</title> |
PASS |
/store/bookstore/book[last()-1] |
Contains |
<title lang=\"en\">XQuery Kick Start</title> |
PASS |
/store/bookstore/book[@category='cooking']/year |
Contains |
2005 |
PASS |
/store/bookstore/book[@category='web'][1] |
Contains |
49.99 |
PASS |
/store/bookstore/book[@category='web'][1]/price/text() |
GreaterThan |
49.98 |
PASS |
/store/bookstore/book[@category='web'][1]/price/text() |
LessThan |
50 |
PASS |
(//store/bookstore/book[@category='web']/price)[1] |
Contains |
<price>49.99<price> |
PASS |
(//store/bookstore/book[@category='web']/price/text())[1] |
Equals |
49.99 |
PASS |
/store/bookstore/book[@category='web'] |
Contains |
2005 |
FAIL |
/store/bookstore/book[price>40]/year/text() |
Equals |
2003 |
PASS |
/store/bookstore/book[price>40]/author |
Equals |
James Linn |
FAIL |
Assert response header
Make an assertion against a response header. In the header name
field, specify the name of the header to look for. The target value
field is dependent on the type of comparator
used. For example, when
using Contains
, the target value
only needs to match some part of
the header value.
Assert response time
Asserts that the response time of the entire request is less than a
specified value, 1
second by default. If the HTTP request takes
longer to respond than the target value, the test will fail. Otherwise
it will pass.
Assert text body
This assertion checks for any text in the response body of the
request. The default value to check for is DOCTYPE
-- you can set
the target to any given text and assert the response body contains
that string. If no match is found the test will fail and if a match is
found it will pass.
Note: the body can be any Content-Type, but will it will be checked as a raw string.
Assert HTML validation
Asserts that the response body contains valid HTML, XHTML, etc markup per the WC3 specification. This is useful for finding invalid DOM and HTML errors in your webpages that may prevent them from loading correctly on some clients.
Assert link check
When enabled, this assertion will check all of the links on an HTML
page and ensure a request to that target is valid. If any of the links
on the page are broken, return a Status Code 404
, etc, the assertion
will fail.
Currently support html tags:
<a>
<script>
<link>
<img>
Built-in assertions
In addition to user-defined assertions, Assertible has a set of built-in assertions known as core assertions. These assertions deal with the default behaviors a test run must satisfy. For example:
Assertible ensures that an HTTP request connects and executes properly:
HTTP Request Success
.Assertible ensures that a valid web service environment is loaded.
Assertible ensures that SSL connections are properly validated, depending on the execution environment's Validate SSL settings.
The easiest way to test and
monitor your web services
Reduce bugs in web applications by using Assertible to create an automated QA pipeline that helps you catch failures & ship code faster.
Get started with GitHubSign up for free