This post will walk through setting up smoke test automation and monitoring for your web services using Assertible.

In this post, I'll describe how to setup automated smoke tests for a RESTful JSON API with Assertible. Smoke tests take less than 5 minutes to setup and can be used to augment your CI pipeline or deployment process by continuously testing your production and staging web applications.

There are 4 steps:

1. Import your web service
2. Create smoke tests for your web app
3. Schedule monitoring and alerts
4. Run smoke tests after deployments

1. Import your web service

All you need to get started with Assertible is a URL or Swagger/OpenAPI spec. When you sign-in to Assertible for the first time, you will see a form labeled Create a new web service.

Enter your URL or Swagger spec into the input, and click Import URL. A preview of the web service and tests that are going to be created will pop up.

In the picture above, I'm calling the /repositories endpoint in the GitHub API.

To finalize the import, click Create service and tests. Once the service is imported, you may need to set up things like auth headers or query parameters to get your tests passing. For example, because I'm using GitHub's free public API, I had to add a header to configure the User-Agent header.

2. Create smoke tests for your web app

Now that the web service is created, you can create basic smoke tests for each endpoint. Smoke tests should be simple and are distinct from more complex end-to-end tests. Ideally, smoke tests should meet these criteria:

• fast execution
• low false/positives
• reproducible
• test core features

To create a new test, click the + button in the top-right of the Assertible dashboard.

Next, you need to add some assertions to validate the HTTP response. Assertible offers many different out-of-the-box assertions for your tests. You can validate things like HTTP status code, response headers, and JSON Schemas.

To create an assertion, click the Assertions tab on the test view:

For smoke testing a basic JSON HTTP response, I will use 3 assertions:

• Status code

  The status code assertion will ensure the HTTP status code is 200 (or whatever it's configured to expect). As you can see, a status code assertion was created and configured automatically when the test was created.

• JSON schema validation

  The JSON schema validation assertion can be used to continuously validate a JSON Schema every time the test is run.

  Click the Add assertion button and select JSON Validation from the drop-down.

  
  

  The assertion is already pre-populated with a valid JSON schema. Simply add your schema to the text box, or build the schema by adding new properties to the existing schema incrementally.

• JSON path data

  The JSON path data assertion uses a JsonPath like syntax to extract data and validate the contents. Assertible even has support for helper functions to check things like the length of an array in a nested JSON object

  Click the Add assertion button and select JSON path data as the assertion type.

  
  

  The new assertion is pre-poluated with the top-level JSON selector . which, when selected, should equal the object {}

  The new assertion will fail unless your API returns a top-level empty object {}. In my case, the object returned from https://api.github.com/repositories is not an empty object.

  This can be clearly seen in the Results viewer.

  
  

3. Schedule monitoring and alerts

Now that you have smoke tests created, the next step is to set up schedules that automatically runs your tests and alert you when failures happen.

• Monitoring

  A good starting point for scheduled monitoring is at least once an hour. To configure schedules in Assertible, navigate to the Monitoring tab. If you did not select Enable hourly monitoring in the initial web service import, you will see a button to Setup a schedule, click that and create a new schedule.

  When you create a schedule, you will see a list with an option to enable/disable tests individually. Make sure your test is enabled. That's it!

  
  

• Alerts

  Next, set up alerts by navigating to the Settings tab and clicking Hooks & Alerts. You can use Slack, Email, and Zapier to alert you when tests fail.

  
  

  By default, email hooks are configured when you create a new test. If you don't want to use email, or want your entire team to recieve notifications, use Slack or Zapier.

  At Assertible, we use Slack for failure notifications and have Zapier configured to create new GitHub issues when specific mission critical tests fail.

4. Automate smoke tests after deployments

The ideal time to run smoke tests against an API is immediately after a deployment. You can do this easily using Assertible's deployments API. Navigate to the Deployments tab and click Bash / Command-line in the sidebar.

The deployments API script works like a trigger. When a new deployment is POSTed to Assertible, your tests are executed automatically.

Integrate the script into your continuous integration pipeline or deployment process and call it immediately after your web has been successfuly deployed.

Assertible keeps a record of the deployment's version, environment, and an optional source code reference. The deployment version is used in test results to identify the source that triggered the run:

You can also view the history of your deployments. Navigate back to the Deployments tab and refresh the page.

Conclusion

That's all there is to it. The steps outlined in this post aren't necessarily specific to RESTful APIs. You can test GraphQL or HTML web services easily too by customizing assertions to fit your needs.

Other resources for automated web service testing:

• Techniques to reduce api testing errors and improve your QA process.
• 4 common API errors and how to test them.
• Automate smoke tests for a go API on Heroku

:: Christopher Reichert