10/25/2016EngineeringCody Reichert

At Assertible, we automatically test every API deployment. Here's why.

Having a reliable process in place for testing and validating API deployments provides a huge amount of confidence when deploying new changes into production. It's the best way to know that your application is ready for your users, and it saves developers time by decreasing the need for manual testing.

In this post I'll discuss our continuous testing process, and how we use Assertible to validate releases with post-deployment testing.

Assertible GitHub Status Check

How we continuously test

At Assertible, our continuous integration/delivery/testing setup is pretty standard:

Running API tests automatically after every deployment quickly validates and provides feedback about the release.

For example, if a pull request is opened, the code will be built and deployed to staging. API tests are automatically run against the new app on the staging environment.

If this new deployment doesn't pass the Assertible tests, the GitHub status check on the PR will fail and tell us the code shouldn't be merged.

Assertible GitHub status check failing

More on the Assertible GitHub integration

By using Github we're able to take advantage of the Assertible/GitHub integration and deployment events.

After the code is built and in the process of being deployed to staging, Assertible receives deployment status events. When the deployment is successful, Assertible runs all tests for that API and updates the status check.

The assertible/deployments repo has instructions on setting up GitHub deployments as a part of your CI with a simple script.

Another neat part of using GitHub status checks with Assertible is that we can see the result of our API tests on the master branch. Any commit made to the master branch means a deployment to production. By looking in the commit history we can see that the API tests were all successful.

Assertible GitHub commit status check

API testing with Assertible

All API tests cases live in Assertible. There are tests set up to validate all API endpoints, test core functionality, and make assertions on response data integrity.

These are some of the tests that are run when a new version of the API is deployed.

Assertible API Testing

Before continuous testing, these all had to be run manually. The manual process is obviously time-consuming and error prone -- no one wants to do that.

With a solid continuous testing workflow in place, manually testing these endpoints is unnecessary. I can sleep sound knowing that everything has already been validated and changes can be pushed to production with confidence.

If a test fails, the status check on GitHub will have a link directly back to the failing test result to easily see why:

Assertible failing test

Uh oh! It looks like we broke some authentication logic in the /tests handler. This is where continuous testing saves our skin and our time (and in some cases our job!).

This is why we test every. single. deployment.

If all of the API tests are passing, we can happily merge knowing that the release has been validated and tested.

Side note! Assertible is free. If you're interested in setting this up yourself, you can get started here.

Continue the discussion

I always enjoy hearing how other teams test and validate API deployments. Tweet at me (@CodyReichert) or us (@AssertibleApp) and let's see what you're working with!

Assuredly yours,

Cody Reichert