Automated API testing with Jenkins

Jenkins is an open-source continuous integration and delivery application that makes it easy to set up automated jobs, like building and deploying code. In this post, I'll explain how you can automate testing your API every time you deploy, using the Assertible deployment testing API.

The most opportune time to run validation checks against your web app is immediately after a new deployment. This helps developers quickly find flaws in an API that may go unnoticed with traditional manual testing methods.

To get started, let's outline the process of testing an API from Jenkins:

• Create tests for your web app
• Install and start Jenkins
• Set up a Jenkins job
• View and manage API tests in Assertible

Create tests for your web app

If you don't already have a web service set up in Assertible, get started free.

First, create a few tests that validate core functionality of your API. For a more in-depth guide on testing your web app with Assertible, check out the Getting Started section. In this post, I'm testing a Node.js API running on Heroku, with these tests:

Running tests from a deployment pipeline is the recommended method to validate new features and bug fixes in your app, and provides a way to immediately find flawed or broken builds, so make sure to create reliable and automatable API tests.

Post deploy testing script

Once you've created a few tests for your web app, find the Deployments tab of your web service. There you should see an example curl command for calling the Deployments API.

This command can be integrated into a CI pipeline or deployment script; when the script is executed, the tests for your API tests will be run. Keep this script handy, as we'll use it in a later section.

Install and start Jenkins

Next, let's set up a Jenkins server where the script will be run. If you already have an installation of Jenkins running, either locally or on a server, you can skip this section.

If you don't already have Jenkins running, you can download and install it for free by following the the Jenkins installation guide.

Set up a Jenkins Job

Once you have Jenkins running, the next step is to create a new job. In Jenkins, a job is basically a set of steps and commands that describe how to build and deploy your application.

• On your Jenkins dashboard, click Create new jobs:

  

• Configure your new job by adding a name, and select Freestyle project as the type:

  

A freestyle project is the simplest way to illustrate automated API testing from Jenkins, but note that you can use any existing Jenkins job, since all you need to do is add a Build Step.

Add a build step

With a Jenkins job created, you can now add a build step. This build step will execute a shell command that calls the post deploy testing script you got in the first section.

• In your Jenkins project dashboard, click Add build step and select the Execute shell option:

  

• Now here comes the fun part! The shell command will call your Assertible deployments script and automatically run your API tests. Grab the deployment script from the first section, it should look like this:

  curl -u YOUR_API_TOKEN: https://assertible.com/deployments -d'{
      "service": "YOUR_SERVICE_ID",
      "version": "v1"
  }'

• Add your script to the text input for the shell command in Jenkins:

  

In a real-world scenario, you will likely be adding this command as a Build Step in an existing Jenkins job. In either case, the script should be run right after your API is deployed.

As a refresher, the post-deploy testing script makes a POST request to Assertible, which will automatically run tests against your API. If you're deploying to a staging or qa environment, you can use the environment parameter to specify which URL your tests will run against. Check out the Deployments API documentation to see more options and examples.

Pro tip! To keep the senstive information in the command provide, like your `ASSERTIBLE_API_TOKEN` and `ASSERTIBLE_SERVICE_ID`, you can use environment variables in Jenkins.

Running the job

Alright! The Jenkins job is set up with a build step to run your API tests. Now, let's run the build to make sure everything is set up correctly.

• On your Jenkins job dashboard, click Build Now:

  

  This will start a build and begin going through the build steps. If the shell command succeeds, you should see a blue ball indicating a successful build.

• Navigate to the console output tab of the build to see the details of the POST request, including the id of the deployment that was created:

  

If you see something like the image above, then congratulations! You now have a Jenkins build that will continuously test your web services. In the next sections, I'll discuss how to manage your API tests in Assertible, set up failure alerts, and add status checks to your GitHub pull requsts.

API test results in Assertible

Back in your Assertible dashboard, find the Results tab of your web service. The results of the tests executed from Jenkins will be available, with all of the details about the assertions:

If you look at the details near the top, you'll see the results were executed via Deployment. Learn more about the different options for running post-deploy API tests in the docs.

Configure test failure notifications

One of the most important parts of continuous testing is getting alerted of web service failures. Assertible has several options you can use to customize how and when you're notified of test failures.

• Email hooks

  Email hooks are enabled by default to send an email when a test fails.

• Zapier hooks

  One last method of receiving alerts is with the Zapier integration. The Zapier integration allows to trigger a Zap when your tests fail, so you can do almost anything the test result data. You can even open an issue on GitHub when API tests fail.

• Slack notifications

  A great alternative to email notifications is to install the Slack integration to send a message to a channel when tests fail. Slack notifications are ideal for teams where multiple people are in charge of responding to web service failures.

  

  
  

Bonus: GitHub status checks for API test results

This is a very basic example of a Jenkins job. In many cases, Jenkins is used to build, test, and deploy code changes from a GitHub repository as a CI/CD server. Assertible can add a GitHub status checks to your pull requests that show the result of your API tests.

If you use GitHub, you can enable status checks for your API tests by connecting your web service to a GitHub repo, and adding the github: true parameter to your deployment script. Learn more about setting up GitHub status checks.

Wrapping up

Using Jenkins and Assertible, it's easy to continuously test your web services after every deployment or on a schedule interval.

More resources and tutorials

• Add post-deploy smoke tests to any Codeship pipeline
• Why you should add API testing to your continuous integration pipeline
• Tutorial: Automate smoke tests for a Go API on Heroku
• Docs: API test automation in Assertible

Have any questions or comments? I'd love to hear it! Hit me up on Twitter or reach out directly.

:: Cody Reichert