One of the most important parts of testing is knowing when there is a failure. If a critical web service is down, has a slow response time, or fails any other assertions, you can set up automation and hooks to alert you or process some other action.

Assertible's automation is extremely useful because of the way assertions work. Not only can you be alerted when a web service is unreachable - but how about when the response time is too high, or an HTTP Header goes missing, or something about the response body has changed? Assertible's automations have you covered.

The currently supported automations are:

Integrations

Integrations are a means to hook your test runs and results into other 3rd party services, like a team chat. Good, timely, and reliable integrations are crucial for teams sharing information.

The currently supported integrations are:

GitHub Deployments

Connecting your Assertible web service with a GitHub repository allows you to automatically run your API tests after every successful deployment.

Assertible GitHub Continuous Testing Integration

GitHub quick start

There are 2 steps required to make use of the Assertible GitHub integration:

  1. Connect Assertible to GitHub

  2. Send deployment events to your GitHub repository

Connect Assertible to GitHub

There are two ways to connect Assertible to GitHub:

  1. Directly from the GitHub integrations directory

  2. From the Assertible dashboard

    • Create a new service and navigate to the "Deployments" tab. You should see an "Add to GitHub" button. Fig 1.0 GitHub Deployments Integration
Setup GitHub deployment events

In order for Assertible to run tests when your web app or API is deployed, it must receive webhooks for your repository on GitHub.

Webhooks can be triggered in two ways:

  1. Through a continuous delivery services, like Heroku and Shippable. These services send deployment events to GitHub which Assertible is able to receive through webhooks.

    For example, Heroku Review Apps, creates a deployment event for all apps spun up for every pull request. These events are received by Assertible which runs tests against the web app for each unique environment.

  2. Use a script to create deployment events from your continuous integration pipeline

    We have released a set of free and open source scripts and and examples for creating deployment events from services like Travis-CI, Jenkins, Wercker, and CircleCI. Check out the assertible/deployments open-source repository for more info.

Remove the GitHub Deployments integration

To remove the GitHub deployments integration, you need to visit any of your web service's deployments tab in the Assertible Dashboard. From the service page, click the Deployments tab. Find the GitHub Deployments integration you want to delete, and click the trash can icon next to it.

How it works

When you connect Assertible to a GitHub repository, it receives webhooks for all repository events related to deployments. Using the webhook payload, Assertible can determine when your web service has been deployed and will run tests against your staging and production environments automatically.

Loading an environment
TL;DR When Assertible receives a deployment event, it checks various fields in the payload for an Environment or Environment URL. If an Environment URL is present in the data, Assertible uses that as the base host which all tests are run against.
Dynamic environment URLs received via the GitHub API will not validate SSL certificate information. However, if the URL is not subject to change (different on each deployment), you can create an environment for the corresponding service and configure SSL validation.


Environment "loading" is the process of identifying the unique url and other HTTP settings which are specific to an instance of your app. For example, your staging environments may all be deployed to a unique environment like https://${branch}.myapp.com or even a static url like https://staging.myapp.com

The method Assertible uses to resolve the test execution environment from a GitHub deployment event can be broken down into a few steps:

  • If the deployment event contains an environment field (the unique name of the environment), Assertible checks your environments for a corresponding name.

    For example, if the GitHub deployment event contains { "environment": "staging", ... } then Assertible will attempt to find a valid environment for the web service named staging.

    This is the primary method used to identify a static environment.

  • Next, Assertible checks for an environment_url in the GitHub deployment status payload. If an environment_url is found, Assertible runs tests against a "transient" (or "temporary") environment.

  • If the environment can't be loaded, then a core assertion failure is raised and the test runs will fail.

    The best way to avoid environment load failures is to create an environment for each static url you deploy your app too (whether staging or production).

    Please feel free to contact us if your use-case requires additional support for loading environments.

See the GitHub deployment documentation for more details

Slack

Assertible is a turn-key testing solution for web apps. Developers and QA professionals use Assertible to write and automate tests for API's, websites, and other web services. Assertible tests and monitors your web service's behavior and health. Learn more.

Using the Assertible integration for Slack, you can:

  • send real-time notifications to your teams channel when tests are failing (or passing!).
  • see a high-level overview of the test results directly in Slack
  • view the results of a test in a single button click

Fig 1.0 Slack Integration

Setup the Slack integration

NOTE: You need a Slack account and a target channel to send messages to (e.g. #dev or #general).

  • Open one of your web service's in the Assertible dashboard.
  • Click on the Settings tab.
  • Select Add New Integration from the Integrations section.
  • Find the Slack Messages integration and select Add to Slack.

    Note: The Slack integration can be customized further using the "Preferences" drop-down.

    Note: This step will lead you to the Slack website to confirm the integration. If you are not redirected directly back to your test page, navigate back to your dashboard using the "Dashboard button in the site navigation.

  • The Integrations section of the web service page should now show a "Slack Integration", mark the checkbox to enable/disable the integrations.

Remove the Slack integration

  • Open one of your test's in the Assertible dashboard.
  • Click on the Deployments tab.
  • Find the Slack Messages integration you'd like to delete and click the trash can icon to completely delete the integration.

Zapier

The Zapier integration makes it possible to send test results to Zapier by making a Zap that triggers when a test is run on your web service. This opens up many possibilities for how you can see and use your Assertible test result data.

Set up the Zapier integration

Beta alert! Our Zapier integration is currently in beta to become a certified app! Be sure to enable the Assertible integration from your dashboard, and send us a message if you have any questions or feedback!


Assertible Zapier Integration

To get started, visit the Settings page of any web service in your Assertible dashboard and find the Zapier hooks section as seen in the image above. From there, follow these steps:

  1. Enable the Assertible app in Zapier

    While the Assertible Zapier app is in beta, you must accept an invitation to use the app. You can find this link on the Settings page of any web service in your Assertible dashboard.

  2. Make a Zap

    After you have enabled the Assertible app in your Zapier account, you can begin creating a zap in Zapier. Start by selecting the Assertible app, and then choose one of the supported triggers.

    On the next page, you'll get a Webhook URL from Zapier that you can use in the Assertible, to create the integration. When you have that Webhook URL, proceed to the next step.

  3. Create the integration in Assertible

    Using the Webhook URL from step 2, you can now create the Zapier integration in your Assertible dashboard. The different options are described below:

    Trigger Zap

    This setting configures when the Zap Trigger wil be executed. This can either be On Test Complete or On Test Fail.

    Webhook URL

    The Webhook URL is given to you by Zapier in Step 2.

    When tests are run via

    The vias option allows you to further restrict when the Zap is triggered. The valid options here are: Deployments, Schedules, Trigger URL, and Dashboard.

Zapier Triggers

Completed Test trigger

The Completed Test trigger fires each time a test completes. You can optionally configure this to only run when tests fail. time a test fails. If multiple tests are executed at the same time (e.g. have the same Run ID), this trigger will fire for each test result independently.

Additionally, this trigger can be configured to run based on who or what executed the test, known as via. For example, via Schedule will call the Completed Test webhook each time tests are executed using a schedule. Other values which can be configured are via Deployment, via Trigger URL, and via Dashboard. Using via helps reduce executing Zaps unknowingly or at undesired times.

Pager Duty

Hang tight! More information on the Pager Duty integration is coming soon!

StatusPage.io

Hang tight! More information on the StatusPage.io integration is coming soon!


...and many more on the way. Have an idea for an integration you'd like to see? Reach out and let us know!

Hooks

Psst.. Currently, Assertible only support emails hooks. Looking for another type of hook? Let us know!

Hooks are the most basic form of test automation that execute an action when a test run completes, like sending an email. Hooks are easy to set up and require little configuration and maintenance.

There are two pieces to every hook: Type and Action. The type determines what type of hook to execute, like an email hook. The action let's you configure when to execute your hook:

  • On test complete: execute this hook every time a test completes
  • On test failure: only execute this hook when a test fails

You can add a new hook from the "Settings" page of any web service. By default, every test on that service will be enabled for the hook. You can enable or disable specific tests by editing your hook from the same page.

Schedules

Note! The paid plans allow more frequent schedules than the free plan. Upgrade today.

Tests can be run on a schedule with flexible frequency, all the way down to every minute. This allows you to set up hands-off monitoring of any web service and be confident that your APIs are being tested regularly. Assertible currently supports the following schedule settings:

Frequency

  • Once per Minute*
  • Once per 5 Minutes*
  • Once per Hour
  • Once per Day

* These frequencies are only available on the paid plans.

Start Date and Time

Choose the date and time that the schedule will start running. This can be now, or any time in the future.

Trigger URLS

Trigger URL's are an easy way to run your tests from anywhere outside of the Assertible dashboard. Nice! This means if you'd like to run your tests programatically, you can do so. Here's a few examples of when Trigger URL's come in handy:

  • Running your tests programatically (eg, from your own test suite)
  • Providing reproducible test cases to other team members.
  • Setting up so anyone on your team can trigger a test run.

A typical Trigger URL can be run with cURL like this:

$ curl -XPOST https://assertible.com/apis/123e4567-e89b-12d3-a456-426655440000/run

Tigger URL's can be run for an entire web service or a single test. For example, the trigger URL to run all tests for a web service will look like:

$ curl -XPOST https://assertible.com/apis/123e4567-e89b-12d3-a456-426655440000/run

The trigger URL for a single test should look like:

$ curl -XPOST https://assertible.com/tests/123e4567-e89b-12d3-a456-426655440000/run

Furthermore, trigger URL's can specify the environment to run all tests against. This is useful for running the same sets of tests against staging and production environments:

$ curl -XPOST https://assertible.com/apis/123e4567-e89b-12d3-a456-426655440000/run?environment=staging

Trigger URL's for an API can also specify a single endpoint to run tests for:

$ curl -XPOST https://assertible.com/apis/123e4567-e89b-12d3-a456-426655440000/run?endpoint=%2Ftests

Note: The response body of a trigger URL is not yet finalized, so although you may use the information, be aware that it is likely the change.