Skip to main content

Running Tests

Leveraging Continuous Integration​

We can choose to run Touca Tests as part of our CI pipeline. Regardless of what CI solution we use, we can add a step to our build pipeline that invokes Touca tests with the appropriate API Key and API URL.

Using Github Actions​

Touca provides GitHub Actions plugins that make it convenient to run your tests as part of the CI pipeline.

actions-setup-touca

GitHub Action for setting up Touca CLI

actions-run-tests

GitHub Action for running Touca tests

Here are two sample steps that you can add to your GitHub Actions workflow:

- uses: trytouca/actions-setup-touca@v1
  with:
    api_key: ${{ secrets.TOUCA_API_KEY }}
    api_url: https://api.touca.io/@/<your-team>
- uses: trytouca/actions-run-tests@v2
  with:
    directory: ./examples/python/02_python_main_api

Where TOUCA_API_KEY is a GitHub Actions secret that you can add from the "Settings" tab of your GitHub repository.

Now every time you push changes to your repository, your tests will automatically run and their results are submitted to the Touca server. You will also see a summary of the test results in GitHub Actions:

If you prefer not to use our GitHub Actions plugins, you can directly invoke Touca tests:

- run: ./local/dist/bin/example_cpp_main_api
  env:
    TOUCA_API_KEY: ${{ secrets.TOUCA_API_KEY }}
    TOUCA_API_URL: https://api.touca.io/@/<your-team>

Or you can run your tests with the Touca CLI:

- uses: actions/setup-python@v5
- run: pip install touca
- run: touca test
  working-directory: 02_python_main_api
  env:
    TOUCA_API_KEY: ${{ secrets.TOUCA_API_KEY }}
    TOUCA_API_URL: https://api.touca.io/@/examples

For a working prototype, check out our own GitHub Actions which includes running sample Touca tests for different programming languages.

Using a Dedicated Test Server​

Running Touca tests as part of the CI is convenient but it has important limitations:

  • Running Touca tests at scale with a large set of test cases could be time consuming, leading to longer build times.
  • Our test workflows may require access to input data that could be large in size and difficult to provision on a build server.
  • Performance benchmarks obtained on a build server are prone to noise.

We can mitigate these limitations using a dedicated test server that can store our input test data and run our tests continuously or on a fixed schedule.

If you prefer running tests on your own infrastructure, you can use the Touca CLI command touca run that takes a recipe describing your software delivery pipeline specification to download and deploy new releases of your software and to run your tests against it. Refer to the documentation for this CLI command for more info.

info

For Touca enterprise customers, we provide the test infrastructure and tooling required to remotely execute Touca tests at scale and based on custom requirements.