Skip to main content

Working as a Team

In the previous document, we learned how to write a simple Touca test and run it to submit data for the current version of our software workflow.

This page provides an overview of how our workflow changes over time in behavior and performance. We can click on each version to inspect the test results submitted for each test case in that version.

We can also review the captured data for any one of the test cases.

But submitting results to a remote Touca server makes more sense when we have large amounts of data that is difficult to compare across versions and overwhelming to interpret manually. Touca server automatically compares submitted test results against previous versions and visualizes all differences to make it easy for us to trace those differences back to their root cause.

In this document, we will learn how Touca server can help us detect differences in future versions of our software, inspect those differences to determine whether they are intended, and take action based on our investigation.

Changing the Code​

We'll keep using the same "Parse Profile" software as in the previous document. Let's make a small change to our code under test.

While we can make any changes to our code under test, for now we are going to replicate a change in behavior by changing one of Alice's courses from Course("math", 4.0) to Course("english", 3.6).

Once we rebuild our software and its corresponding test tool, we can run our test again, this time for version 2.0 of our code.

touca test --revision v2.0

Notice that we are not specifying the list of test cases anymore. When they are not explicitly provided, the SDK fetches this list from the Touca server.


Touca Test Runner
Suite: students/v2.0

 1.  DIFF   alice      (233 ms)
 2.  PASS   bob        (236 ms)
 3.  PASS   charlie    (228 ms)

Tests:      2 perfect, 1 different, 3 total
Time:       1.22 s
Link:       https://app.touca.io/~/demo/students/v2.0

✨   Ran all test suites.

As test cases are executed, the server compares their captured test results and performance benchmarks against the previous "baseline" version v1.0 and visualizes the differences.

We can inspect the differences reported for test case alice to understand the impact of our recent code change.

Promoting Baseline​

The main goal of the Touca server is to make it easy for us to find and understand the side-effects of our code changes. Once we inspect the reported differences for a given version of our workflow, we can decide whether those differences match our expectations. If we determine that the differences are unexpected, we can share the findings of our investigation with our team by adding comments and collaborate to find the root cause of potential defects.

There are times when we may want to change expected behavior or performance of our workflow. We can use the Touca server to promote any given version of our workflow as the new baseline, using the "Promote Version" button in the version overview page.

Promoting a version as "Baseline" makes the Touca server automatically compare future versions of our workflow against that version. Since this may be an important event in the course of evolution of our software, we could make a note to explain our reasons for taking this action. The server shares our explanation with all team members subscribed to this Suite using their preferred notification settings(s).

In the next section, we will take a more detailed look at the functionalities provided by the Touca CLI.