Skip to main content

Writing Tests

Previously, we learned the basic concepts behind Touca and how we can use the Touca CLI to capture the output of a given software to detect changes in its behavior. In this Tutorial, we'll use Touca SDKs to find software regressions by capturing the values of variables and runtime of functions.

Let's suppose we're building a software that takes the username of a student and provides basic information about them.

def find_student(username: str) -> Student:

Where type Student could be defined as follows:

@dataclass
class Student:
username: str
fullname: str
dob: datetime.date
gpa: float

You can find the source code for this example on our GitHub under the examples/<lang>/02_<lang>_main_api directory. If you like to follow along, this is a great time to clone the repository:

git clone git@github.com:trytouca/trytouca.git

The students module represents our code under test which could be arbitrarily complex. It may call various nested functions, perform database lookups, and access other services to retrieve the requested information. Here's a simple implementation to start with:

def find_student(username: str) -> Student:
sleep(0.2)
data = next((k for k in students if k[0] == username), None)
if not data:
raise ValueError(f"no student found for username: ${username}")
return Student(data[0], data[1], data[2], calculate_gpa(data[3]))

We can use unit testing in which we hard-code a set of input numbers and list our expected return value for each input.

from code_under_test import find_student

def test_find_student():
alice = find_student("alice")
assert alice.fullname == "Alice Anderson"
assert alice.dob == date(2006, 3, 1)
assert alice.gpa == 3.9

Unit tests require calling our code under test with a hard-coded set of inputs and comparing the return value of our function against a hard-coded set of expected values. We would need a separate block of assertions for every input to our code under test. This could be prohibitive for workflows that need to handle a variety of different inputs.

Touca takes a different approach:

import touca
from students import find_student

@touca.workflow
def students_test(username: str):
student = find_student(username)
touca.assume("username", student.username)
touca.check("fullname", student.fullname)
touca.check("birth_date", student.dob)
touca.check("gpa", student.gpa)

Unlike unit tests:

  • Touca tests do not use expected values.
  • Touca test inputs are decoupled from the test code.

Similar to property-based testing, Touca test workflows take their input as a parameter. For each test input, we can call our code under test and use Touca SDKs data capturing functions to describe its behavior and performance by capturing values of interesting variables and runtime of important functions. Touca will notify us if this description changes in a future version of our software.

There is more that we can cover but let us accept the above code snippet as the first version of our Touca test code and prepare for running the test.

cd trytouca/examples/python/02_python_main_api
pip install touca

Now we can set our API credentials,

touca login

See touca login to learn more.

And run our tests with any number of inputs from the command-line:

touca test --testcase alice bob charlie
info

In real-world scenarios, our test cases may be too many to list as command-line arguments. Touca SDKs allow you to programmatically declare your test cases for each workflow. See Setting Test Cases to see how.

The above command produces the following output.

We can see our captured data points submitted to the server.

If this was your first ever Touca test, cheers to many more! 🥂