Setting Test Cases
Programmatic testcase declaration
In addition to passing testcases as command-line arguments or letting the SDK fetch and reuse the set of test cases already submitted to the baseline version, sometimes you may want to specify your test cases as code or infer them during the test runtime, based on the set of local files or datasets in a given directory, or rows of a given column in a CSV file, or a nested array of JSON response of a particular API endpoint, etc.
- Python
- JavaScript
- C++
- Java
In Python, you could use the testcases
parameter of the @touca.workflow
decorator which accepts either a list, a generator, or a generator function.
@touca.workflow(testcases=["alice", "bob", "charlie"])
def students_test(username: str):
student = code_under_test.find_student(username)
touca.check("gpa", student.gpa)
Here's the same snippet if we were to use a generator function.
def find_testcases():
for username in ['alice', 'bob', 'charlie']:
yield username
@touca.workflow(testcases=find_testcases)
def students_test(username: str):
student = code_under_test.find_student(username)
touca.check("gpa", student.gpa)
In TypeScript, you could pass a third parameter to the @touca.workflow
function and pass a list or a callback function to the testcases
field.
touca.workflow(
"students_test",
async (username: string) => {
const student = await find_student(username);
touca.check("gpa", student.gpa);
},
{ testcases: ["alice", "bob", "charlie"] }
);
In C++, you could pass a third parameter to the test runner function
touca::workflow
to specify the set of test cases to use for that workflow.
touca::workflow(
"students_test",
[](const std::string& username) {
const auto& student = find_student(username);
touca::check("gpa", student.gpa);
},
[](touca::WorkflowOptions& w) {
w.testcases = {"alice", "bob", "charlie"};
});
You can use lambda captured variables to pass the test cases retrieved by your custom logic.
std::vector<std::string> usernames = {"alice", "bob", "charlie"};
touca::workflow(
"students_test",
[](const std::string& username) {
const auto& student = find_student(username);
touca::check("gpa", student.gpa);
},
[&usernames](touca::WorkflowOptions& w) {
w.testcases = usernames;
});
In Java, we use @Touca.workflow
annotator to declare a workflow. Since
annotations are processed at compile time, settings testcases
as an annotation
parameter could be limitting. Instead, Java SDK provides
Touca.setWorkflowOptions
that can be called within the main function and prior
to calling Touca.run
.
public final class StudentsTest {
@Touca.Workflow
public void findStudent(final String username) {
Student student = Students.findStudent(username);
Touca.check("gpa", student.gpa);
}
public static void main(String[] args) {
Touca.setWorkflowOptions("findStudent", x -> {
x.testcases = new String[] { "alice", "bob", "charlie" };
});
Touca.run(StudentsTest.class, args);
}
}