Image by author
Have you ever encountered software that didn't work as expected? Maybe he clicked a button and nothing happened, or a feature he was excited about turned out to be buggy or incomplete. These issues can be frustrating for users and even cause financial losses for businesses.
To address these challenges, developers follow a programming approach called test driven development. TDD aims to minimize software failures and ensure that the software meets its intended requirements. These test cases describe the expected behavior of the code. By writing these tests in advance, developers get a clear understanding of what they want to achieve. Test pipelines are an essential part of the software development process for any organization. Whenever we make changes to our codebase, we need to ensure that they don't introduce new bugs. This is where testing channels come into play to help us.
Now, let's talk about PyTest. PyTest is a Python package that simplifies the process of writing and executing test cases. This full-featured testing tool has matured to become the de facto standard for many organizations, easily adapting to complex code bases and functionality.
Benefits of the PyTest module
- Improved test and logging reports
Upon execution of the tests, we receive a complete log of all the tests executed and the status of each test case. In case of failure, a full stack trace is provided for each failure, along with the exact values that caused an assert statement to fail. This is extremely beneficial for debugging and makes it easier to trace the exact problem in our code to resolve errors. - Automatic test case discovery
We don't have to manually configure any test cases to run. All files are scanned recursively and all function names prefixed with “test” are executed automatically. - Accessories and parameterization
During test cases, specific requirements may not always be accessible. For example, it is inefficient to retrieve a network resource for testing, and Internet access may not be available when running a test case. In such scenarios, if we want to run a test that makes Internet requests, we will need to add stubs that create a dummy response for that specific part. Additionally, it may be necessary to run a function multiple times with different arguments to cover all possible edge cases. PyTest simplifies the implementation of this using props and parameterization decorators.
Facility
PyTest is available as a PyPI package that can be easily installed using the Python package manager. To configure PyTest, it's good to start with a new environment. To create a new Python virtual environment, use the following commands:
python3 -m venv venv
source venv/bin/activate
To configure the PyTest module, you can install the official PyPI package using pip:
Running your first test case
Let's dive into writing and running your first test case in Python using PyTest. We'll start from scratch and create a simple test to get an idea of how it works.
Structure a Python project
Before starting to write tests, it is essential to organize our project properly. This helps keep things organized and manageable, especially as our projects grow. We will follow a common practice of separating our application code from our test code.
This is how we will structure our project:
pytest_demo/
│
├── src/
│ ├── __init__.py
│ ├── sorting.py
│
├── tests/
│ ├── __init__.py
│ ├── test_sorting.py
│
├── venv/Our root directory pytest_demo contains separate src and tests directories. Our application code resides in src, while our test code resides in tests.
Write a simple program and its associated test case
Now, let's create a basic sorting program using the bubble sort algorithm. We'll put this in src/sorting.py:
# src/sorting.py
def bubble_sort(arr):
for n in range(len(arr)-1, 0, -1):
for i in range(n):
if arr(i) > arr(i + 1):
arr(i), arr(i + 1) = arr(i + 1), arr(i)
return arrWe have implemented a basic bubble sort algorithm, a simple but effective way to sort elements in a list by repeatedly swapping adjacent elements if they are in the wrong order.
Now, let's make sure our implementation works by writing complete test cases.
# tests/test_sorting.py
import pytest
from src.sorting import bubble_sort
def test_always_passes():
assert True
def test_always_fails():
assert False
def test_sorting():
assert bubble_sort((2,3,1,6,4,5,9,8,7)) == (1,2,3,4,5,6,7,8,9)In our test file, we have written three different test cases. Notice how the name of each function begins with the proof prefix, which is a rule that PyTest follows to recognize test functions.
We import the bubble sort implementation from the source code into the test file. This can now be used in our test cases. Each test must have a “say” statement to check if it works as expected. We give the sort function a list that is not in order and compare its result with what we expect. If they match, the test passes; otherwise it fails.
Additionally, we have also included two simple tests, one that always passes and one that always fails. These are just placeholder functions that are useful for checking if our test setup is working correctly.
Run tests and understand the result
Now we can run our tests from the command line. Navigate to the root directory of your project and run:
This will recursively search for all files in the tests directory. All functions and classes starting with the test prefix will automatically be recognized as a test case. From our tests directory, it will look in the test_sorting.py file and run the three test functions.
After running the tests, you will see output similar to this:
===================================================================
test session starts ====================================================================
platform darwin -- Python 3.11.4, pytest-8.1.1, pluggy-1.5.0
rootdir: /pytest_demo/
collected 3 items
tests/test_sorting.py .F. (100%)
========================================================================= FAILURES
=========================================================================
____________________________________________________________________ test_always_fails _____________________________________________________________________
def test_always_fails():
> assert False
E assert False
tests/test_sorting.py:22: AssertionError
================================================================= short test summary info ==================================================================
FAILED tests/test_sorting.py::test_always_fails - assert False
===============================================================
1 failed, 2 passed in 0.02s ================================================================When you run the PyTest command-line utility, it displays the platform metadata and the total number of test cases to be executed. In our example, three test cases were added from the test_sorting.py file. Test cases are executed sequentially. A dot (“.”) represents the test case passed, while an “F” represents a failed test case.
If a test case fails, PyTest provides a traceback that shows the specific line of code and arguments that caused the error. Once all the test cases have been executed, PyTest presents a final report. This report includes the total execution time and the number of test cases that passed and failed. This summary gives you a clear overview of the test results.
Parameterization of functions for multiple test cases
In our example, we tested only one scenario for the classification algorithm. Is that enough? Obviously not! We need to test the feature with multiple examples and edge cases to make sure there are no bugs in our code.
PyTest makes this process easier for us. We use the parameterization decorator provided by PyTest to add multiple test cases for a single function. The code appears as follows:
@pytest.mark.parametrize(
"input_list, expected_output",
(
((), ()),
((1), (1)),
((53,351,23,12), (12,23,53,351)),
((-4,-6,1,0,-2), (-6,-4,-2,0,1))
)
)
def test_sorting(input_list, expected_output):
assert bubble_sort(input_list) == expected_outputIn the updated code, we have modified the test_sorting function using the pytest.mark.parameterize decorator. This decorator allows us to pass multiple sets of input values to the test function. The decorator expects two parameters: a string representing the comma-separated names of the function parameters and a list of tuples where each tuple contains the input values for a specific test case.
Note that the function parameters have the same names as the string passed to the decorator. This is a strict requirement to ensure correct assignment of input values. If the names do not match, an error will be raised during test case collection.
With this implementation, the test_sorting function will be executed four times, once for each set of input values specified in the decorator. Now, let's take a look at the result of the test cases:
===================================================================
test session starts
====================================================================
platform darwin -- Python 3.11.4, pytest-8.1.1, pluggy-1.5.0
rootdir: /pytest_demo
collected 6 items
tests/test_sorting.py .F.... (100%)
=======================================================================
FAILURES ========================================================================
____________________________________________________________________ test_always_fails _____________________________________________________________________
def test_always_fails():
> assert False
E assert False
tests/test_sorting.py:11: AssertionError
=================================================================
short test summary info ==================================================================
FAILED tests/test_sorting.py::test_always_fails - assert False
===============================================================
1 failed, 5 passed in 0.03s ================================================================In this run, a total of six test cases were executed, including four of the test_sorting function and two dummy functions. As expected, only the dummy test case failed.
Now we can say with confidence that our sorting implementation is correct 🙂
Fun practice task
In this article, we introduce the PyTest module and demonstrate its use by testing a bubble sort implementation with multiple test cases. We cover the basic functionality of writing and running test cases using the command line utility. This should be enough to start implementing tests for your own codebases. To improve your understanding of PyTest, here's a fun practice task:
Implement a function called validate_password which takes a password as input and checks if it meets the following criteria:
- Contains at least 8 characters.
- It contains at least one capital letter
- Contains at least one lowercase letter
- Contains at least one digit
- Contains at least one special character (e.g., !, @, #, $, %)
Write PyTest test cases to validate the correctness of your implementation, covering several edge cases. Good luck!
Kanwal Mehreen Kanwal is a machine learning engineer and technical writer with a deep passion for data science and the intersection of ai with medicine. She is the co-author of the eBook “Maximize Productivity with ChatGPT.” As a Google Generation Scholar 2022 for APAC, she champions diversity and academic excellence. She is also recognized as a Teradata Diversity in tech Scholar, Mitacs Globalink Research Scholar, and Harvard WeCode Scholar. Kanwal is a passionate advocate for change and founded FEMCodes to empower women in STEM fields.
