Skip to Content
Course content

340: Property-Based Testing with Hypothesis

Click on the "Edit" button in the top corner of the screen to edit your slide content.

I've spent a huge chunk of my career writing unit tests. Usually, it goes like this: I write a function, I think of three or four "normal" inputs, maybe one weird empty string or a null value, and I call it a day. The problem is that I'm human, and my imagination is limited. I consistently miss the edge cases that eventually crash the app in production at 3 AM.

That's where property-based testing comes in. Instead of me picking the examples, we tell a library called hypothesis what the properties of our code should be, and it spends its time actively trying to break our logic by generating hundreds of bizarre input combinations we'd never think of.

The logic we're testing

Let's build a small utility for an e-commerce app. We need a function that takes a list of order amounts and returns the total, but with a catch: it must ignore any negative values (which we treat as data errors) and cap the total at 1,000,000 to prevent overflow issues in our legacy reporting tool.

def calculate_total_revenue(orders):
    total = 0
    for order in orders:
        if order > 0:
            total += order
    return min(total, 1000000)

If I were writing a standard test, I'd probably just check [10.0, 20.0] and [-5.0, 10.0]. Boring. Let's see what happens when we let Hypothesis take the wheel.

Defining our first property

In property-based testing, we don't assert that f(x) == y. Instead, we assert that f(x) always satisfies some condition. For our revenue function, one property is: "The result should never be negative, regardless of the input list."

from hypothesis import given, strategies as st

@given(st.lists(st.floats()))
def test_revenue_is_never_negative(orders):
    result = calculate_total_revenue(orders)
    assert result >= 0

When I run this, Hypothesis doesn't just run the test once. It generates a variety of lists—empty lists, lists with massive numbers, lists with tiny decimals—to try and force that assert to fail.

The "NaN" trap

I ran the code above, and almost immediately, Hypothesis slapped me in the face. It found a failure. It didn't use a negative number; it used NaN (Not a Number).

In Python, float('nan') > 0 is False, but adding NaN to a number results in NaN. When the function hit a NaN, the if order > 0 check failed (which is fine), but if I had written my logic slightly differently, or if I had a NaN sneak into the total, the final result >= 0 assertion would fail because NaN >= 0 is also False.

Wait, looking at my code, my if order > 0 actually protects me from NaN being added. But Hypothesis found that if the list is [float('nan')], the result is 0, which is >= 0. So why did it fail? I realized I had a typo in my local version where I used if order != 0. The moment I changed it to !=, Hypothesis found the NaN instantly.

This is the "mentorship" moment: Assume your inputs are malicious. st.floats() includes inf and nan by default because that's where real bugs hide.

Tightening the constraints

Usually, we don't actually want to support NaN in our business logic. Instead of changing the code to handle every possible float weirdness, I can tell Hypothesis to generate "sane" floats. I'll also add a second property: the total should never exceed 1,000,000.

@given(st.lists(st.floats(allow_nan=False, allow_infinity=False)))
def test_revenue_bounds(orders):
    result = calculate_total_revenue(orders)
    assert 0 <= result <= 1000000

Now, Hypothesis is generating thousands of combinations of finite floats. If it finds a way to break this, it will perform "shrinking." This is my favorite part of the library. If it finds a list of 100 numbers that breaks the code, it will automatically try to find the smallest possible list that still breaks it, handing you a minimal reproduction case instead of a giant wall of random data.




📋 Practical Task

Exercise: Validating a String Sanitizer

You are building a username sanitizer that removes all non-alphanumeric characters from a string. You need to ensure the function is idempotent. In software engineering, an idempotent operation is one that can be applied multiple times without changing the result beyond the initial application. In other words: sanitize(text) == sanitize(sanitize(text)).

Your Task:

  1. Implement the sanitize_username(text) function using str.isalnum() or a regular expression.
  2. Use hypothesis.given and st.text() to write a test that proves the function is idempotent for any possible string input (including unicode, emojis, and empty strings).
  3. Ensure your test passes by verifying that calling the function twice on the same input produces the exact same output as calling it once.
Rating
0 0

There are no comments for now.

to be the first to leave a comment.