Property-based testing with Hypothesis for edge case discovery

<p>I want to go beyond example-based tests to automatically discover edge cases in my code. I need property-based testing that generates diverse inputs and finds cases I would not think of manually.</p>

<p>Property-based testing with Hypothesis:</p> <div class="highlight"><pre><span></span><code><span class="c1"># pip install hypothesis</span> <span class="kn">import</span><span class="w"> </span><span class="nn">pytest</span> <span class="kn">from</span><span class="w"> </span><span class="nn">hypothesis</span><span class="w"> </span><span class="kn">import</span> <span class="n">given</span><span class="p">,</span> <span class="n">settings</span><span class="p">,</span> <span class="n">assume</span> <span class="kn">from</span><span class="w"> </span><span class="nn">hypothesis</span><span class="w"> </span><span class="kn">import</span> <span class="n">strategies</span> <span class="k">as</span> <span class="n">st</span> <span class="kn">from</span><span class="w"> </span><span class="nn">app.services.tags</span><span class="w"> </span><span class="kn">import</span> <span class="n">normalize_tag</span><span class="p">,</span> <span class="n">validate_tag</span> <span class="c1"># Property: normalized tag is always lowercase</span> <span class="nd">@given</span><span class="p">(</span><span class="n">st</span><span class="o">.</span><span class="n">text</span><span class="p">(</span><span class="n">min_size</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">max_size</span><span class="o">=</span><span class="mi">100</span><span class="p">))</span> <span class="k">def</span><span class="w"> </span><span class="nf">test_normalize_always_lowercase</span><span class="p">(</span><span class="n">raw</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span> <span class="n">normalized</span> <span class="o">=</span> <span class="n">normalize_tag</span><span class="p">(</span><span class="n">raw</span><span class="p">)</span> <span class="k">assert</span> <span class="n">normalized</span> <span class="o">==</span> <span class="n">normalized</span><span class="o">.</span><span class="n">lower</span><span class="p">()</span> <span class="c1"># Property: normalized tag never exceeds 50 chars</span> <span class="nd">@given</span><span class="p">(</span><span class="n">st</span><span class="o">.</span><span class="n">text</span><span class="p">(</span><span class="n">min_size</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">max_size</span><span class="o">=</span><span class="mi">1000</span><span class="p">))</span> <span class="k">def</span><span class="w"> </span><span class="nf">test_normalize_truncates_to_50</span><span class="p">(</span><span class="n">raw</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span> <span class="k">assert</span> <span class="nb">len</span><span class="p">(</span><span class="n">normalize_tag</span><span class="p">(</span><span class="n">raw</span><span class="p">))</span> <span class="o">&lt;=</span> <span class="mi">50</span> <span class="c1"># Property: normalizing twice gives same result (idempotent)</span> <span class="nd">@given</span><span class="p">(</span><span class="n">st</span><span class="o">.</span><span class="n">text</span><span class="p">(</span><span class="n">min_size</span><span class="o">=</span><span class="mi">1</span><span class="p">))</span> <span class="k">def</span><span class="w"> </span><span class="nf">test_normalize_idempotent</span><span class="p">(</span><span class="n">raw</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span> <span class="n">once</span> <span class="o">=</span> <span class="n">normalize_tag</span><span class="p">(</span><span class="n">raw</span><span class="p">)</span> <span class="n">twice</span> <span class="o">=</span> <span class="n">normalize_tag</span><span class="p">(</span><span class="n">once</span><span class="p">)</span> <span class="k">assert</span> <span class="n">once</span> <span class="o">==</span> <span class="n">twice</span> <span class="c1"># Property: valid tag always validates</span> <span class="nd">@given</span><span class="p">(</span><span class="n">st</span><span class="o">.</span><span class="n">from_regex</span><span class="p">(</span><span class="sa">r</span><span class="s1">'^[a-z0-9._-]{1,50}$'</span><span class="p">))</span> <span class="k">def</span><span class="w"> </span><span class="nf">test_valid_regex_always_validates</span><span class="p">(</span><span class="n">tag</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span> <span class="k">assert</span> <span class="n">validate_tag</span><span class="p">(</span><span class="n">tag</span><span class="p">)</span> <span class="ow">is</span> <span class="kc">True</span> <span class="c1"># Property: Wilson score bounds</span> <span class="nd">@given</span><span class="p">(</span> <span class="n">confirmed</span><span class="o">=</span><span class="n">st</span><span class="o">.</span><span class="n">integers</span><span class="p">(</span><span class="n">min_value</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span> <span class="n">max_value</span><span class="o">=</span><span class="mi">10000</span><span class="p">),</span> <span class="n">total</span><span class="o">=</span><span class="n">st</span><span class="o">.</span><span class="n">integers</span><span class="p">(</span><span class="n">min_value</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">max_value</span><span class="o">=</span><span class="mi">10000</span><span class="p">),</span> <span class="p">)</span> <span class="k">def</span><span class="w"> </span><span class="nf">test_wilson_score_in_bounds</span><span class="p">(</span><span class="n">confirmed</span><span class="p">:</span> <span class="nb">int</span><span class="p">,</span> <span class="n">total</span><span class="p">:</span> <span class="nb">int</span><span class="p">):</span> <span class="n">assume</span><span class="p">(</span><span class="n">confirmed</span> <span class="o">&lt;=</span> <span class="n">total</span><span class="p">)</span> <span class="c1"># Precondition</span> <span class="n">score</span> <span class="o">=</span> <span class="n">wilson_score</span><span class="p">(</span><span class="n">confirmed</span><span class="p">,</span> <span class="n">total</span><span class="p">)</span> <span class="k">assert</span> <span class="mf">0.0</span> <span class="o">&lt;=</span> <span class="n">score</span> <span class="o">&lt;=</span> <span class="mf">1.0</span> <span class="nd">@settings</span><span class="p">(</span><span class="n">max_examples</span><span class="o">=</span><span class="mi">500</span><span class="p">)</span> <span class="c1"># Run more examples for complex properties</span> <span class="nd">@given</span><span class="p">(</span><span class="n">st</span><span class="o">.</span><span class="n">lists</span><span class="p">(</span><span class="n">st</span><span class="o">.</span><span class="n">text</span><span class="p">(),</span> <span class="n">min_size</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">max_size</span><span class="o">=</span><span class="mi">20</span><span class="p">))</span> <span class="k">def</span><span class="w"> </span><span class="nf">test_normalize_tags_deduplicates</span><span class="p">(</span><span class="n">raw_tags</span><span class="p">):</span> <span class="n">result</span> <span class="o">=</span> <span class="n">normalize_tags</span><span class="p">(</span><span class="n">raw_tags</span><span class="p">)</span> <span class="k">assert</span> <span class="nb">len</span><span class="p">(</span><span class="n">result</span><span class="p">)</span> <span class="o">==</span> <span class="nb">len</span><span class="p">(</span><span class="nb">set</span><span class="p">(</span><span class="n">result</span><span class="p">))</span> <span class="c1"># No duplicates</span> </code></pre></div> <p>Key points: - Hypothesis generates and shrinks -- when it finds a failure, it minimizes the input - @given specifies input strategies -- text(), integers(), lists(), from_regex() - assume() adds preconditions without affecting example count - Properties should be true for all inputs -- not just specific values - @settings(max_examples=500) runs more examples for important properties</p>