Various improvements in printing test cases by DRMacIver · Pull Request #4694 · HypothesisWorks/hypothesis

DRMacIver · 2026-04-05T18:22:39Z

#4692 made me suspicious so I went digging. I added snapshot tests using syrupy to validate Hypothesis's output on various tests and so we can catch regressions in it, and then I manually reviewed all the output and decided that a lot of it sucked, so I fixed it.

Principle things I fixed:

I removed my previous more limited fix and generalised the logic. Now, whenever we don't need to print as a call because there are comments, and the repr is both valid syntax and shorter, we use that.
I added a _repr_pretty_ for PrettyIter so iterables has good output instead of bad output.
I added logic to strip <locals> from names. This isn't a great solution, because it produces syntax that is valid (and unambiguously refers to the right thing) but won't actually successfully eval because that name is not accessible. I'm not sure there's anything much one can do about this though. I'm just noting that everything we can do here is sortof bad, and I think this is slightly less bad than the status quo (especially given that we have "is this valid syntax?" in a bunch of places in our pretty printing).
I ahem may have got a bit over-enthusiastic and got Claude to write me a custom lambda inliner so that we can have less bad expressions in places where we'd previously have printed a lambda. e.g. "hello" + " " + "world" instead of (lambda x, y: x + " " + y)("hello", "world") or, the motivating example, hashlib.sha1(b'').digest() instead of (lambda b: hashlib.sha1(b).digest())(b'').

Some of these changes it might be best to review commitwise because you can see the corresponding changes they make to the snapshots. In all cases I started with a snapshot of existing code, then made some changes to see how it updated the snapshot.

Liam-DeVoe

Many of these tests are cleanly written as @pytest.mark.parametrize, which will make reviewing this a lot easier! It's a lot of code right now.

DRMacIver · 2026-04-05T18:51:12Z

Many of these tests are cleanly written as @pytest.mark.parametrize, which will make reviewing this a lot easier!

Yeah, that's true, sorry. I'll get them cleaned up once I've got the failing tests I'd missed sorted.

It's a lot of code right now.

FWIW I think there's fairly little non-test code. There's a degree of unavoidable code-quantity in getting reasonable snapshot coverage in place.

I can split it out into multiple smaller stacked PRs if that would be helpful to you though.

…d reprs

When a lambda is used in builds() or map(), try to inline single-use arguments directly into the body expression. For example, (lambda b: hashlib.sha256(b).digest())(b'') becomes hashlib.sha256(b'').digest() Inlining is skipped when parameters are used more than once, when there are varargs/kwargs, when the argument repr isn't valid Python, or when explain-mode comments need the call-style repr. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Consolidate individual _inline() assertion tests into parametrized test_inline_success and test_inline_bail_out, and merge the individual _repr_call tests into the existing test_repr_call_parametrized. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Liam-DeVoe

(first pass, I'd still like to review the logic of when to print as a call vs a repr in more detail)

Liam-DeVoe · 2026-04-13T02:11:59Z


+    def _repr_pretty_(self, printer, cycle):
+        if cycle:
+            printer.text("iter(...)")
+        else:
+            printer.text("iter(")
+            printer.pretty(self._values)
+            printer.text(")")
+



I'm confused why _repr_pretty_ was added. It only differs from __repr__ when there's a cycle. But I'm not sure that is possible in the only place we use PrettyIter (st.iterables), and if it is then we should have a test for it!

No, it differs from repr in a lot of places. e.g. line breaking of the list, if the values have their own pretty repr, etc.

I actually added it because __repr__ was getting skipped by the printing logic and _repr_pretty_ wasn't, but maybe that's no longer true with the new heuristics. It's possible this isn't actually needed to solve the problem I was originally solving, but I do think it solves a different class of important problems.

Liam-DeVoe · 2026-04-13T02:13:37Z

+    try:
+        tree = ast.parse(func_name, mode="eval")
+    except SyntaxError:
+        return None


I've been bitten before by functions like ast.parse throwing all kinds of weird exceptions. For example, I wouldn't be surprised if ast.parse could throw RecursionError. I would prefer a broad except Exception here for safety.

Liam-DeVoe · 2026-04-13T02:15:08Z

+    for repr_str in arg_reprs.values():
+        try:
+            ast.parse(repr_str, mode="eval")
+        except SyntaxError:
+            return None


same here, I'd use except Exception

Zac-HD · 2026-04-13T07:10:59Z

  '''
  Falsifying example: inner(
-      c=test_sampled_from_enum_flag.<locals>.Color.RED,
+      c=test_sampled_from_enum_flag.Color.RED,


I think this is better with .<locals>.

DRMacIver · 2026-04-14T09:24:09Z

@Liam-DeVoe @Zac-HD How would you feel about replacing <locals> with .locals()?

I'm not wed to it and happy to just pull this change. It's arguably worse to have python syntax that doesn't work than invalid python syntax. But I do think given how many of the heuristics in pretty printing check whether it's valid python source it would be good to have valid python source here if we could find some we were happy with.

Liam-DeVoe · 2026-04-14T13:53:08Z

I don't like it. Making up syntax/methods which don't exist but is realistic enough that it plausibly could is the worst of both worlds.

Can we change our checks for is_valid_python_source(s) to is_valid_python_source(s.replace("<locals>", "")?

DRMacIver · 2026-04-14T15:18:33Z

Can we change our checks for is_valid_python_source(s) to is_valid_python_source(s.replace("<locals>", "")?

Hmm. That's not a bad idea.

I think for now though I'm just going to pull it out of this PR. All the solutions seem weird and/or contentious and I think it probably rarely matters.

Revert the <locals> stripping from get_class_name (both Zac and Liam prefer keeping it). Clean up _try_inline_lambda API to return bool instead of None/sentinel, and broaden except SyntaxError to except Exception for safety. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Removed mention of stripping invalid syntax from names.

Liam-DeVoe reviewed Apr 5, 2026

View reviewed changes

DRMacIver force-pushed the DRMacIver/better-reprs-for-all branch 3 times, most recently from a7b4faf to 2823943 Compare April 5, 2026 20:39

Liam-DeVoe reviewed Apr 5, 2026

View reviewed changes

Comment thread hypothesis-python/tests/cover/test_lambda_inlining.py Outdated

DRMacIver marked this pull request as ready for review April 5, 2026 21:44

DRMacIver mentioned this pull request Apr 6, 2026

Add snapshot testing with syrupy #4695

Merged

DRMacIver force-pushed the DRMacIver/better-reprs-for-all branch from d308431 to 22ce429 Compare April 8, 2026 19:59

DRMacIver and others added 6 commits April 8, 2026 21:01

Add pretty repr to PrettyIter so it actually displays right

ed2ce35

Remove primitive-specific rule and instead always prefer shorter vali…

8176130

…d reprs

Strip <locals> from pretty printing

8556437

Add RELEASE.rst for improved repr printing

6360365

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

DRMacIver force-pushed the DRMacIver/better-reprs-for-all branch from 22ce429 to b62162a Compare April 8, 2026 20:01

DRMacIver changed the title ~~Add snapshot testing of Hypothesis outputs, use it to drive improvements to reprs~~ Various improvements in printing test cases Apr 8, 2026

DRMacIver requested a review from Liam-DeVoe April 8, 2026 20:17

Liam-DeVoe added 3 commits April 12, 2026 22:28

simplify typing

4279068

clean up tests

8692604

update stale ambr file

f08713a

Liam-DeVoe reviewed Apr 13, 2026

View reviewed changes

Zac-HD reviewed Apr 13, 2026

View reviewed changes

DRMacIver and others added 2 commits April 16, 2026 13:38

Update RELEASE.rst to remove feature that got pulled

56e2eed

Removed mention of stripping invalid syntax from names.

Conversation

DRMacIver commented Apr 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Liam-DeVoe left a comment

Choose a reason for hiding this comment

Uh oh!

DRMacIver commented Apr 5, 2026

Uh oh!

Uh oh!

Liam-DeVoe left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Liam-DeVoe Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

DRMacIver Apr 14, 2026

Choose a reason for hiding this comment

Uh oh!

Liam-DeVoe Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

Liam-DeVoe Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Zac-HD Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

DRMacIver commented Apr 14, 2026

Uh oh!

Liam-DeVoe commented Apr 14, 2026

Uh oh!

DRMacIver commented Apr 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

DRMacIver commented Apr 5, 2026 •

edited

Loading