Fail Faster: Staging and Fast Randomness for High-Performance PBT
This program is tentative and subject to change.
Property-based testing (PBT) relies on generators for random test
cases, often constructed using embedded domain specific languages,
which provide expressive combinators for building and composing
generators. The effectiveness of PBT depends critically on the speed
of these generators.
%
However, careful measurements show that the generator performance of
widely used PBT libraries falls well short of what is possible,
due principally to (1) the abstraction overhead of their
combinator-heavy style and (2) suboptimal sources of randomness.
We characterize, quantify, and address these bottlenecks.
To eliminate abstraction overheads, we propose a technique based on
multi-stage programming, dubbed Allegro. We apply this technique to
leading generator libraries in OCaml and Scala 3, significantly
improving performance.
%
To quantify the performance impact of the randomness source, we carry
out a controlled experiment, replacing the randomness in the OCaml PBT
library with an optimized version.
%
Both interventions exactly preserve the semantics of generators,
enabling precise, pointwise comparisons.
%
Together, these improvements find bugs up to $13\times$ faster.
This program is tentative and subject to change.
Wed 7 OctDisplayed time zone: Pacific Time (US & Canada) change
13:30 - 15:00 | Property-Based Testing and Test QualityOOPSLA at East Hall 1 Chair(s): Jonathan Bell Northeastern University | ||
13:30 18mTalk | Block Tests OOPSLA Kevin Guan Cornell University, Pengyue Jiang Cornell University, Milos Gligoric University of Texas at Austin, Owolabi Legunsen Cornell University DOI | ||
13:48 18mTalk | Detecting Flaky Tests by Controlling Nondeterministic API Behavior OOPSLA Hengchen Yuan University of Texas at Austin, Jiefang Lin University of Texas at Austin, August Shi University of Texas at Austin DOI | ||
14:06 18mTalk | Random Testing via Runtime Abstract Interpretation OOPSLA DOI | ||
14:24 18mTalk | Testing Theorems, Fully Automatically OOPSLA Segev Elazar Mittelman University of Maryland, Harrison Goldstein University at Buffalo, Leonidas Lampropoulos University of Maryland DOI | ||
14:42 18mTalk | Fail Faster: Staging and Fast Randomness for High-Performance PBT OOPSLA Cynthia Richey University of Pennsylvania, Joseph W. Cutler University of Pennsylvania, Harrison Goldstein University at Buffalo, Benjamin C. Pierce University of Pennsylvania DOI | ||