Mutation testing finds 41% faults missed in young Python libraries

The author mutation-tested 50 Python libraries by planting over 2,300 faults, finding that test suites in young libraries missed 41% of planted faults while mature ones missed 17%. The study demonstrates that mutation testing exposes unwritten behavioral assumptions and unchecked boundary conditions. Furthermore, the author presents an open-source tool, qikly, which automates fault planting, spec-drafting via LLMs, and test generation to help catch real software defects. Open article →