Question for the CS Fediverse: We were talking in a meeting today about test coverage, design by contract, and ensuring you have "enough" tests to cover all of the interesting cases. We lamented that we (in the meeting) don't have good terminology to talk formally about how much of the input space you need to test to have sufficient coverage. Has anyone explored this idea? The notion of equivalence classes (in the input space or the state space) seems very relevant here, so I wonder if there's existing work with insights in that direction.

@jschuster I mean, there's the standard testing technique of equivalence partitioning, but it sounds like you're looking for something with a notion of ground truth completeness...?

@jschuster actually there might be some stuff in Filip Nikšić's work on randomized testing efficacy

@csgordon I think I'm just looking to find some more formal terminology and concepts to talk about otherwise informal notions of "coverage" in general testing. Equivalence partitioning sounds relevant and should give me more good citations to chase, so thank you!

Sign in to participate in the conversation

A Mastodon instance for programming language theorists and mathematicians. Or just anyone who wants to hang out.