Martin Rinard has been putting forward interesting food for thought for a while which just caught my attention. It’s related to the importance of verification tools. In the past many people have evaluated such tools on programs that are in a production state. While this may be successful, exposing bugs in those programs, Martin says that it is likely that those bugs do not matter! After all, the software is in a production state, it is being used. So if the bug was bad, it had been discovered before! What does matter, he says, are all the bugs that have been eliminated before and all the hundreds of man hours that had to be spent on achieving this. I think he is right and I found this very interesting and encouraging for my work on verification tools.
Addendum
If you are interested in this, you might want to read “Understanding the Value of Program Analysis Tools” by Ciera Jaspan, I-Chin Chen, and Anoop Sharma (CMU/Ebay).
