Anyway, here are the benefits of unit testing, without justification or explanation:
- Easier to maintain code
- Avoiding regressions
- Make API clearer, both by describing it and by encouraging contracts.
- Confidence in changes to code
For new contributors, there are other benefits:
- Confidence in changes to code
- Confidence in the validity of your own patch
- Mechanism for exploring internal system behaviour
Some of the costs are:
- Maintaining tests
- Caring about the performance of tests
- Running the tests
- Debugging tests
For new contributors, there are other costs:
- Finding tests
- Figuring out how to run tests
- Figuring out how to write tests
- Learn unit testing
Is this fair? Is this clear? And exactly where was that post by that guy about unit testing being overrated?
On a half-related and slightly exasperated note, many of the conversations I've been having recently have been about costs and benefits. What happened to doing something because it was the right thing?
6 comments:
Yes - Language is very important in this debate. Test are more important for dynamically typed languages (e.g. python), than for statically typed languages (e.g. C#.)
For example, if you rename a method, and you don't change one of the caller of the method (cause you missed it,) the static language will pick this up at compile time, but with the dynamic language, if you don't have tests, it's going to show up later as a bug.
You can focus on what's right only when there is no cost to consider.
Sure, you don't need as many of the unit tests in a language like Haskell, because you can encode some of the constraints on behavior in the type system and the compiler will handle ensure these.
But, I almost think I would be more inclined to do testing of at least the core structures and functions of my applications if I were in Haskell because in fact it sounds like it is easier. With tools like QuickCheck and HPC and some more advanced checkers as well, there is a large toolkit to do so.
I recently have been re-reading Real World Haskell and Chapter 11 Testing and Quality Assurance (http://book.realworldhaskell.org/read/testing-and-quality-assurance.html) makes me feel like the cost vs benefit ratio is definitely more bang for buck.
I also remember watching a talk by Don Stewart (and perhaps another of the xmonad developers, I can't remember right now) about the development of xmonad and the testing that went on. It was very enlightening.
I feel like I'd be more prone to catch the TDD spirit (well, more correctly the attitude that the code isn't done until it is tested) while using Haskell than I have been using Python. I have not, in fact, got into TDD in Python because it does seem onerous.
> What happened to doing something because it was the right thing?
This isn't a moral issue, you won't go to hell if you didn't write any unit tests. ;-)
Something is the right thing if the costs outweigh the benefits. Unit tests clearly add value to the process, and for me it's just as clear that the value is greater than the cost. Most people at least agree that testing (of some kind) reduces the time-spent-per-bug.
I think the debate is really about the opportunity cost of unit testing; could we be doing some BETTER form of testing, if we weren't writing unit tests?
For me, the answer is "maybe, but I haven't heard about it yet".
Cory, I wasn't talking about testing when I was talking about right things, costs and benefits.
Bice, I disagree..
For me a major benefit is that you save time. I.e you don't have to manually test something 100 times until it works correctly. You just run the test and it should be able to automatically do the test in less time than it would have taken to do it manually. With the added benefit that you can redo the test in a months time without remembering exactly how it works.
Another advantage for me is that it gets you thinking about what exactly you are trying to do before you spend the time going in circles of implementation while you are still trying to figure out what you are trying to do.
Especially if you doing it right:
http://en.wikipedia.org/wiki/Behavior_Driven_Development
Post a Comment