My unit tests have taken over my life this weekend. This post is a brief respite…
Unit tests are one of the great successes of modern approaches to programming. If written briefly about them before and Egon has an introduction of how they are used in Eclipse. Unit tests are often seen as part of “eXtreme Programming” though it’s possible to use them more or less independently. Anyway, I was hoping to add some molecule building extensions to JUMBO, and started on the CMLMolecule.class. Even if you have never used Unit tests the mental attitudes may be useful.
I then discovered that only half the unit tests for this class were written. Purists will already have judged me on this sentence – the philosophy is to write the tests before the code. For the CMLMolecule class this makes excellent sense. Suppose you need a routine:
Point3 molecule.getCentroid3()
then you first write a test. You might create a test molecule (formally a “fixture”), something like:
CMLMolecule testMolecule = makeTestMolecule();
Point3 centroid = testMolecule.getCentroid3();
Point3Test.assertEquals("centroid",
new Point3(0.1, 0.2, 0.3), centroid, EPSILON);
I haven’t written the actual body of getCentroid3(). I now run the test and it will fail the assertion (because it hasn’t done anything). The point is I have a known molecule, and a known answer. (I’ve also had to create a set of tests for Point3 so I can compare the results).
You won’t believe it till you start doing it but it saves a lot of time and raises the quality. No question. It’s simple, relatively fun to do, and you know when you have finished. It’s tempting to “leave the test writing till later” – but you mustn’t.
Now why has it taken over my weekend? Because I’m still playing catch-up. At the start of the year JUMBO was ca 100,000 lines of code and there were no tests. I have had to retro-fit them. It hasn’t been fun, but there is a certain satisfaction. Some of it is mindless, but that has the advantage that you can at least watch the football (== soccer) or cricket. (Cricket is particularly good because the cycle between action (ca 1 minute) often coincides with the natural cycle of edit/compile/test).
It’s easy to generate tests – Eclipse does it automatically and makes 2135 at present. If I don’t add the test code I will get 2135 failures. Now Junit has a green bar (“keep the bar green to keep the code clean”) which tells you when the tests all work. Even 1 failure gives a brown bar. The green bar is very compelling. It gives great positive feedback. It’s probably the same limbic pathways as Sudoku. But I can’t fix 2135 failures at one sitting. So I can bypass them with an @Ignore. This also keeps the bar green, but it’s a fudge. I know those tests will have to be done some day.
So 2 days ago JUMBO had about 240 @Ignores. Unfortunately many were in the CMLMolecule and MoleculeTool class. And the more tests I created, the more I unearthed other @Ignores elsewhere.
So I’m down to less than 200 @Ignores now. I’ve found some nasty bugs. A typical one is writing something like:
if (a == null)
when I mean
if (a != null)
This is exactly the sort of thing that Unit tests are very good at catching.
So I here’s my latest test
oh dear! it’s brown. I’ll just @Ignore that one test and…
and it’s GREEN!
(But I haven’t fooled anyone. The @Ignore is just lying in wait to bite me in the future…)
The thing about writing tests beforehand is not just “to be pure”. Tests that you write beforehand tend to be different than tests you write afterwards.
Think about it…the tests that you write afterwards, you write them to pass. They set in stone whatever you’ve already done. If you write them beforehand, they create the specification of the code. They make you consider what you pass into the function, and what it returns. They make you think about corner cases. You almost never write tests for corner cases if you write the tests afterwards…why not? because you’re afraid they might fail (of course, you may be braver than me :-)!
BTW, are you familiar with code coverage tools? These check what percentage of your code is covered by tests. Also, there are some interesting tools for Python that test the tests. In order to check whether your tests are any good, they mutate the source code. If your tests don’t fail when the source code is mutated, then there aren’t much good. Interesting idea!
Hi Noel, Peter,
I find it’s the other way around for me. When I write tests first I tend to write a test that describes the simplest success case (because I’m essentially designing by test). When I test afterwards I feel like I haven’t really done anything if I do that, so I do a bit of input parameter boundary testing, verify exceptions etc. Still easy to fix – I agree that it’s a tough discipline to actually get into the mindset of really putting existing code through the wringer. I’m hoping that peer code review will help here.
I think you’ve got to be *really* careful with code coverage tools – percentage coverage is a really crude metric of test suite quality. There are some bits of code that can’t be tested, some that needn’t be tested, and some that are more important to test than others. Code coverage is useful when used to give basic guidance on which areas need more testing, but it’s too easy to fall into the trap of writing tests just to improve the coverage metric.
Java has a mutation test tool too: http://jester.sourceforge.net/
P.S. Peter doesn’t mention the extra bonus of unit testing whilst watching the cricket; unlike England’s one day team, your unit tests rarely create a commanding lead only to crumble and collapse.
(1) (2) I’m closer to Jim (though this isn’t a major issue here). JUMBO is a library, not an application, and so the functionality is very modular. Maybe I want to write:
molecule.getCentroid3D()
I’m likely to have a clear idea of what this is going to do, though not necessarily the corner tests (== “unusual” cases). So I write the test with a pre-prepared molecule (a few atoms and coordinates that I can check by hand – I have to add N numbers and divide by N – should be able to do that). As I write the actual routine and come across
x = sumx/N
I think “what if N is zero? I might write a test for that – a molecule with no atoms”- return null? Or I think – what if only some atoms have coordinates (this is not uncommon) – perhaps I should throw an exception? Or maybe add a flag to allow the user to direct whether I should just take those atoms with coordinates? So the method and the test evolve as I think.
But If I have a clearly defined task I might take Noel’s route.