The down side of test first, code second

One interesting quirk of Pinboard is a complete absence of unit tests. I used to be a die-hard believer in testing, but in Pinboard tried a different approach, as an experiment. Instead of writng tests I try to be extremely careful in coding, and keep the code size small so I continue to understand it. I’ve found my defect rate to be pretty comparable to earlier projects that included extensive test suites and fixtures, but I am much more productive on Pinboard.

This quote is by Maciej Ceglowski and I can't agree enough.  When writing code I focus on understanding the problem and breaking it down in my mind (and the code) to manageable parts.  I then prototype the code, mess around with naming, strucutre it logically and only when I'm happy do I write tests at the end.  This is the exact opposite of what most test first die-hards would recommend.

I've found several issues with the traditional test first approach that have led me to write code this way:

  • At the start when writing tests you don't understand the problem and must think it through. You invariably forget stuff in the same way you forget to spec out stuff when writing a 20 page spec if developing by the waterfall method.  By writing code first you actually get into the problem and work through exactly what is needed.
  • You don't write code to make the tests pass.  This is dangerous as you lose the big picture of the problem you're trying to solve.  Instead you write code to make a test pass and there's no guarantee that the test suite you write is complete.  In fact nearly every time I've been pairing using test ping-pong the developer finds missing tests.  This highlights the downfall of such an approach.
  • Tests can't be used as an excuse for defects.  I've worked on several client projects with other companies where the party will respond to an issue on a live site by writing a test and fixing it, passing responsability to the missing test (which they didn't write to start with). This is good practice at a technical level but infuriating from the clients point of view.  Why did this happen?  Because you forgot to write a test and missed out some functionality.  By writing the code first you get into the problem and understand it better so are less likely to miss areas of functionality.  As a second benefit it stops you shifting the blame to tests and makes you face up to the client.  This makes you really think things through when writing critical code.

To be clear I do value tests and for two reasons - as a protection against introducing regressions and as a sanity check at the end of writing a feature.  These tests can be easily written by somebody else against your final API with them working through the logic of the problem as well.

How well does it work?  Last years ago I wrote 5ft Shelf which has had 150,000 page views to date, runs on 10,000 lines of code and has had 3 bug reports.  Digital Delivery App has only been live a few months, runs on 4000 lines of code, and has had 1 bug report so far.  These rates are hugely lower than the other 10 or so rails projects I've consulted on (note most of these, but not all, are multi-developer projects which leads to greater bug rates due to developers incorrectly internalising how the application works)

Strongly linked to keeping my defect row low has been a constant drive for simple code above everything else.  This means not using the latest gem because it's fresh off the shelf & cool and not using some eleborate mega encapsulation system because you understand everything better than anyone else could possibly ever.  In the city (financial district of London) they call this "mental masturbation".  Less crudely Paul Battley sums this up perfectly with "narcissism is one of the biggest causes of technical problems".  Simplicty wins above everything else.