To write tests or to not write tests…

At the Bay Area Clojure User Group this evening, a familiar discussion came up. Yea, about unit-testing in Clojure. And someone said, “But Rich Hickey talks of TDD being guard-rails-driven programming… and who wants to drive their car banging into the guard rails?”

Implying that TDD isn’t particularly useful or required in a language like Clojure. 

So here’s what I said then, and here’s what I say now (and yes, this is the official Zolo Labs stand on this): if you’re Rich Hickey, you don’t need TDD or unit-testing. Otherwise, STFU and write tests.

OK, I didn’t actually say STFU.

9 thoughts on “To write tests or to not write tests…

  1. I’ve been exploring the idea of focusing more on specification than tests. From specification we can generate tests, contracts, more documentation, idiomatic examples of code, and more (https://github.com/ohpauleez/sterling).

    I’ll be sharing more at Clojure/West, but the approach feels more like “the Clojure Way” than unit tests. I’ve found it to pair very well with higher-level functional/systems BDD tests.

    • Sounds like an option.

      The essence of my point is just that most of us aren’t anywhere near guaranteeing correctness of our programs. Going the TDD approach doesn’t imply it either, but it moves us towards having more confidence in our software.

      Anything that moves along that path is a Good Thing.

      The dogmatism that some people have towards TDD is just as bad as some people who are “too cool/smart for tests”. And I’ve met both sorts of folks. I like the pragmatic middle ground (as long as it leans towards some form of verification :-)

  2. Amit,

    I wasn’t there but would have liked to hear other arguments. Your post seems have been a hurried one, you start talking about TDD and end up saying start writing tests (unit?). Very different beasts, isn’t it?

    I started my programming with Fortran on punched cards and even after graduating to slightly better but yet primitive environments, there wasn’t a whole lot of effort on unit tests. I worked on compilers and databases. Runtime assertions were a big use and pretty much a weak form of contract-based approach. And we used large test suites to aid verification. We had people helping us test this software. As developers, we were held in high-esteem or low-regard depending on how many avoidable mistakes we made. That kept most us motivated to think of corner cases and come up with robust designs and avoid brittleness.

    I had heard about TCL EXPECT as a great tool for testing and longed for something similar that I could use in our own work. Well, my wish was granted when Java arrived with a bang in JUnit and I embraced it wholeheartedly and pushed it the TDD way, forcing it on others to be our formal specification. Even unit testing gave us the comfort of being able to refactor without worrying how many things would broken in the big picture. But today, I am not so hot on what goes in the name of TDD or unit testing.

    I see too much ceremony in software development. I don’t know how substance of any seriousness gets built today. I think a lot of people are just taking libraries and assembling solutions together. (I know there are plenty of examples to the contrary and I have never done a formal analysis as to the percentages). While I don’t waste my time in the ceremonial unit testing or the largely unsustainable TDD, I do see value in automated testing. Why even test as a separate activity? All my modules perform a self-test when loaded. As I develop code, I capture the test cases I used, right along the code. So, I can be assured that the code is meeting my minimum expectations all the time.

    IMHO, Rich Hickey and others who happen to very influential and have large fan bases, end up with unintended effect. I am sure RH didn’t say testing was unnecessary. But his guardrail analogy has such sex appeal that young impressionable minds end up absorbing it without the critical thought that it was packaged with. So is the case with the proponents of TDD/unit testing. I have worked with people who thought that a unit test must never require a database. The sex appeal of mocking obscured the fact that the verification of the correctness of that unit of code mandated the presence of its processor — the database.

    Here is an example of a great piece of widely used software, TeX. I have perused through the book and I see no unit tests. I see careful thinking and some really great approaches. It is a weird language/notation but the soundness of it is evident in the large number of packages which run on top of it and the longevity of it. I know of no other software of that quality, perhaps emacs,

    So, I have seen it gone wrong either way. Sensational posts don’t help. I don’t know if RH would agree that , if one is RH, no testing is needed. I simply don’t think a human being can keep so many intricately woven details in his/her head. Perpetuating this myth that RH doesn’t need to write tests will only generate some more RH-wannabees. With the software engineers already thinking no end of themselves (does anybody ever try doing serious math or physics before thinking so highly of themselves and software?), I am afraid this is just not a good idea to perpetuate.

    Sorry, my response is longer than your post.

  3. It’s funny, I think Hickey’s “guardrail programming” comment was not so much about testing as it was about the relative simplicity of an immutable, functional language compared to a stateful, “place oriented” language, and the fact that developers working in non-functional, non-immutable languages have no other choice but to resort to massive, comprehensive test suites because once their knitted castles made of collaborating stateful objects with uncontrolled mutation reach some non-trivial size, they’re no longer able to effectively reason about them. Hence, they have no idea how far out the change they’re about to make is going to reverberate. Clojure programmers still write tests (ask Rich or Stu how big the Datomic test suite is), but because we work in an immutable language and prefer pure functions we’re much more confident that we know all of the effects of the change we’re about to make.

  4. I was at the Strange Loop keynote where Rich talked about TDD and how no one would want to drive by banging into the guard rails. The analogy never made sense to me. It made me wonder if he had ever seen TDD as I know it to be. I left the talk with the impression that it was a clever statement, but ultimately a straw man argument.

    The way I do TDD, and the way I was taught, was for the tests to be representative of the spec, and that you write tests to confirm the spec is implemented… but you do it first, instead of after.

    The main advantages of writing them first is that the spec is very clear up front. All parties involved in the software development process are up-front, very clear what is being worked toward. The disadvantage can be that you may not quite know quite how to test it until you’re done. Usually in cases like that, I just hack until I see what the route I want to take, and then add the test as soon as it makes sense what test to write.

    Either way: test first, or test last, I agree with Amit, that having no tests is a recipe for pain.

  5. I’m organizing a hackathon and started doing screencasts with different programming languages (in spanish). In my experience, TDD saves so much time too. Running code fragments in the REPL is great but the larger the program, the more complicated it gets.

    TDDing with Clojure also helps me determining how I want my functions to be used. I could say a lot more but this is plenty.

    So far I’ve created two videos. They are in spanish and the latest one is about implementing a smallish DSL for fictitious quotations. You can see it here: http://goo.gl/Uakpj

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s