It’s all about fast feedback

I’m a big fan of James Coplien.
I appreciate his controversial point of view about Unit Testing, even if I don’t agree with him. Recently he tweeted about this article explaining why integration tests are much better than unit tests.
Let’s explore it together.

Unit test problem

You write code and then run the unit tests, only to have them fail. Upon closer inspection, you realize that you added a collaborator to the production code but forgot to configure a mock object for it in the unit tests.”

In other words he points out that after a refactoring, some tests will fail despite there is no behaviour modification.
But it really depends on how you write your unit tests. When practicing TDD, I’m testing a logical behaviour. Several classes can emerge, I don’t necessarily have one test class for one production class. And when coupled with DDD practices, I can write unit tests without mocking anything, because I test business logic. Also when I strive for applying SRP, I have a few functions per interfaces, meaning one test won’t fail just because I change some interfaces in another place.
That being said, I agree that some refactoring might change some contracts, which requires to change both the production and the test code. My point is: if it is complicated, it is not because of the unit tests.

System tests solution

“While I was enslaved by unit tests, I couldn’t help but notice that the system tests treated me with more respect. They didn’t care about implementation details. As behavior remained intact. they didn’t complain.”
(System tests for the author are almost end to end and include database)

Again, I agree, if it’s a technical refactoring, the tests that use only production code without mocking may be less brittle.
But I also notice that, when a system test goes red after an actual regression, it takes me a while to find out what the actual problem is. Also, when I had some bugs in production, it is harder to reproduce it in my system tests. It encourages me to fix the code without changing the system tests.
And finally, when there is a business change, it hurts much more to change my system tests. I need a few hours just to write the test managing the new behaviour. I have to initialize half of the application just to check 2 lines of code.

 

Pyramid tests is a myth!

“If you’re like me, you were trained to believe in and embrace the testing pyramid.”…”But my experience revealed a different picture altogether. I found that not only were unit tests extremely brittle due to their coupling to volatile implementation details, but they also formed the wider base of the regression pyramid. In other words, they were a pain to maintain, and developers were encouraged to write as many of them as possible.”

I definitely disagree.
I was not trained to believe in the testing pyramid. I was trained to believe in the “get your shit done, faster please”. And using this approach I believed, like the author and many other developers, that system tests have a better ROI.

It’s all about Fast Feedback

So I tried, and it doesn’t work for me because of long feedback (more than a few minutes).
Feedback time is everything when you run your tests hundreds of times per day, on your local machine. And running your tests hundreds of times per day is really important to practice TDD.
Running the tests has to be easy and fast, or it becomes a pain. And when it’s painful, we loose discipline. We start checking Twitter while the tests are running, and that’s the start of the end of our productivity.


Contextual testing

Maybe there are contexts where systems tests have a better ROI than unit tests. But in my context of backend complex application, with lots of business requirements changing every day, it is not the case.
And I’m not speaking theoretically here. I was involved in 6 projects since I started my career. One has no tests at all, two used inverse pyramid, and three used the classic Mike Kohn Pyramid.
No need to talk about the failure of the project without tests.
The two who used the inverse pyramid have commons characteristics: it was really painful to maintain after a few months, and we need one or two developers almost full-time to keep the system tests working. Not mentioning the time we loose to run the tests before pushing our code, meaning that we often pushed without testing and break the build.

The projects I enjoyed the most as a developer, and my customer as well I believe, was when we have a solid unit testing strategy following the classical Mike Kohn Pyramid. We were able to run thousands of unit tests in a few seconds, and keep a few integration tests running only on the server.

To get results in the long run, don’t give up unit tests, at least if you are working in the same context than me.

 

Rework

I discovered DHH during the « TDD is dead» trend with an ambiguous feeling. I highly value (smart) people with different opinions about testing, craftsmanship and software, but at first glance, DHH looks pretty bumptious to me.

Who’s this guy I’ve never heard of before who decides that one of my favourite development tool is dead?

TDD is dead?

First of all I listen to him and his arguments. I always knew TDD has limits, but rarely meet them. In a nutshell, he argues that TDD may hurts design. After some thoughts my feeling is that TDD does not fit well with invasive framework like Ruby on Rails. Does it make TDD or Rails dead? I don’t think so, they are great tools useful in given contexts.

DHH and I are not working in the same context. He’s more in the web fast app context, whereas I’m more in heavy line of business context (usually not even web). The good news is that his thoughts about my work’s style are less biased than people from my context.

Why I read it

I’ve a very simple way to select the books I read: the more I heard about it, the more it goes up in my reading list. I select rework in this way, without knowing DHH was one of the authors (the other one is Jason Fried).
It was a good surprise, and I definitely recognize his style, close of his blog. I also recognize William Zinsser’s influence, something we share. It is a really pleasant book to read.

Content

The content is full of short take-away. I think I would gain more benefice if I read it a few years ago. Some advices sound clearly obvious now. Still I admit it wasn’t this obvious when I discovered the world of software and entrepreneurship in 2009.

 

Favourites take-away

–        Scaling is not mandatory
–        You need less than you think
–        Beware the workaholists
–        Go to sleep
–        Don’t confuse enthusiasm with priority
–        Pick a fight

 

Should you read it?

I think the book stay a must read, especially if you like the direct style of William Zinsser, and/or if you are both entrepreneur and software developer.

It helps me to know more about DHH’s universe.