The systemic failure of the IT industry

Let me tell you a story.
This is a story about a brand new software project, using last technologies and mobile devices, a really important project for BigCorp. They want to use this new product as a vitrine to show how good they are to create mobile applications.

The gold rush

BigCorp invests a lot on this project, quickly a team of 15 people was created. 1 project manager, 1 assistant project manager, 1 scrum master, 3 business analysts, 3 testers 1 architect and 5 developers.
Everybody wants to be in, the global mood was very good, all the requirements were clear, this will be the best project we’ve ever seen.

Sprinting to reality

After 3 weeks it was time for the first release… And the first deadline was missed. Less than half of the expected User Story were produced, and the code was really brittle.
During the retrospective (where project managers and business analysts were missing because they are busy) it was decided to do less automating testing and less refactoring, because it was obviously a lot of time invest in unproductive stuff. We need to deliver more User Stories.

Wall ahead

The project is now 1 year old. Half of the team was replaced. One of the business analysts was fired because he “screwed” up a demo to the company board. The mood is no longer that good. The requirements were not clear enough after all, it’s even hard to know who the users of the system are. The technology is no longer shiny because 2 new frameworks have been created during the year and seem much cooler.
The software is still really brittle, which leads to even less refactoring because “we don’t want to break it”. New attempt to add unit tests were made, but it “costs” too much, this is obviously “not possible for us to do unit testing”. Retrospectives were stopped, because who can afford one unproductive day?

And yet another awful legacy

The project is now 5 years old. No one from the initial team is still working on the project. The team is only 1 developer now, half-time. We ask her to “maintain and fix bugs” that users will find. The code is a nightmare to work with. We have no idea of what is done, and how it’s done.
It creates lots of frustration for the developer, and for the user.

Failure = lesson learned?

Not really, it’s just a normal software project in BigCorp. It’s actually so common that most people in BigCorp think that software projects are always going this way. From developers to the top management, no one is shocked anymore, this is how software project works.

What can we learn from BigCorp?

Software of big companies and governments are usually pretty bad. Because the product is just a mirror of the system that produces it. A system with lots of hierarchy, politics, arbitrary deadline and silos. A system with no idea of the level of communication and collaboration required to achieve great software. A system where the customer does not really exist, because the company is too big to care about them. A system that erases the creativity required for good software.

Most companies have systemic errors. They produce bad software, no matter how skillful the employees are, because the system pushes them in bad habits and make them believe that this is the only way to go.

This explains why it’s so hard to create good software. It’s not only about technical practices and principles, this is also a lot about the system in which this software is produced.

PS: Thanks to Anthony Cassaigne, and on the recommendation of Woody Zuill, I finally read the paper “Nobody Ever Gets Credit for Fixing Problems that Never Happened”. I highly recommand it, many echoes with this article !

 

The violence of software programming

I have no doubt than many jobs are complicated, but I’m convinced that software development is also a really violent one. Mostly because it does not seem violent at all at first sight. But let’s dig a bit in our world to have a better understanding of this “geek job”.

The constant learner

Every seasoned programmer will tell you how important it is to keep your skills sharp. Technologies are running really fast, you have to learn almost daily in order to improve.
Universities are many years late. As a young graduate, you might learn some programming languages, but you will have no clues about how hard building software is. Writing the code is just the emergent part of the iceberg.
In terms of personal investment, this is comparable to what good physicians must do. We need to read many books, to attend conferences, to listen to podcasts, to do regular training…
Everybody acknowledges how hard it is to be a physician, very few acknowledge that being a software developer might require the same kind of learning habits.

The developer paradox

Of course, being hard does not mean it sucks. But most universities and companies explain to us that we can’t stay “developer”. As if it was something to be ashamed of. We need to “evolve” (which often means we need to become manager and fill Excel sheets instead of delivering software). If you don’t take this opportunity to “evolve”, you might stay at the same low salary, which creates a violent paradox.
You must work a lot in order to be good, and you don’t earn much. Basically, you’re considered as one of the lowest roles in hierarchy, even though you’re developing the software that makes the company earn money. This lack of alignment between what you give to the company and what you get as feedback might be so hard that it could lead to burnout. In better case it might just lead to disengagement, either as an old developer or as a new manager. In “best” cases, it leads to deep questioning about your job and how you want to practice it.

Extreme Programming

I once tweeted that XP should not use extreme, because doing good software might not be considered as something extreme. But I think I was wrong, these are extremely violent practices.
Test Driven Development will reveal how bad your architecture and coding habits are. Unit testing will demonstrate how bad you are at predicting software behavior. Pair Programming will show you daily the gap between you and your co-worker. It forces you to reveal how weak you are, because sometimes you don’t remember some simple syntax or shortcuts.
If you can push software each day in production, it will accelerate the feedback loop of unhappy users. If you use mob programming, any malfunction in your team will lead every mob session as a painful disaster of people arguing for ages instead of coding.
These practices are extreme, because following them daily require a deep level of mastery, an egoless approach of your job and a strong ability to communicate at the same time. Three skills that are not common on the same person. Even less common is to find them on each member of a team.

Fighting the violence

How can we be happy as a software developer then?
In theory it’s easy: we want to reduce the developer paradox. In other words, we want to work in places where our job is understood.
In such places, salaries might be higher, but it’s mostly a state of mind. In such places we encourage constant learning, either by helping you to attend conferences or by buying any book you feel useful for your job. We might give you some time at work to practice and learn. In such places we don’t push you to become manager, because we understand the value of seasoned developers. In such places your co-workers will make you feel safe, safe to show that you don’t know everything, safe to fail. In such places nobody will show you how better or higher in hierarchy they are, because they have understood that building software is such a hard thing, that only great team can achieve it. In a great team, everybody feels proud to work with awesome teammates.

If you are working in a place where you can feel this experience, this is great, and your duty is to help more developers to achieve it. Otherwise, do yourself a favor, leave your job and find a better one. The world is full of great companies having a hard time to find good developers.