Why do software projects fail?

I haven’t worked long in the software industry compared to others who write on this topic, but over the last 12 years of my professional software engineering career, I feel like there are some common themes that continuously re-appear. Having built an IT consultancy with my business partners over the last 7 years, mostly by inheriting and fixing many “train wreck” projects, I’ve been in the unique position of being able to hear a lot of retrospectives and to view the results of poorly delivered software projects.

A younger version of myself would have probably blamed “idiot” developers / BAs / PMs etc. but the longer a person works, the more you become aware of your own idiotic mistakes. You seek to learn from your mistakes, you begin a (slow) process of introducing a little humility into your opinions. This process of humility is pretty important to recognising the “smells” of a potentially “rotting” software project and is one of the key tools that allows you to cut out the “off” parts.

The big revelation that I’ve had recently is that the “smells” I speak of fall into 3 major categories of stench and their solutions can’t be found in tools or methodologies. In fact, having been exposed to projects in other engineering disciplines and in business in general, I think it’s fairly safe to say that these 3 forms of “project disease” are probably common in the failure of most endeavours. Of course “luck”, or a lack of it, could also play an important part in the failure of a project, but there’s no point looking at what you could do to overcome that for obvious reasons (if you want to tackle that I suggest you go get a rabbits foot or a four leaf clover).

The 3 Smells of a Potentially Doomed Project

The 3 smells I speak of are greed, laziness, and distrust. All of these are interrelated and one can lead to another, but after much consideration I feel that you can place the negative actions of a team into of these 3 buckets. I could probably end the post here but I feel the need to ramble a little more and explain in more detail exactly what could constitute one of these 3 stinky habits.

I can imagine there are some people who are probably thinking “what a load of crap… my project doesn’t have any of those symptoms” (I would have been one of those people in the past), and to those people I would urge you to stay and hear me out. In the next 3 sections I’m going to go into detail what could constitute an action that is driven by one of these attitudes.

Greed

So I’m not sure how you understand greed, but I’m not referring to just greed of material objects. I’m also referring to greed in other ways such as the attention and adoration of others. Of course greed occurs in a material fashion, but this is a bit more obvious when it occurs. Such an example may be when an employer asks their employee to work unpaid overtime, or alternatively, an employee requests higher pay for providing sub-standard output.

The greed we forget about is when we ask someone to work overtime, not because we don’t wish to pay them (we may not have control of the budget), but because we need to cover our arse because we royally underestimated the budget. The greed here is not in terms of keeping more money for yourself, and it’s not actually asking for more time, it’s actually asking for greater recognition – “wow your team really pulled it together to push this through, great job!”.

This kind of greed is heavily linked to our ego, our pride, and a whole heap of other human emotions and is very primal as it speaks to our need for recognition. This kind of greed I would say is even more dangerous than the monetary and material kind because more often than not, in the heat of the moment, we don’t even realise we’re doing it and are even more protective of it. Money comes and goes, but it only takes a few bad decisions to undo one’s entire legacy – and people will often do whatever it takes to protect that legacy. Rolf Harris and co. know a lot about this subject.

Laziness

Much like greed, laziness could be thought of in a shallow sense as people try to get away with work. In all honesty, those kinds of people are easy to spot and to deal with. I think the bigger issue here is when people are lazy on only certain aspects of their job. I’ve had software engineers work for me in the past, and they’re extremely hard working. They come in early, leave late, and will do whatever it takes to get a project complete. Then when I go to ask for documentation or details of testing regiments, I get nothing. 

In one case of mine it wasn’t because the engineer didn’t have time, as I didn’t set a deadline for the engineer and trusted that the feature would take as long as it needed to, it was because he didn’t really enjoy testing. This kind of laziness is the worst because it’s a big contributor to poor quality software. Unfortunately there are a lot of monotonous tasks in life that most people don’t want to have to do. The difficult part with growing up is realising that if something is to be done properly, it needs to be done regardless of whether you enjoy it or not.

The unfortunate thing is that in most companies we try to deal with this by “structuring” the boring stuff away. We create “testing” teams and hope that they pick up all the bugs in our software because we were too lazy to properly test it. We hire technical writers so that we don’t have to write documentation ourselves. We have business analysts to talk to the business. We hire developers to cut code for us. In truth, we all need to do a bit of all of these tasks. While some will become experts in a particular area and focus the majority of their tasks in that area, it doesn’t mean that we can allow specialisation to continue to absurdity. So often we have a situation where the right hand has no idea what the left hand is doing because neither actually understands what the other does so meaningful communication is actually impossible.

Distrust

The last part of “laziness” where I speak about “meaningful communication” is really where we now head into distrust. You may not trust someone because they got a “shady look”. This is not the kind of distrust I’m referring to. The kind of distrust I’m talking about is where there is the appearance of a lack of consistency in what someone (or some group) says or does. If someone says they’ll meet you next Wednesday at 11:00pm for lunch, but turns up 30 minutes late, you could think of it as misfortune. If they do it again the following week, distrust around their time keeping skills begins to rear it’s head. I’m pretty sure the next time you set a meeting you will probably choose to turn up 15 minutes late as you know you’ll still probably be waiting another 15 minutes for them anyway.

Distrust is sometimes justified, and at other times it isn’t. I think identifying when it’s actually a problem is the really difficult part but is extremely important as the actions you undertake in response to this decision can lead you down a path of destruction or a path of reconciliation. If the distrust is real because it results from either greed or laziness, being blatant about it could be extremely destructive. I would call greed and laziness “shadow” emotions as they don’t like to be brought out into the light. When they are, people often get extremely defensive and will either shut you out or go on the attack. In either scenario, you’re not getting anywhere.

Alternatively, if the distrust is as a result of poor communication, naming it could be very important. In this situation, the fault belongs to everyone rather than one person or group as it is about poor communciation. As the distrust is not driven by a shadow emotion, bringing the topic out into the light will result in constructive discussions about how communication can be improved.

So how do we fix these 3 stenches?

If you’ve made it this far, you would most probably be looking for a fix. I have one but you are probably not going to like it. It goes back to a lesson my mum tried to teach my brother and I when we were kids, and in all honesty she still tries to teach us about it now. You can’t change how other people are, but you can change how you are and hope that you may become an example to others. I know it sounds like a bit of a cop out but I’m going to provide some examples of how you could do this and how I apply my knowledge to trying to improve bad situations.

Greed

Rather than ask the question “why are people so greedy?”, look to yourself and ask “what am I doing that could be resulting in their greed?”. More often than not, the greed other’s exhibit is in fact a response to our own actions. We could be trying to take all the recognition or we could be asking too much of people and their natural response may be to exhibit the same behaviour back.

On the other side of the coin sometimes we need to stop and ask ourselves “am I being too greedy?” and “can I be perceived as being too greedy?”. If we are aware of our own greed we could end up helping a lot of people around us as they no longer need to seek a way to make us aware of our own greed. Also it’s a lot nicer realising it yourself than having Joe blow up in the weekly staff meeting and calling you an “egotistical prick” in front of everyone.

Laziness

I think after what I wrote in greed you can imagine what I’m going to say here. Rather than ask the question “why are other’s so lazy?”, look to yourself and ask “what am I doing that could be resulting in their laziness?”. Not very original, I know, but it’s the truth. Sometimes we shove work onto others because we aren’t too fond of doing it ourselves. If we do that, how can we expect any different from them. Alternatively if we name the fact we don’t enjoy doing it, but work together to try and get it done, perhaps we can find an alternative way to complete the tasks we don’t enjoy in a more fulfilling manner.

Just like with greed, on the flip side you need to ask yourself “could I be perceived as being lazy?”. It’s much nicer to realise and rectify our own issues before they become an issue for others rather than being pulled to the side by your team lead or manager and being told in a nice way that you need to pull it together or your position is on the line. I’ve had that talk before when I was really young and it really dents your ego – definitely not a pleasant experience to be called out for your laziness.

Distrust

Unfortunately this is not as simply tackled as the last two. I kind of mentioned how to approach this problem when I was describing it. If the issue appears to be one of miscommuncation, then naming it straight up and tackling “miscommuncation” head on could most probably solve the issue. If the issue lies at the heart of greed or laziness, then you need to solve the other issues first before you can see any progress in this one.

Where to from here?

I’m forver pondering about this subject so I’m sure I’ll write more posts on the topic as I reflect more on what I’ve written. Ultimately I think a combination of compassion, understanding, humility, and commitment is required to tackle these 3 stenches. At the root of the methods to tackle these issues, I think a very high level of self-awareness would go a long way to solving these problems completely.

I’ve given EXTREMELY simple approaches to how I try to tackle these issues in the projects and workplaces I’ve encountered, but I would interested in the questions people pose to themselves to increase their self-awareness. If you share them in the comments I’ll be sure to add them into the blog.

Uncategorized

6 thoughts on “Why do software projects fail?

  1. Chris:

    Can’t we flip this around or sideways and call it fear, uncertainty, and doubt? I’d say that greed, laziness, and distrust are *active* characteristics. People demonstrating them are doing so intentionally. Fear, uncertainty, and doubt are more passive. Some projects fail because people simply don’t know what they are doing and are afraid to admit it – whether to others or themselves.

    It’s sort of a “glass half full”, “glass half empty” thing. As you and I have discussed, we know that bell curves apply to almost all skills. (Breathing might be an exception.) By that very definition, many people simply aren’t good at things. What you seem to be describing here is those who actually have skills and *choose* not to apply them properly to get things done.

    As for your solutions, it reminds me of the old saying about being a “beacon of light”. It’s not s simple road to walk, though, is it? At least we can aspire to it.

    M.

    • I completely agree with you Marc that fear, uncertainty, and doubt could be another way to define the same issues I’m referring to. Distrust definitely lies somewhere in between uncertainty and doubt.

      A big reason for why I have chosen to go with greed and laziness is because in my opinion these attitudes are far more negative and less socially accepted. This opinion is probably based a lot on my own personal and cultural bias, but I suspect this is a common sentiment held by peoples everywhere.

      This comes back to being a “beacon of light”, as you put it. It’s much easier to realise that greed and laziness in any form are not acceptable and to work towards freeing yourself of these emotions/activities.

      I think fear can often be justified and I don’t necessarily see it as always negative, it’s more about how you choose to react to that fear. This makes fear a grey emotion, rather than one that seeks to operate in the shadows. If someone fears losing their job, that’s totally acceptable – if they choose to deal with that fear by throwing another under the bus, that’s totally unacceptable.

      Unfortunately I don’t think there are any simple solutions to this problem. I think a common misnomer now-a-days is that “technology” is based in hard science. For our complex issues, I think there needs to be a recognition that we need to seek out “cultural technologies” to find a resolution.

    • No I haven’t but after a quick look at your blog I think we are very aligned in our thinking and practices. I’ll be sure to read more on your blog about it and to research it further.

      The tool that I employ, with the organisations that I consult to, is dialogue mapping. At it’s fundamental core, dialogue mapping seeks to increase shared understanding and shared commitment through facilitation and visualisation of dialogue on a shared display. In order for a group / organisation to get the most of dialogue mapping, we seek to create an environment conducive to having constructive conversations.

      It’s the principles that we use to create a “safe holding environment” which I think we are very closely aligned. For instance, what you refer to as “do no harm”, we have a principle that is very closely related that we refer to as “safety to speak truths”.

      Have you got any articles / books you could recommend so that it’s available for others that see this post?

    • Hi Ramu,

      I totally agree with your points about single-loop and double-loop learning as I too appreciate the importance of what Chris Argyris refers to as the “learning organisation”. Appreciating the importance of learning from past mistakes at both the single-loop and double-loops levels is extremely important to future success with projects.

      With that said, the point I’m trying to make here is that I think before even being able to recognise the lessons that could come from even just the single-loop level, never mind at the double-loop level, you may have a barrier to doing this constructively. If at the core of your team dynamics you have the 3 attitudes I mentioned prevalent amongst the people within the organisation, you won’t even be able to do proper single-loop learning. Many of the lessons will be false and their causes will be incorrectly attributed meaning the remedies will be futile.

      Ultimately my point is that prior to even getting to the level of being able to do single-loop or double-loop learning in a useful manner, the foundation needs to be strong. That foundation is having the right values embedded within your team. If there is a sickness at the core, that sickness will remain and continue to have an impact unless it is acknowledged and treated appropriately.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s