Problems in Programmers’ Paradise

Technology is a useful servant but a dangerous master.

— Christian Lous Lange

The State of the Union in software development

I was talking recently to a software company CEO who admitted – with some shame – that he had lost control of his Engineering team. Many of the projects and products in his company were drifting along with seemingly infinite schedules, infinite scope and infinite costs. Given that he was running a large publicly-traded software company, this was a very concerning statement (worse yet, I owned shares in his company, so it was also concerning to me personally). He indicated that, in discussions with his CEO peers, they conveyed the same sense of hopelessness and helplessness. Is it possible that those in charge of controlling the business have lost control of their charges? Or, put another way, are the inmates [1] running the asylum [2]?

In discussions all over the world, I hear a similar refrain from C-level executives all the way down to first-level Engineering managers, as well as Product and Project Managers. If you are a manager or executive in a software company or any enterprise that makes significant use of technology, it is very likely that you feel that you have lost control of the technical people and projects in your organization. If I’m correct with that guess, I would further predict that you feel some or all of the following:

• You’re not comfortable that you know when key projects will be completed or even if they will be completed successfully
• You can’t explain or justify the costs – past, present and future – of the technical projects in your organization
• You don’t entirely understand what your technical teams are doing, partly because you don’t even speak the same language (and I don’t mean the same native tongue, I mean the same vocabulary), and
• You don’t have any idea of how to make any of this better

If this is a genuine problem, how do we go about solving it? This requires us to go back to first principles and treat the process of developing software like any other system – understand the parts of the system, deconstruct how they interact, identify which parts are most important and which parts are causing trouble. Then we find root causes and fix the problems.

In this series of posts, I will do exactly that. I propose that, by the end, we will have an explanatory model for understanding a large portion of technology project failures and – more importantly – a framework for fixing and preventing such problems in the future. It’s a bold claim, one often made by advocates of methodologies such as Scrum [3] and, of course, by many high paid consultants [4] and job candidates for Engineering leadership roles [5].

After I do that over the next few posts, I will present a few real-life examples of common problems so that we can have something to diagnose and discuss. I think that you’ll see some familiar situations in the examples, in the same way that we see a Dilbert cartoon and wonder whether Scott Adams has installed surveillance cameras in our offices.

I believe that when the series is done, you will agree on the power of the models and concepts presented here and you will be eager to put them into practice [6]. I look forward to sharing the ideas with you (and hearing back from you via your comments) as we move ahead.

This post is based on or excerpted from the upcoming book “De-Engineering the Corporation” by Darryl Ricker

Footnotes:

[1] My apologies for suggesting that Engineers are treated like inmates, although in some places I have worked, this wouldn’t be a complete exaggeration.

[2] My apologies for suggesting that Technology companies are like asylums, although in some places I have worked, this wouldn’t be a complete exaggeration.

[3] For the record, I am a huge fan of Agile and Scrum and nothing I will be writing in the series will discredit their potency. In fact, in one post (here), I explain why Scrum actually works.

[4] For the record, I am less of a fan of high priced consultants.

[5] Who, for the record, promise a lot on their way in and often fail in practice because they don’t truly understand how to fix Engineering problems, primarily because they don’t understand the model that I will present here.

[6] Lest you think this model is a whimsical, theoretical one, it’s based on 29 years of experience working in and managing software companies. My specialty is fixing broken Engineering teams and, through the application of the concepts I will describe, I have managed to be pretty successful at that. Having said that, it took 29 years of mistakes and failures mixed in with the successes to get to the point where I am today. As those that work with me today can attest, I continue to make mistakes every day and, subsequently, try to learn from them.