Changelogic weblog

Software development is risk taking

With this post I’ll open next top level topic - handling uncertainty in software development.

Let me list some risks from an average software development project:

• Will the functionality be useful for our customers? (”user risks”)
• Will there be showstopper bugs? Problems with updates? (quality risks)
• Will we hit the due date? Will we be in budget? (”resource risks”)
• (this is not full list, you can come up with your own items)

I’m lining them up in this order because they depend on each other in the same order. If users don’t find our product useful, our product cannot be considered as quality product and we must most certainly spend more money to make it better. On the other hand, if we have resource shortage, our product probably won’t be as good as we’d expect.

Now suppose you have exposed yourself to these risks and run into problems.

Jason Fried: “[You can create] Less Software. Less Software allows you to distribute your time and energy across less features. More attention to less stuff will make that less stuff better.”

Karel Kravik: “Sometimes cutting the scope is really an option. But it’s more usable in product development world and not always possible in custom software development where you have your functionality fixed in contract.”

Angry reader: “I didn’t know this kind of contracts still exist!”

Karel Kravik: “Sorry, they do. Go to your Department of Defense and say you’re now going agile. Gimme €15M and let’s see what we can do.”

The application management case

So when we’re starting to develop or maintain software, we’re always taking risks. Sometimes bigger, sometimes smaller but there is always some amount of it.

Milan Skradin, CIO, Krakozhia Telecom: “Downtime of my applications can not be more than 8 hours in a year under any circumstances. I’m losing thousands in an hour.”

Branko Slavic, project manager, Krakozhia Telecom: “I need to deliver new functionality at least every month.”

Yes, the point can be best made with software maintenance. Picture you signed SLA under these conditions.

Mihajlo D., release manager, Krakozhia Telecom, comments with an evil grin: “Keep in mind that every update takes minimum one hour. This is the absolute minimum.”

Although this post is turning out to be more like a play, we must give word to Kristjan Kanarik, project manager, Webmedia: “The only solution is that every update must be successful. We go through rigorous test procedures as usual, no, heck, we’ll double these efforts. We have well-documented acceptance and release procedures, we have excellent people. We can do it.”

Andrus Tamboom, perfectionist, Webmedia: “We’ll write a test robot.”

Karel Kravik, Changelogic salesman: “They use Changelogic to manage the process.”

Hubert, La Haine: “Heard about the guy who fell off a skyscraper? On his way down past each floor, he kept saying to reassure himself: So far so good… so far so good… so far so good. How you fall doesn’t matter. It’s how you land!”

Of course, you do everything to keep the risk down. Documented procedures, senior staff, doubled hardware. But it can only be a question of few minutes and you’re already breaching this SLA.

The problem is that you’re depending on some random outcome. We may argue that it’s not entirely random, yes, it’s just so complex that to a naked eye it’s indistinguishable from randomness. Suppose some anonymous (wished to stay anonymous) programmer made a mistake writing a shutdown procedure for the database engine you’re using. So the database shutdown takes 14 minutes, cold reboot, recover from backup, an hour and a half altogether instead of usual 2 minutes.

It has nothing to do with the professionalism of your team or the quality of your testing procedures. But it’s very hard to argue it was force majeure, too. Nor does it give you any means to justify the bad performance in your annual report.

You cannot observe the generator

No formal methods exist to manage this kind of risks; this is the kind of randomness where you don’t even know what your chances are.

Nassim Taleb, Fooled By Randomness, basing his example onto Russian roulette: “Unlike a well-defined precise game like Russian roulette, where the risks are visible to anyone capable of multiplying and dividing by six, one can not observe the barrel of reality. Very rarely the generator is visible to the naked eye.”

This is what he calls “unstructured randomness”, contrasting it to structured randomness like we see in, say, gambling, where you typically can calculate your odds.

Or to put it even more simply: we don’t know what we don’t know.

Probably the term “random” is what makes it really hard to accept, so let’s rather use the term “unpredictable”; it doesn’t change the result.

Karel Kravik: “Take any of the risks I mentioned in the beginning, call me amateur, but I say they are quite unpredictable. Users can change their mind, no software can be called bug free, estimates are rarely correct.”

My coworker, whispering: “I can almost hear the angry crowd.”

Karel Kravik: “What’s the problem? If they have accepted agile development for the right reasons, that is, because of better risk management and not because of they can write less documentation, they’re already accepting that software development is somewhat unpredictable.”

Martin Fowler, The New Methodology: “Predictability is a very desirable property. However if you believe you can be predictable when you can’t, it leads to situations where people build a plan early on, then don’t properly handle the situation where the plan falls apart. You see the plan and reality slowly drifting apart. For a long time you can pretend that the plan is still valid. But at some point the drift becomes too much and the plan falls apart.”

Martin Fowler: “Usually the fall is painful.”

Why waterfall doesn’t work

We had an example, quite extreme, from software maintenance, but let me bring another one from development world too.

Nassim Taleb, on ITConversations, author’s free transcript: “To predict the trajectory of one billiard ball - trivial, two - it’s OK, but to predict nine you need to take weight of every person in a room into account, because of gravity effect. […] To predict up to 53, you need to take into account every single particle in the universe.”

Karel Kravik, project manager: “Here is a list of about 400 tasks. We have to give estimates. This is a plan for our next year and a half in project Changelogic.”

Imre Lumiste, senior programmer: “Hmm, OK. Let’s say we implement the 10 with the highest impact first; should I be able to estimate how long will the next 10 after these take (and so on) considering they are depending on each other? Of course I can come up with some numbers, but they aint any better than your own estimates. Or any other guy’s from hallway.”

Karel Kravik, wannabee Dilbert character: “Be professional.”

It could be your project plan.

If nothing more, this is one single example of where your predictions are shaky at best. But if you use a bit of imagination, you’d probably find it more like an example why there is so much agility hype out there instead of highly predictive waterfall.

There will be no grande finale about the Changelogic guys inventing new risk management methodologies here, instead of, maybe next time you’re taking responsibilities, think how big may be the part you don’t know about; how surprising turns your venture might take because of the things you didn’t see coming.

Or at least how much you may be underestimating the error rate of your predictions.

Entry Filed under: Handling Uncertainty