One day, we were discussing some thoughts about defect root cause analysis in our office. “An exception is always a mistake of a programmer,” a project manager explained his rule of thumb to me. “An analyst would never prescribe an exception in the specs, now would he?”
As a former tester and programmer, I wasn’t able to withstand the temptation to run scenarios in my mind where the exception is exactly the result of analyst’s job - mostly unintenionally, though. For example, various mistakes or omissions in interface description can easily cause an exception. Also, specifying changes in one part of a system while neglecting other affected parts can cause the outdated parts to malfunction. A specification of a change request might not fully take into account all the features that had been built into the system before, thus causing hiccups in some less common situations. Whenever some aspect has managed to escape attention of everybody in the line, it’s hard to blame somebody specifically.
Of course, there are many more ways a exception might sneak into existence without any vicious deed from the programmer. An exception could have been caused by system architect by deciding to use a library that turns out to be crippled with bugs. The excpetion could be the fault of the guy who voted to buy those cheap network hubs from the sale in discount store instead of the more expensive ones that had real metal casings and had actually been tested for compliance with gazillion standards. An exception could be caused by some bizarre setting in application server’s configuration that Mike the Server Guy has tried to tune to gain speed. The occasional strange noises from the air conditioner in server room could be the lead to the memory leaks that seem to occur whenever the sun has been smiling all day.
Now, complacently glazing over the ruins of the “rule of thumb” I just bashed, I’d like to argue whether blaming anybody is necessary at all. In a software development team, which one is more important - producing working software, or punishing people for making mistakes? Mind you, learning, development and finding the optimal solution are truth-seeking activities that inherently include making many mistakes. By shooting down the curious explorers we inadvertently tell everybody to keep stomping the old known track and avoid trying anything new, avoid to evolve. While some mistakes will be avoided and production therefore somewhat higher, killing the desire to enhance oneself will hurt a lot in the long run. The software industry evolves way too fast to afford die-hard developers with only wired-in practices from their early days.
In an environment where hunters are bashing bushes looking for witches to burn, fear is the driving force. Dreams of jelled team or happy employees can be buried six feet under.
What to do instead? How to survive without dead bodies hanging from trees as a lesson for all? In my opinion, mistakes in software should be tolerated, but attention must be paid to not miss their educational value. Striving for deep understanding of every single bug and the factors that allowed it to emerge instead of applying “rules of thumb” not only helps to avoid similar bugs in the future, but also helps to craft a better fix for it and gives deeper understanding of the system. In a team, some most interesting or hideous cases could be published for sharing the knowledge - in neutral objective tone, avoiding to vilify any single person. Unfortunately, more often than not the schedules exist in denial of the fact that properly debugging a feature takes more time than writing the first version of it.
I am happy to see this philosophy giving good results in our product, Changelogic. We do acknowledge that mistakes are made during development, thus we have several traps to catch them - change review, task verification, version acceptance. The traps are there to isolate the customer from the bugs that slip in during development. While we do show statistics about how effective each trap is and how fast tasks are processed by the team as a whole, there is intentionally no easy way to get summaries by person. In my mind, seeing a software product forming from the efforts of a team is way more closer to truth than thinking of it as a sum of results of individual developers. After all, successful individuals are no use when the team fails.
September 21st, 2006
What I’ve been saying so far about refactoring probably looks like this:
Karel Kravik: “What happens if you refactor your software? You devote a lot of resource to it, but its current value remains the same and you’re happy if it still works the same way. So why should one do refactoring?”
And as I’ve read out from feedback, people think that now an arrogant salesman speaks out against refactoring.
Actually, that is not the case.
What I’ve been trying to say is that when you refactor your software or do something that does not directly increase its value, like improving maintainability or simplifying future enhancements, you’re not doing something wrong, but you should consider how it affects your risk profile.
Investments into software
A lot of decisions people make in the software scene can be considered as investments, but people making them are not aware of it. What else would you name the process of putting in money (people’s salaries and other expenses) in order to win back more money later (by selling your products and services)?
When developing software product, you can invest into two kinds of assets:
- Tangible assets: new features, new functionality, bug fixing, etc
- Intangible assets: maintainability, scalability, extendibility, reusability, etc
These decisions in turn affect your product’s current value and future value as well as your risk and payoff.
Let’s put these terms onto a basic chart:
The blue part is probably where the most software falls. This picture does not say how valuable a piece of software is, it just tries to guess the proportion of current value and future value, so the centre is not zero but a 50-50 distribution.
The example from the lower left corner could be a script that parses your account statement and calculates how much you’ve been spending on gasoline opposing to Spring framework in the upper right corner.
But let’s go on and see what we read out of this picture.
How the value is perceived?
Your end users see only current value and tend to underestimate future value meaning short-term money is in the lower left quarter.
In the contrary techies want only to invest into future value and tend to underestimate the need for current value.
Customer Guy: “We need this fancy starring feature like you see in GMail tomorrow.”
Architecture Guy: “Can’t do. Our architecture doesn’t support Ajax so we need to deal with it on the architecture level. Let’s get it right at the first time, this way it will be a lot cheaper than the later refactorings.”
Framework Guy: “You have to begin from a good framework. Everything else will follow. I suggest you to take Snooble Web Toolkit.”
The decision-maker must find the right balance for the situation. If you’re developing open-source project or have shitloads of VC money to put at risk, you can enter the scene from the upper right corner, but if your existence depends on the current value, you have to start from lower left quarter.
Risk and payoff
Investments into intangible values protect you in the situations you have been thinking about. Say you invested a lot of money into architecture scaling up to million concurrent users in the first place and now, you suddenly have one million concurrent users, your architecture works and you make a fortune.
The other alternative is that you have invested a lot of money and the tiny chance that there will be million users won’t realize, so the architecture has been a waste.
Karel Kravik: “It would be reasonable to by a lottery ticket when your chances multiplied by the prize are more than the money you actually pay for it. Usually what you see is exactly the opposite, but people still buy them because it’s pocket change and fun.”
Karel Kravik: “I’m sure aspect oriented programming is fun, but introducing it is no pocket change.”
Imre Lumiste: “The same actually applies to the lower left quarter too – if something is cheap and less risky to implement, doesn’t mean the customers actually want it.”
So the decision maker should consider if the money invested into intangible values and the probability of the situations they’ll make money in is really balanced.
Paths you can choose
Every decision toward concreteness increases current value (assuming it’s made by homo economicus) but also narrows the set of paths you could choose between in the future, that is, it might decrease your future value. So abstraction introduces some risks, but also takes down others – in the way, abstraction is a method of handling uncertainty, too.
If you’ve made enough investments to the values we previously called intangible (maintainability, scalability, etc) you are not so vulnerable to fluctuations in the scene. Let’s take the earlier example – thanks to scalable architecture your systems run regardless if you have 10, 1000 or million users.
Karel Kravik, just couldn’t keep quiet: “The problem is if you make money with 10 users.”
A good example of handling uncertainty with abstraction can be seen in amateurish startups (where there quite a lot uncertainty involved) developing systems that want to be everything to everybody.
Karel Kravik: “When Webmedia was little less than half years old and I had just finished my very first project there, we started a ‘universal content management system’, where everything was described with 3 tables. Later we found that we can describe any database with just 3 tables and even later when the XML hype started we saw that the system could be made even simpler – just one table, one column and one record containing all the XML.”
Weebl: “How rare.”
Stories from the wild - Changelogic
- Changelogic started out as in-house pet project for change management, the first version was 3 screens (change list, adding form, editing form) and was called Arendusweb (Development web in Estonian).
- It slowly gained future value as people used it, so we saw it had some potential; we added English version, task and release management processes, new design for user interface, the new name Changelogic was introduced.
- During the previous year Changelogic made a huge investment into intangible values – the client side was rewritten from scratch in Java (it was in Ant XML/Javascript before), it was redesigned so that it would be possible to support other version control tools without much hassle, it’s a lot more maintainable now, it’s whole lot easier to test the client with automatic tests, the logging is improved, web services were ported to WSDL. Almost nothing that would please our users directly, but it was simple to implement Subversion support after that – a tangible value.
- During step 4 (ongoing efforts) we plan to add support for other branching models, configurable workflow, etc.
Karel Kravik: “Almost every meeting someone bashes me for not including some Webmedia specific properties to tasks or things like that. My answer has always been ‘no’ because that will make it applicable only in-house and cut its future value.”
Disclaimer: the curve on the chart is nothing but a curve, a speculation of what it might look like.
Stories from the wild - Araneaframework
- Aranea too started as a quick need for some framework in project X-Gate, it had basic session management, authentication and things like that.
- After X-Gate was finished, the interfaces were cleaned up, bugs were fixed and the package names were changed from xgate to JWLF (Java Web Layer Framework – don’t blame yourself if you haven’t heard about it).
- At the time some other guys felt an urgent need for a library that would make it easier to build standard lists and forms, so you don’t have to paste around the code that displays 10 rows in a list and adds pages if there are more than 10 records to show.
- Later the lists&forms library was merged with JWLF to something called WM-UI (Webmedia User Interface tools), what cut the nasty and resource hungry XML/XSL transformations coming from JWLF, added JSP support and didn’t expect you to use EJB.
- And now the whole project is hundred times refactored, cleaned up, merged with more dirty code and cleaned up once again, documented, abstracted, generalized and is about to be released as Araneaframework 1.0.
Disclaimer: I’m sure people who have actually written code using the frameworks I named, will consider my interpretation oversimplified, so feel free to correct me.
It’s about the direction
It’s actually not very important how you position yourself on the chart I gave; it’s enough if you can choose the quarter.
However, what is important is the direction where your next decisions will take you and if you’re happy with your new risk profile.
September 1st, 2006