Changelogic’s branching model
In a previous issue we discussed the evolution of branching models, we identified 4 levels of sophistication, but we didn’t solve the problems occurring on Level 2.
In this issue we take a look at how Changelogic branching model copes with these problems.
The problems when using the “branch-off on release” model were:
- Difference between mainline and release line may become significant, multiple merges bring even more complexity
- Customers cannot handle the situation when bug are fixed in production but not in latest test version
- Developers are not able nor willing match the risk associated with the change to right code line
Sergey, developer: “I committed the code and sent you the time report. My job is done. I’m pretty tired, I’ll go home now.”
Karel Kravik, release manager: “Where did you commit it?”
Sergey, looking angry: “Whaddaya mean where?? Into CVS!”
To find a relief to the first two points mentioned above, Changelogic deploys technique we call “continuous forward integration”. Don’t confuse it with “continuous integration” which is a lot broader term.
Continuous forward integration
Basically, if you would apply continuous forward integration to branch-off on release model, you get the situation where every bug fix committed to release line is also automatically merged into mainline. As a result mainline has every bug fix that is available in release lines from the moment they are committed and also may have some new functionality.
As the bug fixes are usually quite small pieces of code touching only a handful of files, the double integration does not need much effort. If there are conflicts, they are easier to solve compared to the situation where you have a lot of different bug fixes and you don’t exactly know which line of code belongs to which fix.
Sergey, the angry developer: “You mean that every time I commit something to release line, I have to merge it to mainline too? Come on, this is not optimal! I’m a programmer, not integrator.”
Karel Kravik: “There is a separate role called integrator, but you may also deploy the merge your own code pattern.”
Branko Slavic, project manager, Krakozhia Telecom: “At least if a bug is fixed now, it IS fixed unconditionally, I don’t have to mess around and look for the branches where it’s fixed and where it’s not. Or if it’s fixed at all.”
Karel Kravik, Changelogic salesman: “Actually there is a functionality called ‘version differences’ in Changelogic that enables you to find out what changes integrated between any two versions of your software.”
Early integration
Continuous bugfix propagation also promotes early integration as I understand it.
Karel Kravik, configuration manager: “I think the whole point of early integration is that the earlier you have a likely release package assembled, the earlier you can tackle the possible issues (compare it to the cherry picking philosophy, where you choose the changes shortly before release).”
Or to put it in another words: the earlier you discover problems, the cheaper they are to fix.
I don’t know where the problem is, but having spoken to many people about early integration, I found out in their understanding early integration means that they have no branches at all and they propagate committing (integrating) code as soon as there is any code.
Agile developer: “Have you read James Bach? He says: if it exists, I want to test it.”
Karel Kravik: “No problem, if you want to cooperate with tester, share your private environment or private branch.”
In a way I picture software development, changes with different risk levels cannot be integrated into one line, just because we misunderstand the early integration. I’m fully supporting early integration in the meaning I described - once you have established a release line, you should send your current most likely release candidate to testing, even knowing it’s really not there.
Which brings us to risk levels.
Risk levels (the tofu scale)
Laura Wingerd, Practical Perforce: “Release codelines are highest on the tofu scale; they are firm. They don’t change much, and even the slightest changes to them can impact release schedules because of their rigorous review and testing requirements. Development codelines are soft-they’re changing rapidly, the software in them is farthest from release, and there may not even be tests yet for their newest development.”
Karel Kravik: “Along with assessing the codeline’s ability to absorb risk, we should assess the risk of changes and match them.”
If we think about changes flowing into our application, we see that not all of them have the same impact to our application, if we generalize here a little, we’d get the following risk levels (one can always add, but this is a likely classification):
- experimental
- new functionality
- not critical bug fix
- critical bug fix
On the other hand we have release lines with various maturities:
- new development
- stabilization
- accumulated maintenance
- critical bug fix
All we should do is to recognize the risk level of change and match it to a release lines’ where it should go. It’s not one-to-one relation here and it may have some outside constraint like, say, customers wish “I want to see contracts list in the next week’s release”, but it’s also not so hard to map them. It would probably look like this:
- experimental change goes to new development line
- new functionality goes there too
- not critical bug fixes go to stabilization or accumulated maintenance line depending where the bug is found
- critical bug fixes go to stabilization or critical bug fix line, again, depending where the bug is found
There is one more detail we should pay attention - the release lines maturity changes over time, beginning with the new development and ending with closed, which is actually right after the “critical bug fixes only” level, with the difference that we don’t support it any more.
We’re practicing what we preach - let me bring you an example from project Changelogic itself (that is managed in Changelogic too). At this very moment we have the following developments:
- releases prior to 1.29 are not supported
- releases 1.29 and 1.33 get only very critical bug fixes
- release 1.36 is being actively maintained, including some minor new functionality
- release 2.1 is being stabilized, meaning it’s in production usage in-house
- release 2.2 is where the new functionality goes
- there are also some open changes containing experimental code
If you are looking for morale here, it could be something like this: differentiate between change risks and integrate them early to right release lines, and, if you like, try out the continuous forward integration in Changelogic.
Of course, this model does not come without cost; in the next essay we’ll cover parallelism, isolation and another agile practice “keep it releasable”.
Add comment August 2nd, 2006