• Bytesmiths Editions -- large, archival, fine-art photography on unusual materials
• Bytesmiths Press -- artists' services: web design/hosting, jury slides, giclee reproductions, opening announcements, brochures, etc.
• Champagne Beadworks -- handcrafted jewelry and beadwork
• Crafted By Carol -- handcrafted jewelry and beadwork
• EcoReality, an organization devoted to establishing a sustainable ecovillage
• Ecovillage Newsletter -- Diana Leafe Christian's news of her travels.
• Gemini Gypsy -- Carole Good-Hanson's fused glass frames
• Green Chipper -- light forestry and environmental services.
• Salt Spring Island Society for Community Education -- community education on our island of 10,000.
• Veggie Van Gogh -- two artists' mobile warehouse and living quarters, petroleum-free!
• Veggiemog -- life and times of Kelly O'Toole's Unimog, running on biodiesel

As Dave Thomas of Object Technology International is fond of saying, "Software development is a 'bursty' process." Long periods of time may pass during which nothing is seemingly accomplished, followed by periods of intense development and miraculous productivity.

When one person works alone, this "burstiness" averages out, and its impact is limited to "did I get done what I wanted to in the time I had." However, when the efforts of many need to be coordinated, the 'bursty' nature of development can sink a project, or at least significantly impact its schedule.

Adopt a Spiral Process

Traditional software development follows a "waterfall" process, in which different kinds of activities (such as specification, design, implementation, and test) are allotted sequential time periods, with rigidly defined checkpoints at the end of each phase.

Waterfall works great if you can determine exactly what you want to build, but most modern systems are not so easily specified, or their requirements change during development.

Barry Boehm recognized this weakness, and proposed a scheme by which the various development activities are more tightly integrated. Such a "spiral" process can be classified as iterative , in which portions of a system are re-implemented in each cycle to achieve quality goals, or incremental , in which a system is grown by adding functionality in each cycle. Ideally, these two patterns should be mixed and used as needed.

This spiral process has numerous advantages, mostly in providing the flexibility needed to apply Smalltalk to ill-defined or evolving systems, but there is no free lunch. A spiral process falls down when it is too rigid, or when it must closely follow a corporate waterfall process mandate.

Waterfall is falling out of vogue (overheard at OOPSLA: "If you're not doing incremental development, you're doing excremental development..."), but for well-defined problems with extremely high reliability requirements, waterfall may be more appropriate. If your software is life-critical, you'd better fully understand your requirements up-front, and the flexibility of spiral development is then not as useful.

Avoid Schedule Rigidity

In waterfall, the pre-defined project phases naturally break a project into manageable bits. In spiral, there is no such natural division, and an arbitrary division is often used. These "cycles" are typically assigned the same length, such as 7 weeks.

The functionality assigned to a cycle will be based to some extent on guesswork, especially in an organization's first spiral process project. This shouldn't reflect poorly on the manager, rather it is an acknowledgement of what really happens in most waterfall-based projects! It should help to know ahead of time that the spiral process supports this inherent "truth in planning," and that fuctionality will be shuffled around to different cycles as its meaning and relationships are elaborated.

However, you don't get the full benefit of spiral's flexibility if you are rigid about cycle length. If your project intends to exploit stable periods, some cycles may take less than their allotted time, while few (or none!) should take extra time.

When deciding when to really end a cycle, it's better to postpone functionality than extend the cycle. If a particular feature has not even been started before the last week of a cycle, slip it; don't cram it in!

If you are rigid about both functionality and cycle length, you are simply going to "fail" most of the time -- you deserve a more realistic definition of "success" when working on ill-defined problems!

If you are rigid about functionality, you may end up in the "death spiral," where each extension of the cycle lets you discover problems in the specification or implementation of the functionality, so that you keep trying to squeeze in more work.

The quality of a rigid-functionality cycle's product suffers because the lengthened cycle has shifted emphasis from integrated development to simple coding, at a time when the emphasis should be on integration, testing, and documentation clean-up. The result is a mad push to add a lot of code that will most likely have to be re-written in the next cycle anyway!

Managers who are new to the spiral approach are often reluctant to shorten a development cycle and postpone functionality, due to "waterfall-think" -- "if it doesn't get in now, it'll be dropped altogether and we must have the "DithyWither" function!" Spiral's flexibility means that the relative importance of different features can be continuously re-evaluated, so that if it is really that important, "DithyWither" can be the first thing to happen in the next cycle.

What Is Stability?

Defining stability is like a Supreme Court Justice's definition of pornography: "I can't tell you what it is, but I know it when I see it!" That isn't good enough, though, so here's a working definition: Stability is a condition where features are implemented, integrated, and tested to a planned degree of completion, combined with low rates of recent change.

The combination of planned degree of completion and lack of change is crucial; things that are completed -- including testing -- but have recently changed considerably should not be considered stable. Conversely, incomplete things that have not changed in a long time are not really stable, due to incipient change.

Also note that something does not have to be in its end-form to be stable -- otherwise, the only stable point to exploit would be at project end! Alternating incremental cycles with iterative cycles can provide stable points even during evolution.

Features at a planned degree of completion...

Here's an example of planned completion: John's stuff works with Sue's stuff and both of theirs works with Ed's stuff. Each of them has completed most -- and perhaps all -- of their assigned features for the cycle.

If all of them have completed less than 3/4 of their tasks and it's one week before cycle end, the cycle plan was far too ambitious. If any of them are in this situation, it's time to slip some of their work into the next cycle.

Sometimes, the cycle's last week arrives, and individual modules are complete, but not yet integrated with the entire team's development. If this begins to happen regularly, it's a sign that your project is not practicing continuous integration , but is instead doing "mini-waterfall" development. It may be implemented and tested , but without being integrated , such work is still not stable, and your developers need to review your groupware support, to see if they can do internal integration more frequently.

Low rates of recent change...

If your team is honest and communicating, it's not hard to assess the feature-completion aspect of stability, but you'll need some tool support to accurately assess change rate, sometimes derisively called "code thrash."

With groupware tools such as Team/V® or ENVY/Developer®, it is not too difficult to measure change rate. Even ad-hoc schemes using file-based repositories like SCCS can give you an idea of your change rate, albeit at a gross level.

Using ENVY/Developer information, we measure the ratio of method editions to methods (me/m) between any two app or subapp editions. Obviously, no changes would yield an me/m of 1, but we've seen numbers as high as 20 for highly changing code! Although we've made no formal study, a year or so of gathering this metric leads us to believe that "stable" code will have an me/m below 2.

Like all "sadistics," this one is subject to artifacts and abuse. New Smalltalkers and those unfamiliar with ENVY tend to have inflated me/m rates, not necessarily because their code is unstable, but simply because they insert halt more often, or because they manually revert code back to the way it was, rather than loading a previous edition -- they are merely less efficient users of the environment.

Conversely, a paranoid developer may go out of his way to avoid seeming "unstable" by doing lots of work in workspaces, working on multiple methods at the same time, or not saving methods as often as really needed. Avoid this by building a group culture where measurements are indicators, not juries!

Rather than reducing an entire cycle to a scalar ratio, it would be interesting to "bucketize" method timestamps in a module and plot the resulting histogram against time. A bucket size of one day should be adequate, and will make tallying method timestamps easier. This data gives a manager an indication of intra-cycle stable points, rather than a gross "how stable are we right now" figure.

I'm Stable, What Now?

With modern code management systems, there is no reason for your entire team to "freeze" their development when they near the end of a cycle. Exploiting stability necessarily means dealing with asynchrony within the team.

If John over-estimated his work, and so his stuff is deemed stable two weeks before the planned cycle end, what should he do now?

1. Read news or play Doom,
2. mess around with his code, making it run faster or take less space, or
3. continue with work allotted to the next cycle?

What John wants to do is "1 ," but that usually doesn't work out! What he should do is "3 ," but people are usually afraid of de-stabilizing things by adding new features. What usually happens is "2 ," which often de-stabilizes things as much as "3 " would have!

The key is for John to "checkpoint" his stable work before getting a jump on the next cycle's work. In ENVY, this means making certain that all his stable code is released, and that none of his new code gets released. In extreme cases, this may require working in a new app or subapp edition (or even a different image), although it is best to avoid having more than one editable edition at any given time.

Periods of stability are also ideal points for peer review, although not necessarily on the stable modules! Achieving stability gives developers a chance to plan for their next period of instability, which takes some of the uncertainty away. Design review should be conducted on planned work, and code review should be conducted on stable or nearly-stable work.

Unfortunately, peer review is often done the other way around! There is a strong tendency to review designs once the code is written and stable, when no one is willing to de-stabilize them, and an equally strong tendency to review code that is still in flux, which means very little of the reviewed code will actually end up in the product, since most of it will change. Ignoring stability concerns when planning peer reviews simply wastes the time of the reviewers, and gives a false sense of security.

It's important to be able to recognize incipient instability, or all this flexible planning and asynchronous development goes away. If John suddenly recognizes a design error or discovers a new requirement, he is about to leave stability. If Sue needs to make small changes to a class that handles function "furble," while Ed needs to make a small change to that class to improve behavior for "snotzer," their integration stability decreases, even though their individual changes are benign.

In such cases, it's best to call an early cycle end so that you can capture your stable point before everything comes tumbling down!

Planning For Stability

We hinted earlier that elements of both incremental and iterative spiral methods can be successfully combined. We call this expand-contract development, and it's a life-saver for projects that encounter quality problems related to "rampant featuritis."

In this scheme, periods of added functionality are interspersed with periods whose purpose is to improve the quality of existing functionality. "Quality" is an overloaded term, and any aspect of it could be pursued in intervening periods. Typical quality goals are increased factoring, increased abstraction, increased cohesion, and increased re-use, as well as decreased mass of code, reduced coupling, and reduced variable scope. These periods need not be entire cycles, nor does the entire team need to switch modes in lock step.

Unfortunately, most projects cannot tolerate (or think they cannot tolerate!) budgeting as much as half their time to re-work: "You mean that every six weeks, you're going to take six weeks to re-write what you just did? Why don't you guys just take a ten-minute coffee-break every ten minutes!"

A contract phase by itself does not yield stability, because it is itself changing code. Stability is like fine wine or compound interest: it requires time. By shifting the goal from "more" to "better," the output of a contract phase is much more likely to be deemed stable after one additional cycle than the output of an expand phase.

Conclusion

A spiral development process can be useful for the flexibility it provides to re-evaluate and re-schedule work. Typically, this flexibility results in increased work in a cycle, but rarely results in reduced work in a cycle. This is a missed opportunity!

When a cycle somehow ends up with fewer days than expected, many managers sacrifice quality and stability for functionality, resulting in massive rework in subsequent cycles.

However, with careful attention to points of stability during a cycle, a manager can improve quality, morale, and productivity by shortening cycles as needed, and individuals in the habit of honest self-assessment can help their manager synchronize the group's efforts at the most stable point.