What if Jim Whitehurst developed consumer products? (Interop comments on iterative development)

I'm mulling over the ramifications of Red Hat CEO Jim Whitehurst's keynote at Interop. Here's how InformationWeek described it:

The commercial software industry is failing enterprise customers through overpricing, lengthy development cycles, and products with bloated feature sets that most customers don't use, said Red Hat CEO Jim Whitehurst, who spoke Wednesday during a keynote presentation at the Interop IT Conference and Expo in New York City.

Why?

The main problem, according to Whitehurst, is a commercial development model under which executives, programmers, and marketers get together in an effort to predict what their customers want-and then take five years to build it.

The Register, which also covered the keynote, detailed Whitehurst's proposed solution:

This stands in stark contrast to the iterative and more modest product cycles of providers of open source products and online applications.

Whitehurst compares the iterative model to kaizen, the Japanese business philosophy of continuous improvement that lead to the lean manufacturing techniques first developed at Toyota, and which are now nearly universal in manufacturing.

With lean manufacturing, you work in small groups, you share designs up and down the supply chains, and you make incremental changes in products and improve them gradually. This results in fewer manufacturing defects.

Similarly, the kaizen-like approach to software development embodied in open source projects (as well as in services such as Salesforce.com) have lower levels of software defects than big-bang software products. Whitehurst claimed that he could show that Red Hat's own development efforts had an order of magnitude fewer defects per lines of code than big-bang efforts, thanks to the open source model.

Now in my time as a product manager, we didn't design software to meet the needs of people five years in the future. Frankly, we used a shorter time horizon, though not as short as some of the iterative models that I've seen out there.

But then again, I was not managing a product that was primarily used by IT people. I was managing a product that was primarily used by cops. For some of these customers, they just want to get the software to a point that it works, and then be done with it. I've known of agencies that have refused to upgrade their software for ten years or more - as long as the current software is working, they're happy.

My initial impression - and please correct me with facts if I'm wrong - is that a more iterative development process works better when your customers are technically savvy, or if the iterations are managed in such a way that the customers are never exposed to them. If you're offering a development tool, engineers can handle multiple iterations of the software, and they won't need a glossy four-color manual for every release. If you're offering an enterprise operating system, then automatic upgrades and patches can handle things for most users, and an enterprise IT staff can handle the rest. Even a non-enterprise operating system such as the personal editions of Windows 7 effectively has a monthly release cycle, in which the updates can be handled automatically and don't cause severe disruptions to the way that people work.

But imagine if Microsoft Word were on a more iterative development process, and Microsoft was releasing new versions of Word daily. And let's say that the October 22, 2010 version introduced a new menu item. (Forgive me for my dated references to "menu items," but I use Word 2003 more than I use the later versions of Word.)

What does this mean?

It means that if I walk into a store on November 26 and look at the Word packages available in the store, chances are that I won't know whether the package includes the new menu item or not.

So I go ahead and install it, and I'm asked if I want to get the latest software updates, and I say yes, and the software (and the on-line documentation) is updated. Problem solved?

Well, I need a little more help sometimes, so I go to the Barnes & Noble in Montclair, California (I'm trying to become the mayor, you see) and look at the Microsoft Word books on the shelf. I select the book written by Steven Hodson, which happens to have a publication date of November 2010. What I don't know, however, is that Hodson wrote the book in July, using a beta copy of Word that included some features that made the August 12 release, some that made the August 14 release, and one that was dropped and never released - yet.

After searching in vain for an answer to my menu item question, I do what I should have done in the first place - phone a friend who's more knowledgeable about such things.

"Get OpenOffice," the friend says, "but get it quick before Larry ruins it."

The consumer markets depend upon some level of predictability, and I suspect that people would get very angry at supposed "software churn." Let's face it, people already get mad when Microsoft tells them to perform major operating system upgrades every few years. What if Microsoft started telling them to perform minor upgrades every few days?

Am I wrong here, or does the iterative model fail utterly when applied to the consumer packaged software market?