FOSSLC is a non-profit organization that specializes in technology and know-how to record conferences with excellent quality. Click on the icons below to view great videos from communities we are actively involved with:

 

If waterfall development sucks, why do so many companies cling to it?

02 Oct

In Why open source code can be more secure, I wrote about how shared costs, risks, and different motivations in an open source context can potentially lead to more secure software over time.

When I read Greg Wilson's Overcoming Barriers to Self-Management in Software Teams, a few more thoughts came to mind. The first was how similar some of the issues are to classic critisisms of the waterfall model. The second thing was regarding how open source practices can help software development in practical terms.

In this article, Wilson notes the outcome from some research analysing what works and what doesn't work for (scrum based) software development from a multi-year study at several companies. Read on...

Typically for closed source software - planning and prioritization is an internal process kept secret because of links to corporate strategy and sales interest. Contrasting this, for a functional open source project, the roadmap, prioritization, and other discussion is usually transparent and available to anyone that wants it. This can help clarity and prioritization within development teams. For now, I am not aware of research indicating the closed planning process vs. open plan is better (faster, more efficient, leads to greater success, etc.) though my leaning is towards open as I feel it helps foster collaboration with partners in which the benefits can outweigh the risks.

In a commercial project (closed or open source) schedules are set to accommodate when a customer feels they want the new software. I can't recall many times in my career where this was enough time to sufficiently  accommodate development. I chuckle when I hear all-too-familiar stories of a closed source release being half over before the development team knows what it is they are supposed to create. Before they've even had much to say, the dates and deadlines are set. Sad but true, this happens too often. This is why I prattle on and on to my students about how much the waterfall development model sucks. You're committing yourself up front when you are at the height of ignorance about your work. Any wonder why projects often go over schedule, over budget, or outright fail often? As a better way of doing things, if the customer is willing to roll up their sleeves and be actively involved in the development, testing, and documentation, and constantly re-evaluate the plan, the results can be tremendously better. This does not necessarily require an open source paradigm, but for some reason the active involvement of the end user seems to be more common for open source development.

Have you every played the Telephone game? This is the game where people sit in a circle and one person whispers something in the ear of the person next to them. By the time the circle is traversed, the message no longer resembles the original message. I've noticed this happens all too often as messages are passed up the chain in a large organization. My theory is that no manager wants to rain on their boss's (and boss's boss in turn) parade. Thus the bad news gets toned down, or sugar coated as it is passed up the organization ladder.

An open source project is perhaps more likely to have developers spread across multiple organizations. Thus, it is potentially more difficult to violate autonomy and make planning pointless for developers. It may be easier to voice opinions based on reality when your job is not necessarily on the line. In a closed source scenario, with the costs and risks born exclusively by one firm, giving an estimate that is "too pessimistic" may kill the project if it can't hit the market window. In this situation, the potential for quicker time to market helps directly, though one should not underestimate the value of accurate and honest information. When there is a high degree of transparency, it is much more difficult to see reality distorted. The Eclipse community's long strong of releases on time each year is a good case to point at for evidence that an open source model can help here.

Meetings are one of my sore spots. Earlier in my career I worked at a firm that had a meeting culture. As an architect, I often had 30hrs+ of meetings per week, my "real work" coding, testing, and figuring things out only started towards the end of the day. This company went bankrupt... any questions?! Conversely, in my humble experience, most open source projects have mini-meetings constantly on IRC, over skype or other mediums. The meetings are focused on a particular issue rather than giving status updates. Perhaps in an open source project, a meeting that is not productive will quickly see no one showing up the next time.

So two parting questions... if I am right in my bias that the waterfall model sucks, then why do so many companies cling to it? And secondly, am I correct that some of the tendencies of open source projects help alleviate some of the problems that can arise in software development? What do you think?

Comments

Water Fall Development

Thanks for sharing this article on Waterfall development. I have heard about waterfall development but not used. It was very nice. Looking for more..................Please continue.

Waterfall model sucks! But ...

The waterfall model is indeed the root of much evil out there in the software-developement world. But one should not forget that many (at least the ones I know) agile methods are based on this methods as their most basic fragment: The Iteration. Every Iteration includes the phases of the waterfall model. The phases don't have equal length and importance in every iteration but they are there. So basically one should be careful saying the waterfall model sucks, because that would mean every iteration sucks :-)Please correct me If I'm wrong,Regards,Michael

waterfall development

Waterfall and agile are two unique development models used in interactive and software production. It's likely the debate over which is better will continue indefinitely, but in reality, both approaches boast significant but different benefits. It's generally accepted that agile, specifically, lends itself well to software development and large-scale website projects. Although a novice Project Manager may feel more comfortable working with a waterfall approach because each step is predictable, letting go of the familiar may well be worth it to make the move to an agile model.

Waterfall does suck

Organizations cling to it because they want to pretend they can deliver software on-time, on-budget, and according to a plan. These are people with limited experience developing software for real otherwise they would plan slack for the many items they do not know or they would instead recognize they will be constantly learning and need to re-evaluate and thus use an iterative model.

SNAFU principle

My theory is that no manager wants to rain on their boss's (and boss's
boss in turn) parade. Thus the bad news gets toned down, or sugar
coated as it is passed up the organization ladder.

Congratulations, you've just discovered the SNAFU principle ;-)

Think about the managers

Your manager grew up on waterfall. They are too lazy to learn agile or other development models. Get used to it!

Software development process

Software development process fusion involves taking different kinds of
processes and tools and utilizing a combination on your project to help
you reach your goals. You aren't just using one particular methodology
or school of thought or toolset, you are using a combination of tools
that fits your unique needs on your project to help create value.

Editorial comment: This looks like you copied and pasted this comment from elsewhere. We removed the link to your site. If you want to link to your business, do not comment anonymously - instead register yourself.