All software development goes through some “lifecycle” during its creation. The lifecycle of a software development project starts with a concept someone has for a piece of software and runs through no one needing the software any longer. The creation of software is generally a very structured process and “waterfall” is a common and long lived methodology for creating software.
The diagram below shows the flow of a software project using the waterfall methodology.
The concept of this model is that a review will be held at the end of each phase. If the project doesn’t pass the review, it stays in its current phase until it can. This is a document driven process and handoff to the next phase is usually a piece of documentation.
This methodology is supposed to work well for products that have a stable definition and stable, proven technologies. The concept is to find issues early in the planning process and avoid changes later in the process when they will be much more costly to implement.
This methodology is also supposed to minimize planning overhead because you do the majority of your planning up front. The documentation that goes with each step provides anyone observing the process meaningful information on the status of the project.
This methodology is also supposed to work well with inexperienced teams because everything is very structured and the structure is supposed to help minimize wasted effort. It’s very easy for an inexperienced programmer to get off track into features that weren’t requested but seem neat to the programmer at the time.
I’ve personally done countless projects using this methodology and in my experience it never works as planned. Everything looks good on paper but the results are generally less than optimal.
From my personal experience with this methodology it goes wrong almost from the start. The software development group expects to receive a specification from the “Requirements Analysis” phase that contains all possible features defined. This is a fallacy as it’s never possible to define all features fully from a concept.
Another issue with this methodology is at each phase gate the project is basically thrown over the fence into the next phase. There is typically very little communication on what happened in the previous phase. The documentation is supposed to provide all the detail you need.
This methodology is also very rigid by design and if anything goes wrong in any phase it’s difficult to back up. It’s generally accepted that someone signed off on the last phase so it must have been right. Convincing someone there is a problem can be very difficult.
Here is a list of problems I’ve encountered on a regular basis using this methodology:
- The requirements gathering phase misses requirements or often misinterprets requirements
- The architect hands the project off to the developer and moves on to another project. The detailed design isn’t given enough examination before coding and debugging start.
- The test cases don’t cover enough functionality and the product isn’t tested thoroughly enough.
- System Testing is not done by the end user. The end user will never see the product until it’s deployed.
The issues I’ve listed cause multiple problems at deployment. The last hand off to the user is typically a painful experience. The user gets the software and they are generally disappointed. In our organization the handoff typically looks something like this:
Business Systems Manager: “Hey Bob! Your software is done! We’re on time and complete just as we promised!”
End User: “What happened to the big button that was supposed to be on the main screen that says “Go”?”
Business Systems Manager: “We thought it would be better if that went in the menu up there at the top so we moved it there.”
End User: “So I have to drop the menu down to get to the go button?”
Business Systems Manager: “Yeah, it’s just one extra click and it makes things look much nicer.”
End User: “Well, we have to hit go every ten seconds and it seems like it’ll be a pain to have to drop that menu down to get to it every time but I suppose if that’s what we have to do…”
End User: “So, where is the data entry field to add the notes from our customer?”
Business Systems Manager: “Oh, we decided to make that a drop down. It’s on the next screen.”
End User: “How is going to work as a drop down? These are variable comments from our customers?”
Business Systems Manager: “Well you didn’t say that when we talked. You provided a list of examples and we thought it would be better if we just included the full list in the drop down.”
End User: “Well that’s never going to work! These are random comments and there is no way you have them all covered in a drop down!”
Business Systems Manager: “Yeah but you said…”
End User: “Yeah well that’s not what I meant!”
In my experience this is when “Phase II” of a software development project starts. Phase II was never planned and the development group has moved on their next project but Phase II is suddenly an emergency and must be done right away. Either that or the “bugs” from the newly released software start pouring in.
The waterfall method may work in some organizations but in my experience it doesn’t work well. What most people end up doing is trying to modify the method which typically doesn’t go well either. There are also many common derivatives of the waterfall method. In general, if possible, I would avoid this method and choose another. In all the projects I’ve done I can’t remember one project that was deployed and used with no changes using this method.
I would recommend almost any iterative software development method that involves the end user over this method. In general, iterative methods produce much more user friendly results and need much fewer changes at the end of the development cycle.