Do We Need Another Software Development Methodology?

Do We Need Another Software Development Methodology?

The answer is YES, we do need one. Let me start by asking the mother of all question in software development, and that is why do software projects fail? Most large-scale software projects fail, and this phenomenon is not restricted to large scale projects only. Failures range from the worst-case scenario of projects that are not completed, to projects that are over budget or that fall short of expected goals and objectives. The larger the project, the more likely it is to go belly up, and this often leads to a costly washout for an organization. Even though each less-than-successful project has its own unique causes, I have pinpointed five of the primary factors that underlie most problems:
1. Poor communication
2. Incomplete requirements and specifications
3. Scope issues
4. Inadequate testing
5. Integration

Problem One: Poor communication
One of the main reason projects fail is unclear, inadequate or inconsistent communication between stakeholders. Standard development methods do not include methodologies that support the ongoing and meaningful sharing of information between all project participants. Business users, technical management, outsourced services providers, and individual programmers are often isolated from each other.

Problem Two: Incomplete requirements and specifications
Traditional waterfall development begins with the creation of a comprehensive requirements document. This often massive guideline is designed to provide the blueprint for the entire project. The problem is that, in every project that lasts more than a couple of weeks, specifications will unavoidably change over time. Business requirements change. New aspects of the project come to light. What seemed workable on paper proves untenable in implementation. Despite the almost universal need for changes in requirements and specifications over the course of a project, traditional methodologies do a very poor job in systematically integrating such changes.

Problem Three: Scope issues
One of the most common areas of contention in software projects is “scope creep,” which causes discrepancies in both cost and delivery time. This is not surprising—you would need a crystal ball to anticipate the changes that may be needed in the life of a large-scale software development project whose entire scope can range in months or years. It has been stated that when estimating large projects, you can either be lucky or wrong. Unfortunately, most estimates aren’t very lucky.

Problem Four: Inadequate testing
Traditional development methods save testing to the final phase of a project. This is very problematic because issues with core architecture may not be exposed until a project is almost completed. At that time, correcting such fundamental problems can be very costly or may even require an almost total revamping of the code base. Another factor that adds to testing issues in traditional development is that testing is often under funded and constrained by time. By the end of the project there is usually tremendous time pressure to deliver the completed software, so the first roll-out of the new software is often an unofficial beta test.

Problem Five: Integration
Large projects using traditional development methodologies often fall apart during the integration process. It is often the case that different teams of programmers, who may be geographically dispersed, work in relative isolation on individual parts of the project for long stretches of time. As they finish their work, they typically turn over the code to the project manager who then assigns it to a new team that attempts to integrate the disparate components and make them all work together. Since there had been no consistent communication between the teams during the development process, there are often significant problems at this stage as different interpretations of the requirements or their implementation becomes apparent.