Introducing Software Engineering Practices
It is estimated that one-third of all software projects are canceled before the software is released. Of the remainder, two-thirds significantly overrun their budgets. Research also shows that more than 80 percent of all project errors are committed in the critical analysis and design phase before actual code is written (see also the CHAOS study of the Standish Group at http://www.standishgroup.com/sample_research/chaos_1994_1.php).
Even though these numbers may vary (depending on who publishes them), they show clearly that there is an essential problem with software development. What is the root cause of the problem? How can software development be made more predictable?
Looking at other engineering disciplines, such as civil engineering, we can see that a structured approach seems to be one of the keys for predictability and repeatability and therefore success. Let's take the example of building a house. To build a house, people typically follow a structured approach. First the budget is secured and then the land is acquired, followed by application for the necessary permits before a detailed plan of the house is developed. All these tasks are necessary to build a house within the constraints of time, cost, space required, and options chosen. It is very unlikely that someone would ask a contractor to build a house without agreeing on a plan that considers all the constraints. This approach seems to make sense. It has worked and will work for building many new houses.
Analyzing this kind of approach reveals some of the deficiencies in software development. Often, software developers write a program or code without having any plan in the form of a functional or design description (not to mention a refined, detailed plan). When that happens, the code often does not reflect constraints such as budget, functionality, modularity, reusability, maintainability, and available technologies. It is as if a building contractor were starting to build a house without knowing the location of the house and the architectural plan.
It is at this point that software engineering methodologies come into play. Software engineering methodologies try to structure software development in the same way other engineering practices are structured to make software development more predictable and repeatable, and therefore more successful.
Before introducing software engineering practices we have deliberately avoided the expression "software engineering." That's because if developers do not follow any methodology, there is no software engineering but only hacking. (See also Steve McConnell, After the Gold Rush: Creating a True Profession of Software Engineering, Microsoft Press 1999).
The next section describes how to find a suitable software development model for a project.