Audit and Assessment
It is important to recognize the difference between an audit and an assessment (Zahran, 1997). The IEEE's definition (IEEE-STD-610) of an audit is as follows :
An independent examination of a work product or set of work products to assess compliance with specifications, standards, contractual agreements, or other criteria.
According to ISO documents (ISO 9000-3), the concepts of certification and audit are defined as follows:
Certification, or third-party assessment (referred to as registration in some countries ), is carried out by an independent organization against a particular standard.
The outcome of an audit is in compliance or not in compliance, or pass or fail. Humphrey's view is that "a software process assessment is not an audit but a review of a software organization to advise its management and professionals on how they can improve their operation" (Humphrey, 1989, p. 149). Zahran (1997, p. 149) provides a comprehensive definition of a software process assessment and its objectives according to the maturity framework:
A software process assessment is a disciplined examination of the software processes used by an organization, based on a process model. The objective is to determine the maturity level of those processes, as measured against a process improvement road map. The result should identify and characterize current practices, identifying areas of strengths and weaknesses, and the ability of current practices to control or avoid significant causes of poor (software) quality, cost, and schedule. The assessment findings can also be used as indicators of the capability of those processes to achieve the quality, cost, and schedule goals of software development with a high degree of predictability. (p. 149)
Depending on who plays the key role in an assessment, a software assessment (or audit) can be a self-assessment (or first-party assessment), a second-party assessment, or a third-party assessment. A self-assessment is performed internally by an organization's own personnel. A second-party or third-party assessment is performed by an external party. The assessing party can be the second party (e.g., a company hires an external assessment team, or a company is being assessed by a customer) or the third party (e.g., a supplier is being assessed by a third party to verify its ability to enter contracts with a customer).
In the SEI (Software Engineering Institute at Carnegie Mellon University) terminology, a distinction is made between software process assessments and software capability evaluations because the two differ in motivation, objective, outcome, and ownership of the results. Software capability evaluations are used by the Department of Defense (DoD) and other major customers for selection and monitoring of software contractors or for assessing the risks associated with the procurement of a given product. The results are known to DoD or the initiator of the evaluation, and no member of the organization being evaluated is on the evaluation team. They are conducted in a more audit-oriented environment. Software process assessments, in contrast, are performed in an open , collaborative environment. They are for the use of the organization to improve its software process, and results are confidential to the organization. The organization being assessed must have members on the assessment team (Zahran, 1997). With the move to the Standard CMMI sm Appraisal Method for Process Improvement (SCAMPI sm ) by SEI, this distinction is going away (Software Engineering Institute, 2000). The same assessment method will be used both for internal improvement and external source selection.