Process improvement for small firms: an evaluation of the RAPID assessment-based method

Abstract

With increasing interest by the software development community in software process improvement (SPI), it is vital that SPI programs are evaluated and the reports of lessons learned disseminated. This paper presents an evaluation of a program in which low-rigour, one-day SPI assessments were offered at no cost to 22 small Australian software development firms. The assessment model was based on ISO/IEC 15504 (SPICE). About 12 months after the assessment, the firms were contacted to arrange a follow-up meeting to determine the extent to which they had implemented the recommendations from the assessment.

Comparison of the process capability levels at the time of assessment and the follow-up meetings revealed that the process improvement program was effective in improving the process capability of 15 of these small software development firms. Analysis of the assessment and follow-up reports explored important issues relating to SPI: elapsed time from assessment to follow-up meeting, the need for mentoring, the readiness of firms for SPI, the role of the owner/manager, the advice provided by the assessors, and the need to record costs and benefits. Based on an analysis of the program and its outcomes, firms are warned not to undertake SPI if their operation is likely to be disrupted by events internal to the firm or in the external environment. Firms are urged to draw on the expertise of assessors and consultants as mentors, and to ensure the action plan from the assessment is feasible in terms of the timeframe for evaluation. The RAPID method can be improved by fostering a closer relationship between the assessor and the firm sponsor; by making more extensive use of feedback questionnaires after the assessment and follow-up meeting; by facilitating the collection and reporting of cost benefit metrics; and by providing more detailed guidance for the follow-up meeting.

As well as providing an evaluation of the assessment model and method, the outcomes from this research have the potential to better equip practitioners and consultants to undertake software process improvement, hence increasing the success of small software development firms in domestic and global markets.