Software measurement

Most software reliability approaches classify modules as fault-prone or not fault-prone by way of a predetermined threshold. However, it may not be practical to predefine a threshold because the amount of resources for reliability enhancement may be unknown. Therefore, a module-order model (MOM) predicting the rank order of modules can be used to solve this problem. The objective of this research is to make an empirical study of MOMs based on five different underlying quantitative software quality models. We examine the benefits of principal components analysis with MOM and demonstrate that better accuracy of underlying techniques does not always yield better performance with MOM. Three case studies of large industrial software systems were conducted. The results confirm that MOM can create efficient models using different underlying techniques that provide various accuracy when predicting a quantitative software quality factor over the data sets.

Maintaining superior quality and reliability in software systems is of utmost importance in today's world. Early fault prediction is a proven method for achieving this. Tree based modelling is a simple and effective method that can be used to predict the number of faults in a software system. In this thesis, we use regression tree based modelling to predict the number of faults in a software module. The goal of this study is four-fold. First, a comparative study of the tree based modelling tools CART and S-PLUS. CART yielded simpler regression trees than those built by S-PLUS. Second, a comparative study of the least squares and the least absolute deviation methods of CART. It is shown that the latter yielded better results than the former. Third, a study of the possible benefits of using principal components analysis when performing regression tree modelling. The fourth and final study is a comparison of tree based modelling with other prediction techniques namely, Case Based Reasoning, Artificial Neural Networks and Multiple Linear Regression.

Ensuring quality and reliability in software is important with its growing use in day to day life. Having an estimate of the number of faults in software modules early in their life cycles will enable software project managers to direct testing efforts in those considered risky and reduce the waste of resources in testing the entire software system. Case-based reasoning, abbreviated CBR, is one of the methods which predicts the number of faults in a software. The scope of this thesis is two-fold. First, it empirically investigates the effects of the different factors on the predictive accuracy of CBR. Experiments were done to compare different similarity functions, solution processes, and maximum number of nearest neighbors. Second, it compares the predictive accuracy of CBR models with multiple linear regression and artificial neural network models. The average absolute error and average relative error are used to determine the model with a high accuracy of prediction.

Software quality models are tools for detecting faults early in the software development process. In this research, the TREEDISC algorithm and a general classification rule were used to create classification tree models and predict software quality by classifying software modules as fault-prone or not fault-prone. Software metrics were collected from four consecutive releases of a very large legacy telecommunications system with six subsystems. Using release 1, four classification tree models were built using raw metrics, and another four tree models were built using PCA metrics. Models were then selected based on release 2. Releases 3 and 4 were used to validate the selected model. Models that used PCA metrics were as good as or better than models that used raw metrics. This study also investigated the performance of classification tree models, when the subsystem identifier was included as a predictor.

The ability to efficiently prevent faults in large software systems is a very important concern of software project managers. Successful testing allows us to build quality software systems. Unfortunately, it is not always possible to effectively test a system due to time, resources, or other constraints. A critical bug may cause catastrophic consequences, such as loss of life or very expensive equipment. We can facilitate testing by finding where faults are more likely to be hidden. Case-Based Reasoning (CBR) is one of many methodologies that make this process faster and cheaper by discovering faults early in the software life cycle. This is one of the methodologies used to predict software quality of the system by discovering fault-prone modules. We employ the SMART tool to facilitate CBR , using product and process metrics as independent variables. The study found that CBR is a robust tool capable of carrying out software quality prediction on its own with acceptable results. We also show that CBR's weaknesses do not hinder its effectiveness in finding misclassified modules.

The key to developing high-quality software is the measurement and modeling of software quality. In practice, software measurements are often used as a resource to model and comprehend the quality of software. The use of software measurements to understand quality is accomplished by a software quality model that is trained using software metrics and defect data of similar, previously developed, systems. The model is then applied to estimate quality of the target software project. Such an approach assumes that defect data is available for all program modules in the training data. Various practical issues can cause an unavailability or limited availability of defect data from the previously developed systems. This dissertation presents innovative and practical techniques for addressing the problem of software quality analysis when there is limited or completely absent defect data. The proposed techniques for software quality analysis without defect data include an expert-based approach with unsupervised clustering and an expert-based approach with semi-supervised clustering. The proposed techniques for software quality analysis with limited defect data includes a semi-supervised classification approach with the Expectation-Maximization algorithm and an expert-based approach with semi-supervised clustering. Empirical case studies of software measurement datasets obtained from multiple NASA software projects are used to present and evaluate the different techniques. The empirical results demonstrate the attractiveness, benefit, and definite promise of the proposed techniques. The newly developed techniques presented in this dissertation is invaluable to the software quality practitioner challenged by the absence or limited availability of defect data from previous software development experiences.