Software measurement

Software managers are under pressure to deliver reliable and high quality software, within a limited time and budget. To achieve this goal, they can be aided by different modeling techniques that allow them to predict the quality of software, so that the improvement efforts can be directed to software modules that are more likely to be fault-prone. Also, different projects have different resource availability constraints, and being able to select a model that is suitable for a specific resource constraint allows software managers to direct enhancement techniques more effectively and efficiently. In our study, we use Rule-Based Modeling ( RBM) to predict the likelihood of a module being fault-prone and the Modified Expected Cost of Misclassification (MECM ) measure to select the models that are suitable, in the context of the given resource constraints. This empirical study validates MECM as a measure to select an appropriate RBM model.

In today's world, high reliability has become an essential component of almost every software system. However, since the reliability-enhancement activities entail enormous costs, software quality models, based on the metrics collected early in the development life cycle, serve as handy tools for cost-effectively guiding such activities to the software modules that are likely to be faulty. Case-Based Reasoning (CBR) is an attractive technique for software quality modeling. Software Measurement Analysis and Reliability Toolkit (SMART) is a CBR tool customized for metrics-based software quality modeling. Developed for the NASA IV&V Facility, SMART supports three types of software quality models: quantitative quality prediction, classification, and module-order models. It also supports a goal-oriented selection of classification models. An empirical case study of a military command, control, and communication system demonstrates the accuracy and usefulness of SMART, and also serves as a user-guide for the tool.

In today's competitive environment for software products, quality has become an increasingly important asset to software development organizations. Software quality models are tools for focusing efforts to find faults early in the development. Delaying corrections can lead to higher costs. In this research, the classification Bayesian Networks modelling technique was used to predict the software quality by classifying program modules either as fault-prone or not fault-prone. A general classification rule was applied to yield classification Bayesian Belief Network models. Six classification Bayesian Belief Network models were developed based on quality metrics data records of two very large window application systems. The fit data set was used to build the model and the test data set was used to evaluate the model. The first two models used median based data cluster technique, the second two models used median as critical value to cluster metrics using Generalized Boolean Discriminant Function and the third two models used Kolniogorov-Smirnov test to select the critical value to cluster metrics using Generalized Boolean Discriminant Function; All six models used the product metrics (FAULT or CDCHURN) as predictors.

Reliability is becoming a very important and competitive factor for software-based products. Software metrics-based quality estimation models provide a systematic and scientific approach to detect software faults early in the life cycle, improving software reliability. Classification models for software quality estimation usually classify observations into two groups. This thesis presents an empirical study of an algorithm for software quality classification using three groups: Three-Group Classification Model using Case-Based Reasoning (CBR). The basic idea behind the algorithm is that it uses the commonly used two-group classification technique three times. It can also be implemented with other quality estimation methods, such as Logistic Regression, Regression Trees, etc. This work evaluates the obtained quality with that from the Discriminant Analysis method. Empirical studies were conducted using an inspection data set, collected from a telecommunications system. It was observed that CBR performs better than Discriminant Analysis.

Legacy software systems may go through many releases. It is important to ensure that the reliability of a system improves with subsequent releases. Methods are needed to identify decaying software modules, i.e., modules for which quality decreases with each system release. Early identification of such modules during the software life cycle allows us to focus quality improvement efforts in a more productive manner, by reducing resources wasted for testing and improving the entire system. We present a scheme to classify modules in three groups---Decayed, Improved, and Unchanged---based on a three-group software quality classification method. This scheme is applied to three different case studies, using a case-based reasoning three-group classification model. The model identifies decayed modules, and is validated over different releases. The main goal of this work is to focus on the evolution of program modules of a legacy software system to identify modules that are difficult to maintain and may need to be reengineered.

The increased reliance on computer systems in the modern world has created a need for engineering reliability control of computer systems to the highest standards. Software quality classification models are one of the important tools to achieve high reliability. They can be used to calibrate software metrics-based models to predict whether software modules are fault-prone or not. Timely use of such models can aid in detecting faults early in the life cycle. Individual classifiers may be improved by using the combined decision from multiple classifiers. Several algorithms implement this concept and are investigated in this thesis. These combined learners provide the software quality modeling community with accurate, robust, and goal oriented models. This study presents a comprehensive comparative evaluation of meta learners using a strong and a weak learner, C4.5 and Decision Stump, respectively. Two case studies of industrial software systems are used in our empirical investigations.

Software-based products are part of our daily life. They can be encountered in most of the systems we interact with. This reliance on software products generates a strong need for better software reliability, reducing the cost associated with potential failures. Reliability in software systems may be achieved by using additional testing. However, extensive software testing is expensive and time consuming. Software quality classification models provide an early prediction of a module's quality. Boolean Discriminant Function (BDF), Generalized Boolean Discriminant Function (GBDF), and Rule-Based Modeling (RBM) can be used as classification models. This thesis demonstrates the ability of GBDF and RBM to correctly classify modules. The introduction of the AND operator in the GBDF model and the customizable outcomes for the rules in RBM, enhanced the discriminating quality of GBDF and RBM as compared to BDF. Furthermore, they also yielded better balances for the misclassification rates.

Developing highly reliable software is a must in today's competitive environment. However quality control is a costly and time consuming process. If the quality of software modules being developed can be predicted early in their life cycle, resources can be effectively allocated improving quality, reducing cost and development time. This study examines the C4.5 algorithm as a tool for building classification trees, classifying software module either as fault-prone or not fault-prone. The classification tree models were developed based on four consecutive releases of a very large legacy telecommunication system. The first two releases were used as training data sets and the subsequent two releases were used as test data sets to evaluate the model. We found out that C4.5 was able to build compact classification trees models with balanced misclassification rates.

Time and cost are among the most important elements in a software project. By efficiently using time and resources we can reduce costs. Any program can potentially contain faults. If we can identify those program modules that have better quality and are less likely to be fault-prone, then we can reduce the effort and cost required in testing these modules. This thesis presents a series of studies evaluating the use of Case-Based Reasoning (CBR ) as an effective method for classifying program modules based upon their quality. We believe that this is the first time that the mahalanobis distance, a distance measure utilizing the covariance matrix of the independent variables which accounts for the multi-colinearity of the data without the necessity for preprocessing, and data clustering, wherein the data was separated into groups based on a dependent variable have been used as modeling techniques in conjunction with (CBR).

Reliability is becoming a very important and competitive factor for software products. Software quality models based on software metrics provide a systematic and scientific way to detect software faults early and to improve software reliability. Classification models for software quality usually classify observations using two groups. This thesis presents a new algorithm for classification using three groups, i.e., Three-Group Classification Model using Case Based Reasoning. The basic idea behind the algorithm is that it uses the commonly used two-group classification method three times. This algorithm can be implemented with other techniques such as logistic regression, classification tree models, etc. This work compares its quality with the Discriminant Analysis method. We find that our new method performs much better than Discriminant Analysis. We also show that the addition of object-oriented software measures yielded a model that a practitioner may actually prefer over the simpler procedural measures model.