Test metrics are an integral part of the QA process. It can be defined as ‘Standards of Measurement’. Test metrics are the means by which the software quality can be measured. They provide granular information that helps engineers accurately assess the quality of software builds and figure out what is and isn’t working in the testing process. Metrics are the building blocks of progress and improvement in software creation. These metrics provide the visibility into the readiness of the product and give clear measurement of the quality and completeness of the product.
Definition of Software Metrics given by Paul Goodman: Software Metrics is a Measurement Based Technique which is applied to processes, products and services to supply engineering and management information and working on the information supplied to improve processes, products and services, if required.
Contribution of test metrics to a project
Metrics provide insight into the project’s status in relation to predefined goals. ‘You cannot improve what you cannot measure and ‘You cannot control what you cannot measure’. They can be used to inform decisions about the direction to take in development and testing. Some possible metrics and schedules for a project may include:
Defect cause distribution: Charts can be filled with data on what was responsible for various defects affecting stability, compatibility, performance and other attributes.
Traceability matrix: This arrangement ensures the coordination of requirements and features. It makes test coverage assessment much easier.
Test case effectiveness: This metric takes stock of whether test cases are actually finding defects. It may provide a ratio of test cases that resulted in logged defects to total number of test cases.
Defect discovery rate: As the name suggests, this metric chronicles the rate at which defects were found. It may be compiled on a daily and/or weekly basis depending on the requirement.
Changes to test scope: Just like testing is not stagnant, testing routines and requirements aren’t static too and may change frequently throughout a project. Test scope metrics keep track of how often scope was changed, comparing altered tests and procedures to the total number of items in the regimen.
These different metrics enable correlation of cause and effect, analysis of data points and planning for the rest of the application’s lifecycle. These metrics can be useful for QA teams as a framework for making sense of the vast amounts of results that they examine through each day. In turn, evaluating the project’s progress against given metrics can determine whether it goes to market or is retained for additional refinement and screening.
Type of Metrics
Base Metrics (Direct Measure)
Base metrics constitute the raw data gathered by a Test Analyst throughout the testing effort. These metrics are used to provide project status reports to the Test Lead and Project Manager; they also feed into the formulas used to derive Calculated Metrics.
Calculated Metrics (Indirect Measure)
Calculated Metrics convert the Base Metrics data into more useful information. These types of metrics are generally the responsibility of the Test Lead and can be tracked at many different levels (by module, tester, or project).
Why Testing Metrics?
- Without measurement, it is near to impossible to tell whether the process implemented is improving or not.
- Metrics helps in taking decisions for next phase of activities.
- Metrics help in understanding the type of improvement required and helps in taking decisions on process or technology change.
Importance of Metrics
- Metrics is used to improve the quality and productivity of products and services thus achieving customer satisfaction.
- Easy for management to digest one number and drill down, if required.
- Different Metric(s) trend act as monitor when the process is going out-of-control.
- Metrics provides improvement for current process.
Metric is the foundation for any business improvement. It is a Measurement Based Technique which is applied to processes and indicates level of customer satisfaction, easy for management to accept and drill down whenever required and act as monitor when the process is going out-of-control. It is a standard unit of measurement that quantifies results.
Also read: What is Metrics Analysis?
