Review Data Analysis

Reviews can be analysed based on a number of metrics.  A few of the review metrics are given below

  • Review coverage             : Reviewed Size/Total size
    • For example, if 50 pages out of 100 pages of an artefact are reviewed, then review coverage in percentage is 50 % only. In case 100% review is not possible, the Project Manager or concerned person can identify the critical areas in the work product which need to be mandatorily reviewed.
  • Review Speed                   : Reviewed Size/Total time taken for review
    • For example, if a 1000 Lines Of Code (LOC) is reviewed in 5 hours, then review speed is 200 LOC/Hr.
  • Review Effectiveness    : Defects caught in Review/Total number of defects caught in review and testing
    • For example, if 100 defects were caught in a project including user acceptance testing and 50 of those defects were caught during reviews, then Review Effectiveness will be 50 %
  • Review Defect Density: Defects caught in Review/ Reviewed Size
    • For example, if 100 defects were caught during code review (reviewed LOC =1000), then Review Defect Density will be 0.1 defects/LOC or 100 defects/KLOC

Analysis is not limited to defining and evaluating a metric. From the observed data recommendations must be put forth. Sometimes, the recommendation itself calls for further analysis.

For example,

In a project ‘X’ where the actual number of defects is found to be 50 % more than the estimated.  This scenario triggers further analysis. And during analysis nothing found wrong with preparation or review of the work product. The work product was prepared in detail and reviews were done by experts. The detected defects were all valid too. Now the question arises, whether there was anything wrong with defect estimation. On further investigation, it was found that data used behind defect estimation tool were not similar to that of project type ‘X’.  Thereafter some improvements were triggered within the tool. And more guidelines were added regarding the applicability of the tool.

This is just a sample analysis. Trigger for analysis is found everywhere. The important thing is to identify and take actions on-time.