Category: Quantitative Management

Posted on

Baselines are created for the core “value-generating” processes of the business in the organizations. From the observed measurement data, an organization comes up with various process performance baselines (PPB) periodically. In a software industry, there can be PPBs for coding speed, defect density, productivity, testing speed, review effectiveness etc. Then measurable improvement targets (process performance objectives) are set for the selected processes. Say for example an objective could be to increase the coding speed by 10 % (Definitely some improvement initiatives need to be there to achieve the targets which are above to the current performance). Even, there can be an objective to maintain the performance at current level itself instead of improving upon the same. The process performance objectives are based on

    • the organization’s business objectives
    • the past performance of projects
    • customer requirements

In all these cases organization needs to have a reference value as baseline to know where it is right now. (Otherwise goals will be subjective) Baselines show the current performance measures of an organization. Now assume a case where an organization has just started to collect measurement data. Definitely there won’t be a baseline initially. In this scenario, how the organization will be setting objectives, without a reference baselines

They can have different options as below.

  1. Industry bench marks
  2. The organization can look in to the industry benchmarks. If the organization is performing a similar nature of work as per the contextual information of the industry benchmarks, they can use those values as references for setting targets.
  3. Expert discussions and brainstorming
  4. There might be employees in the organization who are much skilled to come up with some reference values on the critical processes for setting the targets.
  5. Collecting information from similar organization.
  6. The organization can refer baselines of other similar organization via employee contacts and use it as reference values.
  7. Process Performance Models (PPMs)
  8. If there are some prediction models defined, suitable to the requirements of the organization, dependent parameters (y values) can be predicted, assuming organization have some knowledge on the values of the independent parameters(x values)

When a reference value is obtained, organization can build targets upon the same. Once the organization has collected enough data over a period of time, PPBs can be built. Then those PPBs can be used to set targets for coming year. In this way process is continued and baselines are revised on a periodical manner, say on a yearly basis or so.

Add your points if you have got some other methods ‘to set objectives when there is no baseline’

Posted on

In this article, 7 statistical tests are explained which are essential for doing statistical analysis inside a CMMI High Maturity (HM) compliant project or organization.

1       Stability test

Definition:

Data stability for the selected parameter is tested using minitab before making performance baselines.

Steps:

  1. Go to Stat->Control Charts-> I-MR (In variables, enter the column containing the parameter)
  2. From the section for Estimate, choose ‘Average Moving Range” as methods of estimating sigma and ‘2’ as moving range of length
  3. From section of tests, choose specific tests to perform
  4. From the section for ‘Options’ enter sigma limit positions as 1 2 3.

Results:

After eliminating all the out of turn points, the system attains stability and is ready for baseline.

2       Capability test

Definition:

Once the selected parameter is baselined, capability of the same to meet the specification limits are tested.

Steps:

  1. Go to Stat->Quality Tools->Capability Sixpack(Normal), Choose single Column, (In variables, enter the column containing the parameter), Enter ‘1’ as subgroup size),Enter Lower spec and upper spec,
  2. From the section for Estimate choose ‘Average Moving Range” as methods of estimating sigma and ‘2’ as moving range of length,
  3. From the section for Options, enter ‘6’ as sigma tolerance, choose ‘within subgroup analysis’ and ‘percents’, and Opt ‘display graph’

Results:

If the control limits are within specification limits or the Cp and Cpk values are equal to or greater than one, the data is found to be capable.

3       Correlation test

Definition:

Correlation test will be conducted between each independent parameter and the dependent parameter (if both are of continuous data type) in the Process Performance Model.

Steps:

  1. Go to Stat->Basis Statistics ->Correlation (Opt display P values)

Results:

For each correlation test p-value has to be less than 0.05 (or the decided p value within the organization based on risk analysis)

4       Regression test

Definition:

Regression test will be conducted including all the independent parameters and the dependent parameter in the Process Performance Model.

Steps:

  1. Go to Stat->regression->regression; (In response and predictors, enter the columns containing the dependent and independent parameters respectively)
  2. From the section for storage, include Residuals also

Results:

  • p-value has to be less than 0.05 for each factor as well as for the regression equation obtained. (Or the decided p value within the organization based on risk analysis)
  • [R-Sq (adj)] has to be greater than 70 %( or the decided value within the organization based on risk analysis) for ensuring the correlation between the independent parameters and the dependent parameter. Otherwise, the parameter cannot be taken.
  • Variance Inflation Factor (VIF) has to be less than 10. If VIF is greater than 10, correlation test (stat->basic statistics->correlation) will be conducted among the different parameters which are influencing Process Performance Model. In cases where    correlation is high i.e. correlation greater than 0.5 or -0.5, the factors have dependency. In such cases if degree of correlation is quite high one of the factors will be avoided or relooked for new terms.

5       Normality test

Definition:

Normality of the data is tested using the Anderson-Darling test.

Steps:

  1. Go to Stat > Basic Statistics > Normality Test> Anderson-Darling test
  2. In Variables, enter the columns containing the measurement data.

Results:

For the data to be normally distributed, null hypothesis cannot be rejected. For this p value has to be greater than 0.05 (or the decided p value within the organization based on risk analysis) and A2 value has to be less than .757.

6       Test for Two Variances

Definition:

Test for Two Variances is conducted to analyse whether variances are significantly different in two sets of data.

This null hypothesis is tested against the alternate hypothesis (two samples are having unequal variance)

Steps:

  1. Go to Stat > Basic Statistics > 2 Variances.
  2. Opt ‘Samples in Different Columns’. In Variables, enter the columns containing the measurement data

Results: If the test’s p-value is less than the chosen significance level (normally 0.05), null hypothesis will be rejected.

7       Two sample T- Test

Definition:

Two sample T test is used to check whether means are significantly different in two periods for two groups of data.

The null hypothesis is checked against one of the alternative hypotheses, depending on the situation.

Steps:

  1. Go to Stat > Basic Statistics > 2 Sample T
  2. Opt ‘Samples in Different Columns’. In Variables, enter the columns containing the measurement data.( First should be the initial data and second should be current data)
  3. Check or uncheck the box for “Assume equal variances” depending upon the F test results (Two variance Test results)
  4. In the Options, use the required alternative, whether ‘not equal’, ‘less than’ or ‘greater than’.
  5. Put test difference as 0 and confidence interval as 95.

Results:

If the test’s p-value is less than the chosen significance level (normally 0.05), null hypothesis has to be rejected.

Posted on

“Walking on water and developing software from a specification are easy if both are frozen.” –  Edward Berard

Baseline means ‘A Line which is the Base’. The word baseline may refer to surveying, typography, budgeting, pharmacology, Configuration Management, calculations etc.

In an IT industry, baseline mainly implies

  • A Configuration Baseline – A configuration of software, hardware or a process that is established and documented as a point of reference
  • A Process Performance Baseline (PPB) – A datum used as the basis for calculation or comparison

Configuration baseline

As and when a work product is created and ready for review, it should be labeled as ‘ready for review v0.9’. (The version number may change based on the project’s strategy). And at the same time the work product should be available in the configuration tool. Once the review, rework and verification are completed the work product is ready for approval. There should be an identified approving authority for each work product. After the approval, the work product is baselined. (The baseline string can be ‘baselined v1.0’). Whenever a change is triggered in the work product, initially an impact analysis is triggered. Impact analysis helps to understand the change in cost, schedule, competency requirements, affected work products etc. The Change Control Board (as defined in the project plan) should approve the change in order to update the work product. After update, the work product undergoes review, approval and baselining process ( If the change is a minor, the review process can be skipped as defined in the project plan). The baseline string can be ‘baselined v1.1’or ‘baselined v2.0’depending on whether the change is a minor or major. All the change requests should be recorded and tracked for future reference.

If there is no configuration management, many people have to work on a work product that is altering. It creates copy-over problem, which causes versions of files to be lost. The number of uncontrolled copies of files makes it difficult to determine where the latest version really exists or not.

A configuration baseline is the configuration of a service, product or infrastructure that has been formally reviewed and approved, that thereafter can be changed only through formal change mechanisms.

Process Performance Baseline

Project data is recorded in the organizational metric repository. And from the data, performance baselines are created, in most cases the center line and upper and lower control limits from a control chart on that process execution. These baselines are used as a benchmark for comparing actual process performance against expected process performance and are revisited over a defined frequency.

These baselines are used

  • In project estimation process
  • While setting Objectives for the project
  • As inputs to the process performance models

Baseline helps to monitor current project performance and also to improve the accuracy of future estimates.

In short

A Configuration baseline shows the current state of a configuration item while a Process Performance Baseline shows the current performance of a process.

 

Posted on

Causal Analysis and Resolution (CAR) helps to identify and rectify the root cause of a defect or problem. It can be applied to any process area. Analysis can be qualitative, limited to a fish bone diagram or so. And at the same time it can be more advanced involving quantitative data. The steps below show how to quantify the performance of CAR activities.

  1. Define a measurable parameter based on the root cause of the issue identified.
  2. Determine solution for fixing the root cause.
  3. Measure the performance of the parameter before and after implementing the change.
  4. Perform some hypothetical tests like 2 sample T test or paired T test on the measured parameter.
  5. Compare the predicted performance and actual performance of the data.
  6. If there is no statistically significant difference between the predicted performance and actual performance, it can be assumed that the changes are effective.
  7. If there is a statistically significant difference between the predicted performance and actual performance, it can be assumed that the changes are not effective and another CAR needs to be done.

Posted on

Sub processes are components of a larger defined process. For example, a typical development process may be defined in terms of sub processes such as requirements development, design, build, review and test. The sub processes themselves may be further decomposed into other sub processes and process elements. Measurable parameters are defined for these sub processes to analyse the performance of the sub processes. These sub processes are further studied to identify the critical sub processes which are influencing the process performance objectives i.e. PPO. Measurable objectives are set for the critical sub process measures also. PPOs are derived fromBusiness Objectives (BOs).

In the above paragraph, there is a linkage established starting from sub process to BOs. In fact in an organization

Continue reading→

Posted on

PPMs are valid within the data used or building them. We cannot extrapolate the data for prediction. Now we use these PPMs to check the probability of confidence in achieving the targets. That is, we are simulating subprocess parameters which could be outside the valid range of PPM. In that case, how can we expect the PPM to give a convincing result..?

Let me illustrate it with an example .

I have made a productivity PPM, such as Xs are coding speed, expertise index, requirements stability index and Y is the productivity. The data comes from current performance baselines of the organization (stable process). In the PPM, adjusted R-sq factor, VIF, individual correlation factor, everything as required itself. While performed simulation to check, the probability ‘to achieve above the mean of productivity PPB’, >50% observed.

Organizational PPO is to improve productivity from 30 LOC/PD to 40 LOC/PD though some improvements identified in coding process so that increase in coding speed will lead to increase in productivity. Suppose current PPB of coding speed is 25 to 45 LOC/Hr and the increased expected range is 35 to 55 LOC/Hr (proven through some piloting/validation). Now project A want to use this PPM for productivity. Here actually the subprocess performance data is outside the stable limits/outside the valid range of PPM. In such a scenario, the project team cannot use the previous PPM at all. So what is the use of organizational PPM in that case..?

To summarize, if PPO is targeted higher to current PPB and to achieve the same, organization comes up with improvement initiatives with an improved rage on subprocess limits (outside to the valid range), then definitely, the previously defined PPM cannot be used. Or rather in such scenario, organization should come up with a calibrated ppm too..

For Productivity PPM, Code Review Effectiveness (CRE) is a sub process Parameter. CRE, itself is a dependent parameter and hence a sub PPM is built with CRE as the predictor. So while using Productivity PPM, the process shall be composed with CRE PPM first and then the expected range of CRE based on the simulated values shall be put in Productivity PPM.

 

 

 

 

 

 

 

 

 

Posted on

A PM  says to his senior manager, ” There is only a 50- 50 chance to deliver the project on time”. A reply like ‘”you take the risk and go ahead as it is ” would be a rare case especially in case of a project where on time delivery is a critical requirement. Back to the question from where we started, is this probability okay to proceed..?

Similarly, normally improvement initiatives triggered by the organization would lead  to achieve the specification limits (range) of the PPO with a probability of 99.7 % ( or roughly>90 %, considering data variability in Prediction models) ie.to achieve above/below to central value of PPO, probability would be < 50 %. Now if the projects are targeting on the positive side (above to mean in case  ‘ productivity’ is a PPO ), ultimately organization objectives could be attained.  Otherwise reverse situation will happen. So project team would be targeting to achieve above to central value, in normal case. This means that probability of success is only 50 % even with organizational improvement initiatives.

Back to the question from where we started, is this probability okay to proceed..?
Certainly not, in the eyes of senior management.  if a project is targeting for mean as PPO, then probably for success is 50 % only. So either there needs to be some other improvement initiatives inside the projects other than those triggered by the organization or  the 50-50 risk of not meeting the target needs to be accepted by the senior management of the organization.
If additional improvement initiatives are triggered to make probability of success greater, definitely, the current mean would be changed, and in such a scenario project team needs to deduce the internal estimates with current mean.( see more on setting internal estimates)

Now there could be another case like, some of the projects are targeting below to mean due to certain genuine reasons. Then probability of success would be higher. But if the case is repeated among many projects, ultimately organizational objective wont be attained. So there needs to have mechanism to ensure that if any of the projects are targeting below to mean of PPO, there must have another project with PPO above to mean. Normally Engineering process group (EPG) needs to take care of this.