Using SharePoint to Measure the Impact of Change (KPI)

As I discussed in my blog post on Managing Change, it is important to understand the cumulative impact of changes on a project with regards to the scope, schedule, and cost/budget. Throughout the life of a project, there will be changes. The project manager should be able to explain the evolution of the plan from the original baseline to the current baseline, including all approved changes that have been implemented.

Effective use of the change control log (see my blog post on Using SharePoint to Manage Change Requests) provides a tool to track and reconcile changes to the baseline schedule and budget. In addition, use of the Project Status list enables use of SharePoint 2010 Key Performance Indicators to measure and communicate the project team’s ability to manage change at any point in time throughout the project life cycle. In order for the project sponsor and steering committee to make good decisions when approving change requests, they should understand the impact of the changes they have already approved. In addition, project change is one of the key metrics utilized to assess project performance in the project closure process (and identify and implement project improvement opportunities for future product releases / projects).

Creating the Approved Change Request view in the SharePoint list

The first step to establishing the project change metrics in SharePoint is to ensure that you have the appropriate view set-up to support the computation of the metric. The cumulative impact of change on the project represents the total impact of the approved change requests (approved and implemented status) on the project schedule and budget. To support this metric, I create a view in the change request list that is filtered to include only approved and implemented change requests (sample of this list view below).

The screen shot below from the Change Request list view settings illustrates the filters utilized to limit the list to only approved and implemented change requests.

In addition, totals are added to the list view to the Approved Change Request list view to display the sum of the schedule impact (days) and budget impact (dollars). I also added a count of the change requests so you can get a quick feel for the number of change requests that are driving the total change impact. The screen shot below from the Change Request list view settings illustrates setting up the totals that are displayed at the top of the list view.

Creating the Schedule Impact Metric

To create the Schedule Impact metric, select NEW indicator from the Project Status list. Within the indicator set-up, you are prompted for a name of the metric, a brief description of the metric, and comments (I use comments to describe the specifics of the source/calculation). In addition, you are prompted to select the list, and list view (described above) where the change request data is maintained. The screen shot below illustrates the first part of the indicator set-up process.


Then you are prompted to define the schedule impact metric:

  • The metric is calculated using the “Sum” of the “Sched Impact” field in the Approved Change Requests list view.
  • The “lower” the metric value is better.
  • The metric is GREEN when the total schedule impact is X days or lower (in this example I used 10 days)
  • The metric is YELLOW when the total schedule impact is Y days and greater that X days (in this example I used between 11 and 20 days)
  • The metric is RED when the total schedule impact is greater than 20 days.

The screen shot bellow illustrates the definition of the schedule impact metric.


Based upon the information set-up for the Change Request Schedule Impact metric, the screen below illustrates the details displayed for this metric. The biggest challenge is setting up consistent criteria from project to project that effectively defines GREEN/YELLOW/RED for the cumulative schedule impact.

Creating the Cost Impact Metric

The same process is utilized (as described for the Schedule Impact metric) to create the NEW indicator, describe the Cost Impact metric, and select the list/view where the change request data is maintained.

Then you are prompted to define the metric:

  • The metric is calculated using the “Sum” of the “Cost Impact” field in the Approved Change Requests list view.
  • The “lower” the metric value is better.
  • The metric is GREEN when the total cost impact is X dollars ($) or lower (in this example I used $10,000)
  • The metric is YELLOW when the total cost impact is Y dollars and greater that X dollars (in this example I used between $10,000-$25,000)
  • The metric is RED when the total cost impact is greater than $25,000.

The screen shot bellow illustrates the definition of the Cost Impact metric.


Based upon the information set-up for the Change Request Cost Impact metric, the screen below illustrates the details displayed for this metric. Generally the definition of GREEN/YELLOW/RED for this metric is documented within the Cost Management Approach of the Project Management Plan.

Advertisements

PM Foundations – Managing Change

In my experience as a project manager, I have found one of the most critical best practice areas during the project execution phase to be the ability to effectively manage change. Change is an inevitable component of managing a project – nothing works out exactly as planned. The project manager effectively manages change by maintaining the appropriate balance between control and discipline to manage to the baseline plan, and flexibility to adapt the plans to meet customer expectations. Nobody wants to be “that” project manager that leads a project that delivers on-time and under budget, but still has an unhappy customer. Below I describe in more detail the following aspects of managing change:

  • What defines a change in the context of your project
  • What are the sources of change
  • What are the early warning signs of change
  • Establishing a process to manage change
  • Measuring the cumulative impact of change

What Defines Change

By the end of the planning phase, the project baseline is established and approved by the project sponsor and project core team. Project scope, schedule, and cost are all part of the project baseline plan. A change represents a permanent deviation from this baseline plan. The term ‘permanent’ is important. It is possible that there is a deviation from the baseline plan that will be self-correcting and is really only a deviation due to timing (when something has occurred vs. when it was planned to occur).

  • Example of a timing deviation: A deliverable that was completed a week late because the sequence was switched with another deliverable.
  • Example of a permanent deviation: A deliverable that was completed a week late, because it took a week longer than anticipated to be completed (and that week cannot be recovered).

Change is identified and measured based upon the impact on the baseline plan:

  • Scope – Additions/reductions in scope can be either in terms of scope of the product or project – in both cases they change the deliverables articulated in the WBS.
  • Schedule – Planned timing of activities and/or deliverables. In most cases, changes to the schedule are only identified / managed if they impact the project end date or a major milestone.
  • Cost – Addition/reduction to the cost of the project (permanent variance of actual/projected costs vs. planned/budgeted costs)


The following guidelines associated with changes to the project are detailed in the Project Management Plan:

  • There is not one standard definition of a change. Change must be articulated in the context of the specifics of the project. It is important to distinguish between a change and a clarification of expectations.
  • Only changes that have material impacts to the baseline plan will be formalized and managed (the Project Management Plan defines materiality).
  • How the change is determined to be a permanent or timing project impact.

Sources of Change

During project execution, actual events will occur differently than planned, even with the best of plans. It is up to the project manager to proactively identify and manage potential sources of change, and adjust the project plan appropriately. The sources of change generally fall into the following categories:

Customer Needs / Requirements – Adding a product requirement is a common source of change.

  • Adding or changing a product requirement during the middle of development / build / test activities can have even more impacts because of re-work and impacts to other parts of the product.
  • Often the change represents more of a clarification of the original requirement, and not a change in the intent of the requirement. The clarification could still represent more or less work to be completed to satisfy the product requirement.

Technology

  • Implementation of new technology introduces a certain amount of risk / unknowns. During project execution these unknowns may introduce real changes in the work required to complete the implementation (e.g., hardware procurement and set-up, software configuration & customization).
  • An organization’s standard technology platforms may change. If the work required to comply with the new standards is not reflected in the plan, it often represents a change to the project.

Planned vs. Actual Execution – The most common source of change is simply that work is completed in a different manner than originally estimated and planned. The following are more specific examples of this source of change.

  • Inaccurate estimates – Activities take more or less time than was estimated. This is more likely to occur if this type of work had not been done previously by the company or by the resources providing estimates.
  • Change in resources – The skill set associated with the resource completing the work is different that the skill set the estimates were based upon. Changes in resources in the middle of the project introduce factors like ramp-up time, the skill level of the new resource, availability of the new resource, the billing rate of the new resource.
  • Change in deliverables / activities – The addition of or removal of deliverables from the WBS can impact timelines, costs, and quality (e.g. adding training materials). Also, there could just be a process / methodology change that requires new tasks or a change to a task even though there is not a new deliverable. For example, if quality standards change, that could increase the time it takes to perform some tasks.
  • Sequence of work completed (project dependencies) – New or different project dependencies are identified during the project execution that could permanent delays in subsequent activities / milestones.

External Influences – Factors outside the control of the project team often drive changes to the project. Examples of this include:

  • Changes in regulatory or other compliance requirements
  • Changes in other systems / environments that impact the current project
  • Changes in dependent projects
  • Changes in the business environment or economy (e.g., introducing new funding constraints)
  • Reorganizations that drive changes in team members or key stakeholders (this would never happen, right?)

Early Warning Signs

In addition to understanding the potential sources of change for your project, it is also important to understand and monitor key project metrics that help identify potential (or real changes) early on (early warning signs). It is fairly intuitive that the further the project is in the project life cycle the more costly and impactful the change is to the project.


Some early warning signs of change include the following:

Schedule slippage

  • Tasks not started on-time
  • Tasks not completed on-time
  • Tasks with less progress than planned (particularly on larger effort or longer duration tasks)

Variance Trends (Cost / Schedule / Risk / Scope / Quality)

  • Consistent cost variance trends (overall or on specific cost types / categories)
  • Consistent schedule variance trends (overall or on specific project phases / milestones)
  • Actual work effort (or duration to complete) to complete tasks is consistently higher (or takes longer) than planned
  • New tasks or deliverables are identified throughout the project life cycle (pointing to gaps in the planning process)
  • Overall project risk (overall assessment of number, severity, and probability of risks)
  • Defects are identified at a higher than planned rate, or the closure rate is slower than planned

Open Risks and Issues

  • An increase in the number of risks and issues, particularly if they are focused in a specific project area, is a good indication of a potential change / adjustment required
  • Issues that are not closed sometimes point to a project change that is required to close the issue

Change Control Process

The overall goal of change control is to prevent changes from overwhelming a project or taking the project unnecessarily off track. The goal of change control is NOT to prevent all change. Change control enables proactive identification and management of change, in a manner that keeps the project moving in the right direction, towards successful completion/delivery.

The following represent the key processes involved in managing changes to the project:

  1. Identify Change: Either the project manager identifies, or is made aware of a change. Based upon guidelines established in the Project Management Plan (permanent vs. timing, and materiality), the project manager determines whether or not the change should be formalized and assessed.
  2. Assess the Impact: Once the change is identified and it is determined the team needs to assess the impact of the change on the project, the assessment is documented, using a change control tool (i.e., a project impact report).
  3. Review and Approval: Once information about the impacts of the change have been documented, the change should be reviewed by a Change Control Board and either approved or rejected.
  4. Implementation and Verification: Planning and execution of the actions required to implement the change. Implementation includes adjustment to planning artifacts to accurately reflect the new scope, schedule, and budget based upon approved change. Implementation of the change effectively establishes a new baseline for the project.
  5. Follow-up Verification: Perform a follow-up review to confirm that the appropriate actions were completed to implement the change (corrective actions), and that the change had the anticipated impact on the project (time, cost, scope, quality, risk).

Remember that change is inevitable and oftentimes necessary. However, it is important to control change so that changes are properly assessed and only approved changes are implemented within the project.

Assessing the Cumulative Impact of Change

It is important to understand the cumulative impact of changes on a project with regards to the scope, schedule, and cost/ budget. Throughout the life of a project, there will be changes. The project core team, project sponsor, and steering committee members should be aware of the cumulative impact of changes (comparing actual project performance to the original expectations established in the baseline plan).

The project manager is ultimately responsible for project execution against the baseline plans. The project manager should be able to explain the evolution of that plan from the original baseline to the current baseline, including all approved changes that have been implemented. Project managers often will fall back on the excuse that the project is over budget or late because there were project changes, but they cannot quickly explain the changes that account for the primary deviations from the original plan. Effective use of the change control log (see my blog post on Using SharePoint to Manage Change Requests) provides a tool to track and reconcile changes to the baseline schedule and budget.

Reconciling the actual project performance (schedule & cost) to the original baseline plan helps understand the “unexplained” variances. An unexplained variance is the difference between the total schedule or cost variance and the approved project changes / impacts. A project with too high a percentage of unexplained variances is usually an indication of a project with inadequate attention to change control processes. These projects have too many changes that are “flying under the radar”. I am a not a big fan of the term “scope creep” because it represents an excuse vs. an explanation. Good projects managers can pretty precisely describe the difference between the original baseline and the actual results – strive to be a good project manager.

Characteristics of Effective Change Control

Be proactive. The cost of implementing change increases later in the project lifecycle. Utilize project metrics to monitor trends and identify potential changes.

The level of control and rigor around analyzing and approving changes should be appropriately “sized” to both the organization and the project.

  • Project size, complexity, and risk profile are evaluated to adapt the change control processes to the project.
  • If changes have different impacts when they are implemented (higher actual cost or schedule impact), maybe there is not enough rigor involved with the change analysis process.
  • Projects that place more importance on schedules and/or cost constraints tend to require more control around analysis and approval processes.
  • Another consideration associated with change control processes is the visibility / strategic importance associated with the project.

Provide visibility of the change request throughout the change control process. This ensures that everyone involved with the project can easily find out the current status of any changes in the process, as well as the reasons for approving or rejecting specific changes.

Focus on the making sure the change gets appropriately implemented. Project teams often spend all of their time and energy on analysis and approval, and forget to adjust project deliverables to reflect the change.

  • Follow-up to confirm the implementation steps have been completed.
  • Compare the actual impact on the cost and schedule to the estimated project impact.

Using SharePoint to Manage Project Changes & Impacts

One of my favorite questions when interviewing potential project management candidates is, “How do you manage change to your baseline plans.” First of all, many candidates fall into the trap of thinking baseline plans is the same thing as the schedule. Others think that change relates to customers adding features and functions to the product. The easiest way to describe change from a project perspective, is change represents anything that is a permanent (vs. timing) deviation from the original plans that define what is going to be delivered (scope), when it will be delivered (schedule), and how much it will cost (budget).

Using collaboration tools, such as SharePoint, to manage change does not replace the diligence required to identify and manage project impacts effectively, but it does streamline the processes associated with documenting, reviewing and approving the change requests. In addition, SharePoint enhances the ability to effectively communicate project changes throughout the project life cycle.

Creating the Change Request Log

I find it easiest to use a Custom list to create the Change Request Log in SharePoint. The following are the fields I add to this list to improve the identification and tracking of change requests based upon the best practices associated with Project Change Management:

  • Id – System assigned unique identifier assigned for the change.
  • Title – Headline associated with the change request (make sure it is descriptive enough to identify the item from the summary view of the change requests).
  • Status – Drop down selection: Not Started, Documented, Reviewed, Approved, or Implemented.
  • Cost Impact – Numeric field for the estimated impact the change request will have on the project budget.
  • Schedule Impact – Numeric field for the estimated impact the change request will have on the project schedule.
  • Scope Impact – Text field utilized to describe the impact the change request will have on the scope of the project.
  • Comments – Text field utilized to describe the current status of the change request.
  • Attachments – The change request should be supported with more details explaining the request and estimated impacts. A completed project impact document is attached to the change request item.

Below is an example of the Change Request form in SharePoint, and the Project Impact Report template.

cr-input form

cr-project impact report

Managing the Approval Process Using Workflow

Creating workflow for this SharePoint list helps automate the approval and tracking process. At the time the item is added to the SharePoint list, an email is sent to the approver (see below for an example of the email).

cr-email

From this email, the approver can Approve, Request a Change, Reassign, or Reject the change request (see an example of the approval pop-up that is displayed within Outlook).

cr-approval popup

Once the approver performs the approval process, the status of the workflow is updated within the SharePoint list (see the Change Request Workflow column of the All Items view below).

cr-all items

I find it is best to keep the workflow straightforward, with no more than 2-3 approval levels. The benefits associated with approval automation can be lost in time spent on manual intervention if the workflow is too complicated.

Cumulative Impact of Change

It is important to understand the cumulative impact of changes on a project with regards to the scope, schedule, and cost/ budget. Throughout the life of a project, there will be changes. The project core team, project sponsor, and steering committee members should be aware of the cumulative impact of changes (comparing current project performance to the original expectations established in the baseline plan).

The project manager is ultimately responsible for project execution against the baseline. The project manager should be able to explain the evolution of that plan from the original baseline to the current baseline, including all approved changes that have been implemented. Project managers often will fall back on the excuse that the project is over budget or late because there were constant changes, but often cannot quickly explain the changes that account for the primary deviations from the original plan. Effective use of the Change Request Log provides a tool to track and reconcile changes to the baseline plan (and provide as a regular project performance metric).

I create another view within the Change Request list to communicate the cumulative impact of change on the project. This new view filters for only approved or implemented requests, provides a total count of approved/implemented requests, and sums the total schedule and cost impacts. Below is an example of the approved requests view of the change requests.

cr-approved view

Summary

SharePoint provides a great vehicle to capture and track change requests in a timely and accurate manner. In addition, adding workflow to the SharePoint list automates the change review and approval processes. The information captured within this list allows the project manager to communicate effectively around the team’s ability to manage change and accurately describe the cumulative impact of this change on the project.