The need for project status information is common to all levels of the organization. When looking at the status, the report should indicate:
- Is the project / initiative on track with the agreed upon schedule?
- What has been done?
- What remains to be completed?
- When will it be done?
- Are there any risks involved in completing the work?
The purpose of any overview and status report is to highlight areas requiring management attention and to give an easily understood description of the work that has been completed and the work that remains to be done.
Tracking and status reports are used by all levels of the organization. Topic teams and site management may use the reports on a daily basis. Program management will review the work on a regular basis. Executive management will review progress at an interval similar to the program office. However, the information provided to the executive management is often presented by a senior representative of the project organization.
Metrics for Progress Tracking Event Centric Projects
Project managers and responsible executives need a reliable and easy way to interpret a set of metrics so that they can spot issues early enough to avoid major problems and take corrective action. These metrics should be common to all levels in the project organization and universally interpreted and applied.
Most traditional project management software tools focus on work hours or lapsed time as an indicator of progress. For event centric projects, these are not accurate indicators of true progress for two (2) primary reasons:
- Completion of a deliverable item often does not have a linear relationship with either lapsed time or labor hours. Complex deliverables requiring multiple review and rework steps with accompanying lag times skew the relationship between time and actual percent completion for the delivery.
- When providing management or a project team with a project-wide completion percentage with the focus on lapsed time or labor hours, the roll up number will often be misleadingly high when compared to actual deliverables and completed tasks. This is because project management software tools have metrics based on the assumption of an array of sequentially interdependent tasks that is singular in nature. Drug product development, business process re-engineering, and other event-centric projects tend to be made up of a large portfolio of what can be thought of as small independent projects, such as the authoring of an SOP or developing a method, that run parallel to each other. This system of topic-based projects has no true single thread. When rolling up the percent complete of this array of efforts, the total percent complete can be quite high without a single one of the component project deliverables being completed. When management reviews this percent complete value, they can easily get the impression that progress is better then is actually the case. In fact, this roll up technique can often mask problems, since it is in effect an average of peaks and valleys in performance.
The diagram below shows single and multi-treaded projects, each with 60% of the work done, based upon work hours. Only the single threaded project has deliverables completed and nearing completion. The Multi-Treaded project has no completed deliverables. To make matters more complicated, there is no single deliverable that is more than 60% complete. Setting management alerts and progress evaluation criteria purely on completed labor hours or time line will leave an impression that the project is in better shape than it may be.
While no metrics system is perfect, a metrics system that focuses on completion of deliverables tends to give a more accurate picture of progress when looking at an overall project.
Critical Tracking Parameters
Common to all project management systems are the concepts of planned and scheduled completion date. The planned completion date is the original commitment for completion. This is often the promise made to senior management at the time of project funding. The scheduled completion date is when the current schedule, adjusted for upsets, forecasts that the work will actually be done. A third date is also commonly used, the baseline completion date. This is the date on which the original plan projected the work will be completed. A variance parameter is often calculated. This is the difference between the current scheduled completion date and the baseline completion date. Most project managers use this information along with percent complete by hours as their principle metrics.
In event centric projects, especially ones that contain multiple threads, these metrics are insufficient, as was shown in the previous section. Three additional metrics can be defined that allow a project manager, team leader, or executive to get an accurate picture of overall progress and see potential issues developing. They are:
- Overall percent deliverables complete
- Period percent deliverables complete
- Tasks “at risk” ratio
The overall percent (%) complete by task/deliverable is calculated as the ratio of total tasks actually completed / tasks planned to complete. The baseline schedule is used as the basis for planned complete value in the calculation.
The tracking percentage by task/deliverable was chosen since:
- It is a clear indicator of activity closure, and
- It is the only factor that can show action completion and is common to all critical project elements.
It is best that the original plan be designed so that a single task in the plan equates a single measurable, tangible deliverable. The tasks included in these sums must be basic tasks only, not summary tasks.
Overall percent complete by deliverables is calculated for each reporting period. It forms an indicator of actual overall progress vs. planned overall progress to a particular date. When reported as a series of accumulated bar graphs over multiple reporting periods it shows a reasonable picture of general progress.
The trend graph below shows overall percent complete by deliverables over time.
While the trend above shows overall long term trends, it cannot be relied upon as an early indicator of problems because of the averaging effects, especially in project with a large task count. An additional level of sensitivity is needed to operational level management.
Period percent complete provides a level of metric sensitivity that allows project managers and their team leaders to recognize and react quickly to schedule issues.
During each reporting period, a set of deliverables are due. By comparing the planned set with the actual set, the period performance for the sub-project can be measured. By examining the trend of period performance, especially when coupled with the third metric – risk ratio — systemic problems and significant issues can be easily detected. This detection is independent of the total project size or any roll up that may occur when combining multiple sub-projects into an overall program.
| Period % Complete | = | Actual number of deliverables complete in the period that were planned for the period |
| Number of deliverables that where planned to be completed in the period |
The sums are only those tasks planned to be completed in the period. Prior tasks/deliverables should not be included.
When viewing the period complete metric over time, a clear picture of team effectiveness arises.
When managing large projects, it is necessary to react appropriately and not over react or under react to issues. “Bumps in the road” are normal in the course of a project and it is common practice to let team leaders handle those issues. When more significant issues arise, management intervention is necessary. A system of metrics must therefore focus everyone’s attention on important events early. The third metric of the set is intended to do just that.
The “At Risk Ratio” value is derived from the total array of deliverables belonging to the project/topic team. Its intent is to be an indicator of risk, focusing managements attention on significant issues. High “At Risk Ratio” values indicate significant problems. Non-zero risk ratio values that persist over time also indicate significant risk.
As progress is being made, a topic team is expected to close a specific number of tasks. At times, tasks are late. As the total number of tasks completed increases, the total tracking percentage and total percent complete parameters become a less sensitive indicator of problems, simply because of the large numbers involved in the percentage calculations.
The degree of lateness of delayed tasks is a sensitive metric, regardless of task count and project size. It can be used to indicate significant problems in the team or with the portfolio of projects. It is also used in conjunction with the period percent complete value to focus attention on those situations that are less likely to be handled easily by the topic teams.
The At Risk Ratio is intended to be an early warning flag. A ratio is used to attenuate the alerts generated by the Period Complete metric so that small delays, which can be reasonably handled by teams, do not provoke undo management attention while maintaining sensitivity to significant issues.
To calculate an At Risk Ratio, a common definition of an At Risk Delay needs to be established in the project organization. The basis for declaring a task At Risk is that an At Risk task is one that is delayed to the point that a significant effort on the part of the project team is unlikely to bring that task to closure in a time frame that will prevent impact on subsequent tasks. In short, a burst of overtime and weekend work is unlikely to close the task. An example algorithm for declaring a task At Risk is:
At Risk Task is:
A task that is <80% complete and >= 2 days overdue.
An algorithm of this type can be easily implemented in macro language in most project management tools or by using a test on project data imported into a spreadsheet program. By counting the total number of delayed tasks and the total number of At Risk tasks, the At Risk Ratio is defined as:
Total Number of At Risk Tasks / Total Number of Late Tasks Reported as a percentage.
The At Risk Ratio in this report is a representation of the project or project set as a whole. In the trending reports, the At Risk Ratio can be plotted over sequential reporting periods to flag potentially serious project performance issues.
In the appendix is a set of potential project scenarios and a discussion on applying the metrics as an indicator and diagnostic tool.
Reporting
Over the duration of the project, a set of metrics should be published prior to the formal regular project review meetings. In large projects, these period reports can include an executive summary report targeted at senior managers and the project executive, and a set of individual topic team project metric reports targeted at each topic team.
The executive report will usually focus on the entire project and include text descriptions of accomplishments and issues. The report should also include project-wide metrics using the above set and the traditional metrics of planned dates, scheduled dates and variance. Executives overseeing high profile projects will often want to see individual topic team details. In that case, the executive report can contain appended topic team metric details, which will be the same details provided to each topic team leader.
The topic team leader and team, if the that is the custom of the company culture, will get a topic team metrics report consisting principally of four (4) major sections:
- Overview
- Period detail
- Overall detail
- Team trends
The overview section will be similar to the executive overview for the project as a whole. However, it will be focused on the sub-project / topic team project. Below is a sample:
The period detail section shows the tasks / deliverables due and completed for the period as a summary metric and as a period list of deliverables with due dates:
The third section tallies all the tasks/deliverables and shows the origin of the “Risk Ratio” value. The example below uses a spreadsheet to publish the metrics so a series of user selected sorts are available for convenience.
The final page is the single most useful for diagnostic purposes. It shows trends over the life of the project of the three key metrics. You may note that the trend line for the first metric, percent compete, looks similar to an added value curve common in construction projects. It can be interpreted in much the same manner for purposes for diagnostic purposes.
Conclusions
Project management professionals have always looked for a limited set of metrics that can be used to communicate status and risk to project teams and management. Ideally, these metrics not only serve as communication tools, but also as diagnostic tools when schedule issues arise.
Projects tend to be focused on either labor or events. For labor-focused projects, especially single threaded projects, traditional Percent Complete metrics based on labor hours serve the practitioners well. For event- or milestone-centric projects, like those common in the life sciences, labor hours are often not an accurate measure of progress. A metric based on the state of the deliverable is better suited to communicate progress. The paper has shown that by using metrics based on the deliverable states, a project manager, his or her team, and management can oversee a large complex project. The technique shows overall progress accurately and has sensitivity indices that draw attention to potential issues before any significant negative impacts occur. The metrics are suitable for use in a single project and can be equally applied to a set of topic/sub-projects, which are unified into one large program. The unification process does not erode any sensitivity in the metric and is equally applied to all sub-topic areas.