Evaluation Phase - Evaluating Monitoring and Metrics Results

To ensure that process changes are fully and correctly implemented, close monitoring should be performed for a designated period; the length of time varies with the number of possible process actions or transactions that could occur during the monitoring period. Over the long term, performance and status metrics should be collected to identify opportunities for continuous process improvements or the need to make radical process changes.

There are several systems and forms of evaluation used to determine how well your organization performs activities to produce products and services to meet the organization's mission and business goals. The value of the evaluation system to the organization is determined by how it is applied. For example, many companies gather data without matching the metrics with goals, and measure irrelevant processes, outcomes, events, and transactions. If the evaluation procedure measures whether a specific business goal is met, the resulting metric has value; if the metric doesn't measure goal attainment, there is no use to collect it.

The most important metric to collect in a business is the one that measures how well the most critical process for that organization works: The metric or group of metrics must identify specific process successes, failures, and opportunities for continuous improvements. Another critical aspect in evaluating process performance is in monitoring activities that are observable and measurable. This is particularly important when evaluating man-machine-materials (3-M) interfaces. Although each of the components of the 3-M system can be evaluated for how well each works as well as how each works in combination with other components in the system, the evaluation must be made in the context of specific performance requirements that address how each component must perform. Without the requirements and a measure of fulfillment of those requirements, the metric is meaningless.

Process evaluations should start as the process is being built, when it is tested prior to implementation, and under specific conditions of process execution. All repeatable processes begin and end with a specific activity and are, therefore, cyclical. When building a process, a cycle should be tested with a business process simulation software package such as Sigmaflow Modeler or allCLEAR to test the logic flow and completeness of each activity cycle in a repeatable process. A pilot test that executes all functions and activities should be tested by typical task performers before going live. Each activity in the process should be monitored against specific performance criteria under normal and emergency conditions.

The artifacts that are to be annotated or reviewed during the Evaluation Phase are presented by type of project.

Automation "Quick Win" Project Artifacts for Evaluation Phase

Change Management Plan (CMP) (Limited)

Update the CMP to include lessons learned, results of monitoring and evaluation activities, and metrics collected on target process activities. Submit the updated document to the customer.

IT Implementation Automated Forms Routing/Tracking Portal Upgrades

Evaluate ease of use of automated forms functionality use, time in each form state (aging), error rates (wrong data, missing data), and access problems (forms, routing/tracking, portal). Identify the metrics in the CMP that will be collected on a routine basis.

Training Materials

Evaluate the effectiveness of training materials in pilot testing (use, knowledge and skills retention). Update the CMP with pilot test results.

Re-design Project Artifacts for Evaluation Phase

CMP (Full)

Update the CMP based on results of metrics monitoring and trend analysis; include lessons learned and further process improvements identified through evaluation that were not already identified.

Training Materials

Evaluate effectiveness of training materials (use, knowledge and skills retention) in pilot testing. Update the CMP with results.

Be sure to visit my blog on Business Process Management for more information, references, products, and solutions supporting BPM.