Connecting the Business Measure Dots in the ROI Methodology Process Model

by Timothy R. Brock, Ph.D., Director of Consulting Services, ROI Institute 

This article will explain the ROI Methodology process model from a different perspective to help you see how it can help you see and show the value of programs to give executives actionable data to drive the business. It is easy to lose sight of a big picture perspective to guide your thinking about what you are doing and why. We see this disconnect too many times when teaching the ROI Methodology during certification workshops and in university masters and doctorate programs.

Figure 1 shows the 12-step ROI Methodology process that we follow to evaluate programs, projects, and initiatives to determine the valued results they are delivering in return for the resources provided by the organization. You can see the six types of data (i.e., Levels 0 – 5) used in this process model.

Figure 1. 

Where we see the disconnect happening is not connecting the key business measures identified at the Level 0 Input data to the data collection and data analysis steps of the ROI Methodology. Adding a few words and some arrows to this process model helps us connect those dots. That is what we will do in this short article.

The first step of the process model is to start with why to align the program or performance improvement effort we will evaluate to the business. This first step defines the organizational need perspective. This first step is where we learn the measures that matter to the executives that motivate them to act. The executives want to improve underperforming business measures or do better with those measures. They also believe the improvement results should outweigh the costs of getting those results. They expect a positive payoff from acting. These are the Level 5 Payoff and Level 4 Business needs. Let us add these two measures to our process model. Figure 2 is what that looks like in the process model.

Figure 2. Step 1 Level 0 Input Organization Measures

Now that we know why the executives have decided to fund this program and what they expect in return for this funding, we must next determine the right, most feasible actions to take to change behaviors, processes, and policies to improve the identified business measures. Step 2 establishes an operational perspective. We must also pay attention to the program costs to make sure they do not exceed the monetary benefits of the program. Nobody wants to spend two dollars to save one dollar.

If we change something in the workplace or where people take actions that influence those business measures, they must learn what changed and how it affects what they do. This could range from training (because they do not know how to change or what to do) to informational (when they can easily figure out what to do differently).

Finally, we want everyone to perceive the solution in a positive way because it is relevant to them, important for their success, new information, and something they want to do. By knowing the payoff and business needs, we can craft a solution that people will like so they will learn what they must do to be more successful on the job to improve those key business measures. If we don’t know these two organizational needs, we cannot implement a successful operational solution in the eyes of the executives because it did not improve the business measures that they expected it to improve for their commitment of resources to the solution.  

Figure 3 reflects what the process model looks like to identify the needs we identify at this operations step. It includes what people must learn and be motivated to do to change their behaviors and performance. You should also see that we are establishing the baseline for the program’s chain of impact story.

Figure 3. Step 2 Level 0 Input Operations Measures

It is impossible to do step three without doing the first two steps. You cannot plan to achieve payoff and business results if you do not know the results that are expected. Furthermore, you cannot expect success if you do not know what success looks like to the people (aka stakeholders) who can affect whether the program succeeds or fails.

Who are these stakeholders? They are the people affected by the program or paying for it. If the people who are affected by the program are not motivated to do something different to support the program, they will not. If the people who are paying for the program do not get what they expect from the program in return for their resource investment, further funding is at risk. The program must matter to all the stakeholders. Step 3 is where you plan to make that happen if you did a good job at steps 1 and 2. If you do not do a good job at steps 1 and 2 (or worse, skip them), program failure is certain, unless you get lucky or no one is paying attention to results.

At step 3, you write your program objectives that define what success looks like for these stakeholders at each data level using the measures determined by the needs assessment. If the baseline measure is X, you must learn how much the program is expected to improve that measure and by when. This needs-objectives knowledge will influence how the program is designed, developed, and implemented if the program does not already exist. If the program selected for evaluation already exists, we use these objectives to determine how well the program did to satisfy the program objective associated with that measure.

Once we know what the program objectives are, we plan how we will collect the data necessary to determine how well the program did to improve the measures at each level to meet that measure’s objectives. Then you want to plan how you will analyze the collected data. What data? The data that matters to the executives, the impact measures identified at step one. It is these impact measures that are isolated, converted to monetary value, and used in the ROI calculation.  

Figure 4 shows how what is done at step 3 remains focused on the business measures defined at step 1 that justifies the decision the executives made to fund the program. The plan is to collect data to establish the chain of impact story to the impact level. The planned analysis focuses on the impact measures that matter to the executives to justify funding the program in the first place.

Figure 4. Level 0 Input Plan to Deliver “Why” Results

Data is collected as planned with a focus on establishing a chain of impact story about the effect of the program to improve the business measures defined at step 1. The business measure remains the focus on this process model, starting with identifying the need at step one and the impact objective at step 3. A chain of impact story from reaction to impact is conceived at step 3 in the data collection plan.

Finally, the plan to analyze the impact data after it is collected is done to answer how well the program did to meet the impact program objectives to improve the business measures the executives expected at the beginning. The data analysis begins by isolating the impact of the program to improve the business measures defined at step 1. These isolated impact measures are what is converted to monetary value if the measure passes the four-part tangible-intangible test. Figure 5 shows how the business needs remain the focus of the data collection and analysis phases of the process model. 

Figure 5. Level 4 Business Impact Measures and Analysis

So far, we have introduced the extra words in the process model to show the focus is on organizational measures that matter to executives to justify current and ongoing funding. It is time to add the arrows. The arrows at Figure 6 help us see that this process model is an advanced logic model. We start with then end in mind (organization measures that matter to program approvers and funders) and then move to operational measures that must change to improve the organization measures. We then analyze the measures that matter to those program approvers and funders to determine if the program met the program objectives, where it fell short, and to recommend actions to address those shortcomings and even improve those measures where the objectives where met.

Keep in mind that you do not evaluate every program to the impact and ROI levels. If you stop evaluating at a lower level, you do not get to the evaluation data analysis phase. However, you should be able to connect every program to the business at the first two steps. If you cannot, why does the program exist? Even compliance programs are aligned to the business (e.g., avoid fines, stay out of jail, etc.). You can identify the business measures the program influences but may not consider it worth the effort to evaluate it to impact and ROI. You can even write program objectives for each level and limit your data collection to the lower levels. That is fine. Looking at Figure 6, you simply stop collecting evaluation data at the appropriate level and go no further. However, if, at a later date, someone decides they want to evaluate a program to the higher levels, you have your roadmap ready to get there because you have your business-focused work done at the first three steps.

Figure 6. Level 4 Business Impact Measures Trail Through the ROI Methodology Process Model

Regardless of what evaluation level you stop, this process model shows how all roads lead to ROI.