What Role Does TDRp Play in the Measurement Space?

by Dave Vance, Executive Director, CTR

At the annual ATD conference in May, several people asked me what role TDRP plays in the measurement space. Specifically, they wondered how TDRP related to the Phillips ROI methodology and the Kirkpatrick methodology.

Good questions. Think of it this way. TDRP provides a broad framework for measurement and reporting, the “big picture” if you will. One very important reason for measuring is program evaluation, which is the primary focus of both the Phillips and Kirkpatrick methodologies. So, the Phillips (Jack and Patti) and Kirkpatricks (Don and Wendy) are the experts in program evaluation and we have not attempted to duplicate their work. Instead, we suggest you learn more about their methodologies through their numerous webinars, books and workshops.

If program evaluation is only one reason for measuring, what are the other reasons? It turns out the primary reason to measure is to inform, to let others know how many participants, courses or hours are being offered or consumed. Another reason is to monitor results against past results to ensure that current performance is at least as good as past. Finally, the highest-level reason for measuring, but the least often employed, is to manage to ensure results meet specific, measurable plans or targets. So, TDRP focuses on all four reasons to measure while the Phillips and Kirkpatrick methodologies focus primarily on just one: program evaluation.

Another difference is that TDRP provides guidance on selecting measures not just for programs but for the department as a whole.  For example, what percentage of employees have been reached by learning, what percentage of courses are delivered on time, what is the utilization rate for instructors and classrooms, and what is the total cost of the L&D department. These questions do not arise at a program level but are very important to manage at the department level. Typically, the Phillips and Kirkpatrick methodologies do not address these department-wide measures.

An even greater differentiator lies in reporting. TDRP provides a framework for reporting which ties reports back to the reasons to measure. The framework includes five types of reports which go beyond the traditional scorecard and dashboard to include program evaluation reports (good for briefing at the end of a program), custom analysis reports (good for briefing at the conclusion of a statistical research project) and management reports (necessary to run learning like a business and meet targets). This was ground-breaking work which you will not find elsewhere and which is not covered in the many books on program evaluation. In conclusion, TDRP was designed to provide an overarching framework to include all aspects of measurement and reporting. In contrast, the evaluation methodologies focus primarily on evaluation. Both TDRP and the Phillips/Kirkpatrick methodologies also focus on the management of programs, including working with the goal owner and designing for success. So, the real difference remains that TDRP has a much broader focus on all aspects of measurement and reporting, including all four broad reasons for measuring. So, start with TDRP if you want the big picture or dive right into the details of program evaluation if that is all you need right now.