It has been half a century since Kirkpatrick’s seminal tome on measuring the impact of training! Yet the how to’s, why’s, and whether or not to’s abound. While attention to levels 1 to 4 is as robust as ever, there’s more to measurement than meets the eye.
Some espouse measuring return on investment. Others focus on return on expectations. Trends in business measures—such as engagement or customer satisfaction scores—are the focus of others. Intangibles are a recent addition to the “what to measure” list. And, metrics associated with the knowledge, including knowledge and skill tests (and of course, 360s), are making a comeback.
The impact of leadership development has been studied and popularized in books that include The Extraordinary Leader: Turning Good Managers into Great Leaders, by John Zenger and Joseph Folkman (2009) and The ASTD Handbook for Measuring and Evaluating Training by Patricia Pulliam Phillips (2010) just to cite two.
Common sense and current research confirm that leadership development makes a measureable difference. Yet we continue to be eager to measure and to prove the bottom-line impact of our efforts. We capture 360 scores down to the tiniest percentage point. We watch for shifts in data sets (whether engagement, results, or knowledge) that most often are far too small to serve up anything of statistical significance. We create decks, reports, and other tomes that show we are making a difference. Often, we spend more than we spent on the developmental effort to prove the worth of that effort.
Peter Block, recipient of national awards for outstanding contributions in the field of training and development and author of The Answer to How Is Yes: Acting on What Matters (2002), the winner of the Independent Book Publisher Book Award for Business Breakthrough Book of the Year, suggests the “how to measure it” question is an expression of doubt, a question that we turn to when we’ve lost faith and feel the need for control or oversight. Too much focus on the measures draws one away from an underlying focus on values and what’s truly most important. Is the fact that someone’s confidence, commitment and contributions have soared any less impressive that the fact that 89% of newly trained supervisors conduct regular performance reviews? Is the willingness to speak up to power any less impressive that a 0.35 ROI?
While the what to measure and how to measure it debate won’t be decided with this blog, our hope is that a robust conversation follows. Let’s get it started.
Two questions follow:
- What’s the most significant outcome you look for in terms of leadership training? How do you truly know you’ve made a difference?
- What do you measure?
Don’t worry if the two don’t match. Perhaps your responses will enlighten us as a community about the difference we want to make (our values) and how we go about proving our worth (the measures). If there’s congruence, that’s grand. If not, we just might get closer to the reconciliation of what we do with what we measure.
Ivy Savoy, Human Resources Specialist, IRS
Rosaria Hawkins, Ph.D., President, Take Charge Consultants, Inc.