RSS

Monthly Archives: September 2014

How Much Precision is Really Required in Diversity Measurement?

Introduction

Any time I have presented on the subject of measuring Diversity and Inclusion’s impact on the bottom-line, issues of accuracy and precision invariably arise. It has been argued that to measure Diversity activities convincingly or with any high degree of confidence is difficult. Diversity measures based on “report cards” don’t allow for competing hypotheses in assertions such as “managing Diversity leads to increased profitability.” (Some would argue that other efforts could just as easily have caused the profitability to increase).   Another contention is that there is lack of control over thousands of factors that influence a large organization’s profitability. That is, lack of control over factors like inflation, labor market conditions, and cost of money can make it virtually impossible to effectively measure Diversity with accuracy.

In general, this argument is correct, though similar conditions prevail throughout the organization. Everyone knows certain factors are not controllable. The marketing department does not have control over the product or the customer; the finance department does not control the cost of money or inflation. Yet both departments are able to evaluate much of their work quantitatively. If we are willing, we might admit that there are more issues out of control than in the control of any organization. The task of management is to reduce the uncontrollable variables and to instill as much order as possible.

C-Suite management does not require accuracy at the .05 level of statistical significance for general reports of progress in Diversity efforts. In research, precision is critical, obviously. In pharmaceuticals or medicine, extreme care must be taken with procedures and measurement. Results are often required to be statistically valid beyond the .001 level. That kind of measurement with a capital “M” is not required in the operational measurement of Diversity just to get a sense of the progress being made in your Diversity efforts. We are not operating in a laboratory, but in the field, with all the problems inherent in field research and experimentation. Accuracy is necessary, but precision is naturally limited by internal and external conditions. Though we can’t control the variables in the environment completely, we can still come up with usable numbers and “compelling evidence” of Diversity’s contribution to the bottom line. However, there are some exceptions. For example, Diversity measurement leading to Diversity ROI impact does require the use of scientific processes and analytics to demonstrate correlation and causality. A seven-step methodology like the Hubbard Diversity ROI Methodology can be used to highlight these relationships and reveal what drove the results using evidence-based outcomes.

Seven Levels of Data Capture

The Hubbard Diversity ROI methodology captures seven types of data, with the actual Diversity ROI calculation being only one of them. The seven types of data include:

  • Level 0: Needs Analysis
  • Level 1: Reaction, Satisfaction, and Planned Action
  • Level 2: Learning
  • Level 3: Application and Behavioral Transfer
  • Level 4: Business Impact and Consequences
  • Level 5: ROI
  • Level 6: Intangible Benefits. The Intangible Benefits include measures that cannot be credibly converted to monetary values.

Here is a sample of a few of the most-debated questions:

  1. Isn’t Diversity ROI calculations based on nothing but estimates that can be too subjective?

No. Estimates are only used when isolating the effects of a Diversity program or Diversity intervention’s business impact, when converting data to monetary values, and when tabulating program costs. Estimates are used only when other methods are not available or become too time-consuming or expensive to use. Also, estimates are adjusted using “Confidence Level” factors to improve credibility.

  1. How does ROI in Diversity and Inclusion differ from the ROI used by the finance and accounting staff?

The classic definition of ROI is earnings divided by the investment—no matter what the application. In the context of calculating the ROI of Diversity and Inclusion, the earnings become the net benefits from the program/initiative (monetary benefits minus the costs) and the investment is the actual program cost. The difficulty lies in developing the actual monetary benefits in a credible way.

  1. Do I have to be a whiz at finance and statistics to understand the ROI methodology?

No. Most of the basic principles of finance and accounting do not relate to what is needed to develop the ROI in Diversity and Inclusion. However, it is important to understand issues such as revenue, profit, and cost. Only basic statistical processes are required to develop most Diversity ROI impact studies.

  1. Doesn’t ROI cost too much?

No. When external resources are used, the cost for a Diversity ROI study may be as little as 5 percent of the entire project. A large banking group and a large telecommunications company using similar approaches report that the average costs for their ROI studies range from $3,000 to $5,000 per study. The total cost of all evaluation, including selected ROI studies, is usually in the range of 3 to 5 percent of the total project budget.

  1. Does the ROI process reveal program weaknesses? Strengths? Recommendations?

Yes. At low levels, data always capture deficiencies or weaknesses in a process. At the application level, the process requires collecting data about the barriers (which inhibit success) and enablers (which help success). Each study contains a section for recommendations for improvement. You must know what your weaknesses are in order to improve them. In addition, you want to avoid losing strengths that are working for you and providing value.

  1. Is it appropriate to calculate ROI for every program?

No. Only a few select programs should be subjected to evaluation at the ROI level. Ideal targets include programs that are very expensive, strategic, operationally focused, highly visible, involve large target audiences, and have management’s attention in terms of their accountability.

  1. What types of applications are typical for ROI analysis?

The applications vary, but usually they include a range of programs such as Leadership Competencies for a Diverse Workforce training program, Unconscious Bias Training, Measuring the ROI of ERG and BRG initiatives, Multicultural Marketing and Sales Initiatives, Innovation and Diverse Work Team Performance interventions, etc.

How Much Statistical Analysis is Enough?

It is not necessary to introduce heavy statistics to persuade management and employees of the need for progress and accountability in Diversity change initiatives and to highlight their impact on profitability. This does not suggest that “control group”, financial and “mental model” processes should be avoided. It suggests only that the optimum array of Diversity measures should include a blend of data and levels of measurement. In many business situations, activity- and process-based (report card) Diversity measures can play a vital role in the organization’s leadership agenda. Evidence-based outcome measures that go beyond mere “Report Card” (activity-only based) are critical to Diversity being seen as a driver of business performance.

Report card measures of Diversity are like snapshots of the past. They can provide a historical reference to accomplishments or serve as milestones along the path to producing outcomes. Examples: the number of diversity-competent employees, cost per trainee-hour, and turnover by performance level by gender by length of service. These measures provide opportunities for personal feedback regarding achievement and measuring accountability for implementing new diverse workforce change or improvement activities and processes. These report card and evidence-based outcome measures are a key part of a value chain for implementing change in any organization.

The Diversity Value Chain

This value chain is comprised of four basic components:

  • Diversity Activity/Intervention
  • Outcome
  • Impact
  • Value-Added

Generally speaking, all Diversity processes are begun for the purpose of producing value. Any other purpose would be wasteful. One of your objectives should be to develop more effective ways to measure and evaluate changes made in the organization to improve performance based upon Diversity improvement activities/processes, outcomes, impact, and resulting value.

Some typical value chain examples:

Example-1:

Diversity Activity/Intervention

  • Increase diverse talent recruitment sources

Outcome

  • Lower agency rates

Impact

  • Lower hiring costs
  • Jobs filled faster
  • Less need to use temporary help

Value-Added

  • Reduced operating expense
  • Faster human resource to organizational need

Example-2:

Diversity Activity/Intervention

  • Improve diverse work team problem-solving processes

Outcome

  • Reduced Time to Solve

Impact

  • Increase in reasons given in survey for long service by diverse workforce members due to enhanced team collaboration skills and respect for ideas

Value-Added

  • Retention savings as compared to rolling average of previous years

Example-3:

Diversity Activity/Intervention

  • Install succession planning for diverse workforce

Outcome

  • Fewer emergency diverse workforce hires

Impact

  • Less recruitment expense

Value-Added

  • Lower operating expense

For every use of resources to improve an organization using a Diversity activity/process, there should be an improvement in results. We call the result an “outcome”. The difference between this outcome and the previous outcomes before the Diversity process improvement was implemented is the “impact”. The dollar improvement represented by the impact is the “value added”. An example might include changing the talent acquisition methods used to hire diverse workforce talent (activities/process), which shortens the time to fill jobs (outcome).

Time-to-Fill Formula: TF = RR – OD     where:

TF = Time to have an offer accepted

RR = Date the requisition is received (e.g., January 4)

OD = Date the offer is accepted (e.g., February 20)

EXAMPLE

TF = January 4 – February 20

= 47 days

As jobs are filled faster, there is less need to use temporary or contract workers. The cost avoidance can be calculated and a dollar savings computed. If, through the Diversity department’s effort to change acquisition methods, jobs are filled faster, not only does the organization reduce operating expense, but the cost of the human resource product is lowered and moved to market faster. Lower human resource (human asset) acquisition cost and shorter human resource asset delivery time to the organization can create a competitive advantage, especially in light of less successful competitors (based upon benchmarking analysis comparisons).

Thus, report card and outcome-focused measures of Diversity activities and/or processes are vital in order to gain feedback on staff accountability for generating solutions to address such key business issues as inadequate diverse talent acquisition and the like. Without these activities or processes, we could not produce the accompanying outcomes, impacts and results. Even if these activities or processes produced poor results, the Diversity department can gain by knowing what else doesn’t work and then shift its efforts to more productive outcomes.

Creating an Effective Scorecard

In addition to report card and outcome-based measures, it is critical to have “scorecard measures”, a notion popularized by Harvard professors Robert Kaplan and David Norton in their book entitled: “The Balanced Scorecard” (Kaplan and Norton, 1996). In my book, “The Diversity Scorecard: Evaluating the Impact of Diversity on Organizational Performance, 2004, unlike the “report card”, I presented both lead and lag indicators and other analytics of performance and profitability that could be used to highlight Diversity’s impact on organizational performance. Report card measures only tell of past performance (lag indicator). Scorecard measures, on the other hand, reflect both “lead” indicators (such as Diversity climate survey favorable response ratings that indicate if an employee plans to leave within the next six months) and “lag” indicators of Diversity performance such as “high potential Diverse workforce voluntary turnover” that indicates what has happened (after the employee has left the organization leaving little chance to retain a valued human asset).

For the most part, any object, issue, act, process, or activity that can be described by observable variables is subject to measurement. The phenomenon can be evaluated in terms of cost, time, quantity, or quality. The central issue in applying measurement to the Diversity change process is this: to decide what is worth measuring and to agree on the measure as a fair representation of progress and accountability (given field limitations). Often, management will accept progress over perfection.

So, if you are not measuring Diversity’s ROI impact to demonstrate the effect and contribution you are having on the bottom-line, what’s stopping you?

Dr. Ed Hubbard, author of over 40 business books and recognized as the Pioneer and Founder of the Diversity Measurement and Diversity ROI Analytics fields, is an expert in Organizational Behavior, Organizational Analysis, Applied Performance Improvement and Measurement Strategies, Strategic Planning, Diversity Measurement, and Organizational Change Methodologies. He holds a Practitioner Certification and Master Practitioner Certification in Neurolinguistic Programming (NLP), a Neuro-science discipline. Dr. Hubbard earned Bachelors and Masters Degrees from Ohio State University and earned a Ph.D. with Honors in Business Administration.

 

Tags: , , , ,

“Searching For Measurement Processes And Best Practices: Key steps to building an effective Diversity measurement system” Part 1

Although interest in measuring the effects of diversity has been growing, the topic still challenges even the most sophisticated and progressive diversity departments. Diversity Professionals and Practitioners know they must begin to show how diversity is linked to the bottom-line or they will have difficulty maintaining funding, gaining support, and assessing progress. But where do they start? Well, I will weigh in on this topic with a few thoughts in a “two-part” analysis of this dilemma. In this segment (Part One), I will give an overview of a 5 step process to consider and highlight some key background issues. Part Two will detail each step of the 5-step process including possible diversity metrics, formulas, and suggestions. Although measuring Diversity Return on Investment (DROI®) impact is not an exact science, there are a number of valid techniques, tools and reliable methods for translating business performance gains into tangible financial results that C-Suite executives will support and embrace. Let’s get started by addressing the fundamental question…

Can diversity be measured?

Before we look at what to measure, I’d like to address the concern some practitioners have about the validity of measuring diversity results. Some practitioners seem to believe that quantifiable and quality-based measures cannot be applied to the diversity implementation process or a diverse work culture. Others believe that diversity is not a business-focused activity, simply another form of affirmative action regulatory compliance. However workforce and market place demographics make diversity a business and customer issue, as well as a global competitive issue!

Regardless of the events that led to this conclusion or whether this subjective position is valid or not, the fact that the position exists and that some diversity professionals and other business people support it creates major problems. In particular, it sets managing and leveraging diversity apart from the rest of the organization. While peers in other organizational areas are focusing on metrics that reflect their contribution such as sales, reduced costs, profits, income and expenses, those implementing the diversity process often limit their discussion of diversity’s contribution to increased awareness, improved feelings, and increased satisfaction among work groups. Only a select few really show demonstrated, evidenced-based results of Diversity’s impact on organizational performance.

As a result, diversity is not taken seriously. Fewer managers support it in actual practice, such as, sending their workforce to be trained, using Diversity in potential alternatives to solve business unit and customer problems, etc. Even fewer managers structure their workforce to leverage its richness through teaming, implementing strategic partnerships to penetrate key ethnic customer markets, and so on. We know from current organizational practice that diversity initiatives often experience less management support than other business initiatives.

I’m not suggesting that measurement is the sole solution to diversity’s acceptance into the corporate landscape. But measurement of results is a useful tool that allows the diversity practitioner to talk the language of other managers and top management. Remember Diversity activities are not conducted in a vacuum. They are part of an organizational system of processes, activities, and events aimed at delivering “value”, “impact” or both.

Building a measurement system

The creation of an effective Diversity measurement system and “best” practices cannot be a mechanical modeling exercise. It must be preceded by an inspection and utilization of basic business principles. It must focus on organizational and departmental strategic thinking as well as an assessment of the desired quality of work-life. Developing the actual measures is easy compared to the amount of time that should be spent thinking about what is important to the organization’s strategic business objectives and the expectations of the diversity measurement process.

Key steps to building an effective measurement system

Creating an effective Diversity measurement system and process that embodies these concepts involves at least five critical steps:

  • Review the Strategic Business Plan for Needs
  • Formulate Research Questions
  • Design the Study Methodology
  • Collect and Analyze Data
  • Implement Solutions and Communicate Results

Each step in the process logically builds on the previous step which generates an evidenced-based framework that creates a “Best Practice” method for proving Diversity’s link to performance. In the next segment (Part Two), we will explore each step in detail and provide suggestions for their effective use.

Short Bio on the Author

Dr. Edward E. Hubbard is President and CEO of Hubbard & Hubbard, Inc., (http://hubbardnhubbardinc.com), Petaluma, CA, an international organization and human performance-consulting corporation that specializes in techniques for applied business performance improvement, workforce diversity measurement, instructional design and organizational development.

The American Society for Training and Development (ASTD) inducted Dr. Ed Hubbard into the prestigious “ASTD New Guard for 2003”. The July/August 2007 Issue of Profiles in Diversity Journal featured Dr. Hubbard as the “Diversity Pioneer” in Diversity Measurement. In April, 2012 Dr. Hubbard was an honoree at the Inaugural International Society of Diversity and Inclusion Professionals Legends of Diversity Ceremony in Rio Grande, Puerto Rico where he received the Legends of Diversity Award for establishing the “Diversity ROI Analytics” and “Diversity Measurement Fields/Disciplines”. He also received the “Excellence in Global Leadership Award” from the World HRD Congress for 2015. Dr. Hubbard serves on the Harvard Business Review, Diversity Executive Magazine and Strategic Diversity & Inclusion Management (SDIM) magazine Editorial Advisory Boards.

Dr. Hubbard is an expert in Organizational Behavior, Organizational Analysis, Applied Performance Improvement and Measurement Strategies, Strategic Planning, Diversity Measurement, and Organizational Change Methodologies. He holds a Practitioner Certification and Master Practitioner Certification in Neurolinguistic Programming (NLP), a Neuro-science discipline. Dr. Hubbard earned Bachelors and Masters Degrees from Ohio State University and earned a Ph.D. with Honors in Business Administration.

 

Tags: , , , ,