Merit-based reward practices can unintentionally lead to pay disparities based on gender, race, and national origin. Here’s how companies can use data, transparency, and accountability to prevent that.

For their companies to remain competitive and successful, many executives strongly believe that they need to recruit and retain top talent. And to do so, they must foster meritocracies — hiring, rewarding, and promoting the best people, based on their merit. As a result, the most progressive companies have created formal systems for ensuring that job applicants and employees are judged solely by their efforts, skills, abilities, and performance, regardless of gender, race, class, national origin, or sexual orientation. Executives might, for example, take great efforts to show their commitment to meritocracy by implementing performance reward systems that separate performance reviews from pay and reward decisions. But have such approaches helped workplaces become true meritocracies?
In research studying workplace inequality and merit-based pay, I have found that such approaches are no protection against demographic bias. (See “About the Research,”) When managers believe their company is a meritocracy because formal evaluative and distributive mechanisms are in place, they are in fact more likely to exhibit the very biases that those systems seek to prevent. Achieving meritocracy in the workplace can be more difficult than it first appears, and there may even be unrecognized risks behind certain efforts to discourage bias. According to my findings, the very belief that an organization is meritocratic may open the door to biased, nonmerit-based behavior when managers make key individual-level career decisions. In other words, certain gender, racial, and other demographic disparities might persist in today’s organizations not only despite management’s attempts to reduce them but also because of such efforts.
The good news is that establishing a more meritocratic workplace doesn’t require an inordinate amount of time or resources. It is a matter of establishing clear processes and criteria for the hiring and evaluation of employees (or, in fact, any employee career decision). It is also a matter of monitoring and evaluating the outcomes of such company processes, and of bestowing an individual or group within the organization with the responsibility, ability, and authority to ensure that those formal processes are fair. The collection and analysis of data on people-related processes and outcomes — what is referred to as “people analytics” — are key here, enabling companies to identify and correct workplace biases.

About the Research
This article is based on research from three different studies. The first was conducted at a business school at a private university in the United States. The study included more than 400 participants with work and managerial experience. Many of these individuals were MBA students, and the average age was roughly 30, with an average work experience of about six years. The participants were asked to play the role of a senior manager in charge of a small work group of consultants. They had to make managerial decisions about bonuses, promotions, and terminations for these employees at the end of a fiscal year.
For the second study, I investigated the merit-based system at a large service organization with more than 20,000 employees working in several offices in a competitive urban labor market in North America. According to the company’s HR policy manual, performance is the primary basis for all employee salary increases at this company, and a performance appraisal must be completed in order for any employee to obtain a merit pay increase.
For the third study, I analyzed two longitudinal databases containing information on the career history of all support staff for the large service organization from the second study. The first database contained the pay and performance evaluation history for a total of almost 9,000 exempt and nonexempt, nonexecutive and nonmanagement employees from 1996 to 2003, which was before the adoption of the new organizational procedures aimed at increasing accountability and transparency in the company’s performance reward system. This dataset included a total of more than 28,800 employee evaluations made by 2,667 unique direct (or evaluating) supervisors. The second longitudinal database covered the period from 2005 to 2009, which was after the introduction of the new organizational procedures. This dataset contained more than 23,000 employee evaluations made by 2,557 different supervisors.

The Paradox of Meritocracy
When managers believe their company is a meritocracy because of its formal evaluation and reward systems, chances are that it isn’t. I call this organizational phenomenon “the paradox of meritocracy.” To assess it, sociologist Stephen Benard and I conducted experiments involving more than 400 individuals with managerial experience, who were asked to make bonus, promotion, and termination recommendations for several hyphothetical employee profiles based on their annual performance.(1)
We manipulated the gender of the employees being evaluated and altered whether the core values of the company (a hypothetical “ServiceOne”) emphasized meritocracy in evaluations and compensation. For the meritocratic version of ServiceOne, the core values were described with statements such as “raises and bonuses are based entirely on the performance of the employee” and “ServiceOne’s goal is to reward all employees equitably every year.” For the nonmeritocratic version, the core values of ServiceOne instead emphasized managerial autonomy and the regularity of evaluation, using statements such as “raises and bonuses are to be given based on the discretion of the manager” and “ServiceOne’s goal is to evaluate all employees every year.” To ensure that the study participants had read and considered each of the core value statements carefully, we asked them to indicate whether they agreed with each value by placing a check mark on a line next to each statement.
Our findings were consistent across the experiments. When ServiceOne was explicitly presented as meritocratic to a randomly assigned group of managers, they tended to favor a male employee over an equally qualified female individual in the same job, with the same supervisor and the same performance evaluation scores. This bias resulted in larger monetary rewards for men. In one set of experiments, we found that managers who were told that their organization valued merit tended to award men with bonuses that were about 12% higher, on average, than they awarded to equally performing women. Similar but much less significant effects were found for hiring, promotion, and termination decisions, possibly because such decisions would be much more visible — and thus subject to greater scrutiny by others. When ServiceOne was not explicitly presented as meritocratic, the managers awarded the female employees with higher bonuses than they gave to male employees. A plausible explanation for that result is that the managers may have been self-compensating for an assumed bias in the performance evaluation scores, given the language about manager discretion used in this nonmeritocratic condition that they thought might favor male employees.(2)
The paradox of meritocracy may help explain the persistence of gender- and race-associated wage disparities at many organizations today. But what accounts for this paradox? One plausible explanation is that when managers believe their company as a whole is meritocratic, they may become less vigilant about their individual actions, leading them to unintentionally make biased decisions. It might be that, because managers think it unlikely that their actions will be interpreted as prejudiced, they are less prone to guard against being influenced by stereotypes. Moreover, they may have a false sense of confidence that their decisions in such an environment will be fair, objective, and impartial, leading to little self-examination to uncover any hidden demographic biases.(3)
These findings are important for companies that are employing progressive human resources practices to recruit and hire the best talent. They are also relevant for organizations that have increasingly been implementing merit-based (or pay-for-performance) routines as well as performance management systems. Over the years, the trend has swung away from rewarding employees based on seniority and has moved toward rewarding people with job assignments, training opportunities, pay increases, and promotions based on their past performance. My research provides a cautionary lesson for employers with merit-based approaches for attracting, selecting, and retaining their best employees: If not implemented carefully, such efforts can trigger demographic biases.
To study the implications for gender, race, and national origin of such HR systems in practice, I conducted a longitudinal investigation of the translation of employee performance evaluations into compensation decisions at a large service organization in North America.(4)
I analyzed personnel data for 8,900 support staff, which included information from personnel databases as well as from electronic and paper files. I examined the relationship between the merit-based salary growth and promotions of those employees on the one hand, and their performance evaluations on the other. I found that, over the long run, women and minorities received lower salary increases than white men with the same performance evaluation scores, even after controlling for job, work unit, and supervisor effects. In particular, the annual salary growth for women was 0.4% lower than for men, and African Americans and Hispanic Americans received salary increases that were 0.5% lower than equally performing white employees. In addition, workers born outside the United States had annual salary increases that were 0.6% lower than U.S.-born employees. These differences are small, but over time they result in gaps. For example, if “Alice” and “Robert” are equally performing Caucasian employees who both start at a wage of $10 an hour, but he gets a 10% salary increase each year while she gets a 9.96% increase for the same performance, then after 10 years, her hourly salary will be $25.84, and his hourly wage will be $25.94 — 10 cents more per hour.(5)
The company I studied had separated performance appraisals from pay decisions partly to facilitate supervisor feedback to employees to encourage their future career development. Employees were typically recommended for a salary increase or bonus not by their supervisors but by someone at a higher managerial level. Many companies have decoupled the processes of performance reviews and salary compensation. Doing so may be well-intentioned as a way to perhaps increase employee satisfaction and motivation. As I discovered in my study of that large service organization, however, it can introduce the conditions for biases and discriminatory judgments to occur at different stages of the performance reward system. Because subjectivity is usually involved, the typical performance evaluation process is vulnerable to biases with respect to gender, race, nationality, and other personal factors not related to performance. Biases can also affect decisions regarding an employee’s salary increase, promotion, transfer, or termination.

Ensuring Meritocracy in the Workplace
How, then, can companies guard against demographic biases and avoid the paradox of meritocracy in people-related decisions? To answer that question, I empirically tested a potential solution for minimizing the gap in the distribution of performance-based rewards at that large U.S. service organization.(6)   The company had two key areas that needed to be improved. The first had to do with the lack of organizational accountability.(7)   Previously, the company had no procedures to make unit heads (senior managers) accountable for their annual decisions regarding merit-based employee raises. A second, related issue was limited transparency in both the process behind the performance reward system as well as the system’s outcomes.(8)   I did not find gender, race, or foreign nationality disparities in employee promotions or terminations, which are highly visible decisions. I did, however, find such disparities with respect to merit-based pay, which are typically unobservable to managers and employees not directly involved. Thus, the solution was to adopt a set of organizational procedures that incorporated both accountability and transparency into the company’s performance reward system.
The intervention consisted of introducing three key changes to the existing system. First, a performance reward committee was appointed to monitor reward decisions. The committee, consisting of members of the company who were recruited from different divisions, including at least one HR professional, one executive, and a new full-time staff member in charge of compiling (coding and analyzing) data on employee compensation, would be responsible for assessing the fairness of pay decisions. Second, all senior managers had to follow a formalized process for assigning rewards based on employee evaluations. This process required the senior managers to briefly justify how much was awarded to each employee in their work unit. Third, the performance reward committee was granted the authority to modify pay decisions made by senior managers.
The results were noteworthy. For merit-based pay rewards, I found significant reductions in the gender, race, and foreign nationality gaps. In fact, any remaining differences from such biases were negligible. From my follow-up interviews with executives and managers at the company, I found evidence that both the accountability and the transparency mechanisms had been effective in reducing those pay gaps. For one thing, the performance reward committee would send all senior managers comprehensive reports containing descriptive statistics and data analyses about the merit-based pay increases, and that transparency of information helped hold those individuals accountable for their own decisions. As one senior manager remarked, “Annually, I am able to review numbers that have to do with how my pay decisions compare with the decisions made in the aggregate by other work units within my division.”

Implementing Organizational Accountability and Transparency
Other organizations can achieve similar results by using a relatively simple, straightforward process. First, executives can assess the general degree of meritocracy at their company by collecting data on the processes and the decisions concerning new hires, starting salary, merit-based pay, promotions, and other key career outcomes. Such data might already be available at many businesses. At the company I studied, the HR department had been collecting data on variables that could be easily coded from an individual’s resume at the time of application, such as years of work experience and level of education. The company was also collecting information on performance for each employee in a given year, together with longitudinal salary and benefits data for employees during their tenure at the company.
Once the data have been collected and properly coded, the company can analyze that information to uncover any demographic patterns, with other key employment variables being equal. It could, for example, investigate the way performance-based bonuses have been distributed among employees with a particular job title in a given year, comparing employees based on gender, race, and nationality, using some bivariate statistics. In addition, more detailed analyses — using, for example, multivariate regression models — could investigate pay bonuses and other employee rewards as a function of employee, job, and unit-level variables such as job title, manager, work experience, education, work hours, and demographic information. Those variables could be analyzed in addition to employee performance evaluation scores, which should be the most important predictor of bonus pay and other employee rewards in a meritocratic workplace. This use of data to study people-based processes, decisions, and outcomes (often called people analytics) can help companies identify and fix biases in the workplace.(9)
For the large U.S. service organization in my study, I was able to analyze longitudinal databases containing the career history of all of their support staff. These databases contained the pay and performance evaluation history for a total of almost 9,000 exempt and nonexempt, nonexecutive and nonmanagement employees. To test whether gender, race, or nationality (U.S.-born versus born outside the U.S.) had an effect on merit-based bonuses, I estimated a set of multivariate regression models and found a significant gap in the distribution of performance-based rewards based on employee demographics, other things being equal.
As described earlier, correcting for such biases required introducing greater organizational accountability and transparency in the existing performance reward system. Here, organizational accountability is defined as a set of procedures that make certain individuals responsible for ensuring the fair distribution of rewards among employees. Organizational accountability at different levels of the organization can be accomplished in two complementary ways. First, certain individuals can be made responsible for the design and implementation of organizational procedures that managers will use when they make pay decisions (process accountability). Second, certain individuals can be put in charge of monitoring and identifying situations in which managers are not making fair pay decisions (outcome accountability).
Organizational transparency is defined as a set of procedures that make relevant pay data available to certain individuals. This information can be made accessible in different ways to different audiences inside the organization, including HR professionals, managers, employees, and staff members. The information should be managed so that it is up-to-date and available in a timely manner both during and after the implementation of pay policies. Like organizational accountability, transparency can be accomplished in two complementary ways. First, steps can be instituted to ensure that pay distribution processes and criteria are known to certain individuals (process transparency). Second, specific measures can be implemented to make certain pay comparisons possible among individuals inside the company (outcome transparency). Both types of transparency can make disparities more noticeable and thus easier to correct.

Incorporating Accountability and Transparency Into a Pay-For-Performance System: Three Key Dimensions
To create more meritocratic systems, companies should promote organizational accountability and transparency in three key areas: (1) processes and criteria, (2) outcomes, and (3) audiences.

Three Key Dimensions
With the above definitions in mind, I propose three key dimensions for the design of systems that promote accountability and transparency in key organizational people-based systems: (1) processes and criteria, (2) outcomes, and (3) audiences. (See “Incorporating Accountability and Transparency Into a Pay-For-Performance System: Three Key Dimensions.”) The same dimensions also apply to recruitment and selection, access to training and developmental opportunities, promotions, and other key employee career outcomes. Moreover, this framework is general enough that it can be adopted regardless of a company’s existing personnel structure and culture. The goal is to help executives think systematically about the ways in which they can counterbalance the complacency that they might feel when they believe they are working within a meritocracy, while ensuring that the organization is indeed operating as a true meritocracy. In the case of a performance reward system, the different dimensions can be implemented with the following steps:

Step 1. Processes and Criteria: How will performance-based pay be distributed among employees? Companies need to determine who is going to be responsible for setting up the processes and criteria used to evaluate employee performance, how rewards will be distributed among employees based on their performance and other merits, and how transparent those processes and criteria will be across the organization. Some businesses, for example, may elect to tell employees how merit-based bonuses are computed based on exact measures of performance, while others may choose to disclose that information only to senior managers.

Step 2. Outcomes: What rewards are being given to employees? Companies must determine who will collect and analyze the appropriate data, who will be responsible for ensuring that decisions are being driven by merit, and who will get to see the analyses of that data. At the most basic level, the organization should have one individual responsible for collecting, coding, computing, and reporting those numbers and analyses to key decision makers in the organization.

Step 3. Audiences: Who is responsible for and who knows about the pay processes, criteria, and outcomes? Companies need to determine who will be allowed to design and implement changes in the HR structures and processes to develop a meritocratic workplace. Moreover, they should decide on the level of transparency about those processes, criteria, and outcomes both inside and outside the organization.
The decisions that companies make in the above three steps will determine the degree of organizational accountability and transparency in their performance management systems. An organization might, for example, assign just one director to be responsible for ensuring that pay processes and outcomes are fair. Or it could assign a group of individuals (with different affiliations inside, or even outside, the company) to serve as a committee or task force to monitor all pay-based processes, criteria, and outcomes.
Regarding transparency, a company may elect to simply publicize to employees and evaluating supervisors certain routines regarding the process and the criteria behind the distribution of bonuses. Or it could choose to leave the translation of performance into bonuses to senior managers so that employees and evaluating supervisors may not know the pay outcomes of such decisions — that is, high process and criteria transparency but low outcome transparency for employees and evaluating supervisors. In an extreme case (although not that unrealistic for companies that espouse pay for performance), the majority of managers and employees would know about the processes, criteria, and outcomes of the merit-based system. An example of a slightly less open approach would be to allow employees to see the average pay, including salary distribution by quartiles, for particular jobs inside the organization. Many companies already offer this level of transparency by providing employees with information to compare their base salary with the average salary (and range) by job/occupation and perhaps also by level of education, experience, and seniority within their organization.
Setting up a pay-for-performance system that is both accountable and transparent does not require a cumbersome bureaucracy. In fact, it can be quite simply done. In the large service organization that I studied, the accountability and transparency features added to the system consisted of a simple and inexpensive implementation of the framework presented in this article. The company convened and empowered a committee with high accountability and transparency responsibilities, with the members recruited from different divisions within the company. Furthermore, it hired one full-time person to collect and analyze the data to ensure fairness in the process of outcomes regarding the translation of performance into rewards. Employees and evaluating supervisors knew about the processes and criteria behind the evaluation of performance, but they did not get to see how bonuses were distributed by the senior managers. Finally, the committee, with the help of the additional full-time staff member, was responsible for monitoring results by systematically analyzing how bonuses were distributed over time, helping to ensure the meritocratic principles of the organization.

Toward the Rise of Meritocracy
In the past, long-term jobs with predictable career advancement and stable pay were common in large companies. Pay raises were often given on the basis of seniority or granted automatically to all employees at the same percentage levels.(10)   This traditional model of employment has gradually been replaced by merit-based reward systems and other performance management practices, and the hope is that they will lead to more meritocratic and fair workplaces where people are hired and rewarded solely for what they do and not for what they look like. The design and implementation of such approaches, however, can also unintentionally lead to biases based on gender, race, and national origin. In this regard, the pursuit of meritocracy in the workplace — that is, the organizational commitment to hire and reward the best people based on skill, ability, and merit, regardless of who they are, what they look like, or where they come from — is more difficult that it first appears. But it doesn’t have to be that way. As my research has shown, companies can develop into meritocracies by implementing merit-based evaluation and reward systems that have both accountability and transparency. What’s more, doing so doesn’t necessarily have to be a cumbersome, expensive, or daunting effort.

REFERENCES (10)
1. E.J. Castilla and S. Benard, “The Paradox of Meritocracy in Organizations,” Administrative Science Quarterly 55, no. 4 (December 2010): 543-576.
2. This result is consistent with previous research. See, for example, R.E. Petty and D.T. Wegener, “Flexible Correction Processes in Social Judgment: Correcting for Context-Induced Contrast,” Journal of Experimental Social Psychology 29, no. 2 (March 1993): 137-165.
3. See, for example, B. Monin and D.T. Miller, “Moral Credentials and the Expression of Prejudice,” Journal of Personality and Social Psychology 81, no. 1 (July 2001): 33-43; and E.L. Uhlmann and G.L. Cohen, “‘I Think It, Therefore It’s True’: Effects of Self-Perceived Objectivity on Hiring Discrimination,” Organizational Behavior and Human Decision Processes 104, no. 2 (November 2007): 207-223.
4. E.J. Castilla, “Gender, Race, and Meritocracy in Organizational Careers,” American Journal of Sociology 113, no. 6 (May 2008): 1479-1526.
5. Ibid.
6. E.J. Castilla, “Accounting for the Gap: A Firm Study Manipulating Organizational Accountability and Transparency in Pay Decisions,” Organization Science 26, no. 2 (March-April 2015): 311-333.
7. See A. Kalev, F. Dobbin, and E. Kelly, “Best Practices or Best Guesses? Assessing the Efficacy of Corporate Affirmative Action and Diversity Policies,” American Sociological Review 71, no. 4 (August 2006): 589-617. The authors show that employer policies designed to establish managerial responsibility for employee diversity are the most successful policies for increasing the representation of women and minorities in the management ranks of companies (compared to diversity policies targeting managerial bias or social isolation of disadvantaged groups). Along similar lines, field studies have suggested that practices with greater organizational transparency and accountability are associated with more equitable workplace outcomes for employees, irrespective of gender and race. See, for example, T. Petersen and I. Saporta, “The Opportunity Structure for Discrimination,” American Journal of Sociology 109, no. 4 (January 2004): 852-901; and Castilla, “Gender, Race, and Meritocracy.” These field study findings are consistent with considerable experimental work on accountability at the individual level that indicates that gender and racial biases are less likely when decision makers believe they will be held accountable for making fair decisions. See P.E. Tetlock, “Accountability and the Perseverance of First Impressions,” Social Psychology Quarterly 46, no. 4 (December 1983): 285-292; P.E. Tetlock, “Accountability and Complexity of Thought,” Journal of Personality and Social Psychology 45, no. 1 (July 1983): 74-83; and J.S. Lerner and P.E. Tetlock, “Accounting for the Effects of Accountability,” Psychological Bulletin 125, no. 2 (March 1999): 255-275.
8. In the workplace inequality and organizations literature, scholars have also suggested that transparency in decision-making criteria and processes behind the compensation and promotion of employees can help address gender and racial biases in the workplace. See L. Bailyn, “Putting Gender on the Table,” chap. 8 in “Becoming MIT: Moments of Decision,” ed. D. Kaiser (Cambridge, Massachusetts: MIT Press, 2012); W.T. Bielby and J.N. Baron, “Men and Women at Work: Sex Segregation and Statistical Discrimination,” American Journal of Sociology 91, no. 4 (January 1986): 759-799; and S.R. Bird, “Unsettling Universities’ Incongruous, Gendered Bureaucratic Structures: A Case-Study Approach,” Gender, Work & Organization 18, no. 2 (March 2011): 202-230. As described earlier, I proposed (without empirically testing) that organizational pay transparency, as a key complementary mechanism to organizational accountability, may address the observed gender and racial gaps in pay at one company. I argued that pay transparency can make disparities more noticeable and therefore easier for management to correct. Consistent with this transparency argument are my findings that the most visible employee career outcomes — such as being promoted — are less subject to the gender and race disparities; see Castilla, “Gender, Race and Meritocracy.” This finding is in agreement with other field studies; see, for example, Petersen and Saporta, “The Opportunity Structure for Discrimination.” Similar to the conclusions of prior field studies, many scholars have advocated for pay transparency as a preventive measure to address gender and racial inequality in the workplace. For example, law professor Gowri Ramachandran reviewed empirical studies and discovered that those in settings with higher levels of pay transparency (such as state government and unionized workplaces) find lower wage disparities on the basis of gender and race than studies in nonunionized private companies. See G. Ramachandran, “Pay Transparency,” Penn State Law Review 116, no. 4 (spring 2012): 1043-1080; see also M.M. Elvira and I. Saporta, “How Does Collective Bargaining Affect the Gender Pay Gap?,” Work and Occupations 28, no. 4 (November 2001): 469-490.
9. To learn more about these statistical techniques using longitudinal data, see, for example, E.J. Castilla, “Dynamic Analysis in the Social Sciences” (San Diego, California: Elsevier/Academic Press, 2007).
10. T.A. Kochan, H.C. Katz, and R.B. McKersie, “The Transformation of American Industrial Relations” (New York: Basic Books, 1986); see also P. Cappelli, “The New Deal at Work” (Boston: Harvard Business Review Press, 1999).

ABOUT THE AUTHOR
Emilio J. Castilla is the NTU Professor of Management at the MIT Sloan School of Management.

Source: sloanreview.mit.edu
June 13, 2016