Tuesday, August 2, 2022: In Webinar Discussing Its Pay Data Study NASEM Expert Panel Noted Flawed Design, Data Problems, and Recommendations
The expert panel behind the National Academies of Sciences, Engineering, and Medicine’s (NASEM) report evaluating the quality of the limited run EEO-1 Survey Component 2 data collection for reporting years 2017-2018 hosted a webinar to further discuss its method and conclusions. See last week’s WIR: “National Academy of Sciences Released Its Anticipated Pay Data Study; EEOC Commissioners Respond With Notably Different Takes” for details on the report’s conclusions. The following are some highlights of the presentation.
The survey design is antiquated and useless for self-assessment
The current EEO-1 Survey design has not been revised since the mid-1960s panelist Don Tomaskovic-Devey a Sociologist with the University of Massachusetts noted. Employers told them that this design is completely useless for self-assessments of pay. “Employers were really clear on that,” he said.
The job categories and pay bands are too broad due to this outdated design. The current design “does a better job of describing the U.S. economy in 1965 than it does today,” Tomaskovic-Devey observed, adding that it was designed largely for the pre-computer world.
Measurement issues abound
The panel cited several measurement issues of concern, including:
- The measure of pay should be total compensation, not taxable compensation.
- Box 1 on W-2s does not measure all compensation (Conclusion 3-1)
Note: Parenthetical references are to the Panel’s written Report
- Pay bands were overly broad (Conclusion 3-2)
- Job categories were overly broad and outdated (Conclusion 3-3)
- Measures of sex and race/ethnicity did not align with federal best practices and standards, respectively (Conclusion 3-4)
- No data were collected for some Protected Groups (Conclusion 3-5)
- Data on legitimate causes of pay gaps, such as education and experience, were not collected (Conclusion 3-6)
- Firm and establishment identifiers are neither consistent nor unique (Conclusion 4-2
The panel recommended transitioning to individual pay data collection
Transiting to the collection of individual-level pay data would solve many issues, the panel said. Employers already report individual pay data to multiple state and local government agencies. “This is something employers already know how to do and do well,” commented Tomaskovic-Devey.
Collecting individual-level pay data would lessen respondent burden while improving data quality, the panelists agreed. Plus, it solves multiple measurement problems (Recommendation 3-7). Moving to individual data will increase precision and reduce costs, Tomaskovic-Devey said. It also solves a lot of the problems with hours. Individualized data will allow a much better adjustment of the raw pay gaps and can include education and experience. “It’s a much more flexible way to work with the data in the long run,” he observed. The Bureau of Labor Statistics’ transition to individual pay data in the Occupational Employment and Wage Statistics may be instructive to the EEOC, the panel advised.
Moreover, business stakeholders said their reporting burden would go down if the EEOC collected individual data, partly due to enhancements in data collection, said Panel Chair William Rodgers III, who is vice president and director of the Institute for Economic Equity at the Federal Reserve Bank of St. Louis. It would allow employers to tell “better stories at lower cost,” he noted.
The data are valuable
“The U.S. is a little unusual in not having data like this compared to other countries,” Tomaskovic-Devey pointed out. Getting that kind of data will be useful for both the EEOC and employers, said Valerie Rawlston Wilson, director of the Economic Policy Institute’s Program on Race, Ethnicity, and the Economy. These data are “essential in plugging that hole in the information infrastructure necessary for [legal] enforcement,” she said.
EEOC should look to other agencies for examples of how to move forward
The panelists repeatedly noted their recommendation that the EEOC consult with statistics agencies such as the Census Bureau and the Bureau of Labor Statistics, which are already collecting individual-level data, but doing so without the legally-protected class demographic factors that the EEOC would need.
The study did not squarely address confidentiality concerns, but the panelists noted that there are a number of methods statistical agencies such as the Bureau of Labor Statistics and the Census Bureau use to protect confidential information, and the EEOC should work with those agencies. “Other federal agencies have long since solved this problem,” Tomaskovic-Devey said.
University of Maryland Economics Professor Judith K. Hellerstein acknowledged that there may be legal barriers to agencies sharing statistical data and otherwise working together to share information. She also noted that the processing of individual-level data is something the EEOC has not yet done, and it will require more computing power and related resources.
Other notes
Some additional observations of note:
- The panelists mentioned several times that further “cleaning of the data” needs to be done.
- “We are aware that the EEOC is doing an internal assessment around data modernization,” said Tomaskovic-Devey. Therefore, some of the panel’s recommendations that “may seem radical” are not actually that extreme.
- The panel repeatedly and strongly recommended pre-tests, field testing and stakeholder involvement.
- They also noted the report makes clear that the collection is only raw data and that the unadjusted data does not prove any legal violation. This data can only provide initial guidelines as to where there could be discrimination, but it is not evidence of a legal violation.