On December 23, 2022, the New York City Department of Consumer and Worker Protection (DCWP) published updated proposed rules to implement the city’s automated employment decision tools (AEDT) law (Local Law 144). The law conditions the use of automated employment decision tools to screen candidates for employment or employees for promotion within the city on compliance with certain requirements, including the performance of a bias audit, and the furnishing of notifications to candidates and employees. The DCWP issued updated proposed rules following a significant volume of public input concerning an initial version of the rules released on September 23, 2022, and the DCWP’s ensuing decision to postpone enforcement of the law until April 15, 2023, despite the law’s January 1, 2023, effective date.
New Guidance Concerning the Characteristics of an Independent Auditor
In an effort to address an issue that generated a substantial amount of attention, the updated proposed rules provide additional guidance concerning the characteristics of an independent auditor. The initial version of the proposed rules explained that an “[i]ndependent auditor” would be required to be “a person or group that is not involved in using or developing an AEDT.” The updated proposed rules now explain that an “[i]ndependent auditor” means “a person or group that is capable of exercising objective and impartial judgment on all issues within the scope of a bias audit of an AEDT.” Additionally, the updated proposed rules identify three disqualifying characteristics, namely a person or group that:
- is or was involved in using, developing, or distributing the AEDT;
- at any point during the bias audit, has an employment relationship with an employer or employment agency that seeks to use or continue to use the AEDT or with a vendor that developed or distributes the AEDT; or
- at any point during the bias audit, has a direct financial interest or a material indirect financial interest in an employer or employment agency that seeks to use or continue to use the AEDT or in a vendor that developed or distributed the AEDT.
New Guidance Concerning Data Sources That May Be Used for a Bias Audit
Although the examples incorporated in the initial proposed rules indicated that a bias audit could be based upon “historical data,” no specific guidance was provided concerning the meaning of this term or the source(s) of data that could be used for the audit. The updated proposed rules fill this gap by specifying that “[h]istorical data” means “data collected during an employer or employment agency’s use of an AEDT to assess candidates for employment or employees for promotion.” The updated proposed rules further explain that if the AEDT is used by “multiple employers or employment agencies,” an employer or employment agency “may rely on a bias audit of an AEDT that uses the historical data of other employers or employment agencies only if it provided historical data from its use of the AEDT to the independent auditor for the bias audit or if it has never used the AEDT.”
In addition, the updated proposed rules incorporate a definition of “[t]est data” which is explained as “data used to conduct a bias audit that is not historical data.” The updated proposed rules specify that test data may be used to conduct a bias audit only “[i]f insufficient historical data is available to conduct a statistically significant bias audit.” If test data is used, the updated proposed rules would mandate that “the summary of results of the bias audit … explain why historical data was not used and describe how the test data used was generated and obtained.” Underscoring these new components, the updated proposed rules add that the bias audit results would be required to be published “in a clear and conspicuous manner” on “the employment section of the website” of an employer or employment agency and “include the source and explanation of the data used to conduct the bias audit,” along with other information specified in the initial proposed rules, namely, the “distribution date of the AEDT,” the “date of the most recent bias audit,” and a “summary of the results,” which would include “the selection rates and impact ratios for all categories” required to be considered in the audit.
Additional Guidance Concerning the Required Frequency of the Bias Audit
As potentially impacted employers and employment agencies have been aware, the law specifies that the use of AEDT to screen a candidate or an employee for an employment decision is unlawful unless the tool “has been the subject of a bias audit conducted no more than one year prior to the use of such tool.” The updated proposed rules would amplify this requirement by modifying the section of the rules outlining the components of a bias audit to state that “[a]n employer or employment agency may not use or continue to use an AEDT if more than one year has passed since the most recent bias audit of the AEDT.” Like the earlier version of the proposed rules, the updated proposed rules direct that “[a]n employer or employment agency must keep the summary of results and distribution date posted for at least 6 months after last using the AEDT for an employment decision.”
New Guidance Concerning Determination of the Impact Ratio of an AEDT
The updated proposed rules modify in part the definition of the “[i]mpact ratio” to be evaluated in a bias audit. The initial version of the proposed rules defined the term as either “(1) the selection rate for a category divided by the selection rate of the most selected category or (2) the average score of all individuals in a category divided by the average score of individuals in the highest scoring category.” The updated proposed rules revise the second portion of the definition to substitute “scoring rate” for “average score.” The term “scoring rate” is now defined as “the rate at which individuals in a category receive a score above the sample’s median score, where the score has been calculated by an AEDT.” Consistent with this modification, the updated proposed rules specify that “[w]here an AEDT scores candidates for employment or employees being considered for promotion, a bias audit must, at a minimum:
(1) Calculate the median score for the full sample of applicants;
(2) Calculate the scoring rate for individuals in each category;
(3) Calculate the impact ratio for each category[.]”
Despite some changes in wording, the updated proposed rules require the same computations for an AEDT performing a selection function that the earlier version of the proposed rules required. Specifically, the bias audit would be required, “at a minimum,” to calculate the “selection rate for each category” and the “impact ratio for each category” when the AEDT “selects candidates for employment or employees being considered for promotion to move forward in the hiring process,” or where the AEDT “classifies candidates for employment or employees being considered for promotion into groups.”
New Guidance Concerning Information Reported in the Bias Audit
The updated proposed rules also expand the information reported in the bias audit. Whereas the initial version of the proposed rules included examples suggesting that the audit should calculate the impact of the AEDT solely on intersectional categories reported on the EEO-1, the updated proposed rules specify that the bias audit “must separately calculate the impact of the AEDT” on:
- Sex categories (i.e., impact ratio for selection of male candidates vs female candidates),
- Race/Ethnicity categories (e.g., impact ratio for selection of Hispanic or Latino candidates vs Black or African American [Not Hispanic or Latino] candidates)
- [I]ntersectional categories of sex, ethnicity, and race (e.g., impact ratio for selection of Hispanic or Latino male candidates vs. Not Hispanic or Latino Black or African American female candidates).
Additional Guidance Concerning the AEDTs That Trigger Applicability of the Law
One of the key questions for employers and employment agencies relates to the definition of an AEDT. According to Local Law 144, an AEDT is “any computational process, derived from machine learning, statistical modeling, data analytics, or artificial intelligence, that issues simplified output, including a score, classification, or recommendation” that is used to “substantially assist or replace discretionary decision making for making employment decisions that impact natural persons.” The initial version of the proposed rules explained that “to substantially assist or replace discretionary decision making” meant:
- “to rely solely on a simplified output (score, tag, classification, ranking, etc.), with no other factors considered”;
- “to use a simplified output as one of a set of criteria where the output is weighted more than any other criterion in the set”; or
- “to use a simplified output to overrule or modify conclusions derived from other factors including human decision-making.”
The updated proposed rules modify the third element slightly to state that an AEDT will be deemed to substantially assist or replace discretionary decision making if it “use[s] a simplified output to overrule conclusions derived from other factors including human decision-making.”
Next Steps for Employers
The DCWP has scheduled a public hearing on the updated proposed rules for Monday, January 23, 2023, at 11:00 a.m. Interested parties may submit comments on or before the date of the hearing.
Employers and employment agencies that utilize automated decision tools that fall within the scope of the law may wish to continue to plan for how to comply with the law’s bias audit and disclosure requirements, as clarified by the updated proposed rules, and remain alert for the eventual publication of final rules. In addition, employers may wish to monitor guidance from federal agencies and legislation in other jurisdictions where they operate for developments that may impact the use of these tools.
