On December 19, 2023, the U.S. Department of Commerce’s National Institute of Standards and Technology (“NIST”) issued a Request for Information (“RFI”) regarding responsibilities set forth in the Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence (“EO”). Specifically, the EO “directs NIST to develop guidelines for evaluation, red-teaming and more; facilitate development of consensus-based standards; and provide testing environments for the evaluation of AI systems.” Comments responding to the RFI must be submitted by February 2, 2024.
The RFI notes that NIST is seeking input from the public to assist in its “carrying out several of its responsibilities under Sections 4.1 [Developing Guidelines, Standards, and Best Practices for AI Safety and Security], 4.5 [Reducing the Risks Posed by Synthetic Content], and 11 [Strengthening American Leadership Abroad] [… ,]” of the EO. Additional information regarding the “assignments and plans” set for NIST under the EO (including an informative video) can be found here.
In the RFI, NIST expresses its “interest[ed] in receiving information pertinent to” numerous “assignments” established by the EO. While each of the assignments address important topics (and the RFI provides additional detail and examples), below are several that may be of particular relevance.
- Developing a companion resource to the AI Risk Management Framework (“AI RMF”), NIST AI 100-1, for generative AI.
- Creating guidance and benchmarks for evaluating and auditing AI capabilities, with a focus on capabilities and limitations through which AI could be used to cause harm (e.g., autonomy, self-improvement, and access to external resources).
- Establishing guidelines, standards, and best practices for AI safety and security, including appropriate procedures and processes, to enable developers of AI to evaluate cybersecurity effectiveness of AI systems through adversarial testing (red-teaming) to enable deployment of safe, secure, and trustworthy systems.
- Soliciting input concerning topics related to synthetic content creation, detection, labeling, and auditing.
- Seeking information regarding topics related to the development and implementation of AI-related consensus standards, cooperation and coordination, and information sharing that should be considered in the design of standards.
Responses to the RFI must be received on or before February 2, 2024. Developers, researchers, and users of AI systems should strongly consider responding to NIST's request for information and requests that will be solicited by other agencies addressed in the EO over the coming months.
