National Institute of Standards and Technology's new GenAI profile provides guideposts for AI governance professionals to mitigate risks unique to, or exacerbated by, GenAI

On July 26, NIST released a final version of its Generative Artificial Intelligence Profile (GenAI Profile), a cross-sectoral profile of and companion to the AI Risk Management Framework (AI RMF) (for further detail on the AI RMF, see our prior advisory). The new GenAI Profile reflects NIST's recommendations for implementing the risk management principles of the AI RMF specifically with respect to generative AI. This guidance is intended to assist organizations with implementing comprehensive risk management techniques for specific known risks that are unique to or exacerbated by the deployment and use of generative AI applications and systems.

The GenAI profile was mandated by President Biden's October 2023 Executive Order 14110 on Safe, Secure, and Trustworthy Artificial Intelligence (the AI EO). NIST's guidance on AI risk management practices have been widely cited by numerous authorities as a preferred solution for addressing AI risks, and their implementation is evidence of an organization's risk-mitigation steps to support the development of "responsible AI." With the release of this GenAI Profile, organizations can implement NIST's specific mitigation strategies as part of their current AI governance practices to address the unique risks presented by generative AI applications and systems. Indeed, developers and deployers of generative AI applications and systems who adopt these risk-mitigation practices in their AI governance programs may be able to rely on them as a defense against future claims of negligence, bias, or IP infringement asserted by adverse parties or government regulators.

Identifying and Evaluating Generative AI Risks

Many of the risks associated with GenAI are based on the technology's ability to lower barriers of entry to those looking to exploit existing risks or vulnerabilities, and to perpetuate those risks and vulnerabilities more broadly. For example, GenAI tools may be able to synthesize and summarize information relating to the development of harmful weapons. Although much of that information is currently publicly available, GenAI enables quicker access to such information with little cost to potential bad actors. In addition, GenAI tools may facilitate the perpetration of malicious acts and creation of misinformation or disinformation with greater speed and on a larger scale than previously possible.

NIST acknowledges that given the multimodal nature of many GenAI tools, developing effective governance programs for organizations is quite challenging. However, NIST's GenAI Profile provides clear guidance to address these difficulties, such as by implementing tools for additional human review, tracking and documentation, and greater management oversight of GenAI. Risks can also be compounded when organizations license upstream data or models for their own GenAI systems. Details and limitations of such data and models may not be clear, but businesses can attempt to obtain relevant information by including transparency obligations in procurement agreements.

In the GenAI Profile, NIST identifies 12 separate risks that are unique to, or exacerbated by, GenAI.[1] Among the most significant risks identified by NIST are the following:

• Access to information related to chemical, biological, radiological, or nuclear (CBRN) weapons. In the future, GenAI tools may enable malicious actors to more easily access CBRN weapons or relevant knowledge that could be misused to develop such weapons. Although NIST acknowledges that such information is often publicly accessible, large language models (LLMs) could facilitate its analysis or synthesis, particularly by individuals without formal scientific training. Beyond text-based GenAI tools, the GenAI Profile highlights specialized GenAI tools that aid in chemical and biological design as potential sources of risk.
• Confabulation. Commonly referred to as "hallucinations," confabulation refers to the phenomenon where GenAI systems generate and confidently present erroneous or false content in response to prompts. Risks from confabulations arise when users believe and act upon or promote the false information.
• Data Privacy. GenAI system training requires large volumes of data, which in some cases may include personal data. The use of personal data for GenAI training raises risks of unauthorized use, disclosure, or de-anonymization of biometric, health, location, or other personally identifiable information or sensitive data, and promotes the emulation of personal identity, likeness, or voice without permission.
• Environmental Impacts. Training and operating GenAI systems are resource-intensive activities that can result in substantial carbon emissions. "Current estimates suggest that training a single transformer LLM can emit as much carbon as 300 roundtrip flights between San Francisco and New York."
• Information Integrity. GenAI may increase risks of large-scale disinformation and misinformation campaigns by lowering the barrier to entry to generate and distribute content that may not distinguish fact from opinion or fiction or acknowledge uncertainties.
• Intellectual Property. Currently the subject of active litigation, the GenAI profile notes that training GenAI models on copyrighted material may not be considered fair use, and therefore may infringe on copyright and/or trademark rights. In addition, GenAI models may also run afoul of copyrights by reproducing copyrighted works in their outputs.
• Obscene, Degrading, and/or Abusive Content. NIST notes that GenAI tools can ease the production of and access to illegal non-consensual intimate imagery (NCII) of adults and/or child sexual abuse material (CSAM), in particular by GenAI tools' ability to create highly realistic "deepfakes" of real individuals. Such materials can create privacy, psychological, emotional, and physicals harms, and can in some cases be illegal.

Managing and Mitigating Generative AI Risks

The GenAI Profile identifies over 400 specific suggested actions to mitigate AI risks. These risk-mitigation tools can be applied to multiple types of risks and may be utilized by organizations to address differing risks.

In addition, the GenAI Profile identifies overarching themes that were distilled through the course of its drafting. These themes, while not exhaustive, informed NIST's specific recommended actions for managing GenAI Risks in line with the four core functions of its AI RMF: Govern, Map, Measure, and Manage. The themes relevant to GenAI systems identified by NIST are:

• Organizational Governance. Because GenAI systems operate in numerous modalities and can create a wide variety of outputs, a broad set of actors will interact with GenAI systems in widely different contexts of use. As a result, the development and use of GenAI systems may warrant additional human review, tracking and documentation, and greater management oversight.
• Third-Party Considerations. Organizations may choose to incorporate open-source or proprietary third-party GenAI models, systems, or data for various applications. An organization's reliance on third-party GenAI systems may give rise to increased intellectual property, data privacy, or information security risks. Organizations can mitigate these risks by performing due diligence on vendors, applying service level agreements, and complying with AICPA's statements on standards for attestation engagement reports to facilitate third-party transparency.
• Pre-Deployment Testing. The diverse ways and contexts in which GenAI systems may be developed and used complicates the task of pre-deployment risk mapping. Furthermore, NIST notes that current pre-deployment testing processes for GenAI may be inadequate, non-systematically applied, or fail to reflect deployment contexts. To combat these shortcomings, NIST suggests that organizations engage in AI red teaming, perform field testing, and solicit feedback from diverse stakeholders.
• Content Provenance. NIST notes that some GenAI outputs may challenge peoples' ability to distinguish between human-generated content and AI-generated synthetic content. GenAI systems used for content creation may require robust watermarking techniques and corresponding detectors to identify and trace the origins of content and mitigate risks related to information integrity.
• Incident Disclosure. AI incidents are defined in the GenAI Profile using OECD's definitions to be an "event, circumstance, or series of events where the development, use, or malfunction of one or more AI systems directly or indirectly contributes to one of the following harms:
  • injury or harm to the health of a person or groups of people (including psychological harms and harms to mental health);
  • disruption of the management and operation of critical infrastructure;
  • violations of human rights or a breach of obligations under applicable law intended to protect fundamental, labor, and intellectual property rights; or
  • harm to property, communities, or the environment."

Formal channels do not currently exist for reporting or documenting AI incidents. For that reason, NIST recommends that organizations consider developing guidelines for publicly available incident reporting to allow for better understanding of previous incidents and to prevent similar ones in the future. Further, organizations should consider developing guidelines for publicly available incident reporting that include information about AI actor responsibilities and which will help identify AI incidents and the actors associated with such incidents. In this circumstance, the documentation and review of third-party inputs and plugins for GenAI systems are especially important for AI actors.

[View source.]