The federal government is the biggest purchaser in America and that extends to the SaaS space. On September 24, 2024, the Office of Management and Budget (OMB) released Memorandum M-24-18, offering updated guidelines for the acquisition of AI software by federal agencies. Below is a breakdown of some of the issues AI providers must understand to navigate federal acquisitions effectively.
1. Responsible Acquisition of AI
The OMB’s memorandum emphasizes that federal agencies must integrate risk management into every stage of AI acquisition, particularly given AI's ability to impact privacy, civil rights and public safety. AI companies bidding on federal contracts should be prepared to:
- Demonstrate AI Risk Management: Vendors must show their systems can handle unique risks associated with AI development, including biases in data, security vulnerabilities, and the ability to explain and audit decision-making processes. This includes ensuring systems are capable of continuous monitoring and updating as new risks emerge.
- Comply with Privacy and Data Protection Standards: Agencies will expect AI vendors to safeguard personal and sensitive data, ensuring compliance with privacy laws. Companies should integrate privacy-enhancing technologies and secure data practices into their AI solutions.
Federal agencies are required to adopt robust risk management frameworks tailored to the specific nature of AI systems. Key directives for vendors include:
- Generative AI and Biometric Systems: Companies offering AI solutions with generative capabilities (e.g., AI that produces text, images or video) or biometric identification must meet stringent security and privacy standards. The OMB has laid out specific expectations regarding transparency in the training of models, ensuring that training data is lawfully obtained, and that the system performs reliably across diverse populations.
- Addressing AI-Related Bias: Vendors must identify potential biases in their systems, particularly those that could lead to unlawful discrimination based on race, gender or other protected classes. AI systems must include built-in mitigation strategies to reduce harmful bias and ensure equitable outcomes.
The memorandum highlights the OMB’s commitment to fostering a diverse and competitive federal AI market. Key initiatives relevant to AI vendors include:
- Preventing Vendor Lock-In: Federal contracts will increasingly require vendors to offer interoperable AI systems that can be easily integrated with other government systems and technologies. This minimizes the risk of agencies being locked into a single provider and ensures greater flexibility for future AI upgrades or changes.
- Encouraging Innovation: OMB encourages agencies to use procurement mechanisms like modular contracting and pilot programs to engage innovative AI vendors. This means AI companies should consider offering flexible, scalable solutions that can be expanded or refined over time.
4. Cross-Agency Collaboration and Transparency
The OMB memorandum stresses the need for collaboration between federal agencies to ensure best practices in AI acquisition are shared across the government. For vendors, this means:
- Sharing Information Across Agencies: AI providers should expect that once their system is adopted by one federal agency, the government may expect other agencies to have access to lessons learned, data on system performance, and the methods used to mitigate risks such as bias or system failure. This could include disclosure of training data, model performance metrics and compliance with security protocols.
- Continuous Monitoring and Reporting: AI vendors will be required to report performance issues, risks, or breaches to agencies in real-time or as they occur. Additionally, vendors must assist agencies in the ongoing evaluation and auditing of AI systems, especially for rights-impacting or safety-impacting systems.
5. Governmentwide Policies and Compliance
The OMB memorandum reiterates that while AI-specific policies are critical, they do not replace general government procurement laws and regulations, such as the Federal Acquisition Regulation (FAR), cybersecurity and data protection laws. Vendors should be mindful about:
- Compliance with General Procurement Laws: AI companies must meet the same procurement and contracting standards as other vendors, including FAR provisions on competition, cost control and transparency.
- Adherence to Cybersecurity Requirements: Vendors must comply with existing federal cybersecurity standards, including providing secure software development practices, ensuring that AI systems are resilient against attacks and implementing robust encryption for data in transit and at rest.
- Supporting Privacy and Civil Rights: AI vendors must ensure their systems comply with civil rights laws, including those that prevent discriminatory outcomes, and incorporate feedback mechanisms that allow individuals affected by AI decisions to challenge and appeal results.
6. Focus on Generative AI
Generative AI, including systems that create content, poses unique risks that agencies are now required to manage under the OMB's guidance. AI companies must be aware of:
- Transparency of AI-Generated Content: Companies providing generative AI must ensure that outputs such as text, video or images include mechanisms for identifying that the content was created by AI. This may involve embedding watermarks, metadata or cryptographic signatures to ensure transparency.
- Risk of Harmful Content: Vendors should employ safeguards to prevent their AI systems from generating harmful content, such as violent or false information. This includes ensuring training data excludes unlawful materials like child sexual abuse material or non-consensual intimate imagery.
Conclusion
For AI software companies, the OMB’s policies in Memorandum M-24-18 open significant opportunities in the federal marketplace, but also come with increased scrutiny on security, privacy and system performance. By aligning with the OMB’s requirements, AI vendors can tap into the federal government’s expanding investment in AI, particularly as agencies look to modernize their operations with innovative AI solutions that are safe, secure and transparent.
