Scale AI is proud to announce a new collaboration with the National Institute of Standards and Technology (NIST) in the Artificial Intelligence Safety Institute Consortium (AISIC) to develop science-based and empirically backed guidelines and standards for AI measurement and policy, laying the foundation for AI safety across the world.

The newly formed AISIC unites the leading AI companies and developers, academics, government and industry researchers, and civil society organizations in support of the development and deployment of safe and trustworthy AI, and will contribute to priority actions outlined in President Biden’s Executive Order, including developing guidelines for red teaming, capability evaluations, risk management, safety and security, and watermarking synthetic content. This effort will help ready the U.S. to address the capabilities of the next generation of AI models and systems with appropriate risk management strategies. 

“The U.S. government has a significant role to play in setting the standards and developing the tools we need to mitigate the risks and harness the immense potential of artificial intelligence. President Biden directed us to pull every lever to accomplish two key goals: set safety standards and protect our innovation ecosystem. That’s precisely what the U.S. AI Safety Institute Consortium is set up to help us do,” said Secretary Raimondo. “Through President Biden’s landmark Executive Order, we will ensure America is at the front of the pack – and by working with this group of leaders from industry, civil society, and academia, together we can confront these challenges to develop the measurements and standards we need to maintain America’s competitive edge and develop AI responsibly.”

Scale looks forward to working with NIST and other industry leaders to create the next set of methodologies to promote trustworthy AI and its responsible use. NIST has long been a leader in establishing industry-wide best practices and frameworks for the most innovative technologies. Scale applauds the Administration for its Executive Order on AI and the leadership at the Department of Commerce for recognizing that test & evaluation and red teaming are the best ways to ensure that AI is safe, secure, and trustworthy. In doing so, we not only contribute to the responsible use of AI, but also reinforce the United States’ position as the global leader in artificial intelligence. 

Learn more about Scale’s test & evaluation and red teaming initiatives here: 

https://scale.com/llm-test-evaluation