Join top executives in San Francisco on July 11-12 to hear how leaders are integrating and optimizing AI investments for success. Learn more
One of the most effective ways to test an application’s security is to use hostile attacks. In this method, security researchers actively attack the technology – in a controlled environment – to find previously unknown vulnerabilities.
It’s an approach now being advocated by the Biden-Harris administration to help secure generative artificial intelligence (AI). As part of its Actions to Promote Responsible AI announcement yesterday, the government called for public reviews of existing generative AI systems to be conducted. As a result, this year’s DEF CON 31 security conference, to be held August 10-13, will feature a public assessment of generative AI on the AI village.
“This independent exercise will provide researchers and the public with critical information about the impact of these models, and enable AI companies and developers to take steps to address issues in those models,” the White House said in a statement. Edition.
Some of the leading vendors in the generative AI space will participate in the AI Village hack, including: AnthropicGoogle, Hugging facemicrosoft, nvidia, Open AI And Stability AI.
DEF CON villages have a history of advancing security knowledge
The DEF CON security conference is one of the largest gatherings of security researchers in any given year and has long been a venue where new vulnerabilities have been discovered and revealed.
Nor will this be the first time a village has taken aim at DEF CON for a technology that is making national headlines. In recent years, especially after the 2016 US election and fears of election interference, a Voting Village was set up at DEF CON in an effort to look at the security (or lack thereof) in voting machine technologies, infrastructure, and processes.
The Villages at DEF CON allow attendees to discuss and explore technologies in a responsible disclosure model that aims to help improve the overall state of security. With AI, there is a particular need to examine the technology for risk as it becomes more widely deployed in society at large.
How the generative AI hack works
Sven Cattell, the founder of AI Village, responded in a rack that companies have traditionally solved the problem of identifying risk by using specialized red teams.
A red team is a type of cybersecurity group that simulates attacks to detect potential problems. The challenge with generative AI, according to Cattell, is that much of the work around generative AI has been done privately, without the benefit of a red team review.
“The various problems with these models won’t be solved until more people know how to drive and judge a team,” said Cattell.
Specifically, AI Village’s generative AI attack simulation will consist of on-site access to large language models (LLMs) from the participating vendors. The event will feature a ‘capture the flag’ point system where attackers earn points for completing certain objectives that demonstrate a range of potentially harmful activities. The person with the highest number of points wins a ‘high-end Nvidia GPU’.
The evaluation platform on which the event will run is developed by Scale AI. “As the use of basic models becomes widespread, it is critical to ensure that they are carefully evaluated for reliability and accuracy,” Alexandr Wang, founder and CEO of Scale, told VentureBeat.
Wang noted that Scale spent more than seven years building AI systems from the ground up. He claims his company is also unbiased and not committed to a single ecosystem. As such, Wang said Scale is able to independently test and evaluate systems to ensure they are ready to go into production.
“By bringing our expertise at DEF CON to a wider audience, we hope to ensure that advances in base model capabilities coexist with advances in model evaluation and safety,” said Wang.
