White House and top AI developers will collaborate to evaluate generative AI systems at DEF CON 31

The White House, in collaboration with leading AI developers, has organised an event for the public evaluation of generative AI systems at DEF CON 31, a hacker convention scheduled to take place in August. This event aligns with the Biden administration’s AI Bill of Rights and the National Institute of Standards and Technology’s AI Risk Management Framework.

Computer, Electronics, Laptop, Pc, Computer Hardware, Hardware, Monitor, Screen, Computer Keyboard, Furniture, Table, Adult, Male, Man, Person, Mobile Phone, Phone, Desk, Body Part, Finger, Hand, Pen, Tablet Computer

The White House has collaborated with top AI developers, including OpenAI, Google, and Microsoft, to participate in a public evaluation of their generative AI systems at DEF CON 31, a hacker convention taking place in Las Vegas in August 2023. The evaluation aims to push the new generative AI models to their limits and uncover potential flaws and vulnerabilities, such as confabulations, jailbreaks, and biases, which pose challenges for security professionals and the public. The White House Office of Science, Technology, and Policy endorses this exercise, which aligns with the Biden administration’s AI Bill of Rights and the National Institute of Standards and Technology’s AI Risk Management Framework.

Participants will have timed access to multiple large language models (LLMs), including ChatGPT, through laptops provided by the organisers. AI Village aims to grow the community of researchers equipped to handle vulnerabilities in AI systems by conducting the largest red-teaming exercise for any group of AI models.