Realizing trustworthy AI through stakeholder collaboration

Event report

In 2019, the OECD developed the AI Principles as the first intragovernmental standards on AI with the goal of creating a trustworthy ecosystem for AI that benefits society and the planet. Building on the OECD AI values-based principles approach, this workshop considered the enabling environment for stakeholder collaboration.

Recent years have seen a convergence towards risk-based approaches to AI policy and the system life cycle. The approach of adapting policy to risk has four core themes: define, assess, treat, and govern. Different applications and use cases present varied risks, and the right framework has to be able to properly define and classify AI. Based on the classification framework, the assessment has to include different stakeholders and ensure the interoperability of standards. Treating the risks depends on agreement among stakeholders and is often difficult to achieve. Comparing the existing frameworks would improve the assessment of the system and facilitate the implementation of principles. Finally, there are a lot of mechanisms for risk management (from legislation, through experimentation, to standards) but the key point the OECD emphasises is to promote innovation and to base policy on evidence and experimentation. Additionally, applying existing legal instruments and enforcement mechanisms, such as the OECD Due Diligence Guidance for Business could help companies to adapt their existing risk management frameworks to AI systems.

Putting ethical and high-level principles into practice is not a straightforward process. The technical and operational challenges involved are usually discussed in silos. Developers, engineers, product managers, and data scientists discuss these issues somewhat separately from the non-technical policy community. These silos need to be crossed. This set of OECD principles tries to bring together policy and technical issues. As such, principles can serve as a conversation starter and expand into different forums, including engineering competitions, coding competitions, hackathons, and other forums where AI developers gather. 

Another key point is how to balance internationalising standards and make them interoperable. The main challenge of operationalising principles is going beyond abstract high-level terms to more practical multistakeholder discussions. For example, bias has a different meaning to an engineer than to a user. Standards bridge this gap by providing a common understanding of fairness, transparency, well-being, and other principles. Standards also enable metrics of success within the system cycle, as they define the steps in the development of these socio-technical products. A good example is the Standard on Value-Based Design that translates ethical principles into concrete system requirements and defines how a system should behave. 

It was also noted that existing practices in building AI can inform principles: practices can help clarify, refine, and improve the implementation of principles in a way that will help wider dissemination and implementation of said principles. Companies are now looking at and addressing the OECD AI principles as a whole and acknowledging their intrinsic importance in achieving responsible AI. Principles need to interact with each other. For example, privacy and fairness need to interact as AI developers need access to sensitive data to better determine how their AI models are working for people of, for example, different races and ethnicities.

Another practice that can inform principles is the use of regulatory experimentation such as sandboxes. Sandboxes allow simultaneous design and implementation, while also involving the public and bridging the layperson-expert gap. Such participatory tools allow for transparency among stakeholders, solidify trust by engaging the public, and enable collaboration between groups that would otherwise not interact. Lowering barriers within government towards a more participatory process can also reveal value tensions and in turn contribute to building more appropriate systems.

By Andrijana Gavrilovic and Jana Misic

The session in keywords