Google’s efforts to enhance Indian language data and combat AI bias

On 28 June, Google held a developer event in Bengaluru to showcase its Pathways Language Model (PaLM) to Indian developers. Manish Gupta, director of Google Research India, spoke on enhancing the quality of data available in Indian languages and combating AI bias in India and the West. Google is now allowing developers to utilise its APIs to create solutions, and multiple teams, including user engineering units and Google Cloud divisions, already have pre-existing relationships with developers.

In alliance with the Indian Institute of Science (IISc), Google is working on Project Vaani, a digital dataset for Indic languages, where researchers and even large companies may use the open-source dataset. Google’s large language model (LLM) has already developed prejudices in local contexts, resulting in intersectional challenges like gender biases. The research lab is working with IISc to exchange best practices for AI researchers, with an emphasis on underserved languages and recognizing prejudice and misinformation.

Google established the benchmark for local language bias in India due to the technological gap in understanding bias in Indian languages being worsened by language model skills that are inferior in Indian languages compared to more mature languages such as English. The data technology gap is only one example of a gap in understanding bias in the Indian setting.