Joint civil society statement: Structural intervention is required to mitigate the racially discriminatory impacts of emerging digital technologies including AI

As widespread recent protests have highlighted, racial inequality remains an urgent and devastating issue around the world, and this is as true in the context of technology as it is everywhere else. In fact, it may be more so, as algorithmic technologies based on big data are deployed at previously unimaginable scale, reproducing the discriminatory systems that build and govern them.

The undersigned organizations welcome the publication of the report “Racial discrimination and emerging digital technologies: a human rights analysis,” by Special Rapporteur on contemporary forms of racism, racial discrimination, xenophobia and related intolerance, E. Tendayi Achiume, and wish to underscore the importance and timeliness of a number of the recommendations made therein:

1. Technologies that have had or will have significant racially discriminatory impacts should be banned outright. 
   While incremental regulatory approaches may be appropriate in some contexts, where a technology is demonstrably likely to cause racially discriminatory harm, it should not be deployed until that harm can be prevented. Moreover, certain technologies may always have disparate racial impacts, no matter how much their accuracy can be improved. In the present moment, racially discriminatory technologies include facial and affect recognition technology and so-called predictive analytics. We support Special Rapporteur Achiume's call for mandatory human rights impact assessments as a prerequisite for the adoption of new technologies. We also believe that where such assessments reveal that a technology has a high likelihood of deleterious racially disparate impacts, states should prevent its use through a ban or moratorium. We join the Special Rapporteur in welcoming recent municipal bans, for example, on the use of facial recognition technology, and encourage national governments to adopt similar policies. Correspondingly, we reiterate our support for states’ imposition of an immediate moratorium on the trade and use of privately developed surveillance tools until such time as states enact appropriate safeguards, and congratulate Special Rapporteur Achiume on joining that call.

2. Gender mainstreaming and representation along racial, national and other intersecting identities requires radical improvement at all levels of the tech sector. 
   The structural racism and discrimination that the report identifies as endemic to the field of technology (just as it is in many if not all facets of societies around the globe) cannot be remedied if the teams conceiving of, building, and promoting technological solutions do not understand and represent the concerns of those who will be impacted by them. When implemented, these demographic changes must be meaningful: past “diversity” and “inclusivity” efforts in the industry have often been mere tokenizations of underrepresented groups. Meaningful improvement will mean significant changes to industry power structures, funding flows and models, cultural changes in the workplace, and reevaluation of existing and future product lines that may be employed to target racially marginalized communities and other vulnerable populations. [1]

3. Technologists cannot solve political, social, and economic problems without the input of domain experts and those personally impacted. 
   The past several decades have been dominated by “techno-chauvinism,” the idea that technology alone can solve social problems. [2] But as Special Rapporteur Achiume rightly notes, no algorithmic model, no matter how “perfect,” will solve for centuries of inequality. For this reason, tech design and development must include domain experts (including those with first-hand experience alongside those with professional or academic expertise) in a consequential way. Such experts cannot be briefly consulted late in a product’s development, or worse, after its negative impacts are already being felt, but integrated into the design process. Furthermore, they must be compensated for their contributions. This is especially important in sensitive sectors such as education, criminal justice, and social services, where past technologies have been adopted in spite of flaws and biases that were facially apparent to experts in the respective field. Moreover, such experts can help to identify how well-meaning technologies may be distorted or misused in practice, exacerbating social inequities and harms rather than actually addressing community needs. 

4. Access to technology is as urgent an issue of racial discrimination as inequity in the design of technologies themselves. 
   Digital divides illustrate the intersectionality of discrimination in technology, and the COVID-19 pandemic has thrown them into sharp relief. The global South lags the global North in the digital infrastructure that can be employed to provide remote access to medical care, education, and more, but even within so-called “developed” countries, poor and otherwise marginalized communities lack access to these necessary tools. Digital divides are also present within global South states: we have not reached Sustainable Development Goal target 9(c), to bring all those in least developed countries online by 2020. As societies’ reliance on technological interventions increases over time, these divides will deepen in significance and pose life-or-death threats to people around the world. 

5. Representative and disaggregated data is a necessary, if not sufficient, condition for racial equity in emerging digital technologies, but it must be collected and managed equitably as well. 
   We welcome Special Rapporteur Achiume’s call for “States to collect, compile, analyse, disseminate and publish reliable statistical data disaggregated on racial or ethnic grounds.” However, despite important advances in data protection laws globally, adequate standards for non-extractive data collection and governance have not yet been disseminated. The history – and present – of data collection is rife with examples of the exploitation of marginalized populations, and the data that is thus taken is then generally managed in such a way that it can too often be used to further target and victimize. We would welcome, and gladly participate, in an effort to develop such standards, which should include both measures to ensure that data is truly representative and respectful, not reinforcing existing racial and other hierarchies; address the power dynamics between data collectors and those whose data is collected, including around meaningful consent; and provide safeguards to prevent such data from being used by malign actors to further oppress marginalized people.

6. States as well as corporations must provide remedies for racial discrimination, including reparations. 
   The signatories followed with interest Special Rapporteur Achiume’s previous report on the centrality of reparations in remedying the many years of socioeconomic oppression that endured during and continue after colonialism and slavery. In this report, the Special Rapporteur rightly notes that states and companies who have been involved with or overseen the development and deployment of emerging digital technologies that introduced novel instances of racial discrimination or deepened existing inequalities are obligated to participate in remedial processes and provide adequate compensation. Given the interplay between economic privilege and access to technology, as well as the rooms in which it is designed and regulated, we wish to underscore the link between these reports, and to encourage the tech giants whose record profits result at least in part from this inequality to consider acting on their public commitments to non-discrimination. This includes meaningful engagement by states and tech companies to evaluate their role in maintaining racial power structures and taking necessary steps to dismantle them.

Notes:

[1] https://onezero.medium.com/the-seductive-diversion-of-solving-bias-in-artificial-intelligence-890df5e5ef53

[2] https://www.theverge.com/2018/5/23/17384324/meredith-broussard-artifical-unintelligence-technology-criticism-technochauvinism

 

Core signatories:

Access Now

AI Now Institute

Amnesty International

Association for Progressive Communications (APC)

Digital Freedom Fund

Internet Sans Frontières

Rashida Richardson, Visiting Scholar, Rutgers Law School

 

Organisations:

Bangladesh NGOs Network for Radio and Communication

Center on Race, Inequality, and the Law at NYU School of Law

S.T.O.P. – The Surveillance Technology Oversight Project

Asociación TEDIC

AfroLeadership

Bolo Bhi

Digital Welfare State and Human Rights Project at NYU Law

Liberia Information Technology Student Union

ENRICH – Equity for Indigenous Research and Innovation Co-ordinating Hub

Free Expression Myanmar (FEM)

Advocacy Initiative for Development (AID)

BetaNYC

Center for Media Studies and Peacebuilding

Superrr Lab

Homo Digitalis

Fundación Huaira

Rudi International

Hiperderecho

Državljan D

Fundación Karisma

SeguDigital

Fundación Datos Protegidos

Derechos Digitales América Latina

safenet

Iraqi Network for Social Media – INSM

7amleh – The Arab Center for the Advancement of Social Media

Media Institute of Southern Africa-Zimbabwe

RosKomSvoboda

Centre for Human Rights, University of Pretoria

Collaboration on International ICT Policy for East and Southern Africa (CIPESA)

Digital Welfare State and Human Rights Project at NYU Law

Centre for Information Technology and Development (CITAD)

Americans for Democracy & Human Rights in Bahrain (ADHRB)

European Center for Democracy and Human Rights (ECDHR)

Centre for Multilateral Affairs (CfMA)

Paradigm Initiative (PIN)

World Wide Web Foundation

Common Cause Zambia

Right 2 Know Campaign

Unwanted Witness Uganda

European Center For Not-Profit Law Stichting (ECNL)

Defend Digital Me

GreenNet

ASUTIC Senegal

American Civil Liberties Union (ACLU)

Movement Alliance Project

Media Research Association (MEDAR)

L’Association Francophone pour les Droits de l’Homme

DigitalSENSE Africa Media

ITREALMS Media

Código Sur

Bloggers of Zambia

Montreal AI Ethics Institute

Usuarios Digitales

Indoensian ICT Volunteers (Relawan TIK Indonesia)

OpenNet Africa

The Initiative for Equal Rights

Civil Liberties Union for Europe

Thai Netizen Network

Public Knowledge

Coordenação Estadual das Comunidades Quilombolas do Tocantins

AI for the People

European Digital Rights (EDRi)

Intervozes – Coletivo Brasil de Comunicação Social

CIPPIC – The Samuelson Glushko Canadian Internet Policy and Public Interest Clinic

Ranking Digital Rights

Jokkolabs Banjul

AFROTECTOPIA

Refugee Law Laboratory

Hun Consultancy

Amnistie internationale Canada francophone

Instituto Nupef

Open Data Manchester CIC

Privacy International

 

Individuals:

Karine Gentelet, Université du Québec en Outaouais

Professor Ellen P. Goodman, Rutgers Law School

Dr. Sarah Myers West, AI Now Institute/ Data Justice Advocate / HCI Researcher

Dr. Stephen C. Rea, Colorado School of Mines

Dr. Joy Lisi Rankin, AI Now Institute, New York University

Stefaan G. Verhulst, Co-Founder, The GovLab, NYU

Charlie Martial NGOUNOU

Usama Khilji, Bolo Bhi

Ahmed Ansari, NYU

Avis Momeni, civil society actor

Damian Loreti, individual member of APC

Jane Anderson, ENRICH and New York University

Molly Land, Human Rights Institute, University of Connecticut

Mariana Valente

Meredith Broussard, New York University; author of “Artificial Unintelligence: How Computers Misunderstand the World”

Jessica Fjeld, Harvard Law School Cyberlaw Clinic at the Berkman Klein Center for Internet & Society

Mara Mills, Associate Professor, New York University

Ephraim Percy Kenyanito, Global Voices

Parviz Khazaei

Andrew Sporle, The University of Auckland, iNZight Analytics Ltd.

Dr Monique Mann, Senior Lecturer in Criminology, Deakin University

Gayathry Venkiteswaran, media freedom activist, Malaysia

Sabine Poeschmann

Kate Sim, Oxford Internet Institute

John Lunsford

Dr Emily Petherick Loughborough University

Aung Zaw Min

Muheeb Saeed, Media Foundation for West Africa

Daniel Kreiss, UNC Center for Information, Technology, and Public Life

Zeerak Waseem, University of Sheffield

Tonia Sutherland, PhD, Director SOURCE Hawaiʻi, University of Hawaiʻi at Mānoa, Advidory Board, Center for Critical Race + Digital Studies

Janna Huang, UC Berkeley Sociology

Lucy Bernholz, Digital Civil Society Lab, Stanford University

Resident Fellow, Information Society Project at Yale Law School

Vibranium Health Equity

Leyla Gayibova, Teaching Assistant at The Hague University of Applied Sciences

Ben Green, Harvard University

Samuelson-Glushko Professor of Law, University of Ottawa

University of Livingstonia, Malawi

Juliet Nanfuka, CIPESA

Gürkan Özturan, Journalist, Digital Rights & Liberties Analyst

Dr. Farida Vis, Media Department, Manchester Metropolitan University (UK), Director Visual Social Media Lab, Advisory Board Member, Center for Critical Race & Digital Studies

Dr. Simon Faulkner, Department of Art and Performance, Manchester Metropolitan University, United Kingdom

Gaëtan Goldberg (Data Rights)

Mariana Fossatti, APC

Dr. Baobao Zhang, Cornell Society of Fellows; Berkman Klein Center for Internet & Society, Harvard University

Christelle A.

Abebe Chekol, ICT4D Expert

Cynthia Khoo, Research Fellow, Citizen Lab

Rediet Abebe, Harvard Society of Fellows

Daniel Shiffman, New York University, Tisch School of the Arts, ITP/IMA

Lynda Osborne, M.S., Ph.D. Candidate, Educational Psychologist, Human Factors Engineer, Founder of the Consumer Tech Innovation Festival (CTIF) and the Conference on AI Prospects, Perils & Commerce