The AI Now 2019 Report highlights that this lack of effective measures contrasts with the widespread adoption of facial recognition technologies and the emergence of new, particularly intrusive ones, such as the recognition of people’s emotions.
Despite the abundance of proposals on ethical criteria applicable to Artificial Intelligence (AI), systems based on this technology continue to be rapidly deployed in areas of considerable social importance such as health, education, employment, criminal justice, and many others, without adequate safeguard systems or accountability structures (accountability) having been implemented at the same time.
In this context, the development of new emotion recognition systems (affect recognition) stands out in particular — a subclass of facial recognition technologies aimed at detecting aspects such as personality, emotions, mental health, and other inner states of the person, which is particularly intrusive with regard to privacy.
In addition, it must also be borne in mind that the strategic aspects arising from the use of AI are growing: the considerable environmental damage caused by these systems, or their effects on healthcare, where the increasing reliance on AI systems for decision-making can have life-or-death consequences for people. Moreover, these systems are particularly prone to security vulnerabilities.
Furthermore, AI is widening inequality, in various fields and contexts, by placing information and its control in the hands of those who already have power, further disempowering those who do not.
It is therefore not surprising that during the year now ending, social groups, researchers, policymakers, and workers have been demanding that limits be placed on the riskiest and most dangerous AI-based activities.
These are the main conclusions of the AI Now 2019 report, presented by the AI Now Institute, a center based at New York University dedicated to interdisciplinary research on the social implications of AI. The report, which takes a notably critical approach to the effects of these technologies on work and on people’s civil and social rights, analyzes the state of deployment and regulation of these technologies globally, identifies the current situation of their use, and proposes a series of recommendations to solve the problems identified.
Current trends in Artificial Intelligence
The study has also identified the following trends during the year now drawing to a close:
1. The spread of algorithmic management technologies in the workplace is increasing the power asymmetry between workers and employers. AI thereby threatens not only to disproportionately displace lower-wage workers, but also to reduce wages, job security, and other protections for those who need them most.
2. Social groups, workers, journalists, and researchers — not corporate AI ethics statements and policies — have been primarily responsible for pressuring technology companies and governments to put up barriers to the use of AI.
Companies, governments, and social and academic entities are devoting enormous efforts to drawing up ethical principles applicable to AI. But most of them say very little about implementation, accountability, or how these principles are to be measured and applied in practice. At the same time, there is a severe gap between the statement of these principles and their practice.
3. Efforts are being made to regulate AI systems (for example, in the U.S., the Commercial Facial Recognition Privacy Act of 2019, the Facial Recognition Technology Warrant Act, and the No Biometric Barriers Act of 2019, and in Europe, certain aspects of the GDPR), but these efforts are being outpaced by governments’ deployment of AI systems to surveil and control citizens.
4. AI systems continue to be a means of widening racial and gender disparities through techniques such as affect recognition, which lacks a solid scientific basis.
However, instead of acknowledging the scale and systemic nature of the problem, technology companies have responded to the growing evidence of bias and misuse of this technology by focusing mainly on narrow solutions. They have also tried to fix the technical mismatch in the data by working to “fix” the algorithms and diversify the datasets, even though these approaches have proven insufficient and raise serious concerns about privacy and the scope of the consent granted for these purposes. In particular, neither approach addresses the underlying structural inequalities. Nor do they address the growing power asymmetry between those who produce and benefit from AI and those who are subject to AI applications.
5. The increase in AI investment and development has profound repercussions in areas ranging from climate change to the rights of healthcare patients, through to the future of geopolitics and the inequalities being reinforced in the regions of the Global South.
The study highlights the danger of leaving the solutions to these issues in the hands of a small number of individuals and companies, whose purposes and worldview are often at odds with the interests of those who bear the consequences of such decisions.
Recommendations
The report includes a list of 12 recommendations for developers, regulators, and governments:
1. Regulators should ban the use of emotion recognition for making relevant decisions that affect people’s lives and access to opportunities. Until then, companies that develop or use AI should stop using it.
The report stresses that this technology rests on controversial scientific foundations, so it should not be allowed to play a role in decisions that matter to people, such as hiring, insurance pricing, patient pain assessments, or student performance at school.
2. Governments and companies should halt all use of facial recognition in sensitive social and political contexts until the risks are fully studied and adequate regulations for its use are established.
According to the report, 2019 saw a rapid expansion of facial recognition in many areas. However, there is mounting evidence that this technology has serious effects, especially on people of color and the poor. A moratorium should therefore be established on all uses of facial recognition in sensitive social and political domains –including surveillance, policing, education, and employment– where facial recognition poses risks and consequences that cannot be remedied retroactively.
Additionally, this moratorium should include the setting of transparency requirements regarding the operation of these systems, allowing researchers, policymakers, and communities to assess and understand the best possible approach to restricting and regulating facial recognition.
3. The AI industry needs to make significant structural changes to address systemic racism, misogyny, and the lack of diversity in its systems.
4. Research on AI bias must go beyond mere technical fixes to address the policies and consequences of AI use as a whole.
5. Governments must require public disclosure of the AI industry’s climate impact.
6. Workers must have the right to object to the invasive use of AI in the workplace.
7. Tech workers must have the right to know what they are building and to challenge unethical or harmful uses of their AI work.
8. States must draw up broader biometric data privacy laws, aimed at both public and private actors, such as the Illinois Biometric Information Privacy Act (BIPA),
9. Lawmakers must regulate the integration of public and private surveillance infrastructures.
10. Algorithmic impact assessments must take into account the impact of AI on climate, health, and geographic displacement.
11. Machine learning researchers must take into account the potential risks and harms that may arise from its use, and better document the origins of their models and data.
12. Lawmakers must require informed consent for the use of any personal data in health-related AI applications.
Originally published Wolters Kluwer
Carlos B Fernández
