Anormative Black Boxes: Artificial Intelligence and Health Policy

Ismael Al-Amoudi and John S. Latsis (2019)

In: Ismael Al-Amoudi & Emmanuel Lazega (eds.), Post-Human Institutions and Organizations Confronting the Matrix, London, Routledge, pp. 119-142

How will the introduction of AI affect our communities’ capacity to discuss, challenge and decide on the norms governing health policy? We answer this question by first providing a descriptive overview of the basic processes through which AI operates and of how the introduction of AI has started to disrupt the practice of health policy. Doing so allows us to identify and analyse the novel normative problems arising from the development and implementation of AI in healthcare. Our analysis indicates that the way neural networks are trained obfuscates some normative choices, while the way AI qualifications are used may downplay or inhibit the practical wisdom of medical practitioners, patients and policy-makers. While AIs as we currently know them are not capable of participating in normative discussions as discussants, we can expect their outputs to increasingly be the object of normative discussions between humans. In the last section, we speculate on the future. To do so, we reflect on how current developments could lead, through gradual social morphogenesis, to an undesirable outcome; but we also identify strategic themes of struggle for those wishing to shape a different future that leaves room for genuine normative discussion in health policy.

Excerpt: “Organisations and conventions.
As can be seen throughout this chapter, the combination of AI and big data generates unprecedented problems for the normative discussions surrounding health policy. Unless citizens and governments engage in appropriate collective action, we can reasonably expect to witness over the next decade dystopian developments akin to the ones we have delineated previously. But current times are also a pivotal period during which citizens and governments can still attempt to avoid, or at least mitigate, the problems that are otherwise likely to enter the scene. On a more general level, politicians, regulators, NGOs must acquire a better understanding of the AI strategies of the dominant internet companies and the connections between them. As the main developers of algorithms and the owners of the largest and most valuable databases, companies such as Google, Apple or Amazon are in a position that allows them to shape policy in numerous areas. Yet regulation has failed to track their current strategies or anticipate the risks that they present for future societies. They may do so, we propose, by creating organisations and conventions that will limit and harness the use of AI in public health and other areas. Our discussion of the introduction of AI in health policy allows us to make the following suggestions.”

Link to the book