AI and related technologies are increasingly used to create new or newly intense power relations. New, and newly intensified, power relations face presumptive objections grounded in the values of relational equality and collective self-determination. If some folks are unilaterally shaping the shared terms of our social existence, then they have power over us, and stand in hierarchical social relations with us; and if they are shaping our social world, then we, collectively are not. To overcome these objections, we need to know how algorithmic power can be used with substantive justification. But substantive justification alone is not enough. We also need to fathom how to use algorithmic power in a procedurally legitimate way. And we need to ensure that only those with authority to exercise this new power do so.

In this project, which encompasses a book project by PI Lazar, as well as PhD projects by Jake Stone and Emily Leijer, and Honours projects, we explore these questions of substantive justification, procedural legitimacy, and proper authority.

Published and in-progress work lays theoretical foundations by exploring the ways in which AI is used to exercise power, and identifying and vindicating the different justificatory standards such power must meet.

We are engaging the substantive justification of AI power by considering the justification of Automated Influence and the nature of predictive justice, with plans to explore the special obligations of equal concern and respect that flow from exercising governing power, as well as the political philosophy of attention, and the value of digital autonomy.

We have projects focusing on the political value of explanations, and the legitimacy of predictive power, as well as the role of democratic authorisation in the exercise of algorithmic power—exploring the nature of the public/private distinction, and the relative merits of digital constitutionalism vs democratic authorisation.

Many systems that use AI are able to effect significant changes in the world without intervening action by a human decision-maker. What kinds of changes should we want them to make? What would a theory of ethics for AI systems look like? These questions take on new urgency as a new age of Generative Agents—AI Agents built around Large Language Models and Multimodal Foundation Models—is clearly on the horizon.

1. Before asking what we should design machines to do in particular situations, we have to determine where that ‘should’ should come from. Where should we derive these norms, given widespread moral uncertainty and disagreement? Are there other resources here besides just appealing to political solutions?

2. Should we expect norms constraining AI systems to replicate those constraining humans in otherwise-identical choice situations, or are they likely to confront different reasons? What are the implications if machine morality differs from human morality?

3. Is it possible to reliably design these norms into actual autonomous systems? How could we represent moral knowledge in ways that can be reliably acted on by machines? If the capacities of these systems prevent them acting on or understanding nuanced moral concepts, what heuristics and failsafes should we use instead?

4. When does thinking through ethics for AI require us to revisit fundamental questions in normative ethics?

On 1, we have ongoing work in moral and political epistemology drawing on debates over moral uncertainty and moral disagreement, as well as theories of political liberalism.

On 2, we argue that norms constraining AI Agents will often differ from those that would apply to human agents faced with otherwise similar choice situations, due to factors like the different kinds of information available to us, the ability of machines (but not people) to resolutely stick to a plan, the fact that we (but not AI) have a life of our own to lead, and that we (but not AI) can express attitudes of respect or disrespect through our actions. If this is right, then some attempts to answer 1 by using machine learning to infer ethical norms from human behaviour will necessarily fail—they will identify human norms, but be blind to those suited for machines.

On 3, our answers range from more optimistic to more sceptical; though on the sceptical side we argue that the impossibility of (presently) designing an automated moral agent does not imply the impossibility of designing AI systems that have better impacts on the world.

And on 4, we have ongoing projects in normative ethics with special relevance for AI, including the ethics of risk, and foundational questions in deontological moral theory. We also believe that thinking about AI sheds light on the ways in which our standard theories of normative ethics are tailored for the very specific kinds of agents that we are. Once we postulate a different kind of agent, our theories need to adapt also.

As humans, our skills define us. No skill is more human than the exercise of moral judgment. We are already using Artificial Intelligence (AI) to automate morally-loaded decisions. In other domains of human activity, automating a task diminishes our skill at that task. Will 'moral automation' diminish our moral skill? If so, how can we mitigate that risk, and adapt AI to enable moral 'upskilling'? MINT’s contribution to this project—led by PI Seth Lazar with co-director Claire Benn, is to pursue the philosophical dimensions of this question.

Our first task has been to identify areas in which AI interacts with moral skill, in ways that could potentially induce de-skilling. We have focused on the role of AI in New (Connective) Media, and in turn on the implications of New Media for our development of moral skills.

Let’s call a moral skill a cognitive or behavioural capacity that, when exercised well, conduces to morally good behaviour, and which requires skill to exercise well.

AI's biggest impact on these capacities may come through the way in which it structures connective media and digital platforms more generally. In other words, our online lives are shaping our moral skills in novel directions, some of them detrimental.

We are focusing in particular on: the judicious allocation of appropriate attention; role-taking; considerate communication; thoughtful deliberation.

Outside of New Media, we are also exploring ways in which automating decision-making might undermine some other notable moral skills, such as the art of judgment applied in the exercise of discretion.

We cannot ensure humanity’s flourishing in an age of transformative AI through technical means alone. We have seen this many times before. Privacy by design is insufficient—it licences more collective harm by concealing direct individual harm. Fairness by design isn’t enough, often having unfair impacts in practice. Safety by design is the same. Technical AI safety must be complemented with a richer understanding of the sociotechnical systems of which AI is a part. In particular, this means attending to at least four things which existing AI safety research covers inadequately.

(1) Power. Existing AI systems and their immediate successors will concentrate power in very few hands, enabling the unaccountable pursuit of dangerous technological advances. AI will shape the future of power, and power relations will shape the future of AI; we cannot ensure AI safety without understanding each of these dimensions.

(2) Human-AI interaction. Well before any AI system poses a threat to humanity on its own, and even if we successfully resolve catastrophic risks from fully autonomous AI systems, equally capable but less autonomous systems will pose comparable threats when paired with a reckless, negligent, or malicious person. Understanding the dynamics of human-AI interaction requires understanding the whole sociotechnical system (people, systems, context), not just the technical AI component.

(3) Living with transformative AI. Human flourishing with transformative AI requires more than just avoiding the worst outcomes. This requires philosophical expertise, sound judgement, a deep understanding of individual and collective human needs and values, and of the dynamics of human society that will persist even with transformative technologies.

(4) Self-criticism. While there are many vibrant debates within the field of AI safety, there is a dearth of metascientific self-critique, identifying limitations and pathologies, misguided incentives, and power relations. Like any other science, technical AI safety will be strengthened through its own critical examination, in context.

Sociotechnical research on AI must focus on concrete AI systems and present or anticipated impacts. But it must also be sufficiently adaptable to respond rapidly to significant changes in the capabilities of AI systems, as well as new architectures and new deployment avenues. Achieving both of these prerequisites has proved a challenge. 

In particular, existing sociotechnical work on AI is not sufficiently adaptable. Many sociotechnical AI researchers have failed to seriously engage with technical AI safety, or to fully investigate and recognise the upper bound of the capabilities of LLMs, or to acknowledge the novelty and magnitude of the risks that these and more-capable AI systems will pose. In addition, many researchers have tolerated or supported polarising approaches to responsible AI, sorting into ideological factions with associated shibboleths. 

MINT will advance the positive agenda of sociotechnical AI safety by attracting AI safety researchers who recognise the need for sociotechnical insight to make real progress, and sociotechnical researchers who recognise the need to think squarely about more capable, even transformative AI systems. We will achieve this in three ways.

(1) By leading from the front, conducting research that exemplifies what sociotechnical AI safety should be.

(2) By convening researchers working on sociotechnical AI safety from around the world. We are holding standalone workshops in Australia and globally, and will co-convene workshops alongside the major AI and AI ethics conferences, to boost this integrated field.  

(3) By training new researchers, we can build the subfield of sociotechnical AI safety from the ground up. This will include training sociotechnical AI researchers on the new research questions raised by sociotechnical AI safety.