Algorithmic injustices

Criminal justice in the data age ... The lurking dangers when algorithms are used to dispense justice ... Predicting the pattern of potential offenders ... Anthony Kanaan interviews Dr Tatiana Dancy, author of Artificial Justice 

Dancy: examining the algorithms behind digital risk tools

Open any paper and you will be hard-pressed not to find another write up on the revolution that is artificial intelligence. Instantly, rhetorical questions along the lines of, “is your career at risk?” or “will your job still be here in a decade?” come to mind. 

While the binary and superficial style of artificial intelligences like Chat-GPT would seem unsuited to the uniquely human demands, and mysteries, of the legal profession, its use has already pervaded justice systems around the world.

UniMelb Associate Professor Dr Tatiana Dancy's new book Artificial Justice unpicks how predictive algorithms are used in the criminal justice system. 

Dr Dancy, who completed a DPhil in law at Oxford and has previously taught at the LSE, explores how the use of these algorithms across the world - including in NSW - has given prejudice the force of policy.

I spoke to Dr Dancy shortly after the launch of her book to learn more about her research.

Tell me about the origins of this book. What brought your attention to this topic?

The use of algorithms to make decisions is a topic of public interest. What sparked my interest was this focus in the literature on bias and discrimination, both of which are really important aspects of the justice of algorithmic decision making – but that's not all that's captured within "justice."

I wanted to think about what other reasons we might have to be concerned about the use of algorithms that make decisions about us. Are there other things that we might have to think about when we use these algorithms to make important sentencing decisions about people?

What challenges did you encounter in your research?

There were a couple of challenges along the way; one was, what do you do when an area of thinking like "justice" is so dominated by certain frameworks of thinking? 

Instead of starting at the position of, "what is the definition of justice, and how do we figure out what fits within that definition," it was really about, "what is at stake for individuals?" I don't think this is specific to algorithms, this is a question that arises whenever you look at topics within moral theory or legal philosophy, and apply them to real world situations.

Also, there is obviously lots of "pop" writing about algorithms, but what we don't have is a wealth of research to build on when you're writing about them. So that means that you're really thinking for yourself about what matters. 

Do you think that there is a space for these technologies to be used positively?

I don't think that's it's never appropriate to use algorithms. For instance, we use statistical algorithms incredibly effectively in the healthcare system to save lives, to really make a difference to people.

While I don't think it's necessarily inappropriate to use algorithms in the context of criminal justice, we need to be really clear about the factors that are being taken into account, and then how those predictions are turned into decisions – and that is not happening at the moment.

For instance, the Suspect Target Management Plan (STMP) was a police algorithm used in NSW to target certain individuals for the application of higher intensity policing measures.

There was no process of consultation before those powers were used, and there was no publication of the factors upon which those predictions were based. That ended up having ramifications for children as young as ten, and really disparate ramifications for Indigenous peoples who made up about 50% of the program's targets. yet 2-3% of the national population.

There was an investigation and then it was decided that it wasn't appropriate to be using STMP – but that happened after the fact. The STMP program has been withdrawn, but it would be preferable to me if we did this before: we really need to be engaging in these processes before the decision to adopt these types of policies.

You also look to the use of algorithms in other jurisdictions. What broad lessons are apparent in your research into those technologies used abroad? 

One of the broader claims that I am making is that we need to dig a little deeper than just questions on bias and discrimination, because even if these algorithms are applied equally, there still may be problems.

For instance, in OASys [a UK sentencing program], we look at factors about a person's background in a lot of respects, for example whether they are a victim of domestic violence. In COMPAS, used in the US, that includes whether a person's parents separated when they were young. 

I got asked this question in a radio interview, that "it doesn't matter does it, because we've got everybody appearing before the Court and being sentenced in the same way in NSW: we're not doing what others are doing abroad." While that's true to some extent, once somebody enters the corrective system, the way in which they are treated and the kind of question about when they are released is answered by looking at an algorithm in NSW.

What do you hope is going to be the key takeaway of your book for legal professionals.

That for me is where I hope my research will have the biggest impact. Firstly, let's look beyond equality or inequality while considering whether these algorithms are justified, and secondly, that those questions of transparency and accountability are key.

Before we decide as government actors, or as anyone in public roles to adopt an algorithm in a decision-making process, we need to have transparent consultation, we need to have accountability after the fact, and we need to have clear publication of the facts that influence the decisions. 

Artificial Justice is published as part of the Oxford University Press's Technology Law and Policy Series.