Stay informed with free updates
Simply sign up at Social matter Myft Digest – delivered directly to your box.
Does what does it mean that the decision of a car is “right”? So far, public debate has focused mainly on the issue of prejudice and discrimination. This is understandable: most people will expect cars to be less one -sided than humans (indeed, this is often given as a rationale for their use in processes such as recruitment), so it is fair to Pay attention to evidence that they can be one -sided, too
But the word “right” has many interpretations, and “impartial” is only one of them. I found myself in the end of making a recently automated decision, which made me think what it really means to feel that you are treated rightly, and how difficult it can be keeping to those principles in an increasingly automated world.
I have a personal Gmail account I use for correspondence about a book project I am working on. I woke up one morning in November to find out I couldn’t call it anymore. A message from Google said my approach was “globally restricted” because “looks like Gmail was used to send unwanted content. Spamming is a violation of Google policies.” The note said the decision was taken from “automatic processing” and that if I thought it was a mistake, I could have appealed.
I had not sent any spam and could not imagine why the Google algorithm thought I had. This made it difficult to know what to write in the text box “Appeal”, except for a panic version of something like, “I don’t do it (whatever it is)!” And, “Please help, I really need access to my email and my files.” (To my relief, I realized later that I had not lost the entrance to my car.)
Two days later, I heard again: “After reviewing your appeal, access to your account remains limited to this service.” I was not given more information about what I had done allegedly or why the appeal was rejected, but I was told that “if you disagree with this decision, you could present another appeal.” I tried again and was refused again. I did this several times – curious, at this point, as long as this penny loop can continue. A look at Reddit suggested that other people had gone through similar things. Eventually, I gave up. (Google refused to comment on the record.)
Among the regulators, a popular answer to the question of how to make the most “fair” automated decisions is to insist that people ask a person to review them. But how effective is this medicine? For one thing, people are prone to “automation complacency” – a tendency to trust a lot of the car. In the case of the UK post scandal, for example, where the sub-stands were incorrectly charged with theft due to a wrong computer system called Horizon, a judge in 2019 concluded that people in the post office displayed ” a simple institutional obstacle or refusal to consider any possible alternatives to their horizon view. “
Ben Green, an algorithmic justice expert at the University of Michigan, says there may be practical problems in some organizations as well. “Often times, human supervisors are on a narrow schedule – they have many occasions to consider,” he told me. “Many of the cases I have seen are cases where the decision is based on some kind of statistical forecast,” he said, but “people are not very good at making those predictions, so why would they be good at evaluating them?”
After my powerless rage for my email had faded, I discovered that I had a certain amount of sympathy with Google. With so many clients, an automated system is the only practical way to detect violations of its policies. And while feeling deeply unfair I had to play my issue without knowing what caused the system, nor any explanation of the traps to avoid an appeal, I can also see as much detail offers Google for how The system worked, the easier it would be for the bad actors to get around it.
But this is the point. In increasingly automated systems, the purpose of procedural justice – for people to think that the process has been right for them – often runs counter to other purposes, such as the need for efficiency, intimacy or security. There is no easy way to make them trade disappear.
As for my email account, when I decided to write about my experience for this column, I sent E -mail to Google Press Office with details to see if I could discuss the issue. By the end of the day, my access to my email account was back. I was pleased, of course, but I don’t think many people will see it as especially right.
Sarah.oconnor@ft.com
