Everyone knows that we are living an increasing amount of our lives online. But who’s performing the digital reckoning, watching the profiles we create and judging us on that basis? Hassan Masum and Mark Tovey have been thinking about that unsettling question – they are the authors of The Reputation Society: How Online Opinions Are Reshaping the Offline World. They sent this beautiful reflection on the algorithms and other “digital angels” who now watch over us.
For millennia, people have believed that their good and bad deeds were being watched by angels. These angels were thought to add up a person’s virtues and vices to determine their fate in the afterlife.
Today, the angels are digital. These increasingly sophisticated systems live in the servers of search engines and security agencies, data brokers and decentralized bazaars. They are seeing more of our deeds every year—and the judgments they assign us are determining ever more of our fate in this life.
But are these digital angels judging us fairly?
They are certainly observing us carefully. They see our browsing and buying habits, our photos and videos, and our interactions with family, friends, and colleagues. Increasingly, they watch us via Internet-connected things (is that smart TV watching what you watch?).
All this observing builds profiles of us. Being profiled can bring us interesting stories, ads, and contacts. It can hold us accountable if we do wrong.
But it can also hurt us. When rating our creditworthiness or threat level, digital angels may pass or fail us behind the scenes. They may combine data from diverse parts of our lives to deny us insurance or block us from flying—and we may have little recourse.
This invisible discrimination may seem objective since it is algorithmically derived from observations. However, algorithms encode the biases with which they are programmed, and observations can be mistaken or misinterpreted.
After all, the digital angels are autistic. They watch us intimately, but without comprehension. They tote up virtues and vices according to a script that can lack context and understanding.
These scripts will guide tomorrow’s digital angels as they whisper their judgments to the robot servants and drivers who will watch us… and prioritize us. How will someone else’s robot decide how to treat us? Will it judge us by our dress, our ethnic background, our pocketbook, our virtual dossier? Will it decide it can best serve its master by manipulating us with slick sales tactics, or even harming us?
Our reputation in the eyes of machines can profoundly help or hurt us. Our desires and even thoughts can be guessed by them, and used to judge us. We might hesitate to give the digital angels such power over us even if they had the wisdom of Solomon.
But they don’t. They make mistakes. They don’t serve an all-knowing deity—they serve us, with all our flaws and conflicting interests.
Under the command of selfish masters, the digital angels become digital minions. In the hands of repressive states, the digital angels could become digital devils—watching us for suspect behavior, reporting us to rulers, and even disciplining us as the Internet of things moves into the real world.
So, what can you do?
Seek the same accountability from digital angels as you would from public servants. The digital angels should be trustworthy, and they should follow basic principles of justice like oversight and due process.
Code with authority over us should be subject to checks and balances, like being inspected by people we trust. Public servants are constrained by supervision, training, and empathy. We need equivalent constraints for code which acts as bureaucrat, police, or spy.
Since the Magna Carta 800 years ago, we have built up safeguards like habeas corpus to protect us all from victimization. Similar digital due process must be implemented by our leaders and programmers to achieve algorithmic accountability.
Effective digital angels can help us in commerce, civic affairs, and crime-fighting. Yet digital angels should recognize virtue and discourage vice fairly—even mercifully. They should be ruled by our collective ideals, and not by selfish or short-sighted interests.
A decade from now, the digital angels must see us primarily as friends to be advised, rather than pawns to be exploited or sinners to be restrained. Only then will the advantages of having digital angels on our shoulders outweigh the risks.