America’s police are using social media to fight crime, a practice that raises troubling questions.
The fundamental problem with policing via social-media data is that it misrepresents what social networks actually look like on the ground. Despite what techno-evangelists might wish, not all social relationships can be described using computational logic. The problem is structural and epistemological. Like all computer programs, databases are ultimately based on binary logic. If you want shades of meaning, you have to explicitly build that capability into your system. And building nuance is far harder than it seems.
For the kid listed in a gang database, it can be unclear how to get out of it. In the world of human interaction, we accept change through behavior: the addict can redeem himself by getting clean, or the habitual interrupter can redeem himself by not interrupting. We accept behavior change. But in the database world, unless someone has permission to delete or amend a database record, no such change is possible.
This permanence does not necessarily match real-world conditions. Kids cycle in and out of street gangs the way they cycle in and out of any other social group, and many young men age out of violent behavior. Regularly purging the gang database, perhaps on a one-year or two-year cycle, would allow some measure of computational forgiveness.
Now that the Internet is thirty years old, the long-term consequences of information permanence are becoming clear. We also need to acknowledge that computer systems are not a panacea. … We need to put people before programs, and if programs don’t reflect our human values, we need to change the code. And if programmers can’t write code that is fair and just, we should consider relying on people instead of programs.
If our technological systems are entrapping innocent citizens or tampering with the presumption of innocence, should they be used?