Source: The Pollyannish Assumption — Stratechery, by Ben Thompson
Moderating user-generated content is hard: it is easier, though, with a realistic understanding that the Internet reflects humanity — it is capable of both good and evil.
One of the seminal Stratechery posts is called Friction, and while I’ve linked it many times this line is particularly apt:
Friction makes everything harder, both the good we can do, but also the unimaginably terrible. In our zeal to reduce friction and our eagerness to celebrate the good, we ought not lose sight of the potential bad.
This is exactly the root of the problem: I don’t believe these platforms so much drive this abhorrent content as they make it easier than ever before for humans to express themselves, and the reality of what we are is both more amazing and more awful than most anyone ever appreciated.
The point of user reports is to leverage the scale of the Internet to police its own unfathomable scale … That approach, though, clearly isn’t enough: it is rooted in the pollyannish view of the Internet I described above — the idea that everything is mostly good but for some bad apples. A more realistic view — that humanity is capable of both great beauty and tremendous evil, and that the Internet makes it easier to express both — demands a more proactive approach. … alas, being proactive is a sure recipe for false positives.
focus on being neutral … actively seek out and remove content that is widely considered objectionable, … take a strict hands-off policy to everything that isn’t … far more transparency than currently exists … make explicitly clear what sort of content they are actively policing, and what they are not