Source: The Digital Maginot Line | ribbonfarm, by Renee DiResta
The Information World War has already been going on for several years. We called the opening skirmishes “media manipulation” and “hoaxes”, assuming that we were dealing with ideological pranksters doing it for the lulz (and that lulz were harmless).
In reality, the combatants are professional, state-employed cyberwarriors and seasoned amateur guerrillas pursuing very well-defined objectives with military precision and specialized tools.
…
Combatants evolve with remarkable speed, because digital munitions are very close to free. In fact, because of the digital advertising ecosystem, information warfare may even turn a profit. There’s very little incentive not to try everything: this is a revolution that is being A/B tested. The most visible battlespaces are our online forums — Twitter, Facebook, and YouTube — but the activity is increasingly spreading to old-school direct action on the streets, in traditional media outlets, and behind closed doors, as state-sponsored trolls recruit and manipulate activists, launder narratives, and instigate protests.
The combatants want to normalize the idea that the platforms shouldn’t be allowed to set rules of engagement because in the short term, it’s only the platforms that can.
Meanwhile, regular civilian users view these platforms as ordinary extensions of physical public and social spaces – the new public square, with a bit of a pollution problem. Academic leaders and technologists wonder if faster fact checking might solve the problem, and attempt to engage in good-faith debate about whether moderation is censorship. There’s a fundamental disconnect here, driven by underestimation and misinterpretation. The combatants view this as a Hobbesian information war of all against all and a tactical arms race; the other side sees it as a peacetime civil governance problem.
ultimately the information war is about territory — just not the geographic kind. In a warm information war, the human mind is the territory. If you aren’t a combatant, you are the territory. And once a combatant wins over a sufficient number of minds, they have the power to influence culture and society, policy and politics.
The 2014-2016 influence operation playbook went something like this: a group of digital combatants decided to push a specific narrative, something that fit a long-term narrative but also had a short-term news hook. They created content … then activated collections of bots and sockpuppets
…
Since running spammy automated accounts is no longer a good use of resources, sophisticated operators have moved on to new tactics. … Combatants are now focusing on infiltration rather than automation: leveraging real, ideologically-aligned people to inadvertently spread real, ideologically-aligned content instead.
The entities best suited to mitigate the threat of any given emerging tactic will always be the platforms themselves, because they can move fast when so inclined or incentivized. The problem is that many of the mitigation strategies advanced by the platforms are the information integrity version of greenwashing; they’re a kind of digital security theater
…
The key problem is this: platforms aren’t incentivized to engage in the profoundly complex arms race against the worst actors when they can simply point to transparency reports showing that they caught a fair number of the mediocre actors.Platforms cannot continue to operate as if all users are basically the same; they have to develop constant awareness of how various combatant types will abuse the new features that they roll out, and build detection of combatant tactics into the technology they’re creating to police the problem. … They must recognize that they are battlespaces, and as such, must build the policing capabilities that limit the actions of malicious combatants while protecting the actual rights of their real civilian users.
AI-generated audio and video deepfakes will erode trust in what we see with our own eyes, leaving us vulnerable both to faked content and to the discrediting of the actual truth by insinuation. Authenticity debates will commandeer media cycles, pushing us into an infinite loop of perpetually investigating basic facts. Chronic skepticism and the cognitive DDoS will increase polarization, leading to a consolidation of trust in distinct sets of right and left-wing authority figures – thought oligarchs speaking to entirely separate groups.
An admirable commitment to the principle of free speech in peace time turns into a sucker position against adversarial psy-ops in wartime. We need an understanding of free speech that is hardened against the environment of a continuous warm war on a broken information ecosystem. We need to defend the fundamental value from itself becoming a prop in a malign narrative.
We have to move away from treating this as a problem of giving people better facts, or stopping some Russian bots, and move towards thinking about it as an ongoing battle for the integrity of our information infrastructure
More: Common-Knowledge Attacks on Democracy, by Henry John Farrell and Bruce Schneier