You’ve Been Hacked – The Psychology of Disinformation and How to Protect Yourself
By Valerie Tarico ~
Most people genuinely care about truth. I don’t mean that we tell the truth all of the time—though most of us mostly do—but that we very much want to know what is real. Reality can knock you flat if you don’t see it coming.
Misreading the natural world or social cues was often lethal for our hunter-gatherer would-be ancestors, and we ourselves are descended from the ones who got it right. Truth-seeking, in other words, was written into our genetic code long before it was written into our moral and legal codes. Why, then, is it so easy for social media flurries, conspiracy theories, religions, viral ideologies, or political disinformation campaigns to get us believing utter bullshit?
A volunteer at a “Crisis Pregnancy Center” tells a young pregnant woman that abortion causes cancer.
A cat lover insists that neutered cat colonies work to eliminate feral cats.
A Youth for Christ leader tells teens that atheists are unhappy.
A conservative Facebook friend says that migrants at the border are mostly gang members and criminals.
A liberal Facebook friend says they’re mostly fleeing near-certain violence or death.
Each of these claims is both falsifiable and false, and when we see other people embrace and spread ideas or “facts” that seem obviously false to us, we often assume they either don’t care about truth or are flat-out liars. But that is lazy thinking, and more than a little arrogant. Yes, people on the other side are less than fully truthful. So are you. And yes, some people are habitual liars or even sociopaths. And yes, people who aren’t sociopathic sometimes decide for one reason or another that the end justifies the means. But when people spread falsehoods—even stuff that seems like transparent hooey from the outside, they mostly believe what they say. A lie can travel half way around the world while the truth is putting on its shoes, so goes the oft repeated saying. But lies travel only when people believe them. That is why most propaganda uses partial or decontextualized truths to create misconceptions and distort priorities.
Truth-Seeking Opens Vulnerabilities.
Those who professionally traffic in partial truths and disinformation—like some ideologues, political strategists, marketing experts or corporate- and state-level information warfare specialists— analyze and exploit our desire to know what is real. One common and highly effective way they play us is by convincing us that only we and a small group of enlightened folks like us have the inside scoop.
Human beings are social information specialists, and we gain standing with each other by sharing insider knowledge. Gossip is a great example, and provides the basis for social marketing. OMG, have you heard??! But so are conspiracy theories, whose believers wind their way through rabbit warrens of logic to find secret knowledge that has been deliberately buried—which they then publish in a-mazing detail. Religious ideologies teach that they alone hold the answer to life’s mysteries. Some spawn secret societies or hidden rituals, while external symbols like a silver cross necklace, a yarmulke, or a hijab signal the embrace of exclusive Truths, and thus raise status in the eyes of fellow believers.
Social media provide fertile ground in which insider knowledge takes root, and—as we all know—falsehoods (especially partial truths masquerading as whole truths) spread like invasive weeds. Some of these falsehoods are deliberately seeded by ideologues, propagandists, disinformation specialists; others spring up naturally from the constant shift and flow and recombination of information in the human ideosphere. Once seeded, all viral bullshit exploits weaknesses in how the human mind determines what’s real. These include:
Tribal Boundaries—Social networks largely determine what information flows past us.
Identity Filters—Who we are, meaning our genetics and lived experience, our interests and values, our cognitive strengths and weakness, and our emotional makeup all play roles in determining what gets our attention.
Delightful Surprises—We are drawn like proverbial moths to the unexpected and counter-intuitive, which trigger the thrill of discovery and secret knowledge.
Thinking Fast—In the face of competing priorities and overwhelming quantities of information coming at us, we fall back on intuition, gut feel, and other cognitive shortcuts, only rarely applying higher order reasoning to carefully scrutinize a set of propositions or evidence.
Groupthink—Whenever possible, again for the sake of efficiency in information processing, we let other members of our tribe do the analysis for us and consider the task done.
Authority Hierarchies—We bypass thinking altogether by looking to trusted authorities which can be individuals, institutions, or sacred texts—not necessarily relevant experts—and accept as fact what they say.
The Consensus Shortcut—We treat the preponderance of opinion as if it were a preponderance of evidence.
Saturation Seduction—We succumb to messages that are repeated, ubiquitous and consistent whether they are backed by evidence or not.
Motivated Reasoning—Even when we do take the time to construct an analysis of our own, we often start with what we want to be true, what fits our worldview or what benefits us directly, and then reason backwards from there.
Confirmation Bias—Once true believers or disinformers get us to believe something, then we ourselves often take over their work, ignoring or rejecting information that might show us wrong.
Pleasures of Superiority—Certain emotions including the feelings of righteousness, moral disgust, or jeering laughter close us off to people and ideas who might challenge us.
Panic—When flooded with emotion—especially fear—higher-order reasoning vanishes and we simply stampede.
Bolster Your Defenses
It is impossible to get away from these dynamics, but there are things we can do to minimize their power. What, you ask? As odd as it sounds, religion may offer some ideas.
When people spread falsehoods—even stuff that seems like transparent hooey from the outside, they mostly believe what they say.Viral religions that make dubious truth claims often include a set of rules, scripts, advisories, structures and dogmas aimed at preventing defection. In other words, they outline the exact opposite of what one should do if one is actually interested in figuring out what is real.
Here are some of the advisories from Bible-believing Christianity, the religion of my youth: Have faith that what the Bible says is true and you will find evidence that fits. Believe and be saved. One Way. Lean not unto your own understanding. Be wary of secular and scientific expertise. Recite statements of faith. The fool has said in is heart there is no God. Trust and obey, for there’s no other way. Outsiders are conduits for evil, and they will try to tempt you. Avoid spiritual pornography (writings that take a skeptical point of view). Don’t be “unequally yoked” with unbelievers. Be in the world, not of it. Preach the gospel to every creature. Doubt is weakness; pray it away. Feel the love of Jesus and the power of the Holy Spirit flowing through you. Surround yourself with fellow Christians. Come to church every Sunday morning (and Sunday evening, and Wednesday evening, and maybe Tuesday night Bible Study).
Worship services are structured to elicit emotion and a sense of unity that causes people to suspend disbelief, and repetition cements the deal. In past generations, physical architecture evolved to help with this; the vast spaces inside of cathedrals disoriented the cerebellum, triggering feelings of transcendence. But now most denominations simply rely on the cadence of familiar mutually-reinforcing stories and precepts delivered by familiar authorities in the company of like-minded people who envelop the believer in a warm, supportive and morally-superior community of people who have got it right.
To the degree that this fits your Facebook friend list or Twitter feed or news and entertainment choices or activist network, my former religion, as a highly successful mind virus, points inadvertently toward a cure:
Get yourself out of the sanctuary so that you can shake off the sweet, soothing lull of the choir and escape saturation seduction. Find silence.
Trust doubt. Identify real experts and expertise, but question orthodoxies—especially those with strong emotional appeal. Be wary of salvific Truths preached by admired authorities. Deploy your capacity for critical thinking—not just when you want to denigrate evil outsiders and their obvious falsehoods, but rather in those times and places where you congregate with people who share your values and views. Voice misgivings. Tolerate uncertainty.
Lastly, reject the oh-so-satisfying idea that all heretics must be evil. Seek the company of people who don’t think like you. Listen for kernels of truth and wisdom spoken by enemies of your tribe. They may not be able to break out of their disinformation silos, but you, at least, can break out of yours.
Valerie Tarico is a psychologist and writer in Seattle, Washington. She is the author ofTrusting Doubt: A Former Evangelical Looks at Old Beliefs in a New Light and Deas and Other Imaginings, and the founder of www.WisdomCommons.org. Her articles about religion, reproductive health, and the role of women in society have been featured at sites including The Huffington Post, Salon, The Independent, Free Inquiry, The Humanist, AlterNet, Raw Story, Grist, Jezebel, and the Institute for Ethics and Emerging Technologies. Subscribe at ValerieTarico.com.
Once seeded, viral misinformation exploits weaknesses in how the human mind determines what’s real.
Misreading the natural world or social cues was often lethal for our hunter-gatherer would-be ancestors, and we ourselves are descended from the ones who got it right. Truth-seeking, in other words, was written into our genetic code long before it was written into our moral and legal codes. Why, then, is it so easy for social media flurries, conspiracy theories, religions, viral ideologies, or political disinformation campaigns to get us believing utter bullshit?
A volunteer at a “Crisis Pregnancy Center” tells a young pregnant woman that abortion causes cancer.
A cat lover insists that neutered cat colonies work to eliminate feral cats.
A Youth for Christ leader tells teens that atheists are unhappy.
A conservative Facebook friend says that migrants at the border are mostly gang members and criminals.
A liberal Facebook friend says they’re mostly fleeing near-certain violence or death.
Each of these claims is both falsifiable and false, and when we see other people embrace and spread ideas or “facts” that seem obviously false to us, we often assume they either don’t care about truth or are flat-out liars. But that is lazy thinking, and more than a little arrogant. Yes, people on the other side are less than fully truthful. So are you. And yes, some people are habitual liars or even sociopaths. And yes, people who aren’t sociopathic sometimes decide for one reason or another that the end justifies the means. But when people spread falsehoods—even stuff that seems like transparent hooey from the outside, they mostly believe what they say. A lie can travel half way around the world while the truth is putting on its shoes, so goes the oft repeated saying. But lies travel only when people believe them. That is why most propaganda uses partial or decontextualized truths to create misconceptions and distort priorities.
Truth-Seeking Opens Vulnerabilities.
Those who professionally traffic in partial truths and disinformation—like some ideologues, political strategists, marketing experts or corporate- and state-level information warfare specialists— analyze and exploit our desire to know what is real. One common and highly effective way they play us is by convincing us that only we and a small group of enlightened folks like us have the inside scoop.
Human beings are social information specialists, and we gain standing with each other by sharing insider knowledge. Gossip is a great example, and provides the basis for social marketing. OMG, have you heard??! But so are conspiracy theories, whose believers wind their way through rabbit warrens of logic to find secret knowledge that has been deliberately buried—which they then publish in a-mazing detail. Religious ideologies teach that they alone hold the answer to life’s mysteries. Some spawn secret societies or hidden rituals, while external symbols like a silver cross necklace, a yarmulke, or a hijab signal the embrace of exclusive Truths, and thus raise status in the eyes of fellow believers.
Social media provide fertile ground in which insider knowledge takes root, and—as we all know—falsehoods (especially partial truths masquerading as whole truths) spread like invasive weeds. Some of these falsehoods are deliberately seeded by ideologues, propagandists, disinformation specialists; others spring up naturally from the constant shift and flow and recombination of information in the human ideosphere. Once seeded, all viral bullshit exploits weaknesses in how the human mind determines what’s real. These include:
Tribal Boundaries—Social networks largely determine what information flows past us.
Identity Filters—Who we are, meaning our genetics and lived experience, our interests and values, our cognitive strengths and weakness, and our emotional makeup all play roles in determining what gets our attention.
Delightful Surprises—We are drawn like proverbial moths to the unexpected and counter-intuitive, which trigger the thrill of discovery and secret knowledge.
Thinking Fast—In the face of competing priorities and overwhelming quantities of information coming at us, we fall back on intuition, gut feel, and other cognitive shortcuts, only rarely applying higher order reasoning to carefully scrutinize a set of propositions or evidence.
Groupthink—Whenever possible, again for the sake of efficiency in information processing, we let other members of our tribe do the analysis for us and consider the task done.
Authority Hierarchies—We bypass thinking altogether by looking to trusted authorities which can be individuals, institutions, or sacred texts—not necessarily relevant experts—and accept as fact what they say.
The Consensus Shortcut—We treat the preponderance of opinion as if it were a preponderance of evidence.
Saturation Seduction—We succumb to messages that are repeated, ubiquitous and consistent whether they are backed by evidence or not.
Motivated Reasoning—Even when we do take the time to construct an analysis of our own, we often start with what we want to be true, what fits our worldview or what benefits us directly, and then reason backwards from there.
Confirmation Bias—Once true believers or disinformers get us to believe something, then we ourselves often take over their work, ignoring or rejecting information that might show us wrong.
Pleasures of Superiority—Certain emotions including the feelings of righteousness, moral disgust, or jeering laughter close us off to people and ideas who might challenge us.
Panic—When flooded with emotion—especially fear—higher-order reasoning vanishes and we simply stampede.
Bolster Your Defenses
It is impossible to get away from these dynamics, but there are things we can do to minimize their power. What, you ask? As odd as it sounds, religion may offer some ideas.
When people spread falsehoods—even stuff that seems like transparent hooey from the outside, they mostly believe what they say.Viral religions that make dubious truth claims often include a set of rules, scripts, advisories, structures and dogmas aimed at preventing defection. In other words, they outline the exact opposite of what one should do if one is actually interested in figuring out what is real.
Here are some of the advisories from Bible-believing Christianity, the religion of my youth: Have faith that what the Bible says is true and you will find evidence that fits. Believe and be saved. One Way. Lean not unto your own understanding. Be wary of secular and scientific expertise. Recite statements of faith. The fool has said in is heart there is no God. Trust and obey, for there’s no other way. Outsiders are conduits for evil, and they will try to tempt you. Avoid spiritual pornography (writings that take a skeptical point of view). Don’t be “unequally yoked” with unbelievers. Be in the world, not of it. Preach the gospel to every creature. Doubt is weakness; pray it away. Feel the love of Jesus and the power of the Holy Spirit flowing through you. Surround yourself with fellow Christians. Come to church every Sunday morning (and Sunday evening, and Wednesday evening, and maybe Tuesday night Bible Study).
Worship services are structured to elicit emotion and a sense of unity that causes people to suspend disbelief, and repetition cements the deal. In past generations, physical architecture evolved to help with this; the vast spaces inside of cathedrals disoriented the cerebellum, triggering feelings of transcendence. But now most denominations simply rely on the cadence of familiar mutually-reinforcing stories and precepts delivered by familiar authorities in the company of like-minded people who envelop the believer in a warm, supportive and morally-superior community of people who have got it right.
To the degree that this fits your Facebook friend list or Twitter feed or news and entertainment choices or activist network, my former religion, as a highly successful mind virus, points inadvertently toward a cure:
Get yourself out of the sanctuary so that you can shake off the sweet, soothing lull of the choir and escape saturation seduction. Find silence.
Trust doubt. Identify real experts and expertise, but question orthodoxies—especially those with strong emotional appeal. Be wary of salvific Truths preached by admired authorities. Deploy your capacity for critical thinking—not just when you want to denigrate evil outsiders and their obvious falsehoods, but rather in those times and places where you congregate with people who share your values and views. Voice misgivings. Tolerate uncertainty.
Lastly, reject the oh-so-satisfying idea that all heretics must be evil. Seek the company of people who don’t think like you. Listen for kernels of truth and wisdom spoken by enemies of your tribe. They may not be able to break out of their disinformation silos, but you, at least, can break out of yours.
Valerie Tarico is a psychologist and writer in Seattle, Washington. She is the author ofTrusting Doubt: A Former Evangelical Looks at Old Beliefs in a New Light and Deas and Other Imaginings, and the founder of www.WisdomCommons.org. Her articles about religion, reproductive health, and the role of women in society have been featured at sites including The Huffington Post, Salon, The Independent, Free Inquiry, The Humanist, AlterNet, Raw Story, Grist, Jezebel, and the Institute for Ethics and Emerging Technologies. Subscribe at ValerieTarico.com.