Welcome, friends, to the fourth issue of In Pursuit of Quality. I'm doing my best to keep these brief and interesting/entertaining. Thank you for reading, sharing, and commenting.
This week picks up on a theme that's been gnawing at me for a long while: The Truth.
Don't worry! This is not a vilification of fact-adverse politicians (although it could be). Nor does it focus on how it has become increasingly hard to rely on social feeds to give us honest updates about real-time events (this piece does a good job of that). It's about our human relationship with truth. And how we use our understanding of truth to make sense of our work and lives.
Statistical Truth.
This has been a journey for me. I was raised in a tiny household where — through no fault of her own — my sole caretaking parent had a tenuous relationship with facts. Before Google, I had to verify her every word through childish archival research and old-fashioned gossip. Was my grandfather really a school teacher? No, he drove taxis. Did she really want to have a kid? The jury's still out.
So maybe it's no surprise that I became obsessed with understanding how to find 'truth.’ I invested in learning and was the first in my family to go to college. Then I entered a Ph.D. program in cognitive psychology, focusing on how humans (and machines) process language. That's where I learned the discipline of scientific objectivity. People who participated in my experiments became subjects. To get published in psychology journals, I was in pursuit of statistically significant results.
Generating statistically significant results is the one way to gain fame as a social scientist. In psychology, statistical significance shows whether a result in a study is likely to be true or if it might have occurred by chance. This is represented by a p-value indicating the percent chance of getting a particular result due to noise or randomness. The most commonly used threshold is a p-value of 0.05 or less. If your p-value is bigger, you're stuck with a null result — and null results don't get published.
What could possibly go wrong here? Surely, we can rely on the academic community to uphold the integrity of experimental findings. Right? Not so fast. Turns out even researchers might be willing to bend hard facts into wishful fictions. I think this is driven by a pursuit of quantity — more publications, more notoriety — rather than the pursuit of quality scientific results.
Academic Heroes.
Dan Ariely was a hero of mine. I first learned about him when I was in academia myself, where I came across a draft of his paper about sexual arousal and decision-making. This is the only published academic paper that contains this gem of a sentence:
"It is important to note that all the subjects completed the sessions, and no one reported that they accidentally ejaculated during the session (subjects were instructed to press the tab key if they ejaculated, which would have ended the session)."
Partly due to those daring experimental forays, Ariely has become a bit of a superstar. He holds a top position at Duke, and he's written three best-selling books – including one about dishonesty. He even co-produced a 2015 documentary called (Dis)Honesty: The Truth About Lies. Turns out he might know more about lying than we thought.
Just a few weeks ago The New Yorker published an article about Ariely and his frequent collaborator, Francesca Gino, detailing the claim that both have fabricated data. They are foremost experts on why we lie. And now, Ariely faces multiple accusations of data fraud which has resulted in an academic suspension. This sucks for a few reasons.
First, I'm bummed that any serious scientist would engage in p-hacking, which is when researchers manipulate data in a way that increases the likelihood of getting a statistically significant result. But I get it as a part of a thoughtless pursuit of quantity. More significant results leads to more publications which leads to more fame, more podcast appearances, more money. Yet it pollutes the academic waters by leading other researchers down errant paths, calling into question the whole scientific process. What science can we trust?
Integrity Matters.
That leads to an even bigger question. If we can't rely on researchers to give us facts, where can we find any semblance of truth? Politicians lean on rhetoric over facts. Social media is awash with deep fakes laced with vitriol. Trust across our institutions – from business and media to government and education – is buckling. This is not what quality outcomes looks like.
So that's the call to action here. We can't be in pursuit of quality – better human outcomes – until we demand truth where we can find it. We will not achieve true quality outcomes until we commit not to fall prey to the temptations of short-term gains that can lead to terrible longer-term outcomes for ourselves, for our work, and for society at large.
There's a sea of subjectivity out there. That makes things murky. But I expect honesty from the people I talk to. I expect truth from the media (but never social media). I expect our institutions to have integrity. And I count on the scientific community to be honest about reporting facts that help us all achieve better outcomes. Especially – especially – if those very researchers study the black art of deception. Is that too much to ask?
One action we can all take is to be bold about calling each other out on bullshit. This isn't about policing; it's about demanding human integrity. If something doesn't seem quite right, why not dig a little deeper? This can be scary. Yet it's that very timidity that has allowed social scientists like Ariely – and many others – to get away with publishing faulty research. This might lead to short-term quality for them. But ultimately worse outcomes for everyone, especially those who get caught.
I look forward to your thoughts.
With warmth, gratitude, and respect.
Honesty is such a lonely word
Everyone is so untrue
Honesty is hardly ever heard
And mostly what I need from you
Your insights always give me plenty to consider. Of course it is bold to ever demand honesty. Humans are inherently biased toward their own comfort, and very good at fooling themselves before uttering a word to anyone else--as you illustrated. Can we become too jaded to expect the best from one another? I hope not. On a related note, I would love to retire the word 'authenticity.' It rings phony now--especially in marketing. I can't even bring myself to say it anymore in relation to my work. I usually choose 'genuine' instead. I like this focus on integrity. If the powerful weren't so perfumed by their own privilege, we'd have an easier time sniffing out deception. I think it would be good to view this through a social justice lens as well, especially if we are seeking institutional change. Keep up these thoughtful posts!