Objectivity, Rigor… and Irony.

Back in September the Urban Institute published a ‘blog post arguing that “Equitable Research Requires Questioning the Status Quo”. The author—Lauren Farrell—argued that both “objectivity” and “rigor” were “harmful research practices” that should be rectified.

Not surprisingly, this has generated a flurry of responses from persons with more conservative leanings eager to defend “objectivity” and “rigor” in research. Also not surprisingly, some of these responses have been hyperbolic. Writing for Persuasion, Zaid Jilani claimed that these claims were “emblematic of the struggle between truth and social justice that is taking place across many left-leaning institutions in the United States.”

There’s a certain irony to this response to Farrell’s argument, for in their rush to defend objectivity and rigor many of their putative proponents have abandoned both.

Farrell writes that an appeal to objectivity

“…allows researchers, intentions aside, to define themselves as experts without learning from people with lived experience.  Objectivity also gives researchers grounds to claim they have no motives or biases in their work. Racism, sexism, classism, and ableism permeate US institutions and systems, which, in turn, allows for research that reproduces or creats racist stereotypes and reinforces societal power differences between who generates information… and who is a subject…”

With respect to rigor, she writes that

“…researchers often define rigor as following an established research protocol meticulously instead of ensuring data are contextualized and grounded in community experience”. This understanding of rigor, she holds, “does not guarantee trustworthiness or accuracy.”

It might be that some of Farrell’s critics have been “triggered” by her use of certain words (“objectivity”, “rigor”) without paying attention to how she is using them or her intended message. If these terms are removed, Farrell’s claims become utterly anodyne. To paraphrase:

“…researchers should not define themselves as experts without learning from people who have direct experience of the subject the researchers are studying. They should recognize that their work might be motivated by pretheoretic commitments or be biased in some way…” 

And

“…researchers should recognize that established research protocols might not lead to results that are trustworthy or accurate unless the data that they gather are placed in proper context and accurately reflect the experiences of the persons who are the subjects of research.”

But these claims shouldn’t be controversial. Consider how they’d apply to Nancy MacLean’s much criticized work Democracy in Chains. (MacLean argues, in brief, that James Buchanan and public choice theory was at the heart of a stealthy conspiracy funded by the Koch Brothers to protect the privilege of rich white men.) For Farrell, MacLean should not have defined herself as an expert in this area without learning from people who knew and worked with Buchanan to ensure that she was getting her facts right. She should also recognize that her interpretation of documents and events might be biased by her ideological antipathy to libertarianism—and taken pains to correct this. And she also should have recognized that despite her extensive documentation of her sources her adherence to this established research protocol of her discipline (history) is no guarantee that her work will be trustworthy or accurate unless her data is placed in proper context and reflects the experiences of those who are the subject of her research.

But isn’t the problem with Democracy in Chains that MacLean failed to be objective and rigorous? If so, how could her methodological failures support Farrell’s rejection of “objectivity” and “rigor”? The answer is simple. Although Farrell takes herself to be rejecting “objectivity” and “rigor” what she is really arguing against are appeals to these values that mask poor research methodology.

To be sure, she should have been clearer about this. But to rush to condemn her for rejecting “truth” in favor of “social justice” on the basis of this ‘blog post is to commit the very errors in research that she decries.

Vaccines, Science, Judgement, & Discourse

My very first entry into this blog—back on July 2, 2020—was about wearing face coverings because of Covid. That was fairly early into the pandemic, but I think the post has aged very well and I still stand by it.  It seems clear that when there are many cases of a serious new infection, people should wear masks if they go into an enclosed space with lots of unknown others. I also think, though, that it would be wrong to have government mandates requiring that people wear masks (except in places, like nursing homes, where the occupants would be at a known and significant risk) and that private businesses should decide the policy for their brick and mortar operations, just as individuals should decide the policy for their homes.  There is nothing inconsistent in any of that.

Similarly, it seems to me that everybody who can, should want to be inoculated against serious infections (having had the actual infection is likely sufficient). Again, that doesn’t mean that it should be government mandated. (I’m so pro-choice, I think people should be able to choose things that are bad and foolish; I don’t think they should be able to choose things that clearly cause harms to others, but neither the vaccine nor its rejection by an individual does that, so far as I can tell.) We shouldn’t need government mandates to encourage us to follow the science.  So let’s discuss that.  

Acetylsalicylic Acid alleviates headaches, fevers, and other pains.  I don’t know how that works.  Here’s a guess: the acid kills the nerves that are firing.  I actually doubt there is any accuracy in that guess at all, but it doesn’t matter.  I don’t need to know how aspirin works.  I know it works and is generally safe so I use it. How do I know this?  It’s been well tested, both by scientists and by tremendous numbers of people throughout the world.

Now, I actually think I have a better sense of how vaccines work than how aspirin works, though I doubt that holds for the new mRNA vaccines and I realize I could be wrong.  Again it doesn’t really matter.  I’ll use them nonetheless—and for the same reason. The fact is that most of the time, most or all of us simply trust in science.  We use elevators, escalators, cars, planes, trains, clothing with new-fangled fabrics, shoes with new-fangled rubber, foods with all sorts of odd new additives, etc.—all of which were developed with science.  And we don’t usually let that bother us.  

What seems to me foolish in standard vaccine refusal is roughly the same as what seems foolish to me in opposition to using the insecticide DEET in areas where mosquitoes carry malaria, which kills many people. It’s true that the DEET causes some significant problems, but it is unlikely that those problems are worse than the many deaths that would result without it.  This seems clear just based on historical use of the chemical. Similarly, vaccines may cause some problems but the (recent) historical use suggests pretty clearly that they save lives.

Of course, there are always mistakes.  Science is constantly evolving—it is more of a process, after all, than a single state of knowledge.  Scientists make mistakes.  Worse, sometimes scientists bend to their desires and sometimes industries have enough financial power to change the way science is presented. (Looking at you, sugar Industry!) Given that and a personal distrust of government, I certainly understand when people want to wait for evidence to settle.

A drug or other scientific advancement used too early may well turn out to be more problematic than its worth.  But aspirin has been well tested.  And vaccines have been well tested.  Even the recent Covid vaccines have been well tested.  The fact is you are far more likely to die from Covid if you are unvaccinated than if you are.  Granted, the odds of dying either way are thankfully slim for most of us.  But what people are now faced with is a free and easy way to avoid (a small chance of) death.  Admittedly, it’s possible that in 20 years we’ll learn that these new vaccines cause cancer or such.  But scientific advancement will continue and the fight against cancer is already far better than it was any time in the past.  So the option is between a free and easy way to avoid a chance of death or serious illness now combined with some chance of added problem later that we may know how to deal with and, well, not avoiding that.  Maybe this is a judgement call, but the former seems pretty clearly the better option in standard cases.  (Other downsides, so far as I can tell, are mostly fictitious.  If you’re worried about a computer chip embedded in the vaccine, for example, realize you could have had one put in you when you were born.)

About it being a judgement call. Consider using a GPS.  Some people just slavishly listen to the directions from their GPS. Unfortunately, this can have pretty bad results.  Other people refuse to use a GPS at all, perhaps thinking they have to do it on their own. For me, the GPS (in my phone) is a tool that is helpful to get where I need to go when I can’t really remember all the directions well or simply don’t trust my ability to do so. Still, I listen to the GPS and sometimes override its directions, for example, if I think it’s going in an unsafe way or a way that’s likely to cause more problems.  Here too, judgment is needed.

Unfortunately, we all seem to think we individually have great judgment even though it’s obvious that not all of us do.  Or perhaps better, none of us do all of the time.  Sometimes one has to recognize that we have to trust others to know better than we do.  

So, what should we do?  We should each try to be honest with ourselves about whether our judgment is likely to be better than those telling us to do other than we would choose. We should listen to people who are actually able to consider all of the relevant evidence.  Because it’s unlikely that any single source of information will always be completely trustworthy, we should likely listen to variety of generally trustworthy sources. 

We need to find people we can rely on—mentors or people recognized as experts in the relevant field—and take their views seriously.  This may simply push the problem back a step: those whose judgment lead them to make bad choices may simply choose to listen to other people with similarly bad judgement.  That is a real problem worth further investigation.  My only suggestion here is to trust those who are leading good lives and who have the trust of their professional peers.  I don’t pretend that is sufficient, but can’t say more here except to note that we can only hope to get better decisions, for ourselves and others, if we have better discussions.  To that end, see this postAlso, realize that if people would in fact standardly make better decisions (in part by having better discussions prior to making decisions), there would be less call for government intervention.  Indeed, if we had better conversations across the board, we would have less people wanting government intervention.  Realizing that those who have suffered through COVID are inoculated, for example, should stop others from trying to pressure them to get vaccinated.


Thanks to Lauren Hall, Connor Kianpour, and JP Messina for suggesting ays to improve this post.