All posts by James Stacey Taylor

Objectivity, Rigor… and Irony.

Back in September the Urban Institute published a ‘blog post arguing that “Equitable Research Requires Questioning the Status Quo”. The author—Lauren Farrell—argued that both “objectivity” and “rigor” were “harmful research practices” that should be rectified.

Not surprisingly, this has generated a flurry of responses from persons with more conservative leanings eager to defend “objectivity” and “rigor” in research. Also not surprisingly, some of these responses have been hyperbolic. Writing for Persuasion, Zaid Jilani claimed that these claims were “emblematic of the struggle between truth and social justice that is taking place across many left-leaning institutions in the United States.”

There’s a certain irony to this response to Farrell’s argument, for in their rush to defend objectivity and rigor many of their putative proponents have abandoned both.

Farrell writes that an appeal to objectivity

“…allows researchers, intentions aside, to define themselves as experts without learning from people with lived experience.  Objectivity also gives researchers grounds to claim they have no motives or biases in their work. Racism, sexism, classism, and ableism permeate US institutions and systems, which, in turn, allows for research that reproduces or creats racist stereotypes and reinforces societal power differences between who generates information… and who is a subject…”

With respect to rigor, she writes that

“…researchers often define rigor as following an established research protocol meticulously instead of ensuring data are contextualized and grounded in community experience”. This understanding of rigor, she holds, “does not guarantee trustworthiness or accuracy.”

It might be that some of Farrell’s critics have been “triggered” by her use of certain words (“objectivity”, “rigor”) without paying attention to how she is using them or her intended message. If these terms are removed, Farrell’s claims become utterly anodyne. To paraphrase:

“…researchers should not define themselves as experts without learning from people who have direct experience of the subject the researchers are studying. They should recognize that their work might be motivated by pretheoretic commitments or be biased in some way…” 

And

“…researchers should recognize that established research protocols might not lead to results that are trustworthy or accurate unless the data that they gather are placed in proper context and accurately reflect the experiences of the persons who are the subjects of research.”

But these claims shouldn’t be controversial. Consider how they’d apply to Nancy MacLean’s much criticized work Democracy in Chains. (MacLean argues, in brief, that James Buchanan and public choice theory was at the heart of a stealthy conspiracy funded by the Koch Brothers to protect the privilege of rich white men.) For Farrell, MacLean should not have defined herself as an expert in this area without learning from people who knew and worked with Buchanan to ensure that she was getting her facts right. She should also recognize that her interpretation of documents and events might be biased by her ideological antipathy to libertarianism—and taken pains to correct this. And she also should have recognized that despite her extensive documentation of her sources her adherence to this established research protocol of her discipline (history) is no guarantee that her work will be trustworthy or accurate unless her data is placed in proper context and reflects the experiences of those who are the subject of her research.

But isn’t the problem with Democracy in Chains that MacLean failed to be objective and rigorous? If so, how could her methodological failures support Farrell’s rejection of “objectivity” and “rigor”? The answer is simple. Although Farrell takes herself to be rejecting “objectivity” and “rigor” what she is really arguing against are appeals to these values that mask poor research methodology.

To be sure, she should have been clearer about this. But to rush to condemn her for rejecting “truth” in favor of “social justice” on the basis of this ‘blog post is to commit the very errors in research that she decries.