Novella on Anecdotes, Anomalies, and the Importance of Context
Steven Novella has thought a lot about thinking. I offer in evidence the following post from his Neurologica blog: The Context of Anecdotes and Anomalies.
The problem with anecdotes is that they are subject to a host of biases, such as confirmation bias. They are easily cherry picked, even unintentionally, and therefore can be used to support just about any position. For every anecdote, there is an equal and opposite anecdote.
I really liked it, and heartily recommend the whole thing to the friendly local conspiracy theorists. Unfortunately, I also predict that they will fail to recognize it as a valid indictment of their epistemological shortcomings. Oh, well.
December 7th, 2010 at 1:49 pm
JCB, I suspect that was aimed at least partly at me. Thank you, I appreciated the article and particularly agree with the concluding paragraph. Novella makes some valuable points, but is also sloppy in his reasoning.
JBC, you said: “I also predict that they will fail to recognize it as a valid indictment of their epistemological shortcomings.” Might the opposite also be true, that in some cases it shows a valid indictment of supposed non-conspiracy theorists’ own epistemological shortcomings? Also, in identifying valid conspiracy theorists’ shortcomings, might Novella’s comments be helpful to better enable the conspiracy theorists’ to focus their conspiracy investigation without employing those shortcomings?
I apologize that, as you predicted, from my perspective the relevance to me is not entirely clear. Please explain, but first observe three of the points that I see:
Point #1
Shcb, JBC, Craig, etc., (when not using ridicule or ad hominem attacks on persons like Dr. Steven Jones, Dr. Jim Hoffman or David Ray Griffin) use anecdotes such as the Bush administration’s bumbling as evidence that a complex conspiracy like 911 would be impossible to pull off successfully under their watch. No testing or controlled observations or experiments of that are possible.
(The counter-anecdotes are that the Bush admin was more than competent enough to succeed in overt and covert lobbying to achieve their pre-meditated goals to attack Afghanistan and Iraq. But that’s irrelevant.)
“In short – anecdotes are useful for generating hypotheses, but not for testing them. Problems arise when anecdotes are used to support a hypothesis or claim, rather than just to raise a possibility to be tested.” – Novella
So, you guys, please put aside your anecdote based objections long enough to call for an honest and thorough investigation of 9/11 anomalies. That’s all that I ask.
Specifically (and as an aside), there are many anomalies and other leads, but the key point is that they were not investigated honestly or thoroughly and it appears to many the reticence in investigating them was because they did not fit the official hypothesis closely enough (e.g. tunnel vision imposed in the official analysis by analyzing only the events leading up to the moment of collapse and purposely ignoring all evidence of the collapse itself, failing to investigate NIST’s incongruous sample of super-heated steel or consider evidence of molten metal, Phillip Zelikow shutting down investigations that seemed to lead to Saudi connections with the hijackers, insider trading investigations called off when the trail led to others besides Islamists, etc. It may not be possible to investigate with the rigor of Science since there are few opportunities for repeating experiments and using blind controls, but the least we should expect is a thorough and open-minded investigation conducted at a level of competency comparable to a serious criminal investigation. What we got instead was is a fairy tale and instructions to disregard the laws of physics.
Point #2
The saddest part is that the Novella article then picks an “example” / anecdote to malign 911 truth and by implied association belittles all of 9/11 truth:
“For example, 911 truthers will point to apparent anomalies in the debris of the jet crash into the Pentagon as evidence for a conspiracy. The unstated premise is that a casual observer would have a good instinctive idea of what such a debris field should look like.”
Yes, that anecdote is a true description of some 911 truthers and some arguments, but there are at least as many equal anecdotes to the contrary. If the reader was paying attention to Novella and tended to agree with him in the first place, then (if they can think for themselves) they should realize the incredible irony that Novella so readily falls into the erroneous thought process that he argues against. (I.e. picking an anecdote in an attempt to prove a point.)
Point #3
Novella groups 9/11 truth, JFK investigation, and moon landing hoaxes together which suggests an ulterior motive to rank unconventional and unpopular investigations into the same box of crazy nuts. However, an understanding of Logic (that forms the basis of Western civilization’s epistemological triumphs) tells one that the truth or falseness of any one of those separate conspiracies has no bearing on the others. By the time most readers get to those passages in the article they will be primarily relying on their emotional response (“Yea, that sounds right!” – their critical thinking skills are on the back burner) and so they absorb far more of the implied association between the various conspiracy theories presented than logic would dictate. That seems to be reflected in JBC’s introductory tone: “heartily recommend the whole thing to the friendly local conspiracy theorists.”
Uh huh. I heartily recommend it to everyone as a study about reinforcing peoples’ biases against conspiracy theorists by masquerading as observations and pleas for epistemological straight thinking.
December 8th, 2010 at 5:49 am
One small point Knarly, when we say that Bush didn’t have the intelligence to pull something like this off that isn’t an assessment of the situation it’s a jab at liberals. How can someone be an idiot and pull off the biggest hoax in the history of the world, that we don’t think George Bush was an idiot, but the left does and somehow they want to have it both ways.
I think this is a good article and it shows some of the false belief systems of those that don’t believe the hype of global warming is real, global warming is real, man plays some small part in the warming, but the hype surrounding this issue is just as real as the global warming itself. Bad science was used. Almost every report that has been issued on the subject has agreed with that assessment, and yet the hype deniers can’t see through their rose-colored glasses. They have the perfect vehicle to impose massive socialism on the entire world and that is just too good to be true. Mann used bad math, he used sloppy record-keeping, this computer codes are a mess, and he cherry picked specimens for his tree ring analysis. All of these things are exactly what this article says good science should not do, and yet the hype deniers can’t see through the rose-colored glasses.
December 8th, 2010 at 10:28 am
Wow that is a small point. Since you brought it up I’ll deal with it, but I’ll do so quickly because it is also irrelevant.
Novella makes clear that anecdotes have a limited use, the implication being that people who use them otherwise are either ignorant of that or are being disingenuous.
You say that your reason (for using one as a pillar of your argument against 911 truth) is based primarily on spite arising from liberals calling Bush an idiot and on the false premise that truthers say Bush was the mastermind*. That illustrates the fundamental dishonesty of your rhetoric. That may be one reason why people like Smith think you are an idiot. But you are clever at diverting attention, so now I shift the focus back to the grazing grounds.
As I said before, guys please put aside your anecdote based objections long enough to ensure we get an honest and thorough investigation of the 9/11 anomalies. That’s all that I ask.
Note: * The common belief is that some of the “masterminds” were likely among the signatories to the “Project for a Mew American Century”, not Bush.)
December 8th, 2010 at 10:57 am
You’re not getting my point, it’s not even antidotal, it has nothing to do with the discussion so it really doesn’t belong in your piece, unless the person making the point were using it to change the subject, but you aren’t talking about a specific argument, you are making a general point. I think you are unwittingly doing what this discussion says not to do, you have made an invalid point and then built at least a small portion of your argument around it. I’m not criticizing you here, this is why the peer review process is so important, whether it is the actual scientific review process or just having your wife proof read your Christmas letter. I found that flaw, I may be right, you may be right or we may both be partially right. Maybe what you wanted to say was right you just didn’t say it very well, you were too close. That is one of our complaints with the AGW reports, they seem to lack rigor in changing things that are obviously wrong, is it a conspiracy? Probably not.
December 8th, 2010 at 11:29 am
“antidotal”? You’re like talking to archie bunker, except worse.
December 8th, 2010 at 11:50 am
voice recognition software without proofreading
December 8th, 2010 at 11:52 am
like I said, its worse.
December 8th, 2010 at 12:50 pm
yeah, Archie could do a retake
December 10th, 2010 at 10:42 am
BTW, I am speaking of the anecdotes provided related to the Bush administration’s bumbling. In the interest of brevity I omitted digging up specific examples and repeating them here as most people would still get (Novella’s) point that such anecdotes are not useful for testing hypothesis.