False Argument #33: Extraordinary Claims Require Extraordinary Evidence

Is there anybody out there who hasn't heard some debater quip, "Extraordinary claims require extraordinary evidence?"

I'm betting not.

How many people actually stop and consider the rhetorical device they're using?

I'm betting not that many, else we'd hear it much less!

At any rate, I've got a very simple and straight-forward example of an instance where the claim, "Extraordinary claims require extraordinary evidence" can easily be shown false.

Take the claim that some person climbed a tree, for instance. I doubt there is anybody out there who would deny this claim's ordinariness, as people have been known to climb trees since antiquity. If I were to make the claim that some person climbed a tree, what sort of evidence would a reasonable individual require?

I'm willing to bet your answer to that question would be comprised of one or more of the following:

1) some kind of hard evidence, e.g. an authentic photograph or video footage;

2) reliable testimony from a trusted source.

If your answer is comprised of one or more of the aforementioned, then I'm going to argue that it's fair to call those examples of "ordinary evidence."

Next, take the claim that some person climbed one of the 88-story Petronas twin towers in Malasia. I doubt there is anybody out there who would deny this claim's extra-ordinariness, as such feats are, well... simply not ordinary. If I were to make the claim that some person climbed one of the 88-story Petronas twin towers in Malasia, what sort of evidence would a reasonable individual require?

I'm willing to bet your answer to that question is going to be the same as your answer to the previous question: either some sort of hard evidence, reliable testimony from a trusted source, or both.

As such, I'm going to argue that the person who makes the argument, "Extraordinary claims require extraordinary evidence" either misuses the term extraordinary, or attempts to increase the claimant's burden of proof without justification.

At least without emendations, "Extraordinary claims require extraordinary evidence" is a false argument.

Comments

[Shema]

cl, MS shared your blog with me, for which I'm grateful. Good thoughts lingering in these parts...I will now spend the evening brainstorming extraordinary claims, testing your reasoning posited above. Thank you!

[Dominic Saltarelli]

Not to side track you too much, but I've seen this often enough before to notice that this sort of thing represents a fundamental difference between people biased towards religious belief and those who are not.

When considering the question of 'evidence' and what constitutes 'evidence', I've noticed there's a distinct fork in the road, where the theists go one way and the rest go the other.

The forks are the assumptions that either 1) there are types of evidence; or 2) there is only one type of evidence. Depending on one's unconscious assumption, people go with one of them, and lock horns with people who go the other and no one gets anywhere. It's that "talking past each other" phenomenon that's all too common.

Regarding the idea that some evidence is ordinary and other is extraordinary, I've noticed the theists break evidence up into classes and show that none of the classes are extraordinary (the post above being a prime example). This falls on deaf ears to those for whom breaking evidence up into classes is nonsense on its face.

For people like myself, the assumption is that there is only 1 type of evidence, information. It is the content therein that lets you distinguish between the ordinary and the extraordinary. To use an analogy, replace 'evidence' with 'books'. What distinguishes an ordinary book from an extraordinary book is the content of the information therein, they're still just books by definition.

Take for example this bit from DD's blog:

“Extraordinary evidence” does not mean that we need a fundamentally different type of evidence, it means natural evidence which is consistent with the extraordinary claims being made, such as observable and verifiable smashing of vases without the involvement of any natural agents or forces. Evidence of poltergeists, or levitation, or telepathy or other such things would not be “extraordinary evidence” in the sense of being something other than measurement, observation, correlation, and so on. It would simply be natural evidence consistent with the extraordinary phenomena of poltergeists, levitation, telepathy, and so on.

(sorry about the link, I recall they were pretty hostile over there, don't want to drudge up too many bad memories)

[TaiChi]

I must admit, I'm not sure quite what people have in mind when they use this chestnut - it depends upon what one calls 'extraordinary'. I do think claims which contradict our background theories about how the world works require extraordinary evidence, though - if these are what is meant by 'extraordinary', then a man climbing the Petronas Towers is not extraordinary in the relevant sense, and so this claim does not require extraordinary evidence. Miracles, on the other hand, would.

[woodchuck64]

I take "extraordinary claims require extraordinary evidence" to mean extraordinary claims require a similar degree of extraordinarily GOOD evidence; that is, the quality of the evidence is in some sense proportional to the inverse probability of the claim.

For tree climbing, word of mouth is good enough evidence for me.
But for Petronas tower climbing, I need evidence of a little higher quality: a photo or maybe a French driver's license with "Alain Robert" on it.

I'm not certain "extraordinary evidence" should always be taken to mean "extraordinarily unusual evidence". I think it can just as well mean "extraordinarily high quality evidence".

[Lifeguard]

I totally get your point, Cl, but I sort of lean towards Woodchuck regarding the quality of the evidence-- it has to leave little room for doubt that the extraordinary claim is true.

For instance, I could make the extraordinary claim that I made flat footed jump to the top of one of the Petronas towers and offer as proof a photo of me at the top. I think you would be fully justified in requiring better evidence than a photograph of me at the top, since all that proves is that I've BEEN to the top. Even a video of me actually leaping to the top could be questioned based on the availability of photo-shopping technology and what not.

The evidence I offer in either case is evidence-- it's probative of my claim, but in either case, you would probably want to see something more definitive to be satisfied that, (given the extraordinary nature of my claim) there was no other less extraordinary explanation for what you're seeing in the evidence (i.e., that the photograph has been tampered with or that I was not helicoptered to the top of the tower for a staged photo).

The simple fact that the photos or video are probative doesn't matter, because the probative value of those pieces of evidence are insufficient-- they are not enough to purchase your believing my extraordinary claim even if they TEND to prove it (if I can prove I have been to the top, then I've proven one necessary element of my claim). They don't, however, prove ENOUGH to satisfy you that my claim is true.

I'm guessing the only way I could convince you would be to do it right in front of your face-- an ordinary manner of proof, sure, but possessing far more probative value than you might demand for a less extraordinary claim.

For example, if I made the less extraordinary claim that I have met Willie Nelson, you might be satisfied with a photograph, even though the possibility of photo-shopping still exists, because that claim does not beg as much credulity on your part as my having made a flat footed hop to the top of one of the Petronas towers. Now, if I said he and I are great friends and smoke pot together the second Wednesday of the month, a photograph and a roach might do something to convince you that it might be true, but you would probably want something more before you believe that it is.

So, I get your point about the evidence being ordinary in either case, but I don't think that "extraordinary" necessarily means an unusual or atypical form of evidence, but a piece (or pieces) of evidence that have an unusually high degree of probative value. At the very least evidence with higher probative value than one would expect for a less extraordinary claim.

[Sabio Lantz]

Or maybe something like this gets to the point:

Higher stake claims demand higher levels of evidence.

[Sabio Lantz]

How do I follow threads on TypePad? I didn't see something to check? Or is it automatic?

[cl]

Shema,

Thanks for stopping by. Let me know of any noteworthy findings :)

Dominic,

Though I'll readily agree that all evidence distills to information, and in that sense, there is only "one type" of evidence, beyond this superficial distinction I can't agree with you. As I've argued here before, I take 1: there are types of evidence:

Genuine or conclusive evidence lends well to incontrovertible conclusions. On the contrary, inconclusive evidence cannot reliably sustain incontrovertible conclusions. Several pieces of inconclusive evidence pointing to a conclusion carry greater weight than just one piece. (cl, Thoughts on the Nature of Evidence)

There are many types of evidence. What I'm objecting to is the atheist who raises the burden of proof post hoc, simply because they find the claim incredible. That strategy strikes me as an insulation from cognitive dissonance.

I'm confused as to why you cited DD here, as, it seems he's making my point:

“Extraordinary evidence” does not mean that we need a fundamentally different type of evidence, it means natural evidence which is consistent with the extraordinary claims being made, such as observable and verifiable smashing of vases without the involvement of any natural agents or forces. (DD, emph. mine)

That's exactly what I argue should be accepted: evidence which is consistent with the extraordinary claim. Again, DD:

Evidence of poltergeists, or levitation, or telepathy or other such things would not be “extraordinary evidence” in the sense of being something other than measurement, observation, correlation, and so on. It would simply be natural evidence consistent with the extraordinary phenomena of poltergeists, levitation, telepathy, and so on.

I'll be back later to address the others' comments. Maybe in the meantime you could help me out by clarifying exactly where we agree or disagree?

[cl]

woodchuck64,

I take "extraordinary claims require extraordinary evidence" to mean extraordinary claims require a similar degree of extraordinarily GOOD evidence; that is, the quality of the evidence is in some sense proportional to the inverse probability of the claim.

I understand, and don't really have an objection to that per se, because I myself argue that conclusive evidence should be preferred to inconclusive evidence, but here's the problem as I see it [in the context of (a)theist debate]: the approach in question allows the atheist to simply draw lines in the sand and increase the burden of proof with no justification other than their opinion that some claim is extraordinary. To me, that's not sound methodology. It invites the very same bias science attempts to dispense with. We should require exactly the same quality of evidence whether the claim is ghosts or atomic structure: conclusive evidence.

Lifeguard,

Well, well! Long time no hear. Hope you're well.

...I could make the extraordinary claim that I made a flat footed jump to the top of one of the Petronas towers and offer as proof a photo of me at the top. I think you would be fully justified in requiring better evidence than a photograph of me at the top, since all that proves is that I've BEEN to the top.

I agree: such evidence would be inconclusive.

Even a video of me actually leaping to the top could be questioned based on the availability of photo-shopping technology and what not.

Ah, now we're getting somewhere. Again, I agree: such a video could be doubted, but guess what? Anything can be doubted. So how does the claimant circumvent the natural human tendency to shield oneself from cognitive bias via unjustified, post hoc increases to the burden of proof?

As an aside, this is why I find atheists like jim and SI [who ask for things like videotaped miracles] incredibly naive and intellectually immature. If people don't believe Criss Angel, who pulls his stunts in the flesh, why would they believe a videotaped limb regrowth?

The evidence I offer in either case is evidence-- it's probative of my claim, but in either case, you would probably want to see something more definitive to be satisfied that, (given the extraordinary nature of my claim) there was no other less extraordinary explanation for what you're seeing in the evidence (i.e., that the photograph has been tampered with or that I was not helicoptered to the top of the tower for a staged photo).

I want to see conclusive evidence, regardless of the nature of the claim. It's that simple. I've noticed that the closer one gets to disproving certain atheist's beliefs [or lack of beliefs or whatever] the higher the bar gets raised. That's what I'm objecting to. To say, "Well, we must raise the bar when the claim is extraordinary" is to load things with the atheist's opinion as to what constitutes an extraordinary claim.

I agree that meeting Willie Nelson "does not beg as much credulity on [my] part as [your] having made a flat footed hop to the top of one of the Petronas towers." Still, in both cases, all I'm asking for is conclusive evidence. If conclusive evidence has been provided for both claims, I'm not going to ask for "more conclusive evidence" just because I'm biased against the possibility of one of them.

In short, our method needs to be objective, and fairly applied.

Sabio,

I'll look into the RSS for comments. I can't remember if Typepad offers that or not.

Higher stake claims demand higher levels of evidence.

I get the gist of what you're offering, but I have the same objection: who becomes the arbiter of "higher stakes?" The whole point of science is to keep human opinions as far away from the research as possible. If we start by structuring the research according to human opinions [i.e. one's opinion that some claim is extraordinary], isn't that a flawed approach? Then again, I would grant that there are times when it is "more than opinion" to call a claim extraordinary. For example,

For everyone / anyone:

What type of evidence would you require before you believed that I saw a de-capitated man re-capitate himself and wander into the bar for a drink? Why?

[Sabio Lantz]

@ CL
The person who gets to be the arbiter of "higher stakes" is the individual of course.
Take for instance if I tell you there is a drug to cure your bald head. Well, some individuals may want lots of evidence before ingesting an unknown drug, others not.

Let's say I have evidence that a God lives in the tree in the woods and will save your soul if you lay acorns before it. Some may indeed need not evidence and some may want lots.

So I think the folks who would say this are saying, "The cost of acting on your proposition is too high for me at your level of evidence."

Pretty simple I think.
Concerning your de-capitated chap -- I wouldn't believe you with a video or witnesses, I'd probably have to see it and even then I'd doubt it I were tricked. But the trick here is that I really wouldn't care. Believing such a thing would risk none of my stakes.

(Yeah, without ability to follow your threads, I will probably rarely be able to come back -- have you ever thought of switching over to WordPress?)

[Sabio Lantz]

I realized I have two accounts with Typepad (too complicated): one under Sabio Lantz and one under Triangulations. But it won't allow me to use the same e-mail in both. So I am trying this. Maybe it will work better.

[Dominic Saltarelli]

It's not that we disagree on this point. It's why I said my post may be a bit of a sidetrack. The assumptions I pointed out, I think are both perfectly reasonable ones. It's just a natural bias that points people towards one or another, and this natural bias is what leads to irreconcilable differences when the topic of evidence arises.

What I'm trying to do is recognize the bias for what it is and air it out before charging forward and making the same tired old arguments that keep getting made time and time again.

[Lifeguard]

1) "So how does the claimant circumvent the natural human tendency to shield oneself from cognitive bias via unjustified, post hoc increases to the burden of proof?"

If it's a natural human tendency, then all claimants is shit out of luck unless they happen to be arguing with someone who is already inclined to believe them. The flip side of your question, however, is how does the claimant circumvent the natural human tendency to shield oneself from cognitive bias via unjustified increases to the probative value of their evidence?

For example, "I've noticed that the closer one gets to disproving certain atheist's beliefs [or lack of beliefs or whatever] the higher the bar gets raised." There are two sides to each of those discussions, and the natural human tendency towards cognitive bias is present in ALL of us.

How do you know that you're getting closer to disproving a certain atheist's beliefs as opposed to succumbing to the natural human tendency towards cognitive bias when it comes to assessing the strength of your own argument?

The solution-- the only real way around it-- is for the individual to police HIM OR HERSELF INDIVIDUALLY and STOP ASSUMING that the other is the only one prone to what is, in fact, a natural human tendency to overestimate the strength of one's own position and devalue the other. That means not assuming you have disproven or proven anything and simply discussing the evidence from your perspective.

You went on to say that "To say, 'Well, we must raise the bar when the claim is extraordinary' is to load things with the atheist's opinion as to what constitutes an extraordinary claim." I think that's almost right to the point, and I spent a good part of the weekend wondering why you hadn't just made this the real focus of your post, perhaps even arguing that claiming the universe came from nothing is a more extraordinary claim than claiming that a supreme being made it.

Again, all I can really say is that people have what I at least think are understandably different ideas of what constitutes extraordinary when you're talking about the creation of the universe or the existence of a supreme being, but not so much so when it comes to Criss Angel or a decapitated man. Whether there's a supreme being who created it or the universe coalesced from an eternal and impersonal amalgamation of matter and energy, either way it's pretty extraordinary.

So whatever that's worth...

[cl]

Sabio,

Thanks for putting some flame under me arse. I think I'm pretty close to having a comment feed implemented.

So I think the folks who would say this are saying, "The cost of acting on your proposition is too high for me at your level of evidence."

Perhaps some do mean that. Imagine a person who emails you asking for $1,000 and promising to turn it into $10,000. The principle you just described is the same principle that prevents most rational people from giving the person $1,000: the risk is too high, given the evidence. However, put these same rational people on Wall Street, and things change. I think these facts say more about the mindset of individual than the truth of the claim being evaluated.

If we're talking about science, we need to keep personal criteria out of the discussion, don't you think? Meaning that, both sides should pursue the truth regardless of the personal aversions towards / against it. For example, if, instead of the suspicious email, we were discussing evidence for ghosts, and you had a strong aversion to the belief that ghosts exist, all you have to do is label ghosts "extraordinary" and that allows you to [unjustifiedly, IMO] raise the burden of proof - when in reality - all we need for ghosts is conclusive evidence. Just as conclusive evidence is all we need for any other claim.

[cl]

Dominic,

What I'm trying to do is recognize the bias for what it is and air it out before charging forward and making the same tired old arguments that keep getting made time and time again.

Then we're in the same boat. I recognize the bias for what it is. I agree that we need to find a way to circumvent it before charging ahead. I believe elimination of rhetorical devices is a great place to start. "Extraordinary claims require extraordinary evidence" lends all-too-easily to rhetorical device. It allows anybody to increase the burden of proof, based on nothing more than their opinion that some claim is extraordinary. Well, the claim that the Earth actually revolves around the Sun was pretty extraordinary - many moons ago. Yet, all it took to prove this was conclusive evidence.

[cl]

Lifeguard,

First, I apologize. I made a typo. The paragraph,

So how does the claimant circumvent the natural human tendency to shield oneself from cognitive bias via unjustified, post hoc increases to the burden of proof?

...should have read,

So how does the claimant circumvent the natural human tendency to shield oneself from cognitive dissonance via unjustified, post hoc increases to the burden of proof?

Not sure how much that changes our discussion, it at all, I just felt you should know what I originally meant.

The flip side of your question, however, is how does the claimant circumvent the natural human tendency to shield oneself from cognitive bias via unjustified increases to the probative value of their evidence? [...] How do you know that you're getting closer to disproving a certain atheist's beliefs as opposed to succumbing to the natural human tendency towards cognitive bias when it comes to assessing the strength of your own argument?

That's easy: by limiting oneself to conservatively stated beliefs / conclusions. Getting back to our old buddies jim and SI, if I were telling them that the video game incident was proof of God, or even evidence for God, then the person who accuses me of an unjustified increase to the probative value of my evidence would be making a justified accusation. Since I don't offer the incident as "proof" of anything, or "evidence for God," how might one justify the claim that I'm unjustifiedly increasing the probative value of my evidence?

So, maybe you can answer my question, now that I've answered yours? Since anything can be doubted, how do hold people accountable?

That means not assuming you have disproven or proven anything and simply discussing the evidence from your perspective.

I'm not assuming that I've proven or disproven anything. I am stating confidently that I - along with witnesses - have experienced an event that [to date] has not been explained sensibly through a naturalist paradigm. The evidence is consistent with claims about the nature of paranormal / spiritual phenomena. Consider jim and SI's response: "you're either lying or mistaken!" They could say that about anything. Even if they experienced what I experienced.

...I spent a good part of the weekend wondering why you hadn't just made this the real focus of your post, perhaps even arguing that claiming the universe came from nothing is a more extraordinary claim than claiming that a supreme being made it.

Because, although it might better prove my point, that would be hypocritical. That would be an instance of exactly the error I'm seeking to eliminate: the error in which somebody begins ostensibly impartial research from their own opinions about what is or is not extraordinary.

Granted, there are claims that deserve the label extraordinary, IMO. Your flat-footed jump is one of them. Note that the only reason we can justify our claim that that claim is extraordinary is because we have prior knowledge of the natural limitations of human ability.

[Lifeguard]

"Since anything can be doubted, how do hold people accountable?"

Honestly, I'm not even sure what you mean by "accountable" in this context, so let's take a very simple example. I claim the sun rises in the East and sets in the West and someone says "You can't prove that." So I say, "Okay, let's check it out tomorrow," but tomorrow, when the sun does rise in the East and sets in the West, this person accuses me of using a fake compass. So I say, "Alright, tomorrow you bring YOUR compass." They do, but tomorrow he says I have a hidden magnet on me that I'm using to screw with his compass, and so on and so on... he always comes up with some reason other than his being wrong for why the mounting evidence consistently points the other way no matter what concessions I offer him when gathering data.

At some point, more and more people observing this would agree that my interlocutor is being unreasonable skeptical of my claims.

Now imagine we're not talking about the sun rising and setting, but... uh... something far more controversial like the creation of the universe, the meaning (if any) of life. Things that are not readily observable for which there is more than one reasonable explanation. How do you judge when someone is being TOO skeptical?

I'm not sure you can unless, as you state, you scale down your claims to the point where they essentially tell us nothing.

Regarding one of your arguments you wrote that the phenomenon you and others experienced amounts to something that "has not been explained sensibly through a naturalist paradigm" as opposed to being evidence or proof of God's existence. You describe it as merely "consistent with claims about the nature of paranormal / spiritual phenomena."

Personally, I think anyone who disputes the latter formulation probably either doesn't trust you or is being unreasonable. In either case, it's probably because they think you're on your way to pulling some intellectual slight of hand that will enable you to get a lot of meaningless rhetorical mileage out of the rather mundane concession that something is merely CONSISTENT with CLAIMS about the paranormal.

As for "an event that [to date] has not been explained sensibly through a naturalist paradigm," you're going to run into what I think are understandable disputes about what constitutes an "explanation," what makes an explanation "sensible," and whether claims about the paranormal amount to a better explanation or indeed even amount to an explanation at all.

And so we are right back where we started from-- everyone is going to have a different threshold for credulity depending on the nature of the event and the explanation proferred. It's the same as people having different levels of risk tolerance. AT BEST, you can probably get a concession about the consistency of the proof with an alternative explanation, but that's about it.

And let's be honest, most folks out here blogging about this stuff aren't looking for concessions about the consistency with the evidence with theist or atheist explanations. They want to convince people that they are right and the other is wrong, and one doesn't undertake that endeavor unless they believe they ARE right, which means they are already emotionally anchored to their position in such a way that substantially increases the likelihood that they are biased.

The only thing we know for certain about the big questions is that certainty is hard to come by.

[cl]

Lifeguard,

I claim the sun rises in the East and sets in the West and someone says "You can't prove that." So I say, "Okay, let's check it out tomorrow," but tomorrow, when the sun does rise in the East and sets in the West, this person accuses me of using a fake compass. So I say, "Alright, tomorrow you bring YOUR compass." They do, but tomorrow he says I have a hidden magnet on me that I'm using to screw with his compass, and so on and so on... he always comes up with some reason other than his being wrong for why the mounting evidence consistently points the other way no matter what concessions I offer him when gathering data.

That's exactly what I mean. How do we hold those types of people accountable to logic? I'm willing to accept the possibility that we can't.

...I think anyone who disputes the latter formulation probably either doesn't trust you or is being unreasonable.

I agree.

In either case, it's probably because they think you're on your way to pulling some intellectual slight of hand that will enable you to get a lot of meaningless rhetorical mileage out of the rather mundane concession that something is merely CONSISTENT with CLAIMS about the paranormal.

When the question is, "Do you have evidence for [i.e. consistent with] spiritual or paranormal claims," how is that a rather mundane concession? Shouldn't such put at least a dent in the skeptic's immovability?

Or, should we simply do as I do, and say, "Well, you can lead a skeptic to water, but you can't make them drink?"

The only thing we know for certain about the big questions is that certainty is hard to come by.

So then, why do people like SI and jim and PhillyChief portray such certainty in their claims that "there is no evidence for spiritual claims?" Why aren't they more willing to say something like, "Now that's interesting... there's something that actually does confront my views head-on?"

I tell you the truth, that's all I was ever asking for.

[Shema]

"Thanks for stopping by. Let me know of any noteworthy findings :)"

Speaking of noteworthy, cl, Beethoven came to mind. While he is/was an extraordinary composer, his example of extraordinary doesn't refute the *theme* of your argument. :)

* UFO’s exist?
* Obama is a Messiah figure?
* Jesus walked on water?
* The Ramones played faster than any band prior to their masterful contribution to music?

I see problems with opinions as well as shades in our interpretation of facts—problems that elucidate the biases of people. In addition, it’s near impossible to collect a large enough sample of evidence to form a valid conclusion about a number of claims, whether extraordinary or not.

* The origin of our planet? No human observed the formation, and no human has conclusive evidence. Both atheists and “Christians” must cling to faith. Believers appeal to the numinous, His Word, and evidence found in creation while atheists point to naturalism and skepticism (which is a form of faith, in my opinion).

While I'm curious like others here about definitions and semantics, I’ve not thought of a single example that disproves your argument. The examples merely demonstrate in general that people believe what they want to believe and appealing to extraordinary evidence shelters those who request such evidence from considering the basis and bias of their beliefs.

Perhaps…

[cl]

Shema,

The examples merely demonstrate in general that people believe what they want to believe and appealing to extraordinary evidence shelters those who request such evidence from considering the basis and bias of their beliefs.

I couldn't agree more, and would add that it's a human error, not necessarily a theist or atheist one.

[Dominic Saltarelli]

cl:

Then we're in the same boat. I recognize the bias for what it is. I agree that we need to find a way to circumvent it before charging ahead. I believe elimination of rhetorical devices is a great place to start. "Extraordinary claims require extraordinary evidence" lends all-too-easily to rhetorical device. It allows anybody to increase the burden of proof, based on nothing more than their opinion that some claim is extraordinary. Well, the claim that the Earth actually revolves around the Sun was pretty extraordinary - many moons ago. Yet, all it took to prove this was conclusive evidence.

Not sure if you're saying conclusive evidence is different from extraordinary evidence in this case. 'Conclusive' still leaves one with the freedom to set the bar at what actually counts as 'conclusive', leaving you with the same problem that arises when resorting to a rhetorical device.

Personally, I think the best way to rephrase Mr. Sagan to satisfy those who take the left hand fork, that is, bias towards classifying information, such as into:
authority - intuitively understood divine edict, scripture, textbook, priest, scientist, etc...
first hand observation - heard or saw
secondary - video or picture
tertiary - blog post, newspaper article
and so on...

..would be to say "extraordinary claims require evidence of the extraordinary that's consistent with everything else you're fairly sure is true".

Those of us who automatically take the right hand path, that of evaluating evidence based on its actual content rather than its type to determine whether it is conclusive, believable, ordinary, or extraordinary, implicitly understand this and don't see the problem with the statement as originally phrased. After all, Sagan has been in the atheist/materialist camp, so to speak, from the get go.

I don't think you've been the victim of any goal post moving in the past when this phrase has been trotted out, it's just that you were never told where the post was to begin with, and whoever you were arguing with couldn't see past his/her own internal, personal bias, to spell it out.

[Leah]

I can't get your trackback link to work for me, so I just wanted to let you know that I found this post very helpful and gave it a hat tip on my blog here: http://www.unequally-yoked.com/2010/06/eye-has-not-seen-ear-has-not-heard.html

[Sabio Lantz]

Darn, I just wrote a long reply, but when I had to "sign in", I lost it. TypePad is a royal pain.

[Sabio Lantz]

@ cl
On Common Sense Atheism you just accused Luke of not being generous in understanding an opponent by not following his own Golden Rule. You wanted him to try and put an opponent's view in the best light before arguing.

Could you do the same for the claim that "Extra-ordinary claims demand extra-ordinary evidence". A chap in Leah's site (above) wrote a good condition to supplement the claim: cumulative evidence.

For instance, if you claimed to own a Honda, I would require little evidence for several reasons.

(1) i know lots of folks own Hondas (cumulative evidence - outside the particular claim)
(2) I have no reason to distrust you to date (cumulative evidence)
(3) I have little at stake in your claim (following my version that: "Higher stake claims demand higher levels of evidence.")

So I can believe easily.
But if you told me that "If you jump out that 17th story window, my flying saucer will pick you up and give you the ride of your life", well, for all the above reasons, I would demand more evidence.

So, cl, if you see even the slightest hint of truth in what the terse saying is hinting at, could you use the golden rule and try to write a version that captures that truth.

For language is a slippery thing -- we should try to help each other. (I admit hypocricy for my admonition but it is a good guiding principal, I think)

Looking forward to your version unless you feel it is totally shipwrecked and points at no truth.

[cl]

Dominic,

'Conclusive' still leaves one with the freedom to set the bar at what actually counts as 'conclusive', leaving you with the same problem that arises when resorting to a rhetorical device.

I disagree. Let's say that on a Sunday evening you go for a vacation to some foreign country, and you stay in a hotel. In the middle of your first night's sleep, the police yank you out of bed and accuse you of murder, because a blood trail led them back to your room, and some lady says she saw "the killer" enter your room. The police - for whatever reason - aren't buying your appeals to innocence, so I, your faithful attorney, fly in to assist. I arrive at your interrogation Monday afternoon, at 4:00pm. I ask to see the physical evidence associated with the case, and immediately note that rigormortis in the victim's body has dissipated. I state that it would have been impossible for your to be the murderer, because this victim has been dead for at least 48-72 hours, and you've only been in their country 12 hours.

Would you say that one is "allowed to set the bar" and deem the evidence "not conclusive enough" in this case? Why or why not?

[cl]

Leah,

I apologize for Typepad's quirkiness. Also, thanks for the link over there. Much obliged. I will add my sentiments to your post as time allows, as well as check out the rest of your site. You've piqued my interest, primarily because of the word economy and precision in your writing. I believe those qualities can take a writer far, regardless of education.

Sabio,

I'm sorry about your loss, but believe me, I've lost many more. That's why I don't use the Typepad comment window anymore. Writing comments in Notepad then cutting-and-pasting insures against Typepad's mysterious tendencies towards comment vaporization. I'm aware that Typepad is a royal pain. In my experience, all blogs are. I've been meaning to take a weekend and code my own for some time now, and every comment like yours (and Leah's about trackback links not working) adds to that motivation. That said,

On Common Sense Atheism you just accused Luke of not being generous in understanding an opponent by not following his own Golden Rule. You wanted him to try and put an opponent's view in the best light before arguing.

Where the hell did you get that from? I accused Luke of a Courtier's Reply, because, for the past six months, his stock response to my questions about desirism has been either, "You obviously don't understand the theory," or, "I don't have the time to answer your questions."

Anyways, you imply that I've not attempted to put "extraordinary claims require extraordinary evidence" in the best light possible. I disagree, but, at least so I can understand you and prevent these types of misunderstandings in the future, what words of mine gave you that impression?

So, cl, if you see even the slightest hint of truth in what the terse saying is hinting at, could you use the golden rule and try to write a version that captures that truth.

What "golden rule" are you alluding to? Also, why should I supply another when you didn't even acknowledge my last version? I said, "...all we need for ghosts is conclusive evidence. Just as conclusive evidence is all we need for any other claim."

Or, how about, "claims require conclusive evidence, regardless of opinions about their extra-ordinariness?"

What I'm looking for here is a standard that can't be waffled due to cognitive bias. As worded, the late Mr. Sagan's maxim allows for an IHOP's worth of waffling.

Also, I expressed my objection to your wording. Aside from the non-trivial ambiguity inherent in the wording, to say "Higher stake claims demand higher levels of evidence" is to descend right back into subjectivity. Your wording seems to invite the very bias I'm seeking to curtail. Regarding my re-capitated man, you said,

Believing such a thing would risk none of my stakes.

I don't see the value in using "personal risk" as a criteria for standards of evidence, because wherever "personal risk" is high the tendency towards slothful induction is likely high as well, don't you think?

[NFQ]

As the "chap" who wrote that comment over on Leah's blog, I'd like to back up what Sabio is getting at here about cumulative evidence.

I agree with you that, as it is worded on face, "extraordinary claims require extraordinary evidence" is a silly rule. Claims require evidence. One would hope that you don't require more evidence just for things you decide beforehand are too weird to be true.

But Sabio is raising an important point about cumulative evidence. We have a lot of evidence built up over time for the plausibility of certain claims based on the ordinariness of certain states of affairs. In this sense, I would require more evidence right now if you claimed to have a daisy growing out of the top of your head than if you claimed to have blond hair. People can have blonde hair -- I have amassed considerable evidence for this. That is in fact exactly what we mean when we say that something is "ordinary."

So the total evidence required would be roughly the same whether the claim is "ordinary" or "extraordinary." But the evidence demanded in response to the claim, demanded of the person making that claim, would be much higher in the case of something "extraordinary" because it doesn't have the head start of all that cumulative evidence.

[Dominic Saltarelli]

cl,

Would you say that one is "allowed to set the bar" and deem the evidence "not conclusive enough" in this case? Why or why not?

"In this case."
Good that you said this. Context is king. In this case, no, a person cannot set their own personal bar for what counts as conclusive in this particular case.

But, keeping context in mind, it must be noted that there's nothing extraordinary about murder, and the topic is regarding extraordinary claims. It's conclusive in this case because it's consistent with everything else we're fairly certain is true (arrival time in the country and time it takes for rigor mortis to dissapate). Run the same armchair experiment with an extraordinary claim and you'll suddenly find what counts as conclusive for different people to be quite plastic.

[cl]

NFQ,

It seems the distinction we're converging on is something like this: Ordinary claims don't contradict background knowledge (cf. owning a Honda, having blond hair). I bet we can all agree on that. However, I think when Sagan uses "extraordinary," he really means something like, "claims that contradict background knowledge" (cf. flat-footed jump to the top of a Petronas tower, growing daisies out of one's head). The problem is that a claim can be "extraordinary" without contradicting background knowledge (e.g., I found a million dollars on the ground).

However, no matter which way you cut it, the fact remains that in court of law / scientific context, all claims require conclusive evidence. Sagan's wording allows for people to get the wrong idea, which you allude to here:

One would hope that you don't require more evidence just for things you decide beforehand are too weird to be true.

I agree.

...Sabio is raising an important point about cumulative evidence.

I understand. My point is that, "claims require conclusive evidence" respects Sabio's point.

In this sense, I would require more evidence right now if you claimed to have a daisy growing out of the top of your head than if you claimed to have blond hair.

I'm not sure that the quantity of evidence would be different in either case. I'm taking a bit of liberty here, but I'm willing to bet that in either case, all you require is conclusive evidence. In the case of blond hair, we already have it. In the case of daisies growing out of heads, we don't. However, this doesn't change the quantity of evidence required, so I'm hesitant to reformulate Sagan using a word that signifies quantity. The quality of the evidence is what concerns me, and conclusive evidence is conclusive evidence.

Of course, that's what you're getting at when you say,

So the total evidence required would be roughly the same whether the claim is "ordinary" or "extraordinary." But the evidence demanded in response to the claim, demanded of the person making that claim, would be much higher in the case of something "extraordinary" because it doesn't have the head start of all that cumulative evidence.

It seems to me that we agree, although, again, I'm not sure that "higher" is the right word. It's just that the person claiming daisies has more work cut out for them.

Dominic,

In this case, no, a person cannot set their own personal bar for what counts as conclusive in this particular case.

Can you provide an example where somebody could set their own personal bar for what counts as conclusive?

...keeping context in mind, it must be noted that there's nothing extraordinary about murder, and the topic is regarding extraordinary claims.

Actually, the context of our last exchange was "conclusive vs. extraordinary."

Run the same armchair experiment with an extraordinary claim and you'll suddenly find what counts as conclusive for different people to be quite plastic.

Supply an example of an extraordinary claim and let's try.

[Dominic Saltarelli]

cl,

Supply an example of an extraordinary claim and let's try.

English speaking dinosaurs.

[cl]

English speaking dinosaurs.

We can't test that. How about, "I can walk on water?" Certainly that's extraordinary and testable, right? If you agree, let's say I walked across a pool in front of witnesses who confirmed the absence of any kind of material support that would allow for suspension.

Also: Can you provide an example where somebody could set their own personal bar for what counts as conclusive? I deny your claim that conclusive leaves us with the same problem as extraordinary.

[Dominic Saltarelli]

cl,

We can't test that. How about, "I can walk on water?" Certainly that's extraordinary and testable, right? If you agree, let's say I walked across a pool in front of witnesses who confirmed the absence of any kind of material support that would allow for suspension.

Also: Can you provide an example where somebody could set their own personal bar for what counts as conclusive? I deny your claim that conclusive leaves us with the same problem as extraordinary.

Can't test whether Jesus rose from the dead either. Which is one of those extraordinary claims requiring extraordinary evidence that birthed the rhetorical device in question. But, testable claim, fair enough.

Ok then, how about the claim that "communism is the best type of government". We still have communists today who make this claim, in spite of, you know, the 20th century.

[cl]

I see a need for another distinction. In the Communism example - at least in the absence of strict criteria delineating best - we're evaluating an inherently subjective claim. When I say conclusive, I refer to factual or objective claims: i.e., the time a person was murdered and its relation to a defendant's guilt or innocence; or, the fact of a man freeclimbing an 88-story building. Another example would be the identity of the person who left DNA evidence at the crime scene. Those are all claims in which the word conclusive has meaning. Your claim of English speaking dinosaurs would also qualify as an objective or factual claim under this rubric (it's just not testable). Personally, I wouldn't invoke the criteria of conclusiveness for opinion-based claims.

Getting back to the last comment: would you consider what I offered conclusive evidence that I've walked on water? Why or why not?

[Dominic Saltarelli]

Ok, gotcha. The only kinds of claims on the table right now are those that can only be defined in terms of some objective measurement. I had a broader category of claims in mind when I made my original statement. We'll just use your walk on water example, then.

To answer the question:

Getting back to the last comment: would you consider what I offered conclusive evidence that I've walked on water? Why or why not?

I would, the testimony of the individuals (we're assuming that I trust them as good sources, yes?) would be good enough for me. Basically, since I don't have anything vested personally in whether you can walk on water or not, my threshold for believability would be met. I would simply shelve it as something "quite odd" and continue to marvel at how much I don't know about the universe.

Now, if I may take it a step further, lets say one of your fans (what were their names, phillyChief, or SI, something like that?) heard the same testimony. I'm pretty sure they'd cry "bollocks" at the evidence provided and wouldn't find it conclusive at all, even though it's good enough to convince me. They may demand to be personally swimming underneath while you walk before being convinced.

So there you have it, a sliding scale for conclusiveness. Unless you have a few more caveats to throw in, that is.

[Leah Libresco]

So, let's contrast the claim that you can walk on water with the claim that you can walk on pavement.

Pavement case:
I might not ask you for any additional evidence here. I already have the prior knowledge that other people who are physically similar to you can walk on pavement. I also have prior knowledge about the reliability of your hearsay claims in the past on similar subjects. Completely reasonable for me to accept at face value.

Water case:
Now I’m going to ask you to show me more evidence. Most of my requests are going to turn on the fact that this claim doesn’t accord with my knowledge-to-date of water and physics in general. I know people have made this claim before and have been shown to have faked it.

So what can I ask you for?

Certainly not video evidence, since I’ve seen movies.
Watching you from a distance would still leave me suspicious (magicians do live shows, too).
Testimony of other viewers is no better than my own viewing.

I really see two possibilities:
- I observe up close, preferably on piggy-back, after checking out both you and the body of water for tricks.
- I take the word of a group of skeptical expects who I trust will apply the same standards of evidence I would, but who are more proficient at evaluating the data.

Even after either of the above two things, I would still be skeptical, but I would have to let your claim stand for now.

Now, obviously, I wouldn’t ask you for anywhere near as much evidence in the pavement case (though this would be sufficient evidence to prove the pavement case as well). Nothing I’m asking for is in a radical class of ‘exceptional evidence,’ it’s just out of the class of casual or hearsay evidence.

Also, I may just not be properly understanding TypePad, but is there a way to recieve emails when there are further comments on posts? I would have liked to jump back in on this discussion earlier.

--

Leah @ http://www.unequally-yoked.com

[woodchuck64]

CL:

let's say I walked across a pool in front of witnesses who confirmed the absence of any kind of material support that would allow for suspension.

A witness would confirm the absence of material support, but I don't think a reliable witness would confidently report after just one test that you are really able to walk on water. Rather, I think he/she would acknowledge that it certainly appears to be a neat trick, but several more rounds of experimentation, analysis and thought is required before jumping to any conclusions. After a week of that without finding a natural explanation for your ability, I would expect growing interest from the scientific community and the performing of increasingly more rigorous and expensive tests (with your permission of course). If in a month's time no plausible natural theories are formulated to explain your ability in a way that renders it a complex illusion, I would start to suspect you really are walking on water.

But this is still an informal calculation based on a rough idea of the likely competence of the scientists involved, how quickly they would be inclined to accept your ability at face value, and a vague idea of how much effort it takes to reasonably rule out all unknown (as opposed to known) water walking illusions.

[keddaw]

Any claim that goes against my current worldview requires not extraordinary evidence, but acceptance on a level I agree with.

For example, if someone claimed that quantum particle A could spontaneously turn into quantum particle B I would require scientific acceptance of the principle, but not actual evidence that I would look at. If someone claimed they could turn water into wine I would require actual evidence of it since it works at a scale I *cough* understand and it impacts my worldview.

Similarly, I would require extremely good evidence for an out of body experience since it is outside my worldview, but an extremely religious person would accept this at face value since it affirms their belief in an afterlife.

[Atheist.pig]

English speaking dinosaurs.

http://www.metacafe.com/watch/1934032/speaking_parrot_attending_the_door/