Apropos, given recent discussions:
"Professional medium Patricia Putt was last week subjected to a rigorous scientific test of her powers as the first stage of her bid to claim a $1m prize from the James Randi Educational Foundation."
And was Mrs Putt, who is apparently a dab hand at removing evil spirits from houses, the very first person in the history of the One Million Dollar Paranormal Challenge to pass the preliminary test?
Mrs Putt claims to have the ability to psychically read people just by "being near them and hearing their voices". So ten volunteers were disguised, and faced away from her while reading from a set text, and Mrs Putt produced personalised psychic readings for each of them. The volunteers then perused all of the readings and picked the one which they thought best described them.
If five or more subjects picked the right reading, Mrs Putt would have passed the preliminary test and been able to take the final, more rigorous, million-dollar test.
(Which isn't to say this preliminary test was sloppy. As with every other preliminary test, the protocol was well-thought-out, with thorough precautions to avoid even quite subtle cheating.)
Precisely none of the subjects picked the reading that was allegedly about them.
To Mrs Putt's credit, she didn't then do what so many failed contestants have done - say the testers must have been cheating, the challenge is rigged, Randi is a child molester, blah blah blah.
Some failed challengers are all smiles when they leave and then immediately start pumping out press releases about how the scientific method is a Satanic plot, but Mrs Putt's Web site has no such updates (there seem to have been no updates at all for a few years, actually). Randi has also had to endure loud complaints from various people who never even made it to the preliminary test, because they refused to say exactly what it was they proposed to do, or proposed a test that could reasonably be expected to result in serious injury or death, or wouldn't say what would constitute failure. Or, most commonly, because they just broke off correspondence with Randi, since they had no time to talk to him between all their blog and Usenet posts about how Randi is too cowardly to face them.
But Mrs Putt took her defeat with dignity. For a few days. Then she gave in to temptation, as is explained in a stop-press at the end of the Guardian article. She announced that because the subjects were "bound from head to foot like black mummies, they themselves felt tied so were not really free to link with Spirit making my work a great deal more difficult".
(The subjects were actually wearing a ski mask and wraparound shades, and draped with a graduation gown. None reported feeling "tied".)
The fact that not a single one of the subjects picked the right reading opens a well-known psychic escape hatch, which goes something like: "In the pressure of the test, my powers must have fouled up and started working in reverse! Surely if I failed every single round of the test, when you would have expected me to get some right by random chance, that still indicates that something spooky is going on!"
Mrs Putt hasn't tried that one (yet), but I think it's instructive to run the numbers to see just how unlikely this event was, assuming everything was just random chance.
In the Putt test, each subject got their own set of all ten of the readings to choose from. This is good, for two reasons.
One, if they were all choosing from one pile of ten readings, then Subject 3 (say) could take Subject 7's reading, making it impossible for Subject 7 to pick the right one.
And two, it meant I didn't have to tie myself up in conditional-probability knots to figure out how likely it actually was that no subjects would choose the right reading by random chance.
As I've discussed before, this is also the way to go about calculating how likely it is that one or more subjects would choose the right reading randomly, because that probability is the inverse, or "complement", of the probability that none would.
If you've got ten subjects each randomly choosing from ten readings, one of which is "theirs", the probability that each individual subject will choose the wrong reading is 9 out of 10, or 0.9. Since there are ten subjects, the probability that no subject will get the right reading by random chance is 0.9 to the power of 10, or 0.349. Which means the probability that at least one subject will get the right reading is 0.651.
So it's only slightly odd that nobody picked the right reading by accident. The possibility still remains that Mrs Putt actually did do some "reverse psychic-ology" and produce readings that were completely unlike the person she was attempting to describe, and that the zero result thereby does indicate something odd going on. But given that the same test done entirely at random will give you this same zero-hits result seven times out of every twenty, the result doesn't look peculiar at all.
(The chance that all of the subjects would randomly choose the right reading is 0.1 to the power of 10, or one in 10,000,000,000. Mrs Putt would only have needed five or more of the subjects to randomly choose the right reading in order to pass the test, though. I leave calculating the probability of that as an exercise for the reader.)
14 May 2009 at 2:54 pm
My calculations say the probability of her succeeding by chance is about 1 in 365.
14 May 2009 at 4:29 pm
I get 1 in 612 or more accurately this binomial probability calculator gets 1 in 612.
http://faculty.vassar.edu/lowry/binomialX.html
14 May 2009 at 4:35 pm
Using my fuzzy probability math I put her chance of getting 5 out of 10 by random as exactly fucking slim.
14 May 2009 at 5:51 pm
There's now an update on the randi blog where Putt claims:
"I'm not in the least disappointed that the results did not go my way. I was stunned at first but when normal thought re-entered my head I realised that I was never going to win the barriers presented in the protocol were too much even for me to surmount,"
In other words, it wasn't a failure, just a result of the test setup being unfair. (Never mind that she agreed to it beforehand...)
14 May 2009 at 5:52 pm
Heh. I've had experiences that more or less add up to "psychic" powers, but I still can't explain it, and think that, given that I can't explain it and apparently neither can anyone else, that anyone who claims a "psychic" or similarly preternatural power for fame or profit is, at best, a bit soft in the head, and at worst, a filthy scammer. My take on it is this -for instance I've had dreams that eerily predicted future events. Coincidence? Possibly. The unconscious mind connecting existing information in ways the conscious mind could not/did not - likely. Can I do it at will? Hell no. Can I tell you for a fact how or why it happens? Nope. So I couldn't rightly attempt to sell it, quantify it, or otherwise market it. But I can tell you, for a fact, how big a colour television set is or how many HDMI ports it's got, so I feel perfectl comfortable and justified in selling said television set, so I think I'll stick to that. Hell, I've even had "precognizant" experiences while awake -but I'd again chalk them up to the "lizard brain" processing information in a way I don't perceive any more than I percieve the signals that keep my heart going, yet they exist, and I don't call them magic any more than I do what appears to be a precognizant experience.
14 May 2009 at 6:00 pm
Sorry for the double-post but Popup's comment wasn't there when I posted -I think Putt'statement proves my point: clearly subconcoius processing is going on with available information; when the available information is so limited as to be effectually nil, even the most advanced processing cannot do anything useful with it, and the "magic" will not happen. If your first thought is "ZOMG Magic!!1" for an as-yet-unexplained experience, then I have a lovely landmark to sell, cash only, small bills please.
14 May 2009 at 7:22 pm
Her comment as quoted by Popup pretty much amounts to
"I failed, because it was too hard."
I'd be satisfied with that. Particularly given her stated preference for a more diverse group; as the article mentions, with a more diverse group of 10 people, I bet even I could be "psychic"!
14 May 2009 at 8:44 pm
Red October: I've experienced that sort of thing. Something happens and you go "Huh...I'm pretty sure I dreamt that would happen." I always just assumed my memory was faulty - that I dreamt something sufficiently similar that I automatically connected the two. Since I never fully remember dreams, I assumed the combination of "How would I have predicted this?" combined with "Fuzzy dream where something similar happened" just added up to some nice deja vu. Your hypothesis is very complimentary toward my lizard brain, though. Thank you.
14 May 2009 at 10:08 pm
Red October - You've given me an idea.
Fortune telling by crocodile. O(of course there may be some disclaimers to sign. No responsibility for loss of limbs and such)
14 May 2009 at 10:17 pm
Not that I have any objections to the test, nor do I in any way believe that she would have been successful; but wouldn't it be more objective to have some close friend or relative of the subject choose the most suitable reading? What one thinks about oneself and what one actually is like may be quite different.
15 May 2009 at 1:50 am
mutercim - I would tend to agree with you there... but it's important to remember that Mrs Putt had complete approval over the specifics of the test. She was completely OK with the methodology and setup prior to the test.
Now, post test, she claims it couldn't be done because everyone felt "tied up"... only she never even asked any of the participants if they actually felt "tied up". Apparently her psychic abilities are enough to discern the comfort level and state of mind of the subjects but not enough to do what she claimed she could. Go figure.
15 May 2009 at 8:40 pm
More to the point, her psychic abilities didn't pick up on the obstruction until after she was told she had failed the test. Hindsight much?
16 May 2009 at 10:56 am
Being a devils advocate, if it were real we understand so little how it works it could be very easily to unknowingly break it. Imagine if you had a cordless phone with no understanding at all of how it worked and run a test with the handset behind a wall from the basestation. Result, cordless phones don't work! Because you had no idea that you're using a concrete wall that is blocking the signal.
I just like to be devils advocate sometimes.:)
16 May 2009 at 4:02 pm
I work at a Radio Shack. People who have no idea how cordless phones work are more real than psychic powers! I get people who have no idea that they need batteries, need to be plugged in (to mains or telco or both), have limited range (they try to place calls from the lone handset in the store to test its new, uncharged battery!), etc. Proof positive that the "anything I don't understand must be supernatural" line of thinking is very strong. Is it possible to have a predictive thought or dream or even beyond -yes. Just because you have no explanation for it doesn't mean that it's without rational explanation.
In child psychology, there is a test for developmental level where two screens are set up in front of a child and an object moved first behind one, and then from behind the first to behind the second. Sufficiently developed children will look for it behind the second, younger/insufficientlty developed children will look behind the first. From the thought process of the "Zomg magic!" people, you'd think they'd still look behind the first screen!
16 May 2009 at 9:47 pm
You don't need an understanding of how a cordless phone works to figure out that it won't work more than 20 metres from the base station or on the other side of a big metal object. Those things will come up in everyday use. Likewise it's odd that psychics never run into these sorts of complications (inability to read people wearing academic gowns, inability to read people facing the other way, inability to read people when the readings are written down afterwards...) after the fact. And they never notice the interference until after the failure is confirmed by the testers! Then it's clear as day somehow.
16 May 2009 at 9:47 pm
Err, never run into these sorts of complications until after the fact, i.e. after the test.
17 May 2009 at 11:18 am
Alex:
There's also the fact that the test subjects frequently think they did well right up until the results are read out. Mrs. Putt had the option of replacing people she found difficult to read, but apparently gave no indication anything was wrong until being "gobsmacked" by the result. In the cordless phone analogy, this would be the equivalent of not being able to tell the difference between standing next to the base and standing inside a Faraday cage.
19 May 2009 at 4:46 am
If I were Ms. Putt I'd say something to the effect that people can't pick their own descriptions because they'll pick what they want to believe rather than what's correct.
Maybe the test took that into consideration. I don't know, as in grand Internet tradition I didn't read any of the links.
Randi should have had ten married men psychically read by Ms. Putt and then had their wives pick the right description. I don't know if it would have been scientifically valid, but it would have be an atomic laff riot.
19 May 2009 at 5:54 am
I think any type of picking descriptions leaves too much vagueness in it. But, to create a perfect valid and objective test would require more of an understanding of how it works. Which at the moment we don't have. (Ignoring that it doesn't actually work at all)
19 May 2009 at 9:29 pm
Actually, I think that psychics are pefectly happy to tell you exactly how it works, but great difficulty telling you what it does.
I mean, every psychic, regardless of their application, knows how their process works. A psychic ability gives them the information. Now, this is an unsatisfactory answer if you don't believe in psychic powers, but otherwise, it's perfectly valid. Suppose that somebody asked how you were able to talk to Aunt Sally when she's on another continent, and you told them you used the telephone. That'd be perfectly satisfactory, even though it's no more enlightening. You could go into technical detail about how the data is transmitted, but you don't need to do so. We can take "I use a telephone" as your explaination for your ability to communicate with Aunt Sally over huge distances. Likewise we can take "it uses psychic powers" as the psychic's answer for how they can receive secret information, without going into any technical detail about how the underlying psychic power does its thing.
The part that gives psychics problems is deciding just what their powers do. If someone wants to know what, exactly, my special long-distance-telephone ability does, I can explain that it lets me communicate with others at a great distance so long as they are awake, standing near their own telephone, and paying attention. And that's something you can objectively test me on, with no understanding of how the phone itself works. My distance-speaking ability can be proven.
Psychics don't have any clue what their ability does. Sometimes, their ability lets them know that the body is next to some water. Sometimes it turns out that it lets them know the environment is wet. Or they're crying. There's water in there somewhere. Or the opposite of water. Or it's the answer on Tuesdays and the opposite of the answer on Thursdays. How do you test them when they're not even sure what their ability does, and it might actually do the opposite of what they tell you it does?
The uncertainty in what their psychic powers do, rather than any uncertainty in how they work, is the issue.
20 May 2009 at 4:10 am
@Alex - I call BS. Psychics are generally very specific about what they can do. Witness this test. The testee knew exactly what to test for and was able to define the problem. The difficulty is that they simply cannot do what they freaking claim to do! In fact, they cannot do anything other than prey on and mislead the gullible and provide entertainment for the rest of us on late night TV.
20 May 2009 at 12:34 pm
I agree with both of you :-).
Yes, psychics are often happy to make strong, definite, testable claims about their abilities, especially in advertisements. But when someone actually tries to pin their claims down, they almost invariably start the waffling Alex is talking about. Paranormal claimants are the kings of the ad-hoc explanation.
You can side-step this whole problem by just looking at psychics' results, though. Do they win lotteries? Do they find kidnapped children? Do they tell us what the moons of Neptune look like before we've sent a probe there? They certainly try to do all of those things, but they do not seem to achieve better than chance results.
21 May 2009 at 12:39 am
mlipphardt is right of course, they often do make strong predictions. I guess I'm so used to assessing them from the unassailably flaky positions they retreat to as soon as you test them.
22 May 2009 at 3:40 am
@alex, I am always right. Ask my teen daughter! My chi tells me the answers and my spirit guides help me translate the astral flux.