Polygraphs: The evidence

[CFLarsen]

In this thread (http://forums.randi.org/showthread.php?t=102225) about how polygraphs purportedly revealed that over 80% of atheists really believe in God, skeptigirl had this to say about the validity of polygraphs:

Just to dispense with the polygraph discussion, (though I doubt it will, perhaps if not the mods will see fit to split the thread), it doesn't matter if the polygraph is not good enough for court, because the majority of the answers would be correctly determined. Therefore the claim ALL 100 atheists were lying or all the atheists in the above story were lying and really believe in gods would not be entirely discredited on the basis of the polygraph reliability alone.


Here's a very thorough evaluation of polygraph testing. THE POLYGRAPH AND LIE DETECTION; Committee to Review the Scientific Evidence on the Polygraph; Board on Behavioral, Cognitive, and Sensory Sciences; and Committee on National Statistics; Division of Behavioral and Social Sciences and Education; NATIONAL RESEARCH COUNCIL OF THE NATIONAL ACADEMIES (http://www.nap.edu/openbook.php?isbn=0309084369) and on this page (http://www.nap.edu/openbook.php?record_id=10420&page=123) is a graph of results showing
FIGURE 5-1 Sensitivity and false positive rates in 52 laboratory datasets on polygraph validity.

NOTES: Points connected by lines come from the same dataset. The two curves are symmetrical receiver operating characteristic (ROC) curves with accuracy index (A) values of 0.81 and 0.91.

involving naïve examinees untrained in countermeasures: for such examinees and test contexts, the polygraph has an accuracy greater than chance. Random variation and biases in study design are highly implausible explanations for these results, and no formal integrative hypothesis test seems necessary to demonstrate this point.

Second, with few exceptions, the points fall well below the upper left-hand corner of the figure indicative of perfect accuracy. No formal hypothesis test is needed or appropriate to demonstrate that errors are not infrequent in polygraph testing.

The entire report is available at the link.

If a polygraph is wrong 2% or 5% or 10% of the time one can conclude out of 100 answers the truthfulness of 98 or 95 or 90 respectively will be correctly determined to be truthful or not. Since the OP is talking about a polygraph determining that 80 atheists were lying, then were it true, which there is no evidence it is, then the majority of those atheists would indeed be lying.

How the question was phrased would also need to be taken into consideration before drawing the lie-for-Jesus conclusion implied in the urban myth as well.

The report's conclusion says:

We have reviewed the scientific evidence on the polygraph with the goal of assessing its validity for security uses, especially those involving the screening of substantial numbers of government employees. Overall, the evidence is scanty and scientifically weak. Our conclusions are necessarily based on the far from satisfactory body of evidence on polygraph accuracy, as well as basic knowledge about the physiological responses the polygraph measures. We separately present our conclusions about scientific knowledge on the validity of polygraph and other techniques of detecting deception, about policy for employee security screening in the context of the U.S. Department of Energy (DOE) laboratories, and about the future of detection and deterrence of deception, including a recommendation for research.
Source (http://www.nap.edu/openbook.php?record_id=10420&page=212)

skeptigirl,

Please present your evidence that polygraphs are able to tell when you lie or not.

[Skeptic Ginger]

What is that link if not evidence? I don't understand your problem. It is an in depth detailed analysis of the polygraph.

Holy cow just looking at the title should tell you that:

THE POLYGRAPH AND LIE DETECTION; Committee to Review the Scientific Evidence on the Polygraph; Board on Behavioral, Cognitive, and Sensory Sciences; and Committee on National Statistics; Division of Behavioral and Social Sciences and Education; NATIONAL RESEARCH COUNCIL OF THE NATIONAL ACADEMIES

What more do you want?

[CFLarsen]

Well, you can start with explained what you should have read instead: Namely, the conclusion.

How is that evidence that polygraphs are able to tell when you lie or not?

[Skeptic Ginger]

The discussion was about the results of 100 polygraphs. You cannot tell which of the hundred results are wrong but you can make a general statement that the majority of the results are not wrong.

It's called sensitivity and specificity. I use it every day interpreting lab results. Some results are erroneous but I can determine with whatever degree of confidence the specificity and sensitivity is that the results from my patient's lab tests are reliable.

[Lonewulf]

This should be fun...

[Skeptic Ginger]

Hypothetically, say you asked 100 theists if they really believed in god and they all answered yes and the results showed they were all lying. If the polygraph gives valid results 80% of the time (without trying to fool it the report said the results were valid close to 100% of the time but say it was only 80%), then 80 theists would be lying. You could not tell which 80 were lying and which 20 were not lying but you could conclude 80 were lying based on the average validity of the polygraph results.

I really don't understand why the two of you don't understand the scientific concept of sensitivity and specificity.

[Lonewulf]

Hypothetically, say you asked 100 theists if they really believed in god and they all answered yes and the results showed they were all lying. If the polygraph gives valid results 80% of the time (without trying to fool it the report said the results were valid close to 100% of the time but say it was only 80%), then 80 theists would be lying. You could not tell which 80 were lying and which 20 were not lying but you could conclude 80 were lying based on the average validity of the polygraph results.

I really don't understand why the two of you don't understand the concept of sensitivity and specificity.
I don't get why you don't understand the simple concept of interpretation that people brought up, but whatever.

[Skeptic Ginger]

What concept is that, Lonew? I really don't have a clue what you are pissed about or what you are talking about here. Did you see my post about the off topic matter in the dungeon banished thread?

[CFLarsen]

The discussion was about the results of 100 polygraphs. You cannot tell which of the hundred results are wrong but you can make a general statement that the majority of the results are not wrong.

It's called sensitivity and specificity. I use it every day interpreting lab results. Some results are erroneous but I can determine with whatever degree of confidence the specificity and sensitivity is that the results from my patient's lab tests are reliable.

Hypothetically, say you asked 100 theists if they really believed in god and they all answered yes and the results showed they were all lying. If the polygraph gives valid results 80% of the time (without trying to fool it the report said the results were valid close to 100% of the time

Hello??

The whole idea of the polygraph is to detect if people try to fool it.

but say it was only 80%), then 80 theists would be lying. You could not tell which 80 were lying and which 20 were not lying but you could conclude 80 were lying based on the average validity of the polygraph results.

I really don't understand why the two of you don't understand the scientific concept of sensitivity and specificity.

But we are not talking about this concept. We are talking about whether or not polygraphs are able to tell when you lie or not.

You claim that they are. The conclusion of the study you pointed to clearly says the opposite of what you claim it says.

So, where is the evidence?

[Lonewulf]

What concept is that, Lonew?

You weren't reading in the thread it was discussed in? Thought not.

How does a lie detector work? Do you know? It monitors your body to see if it reacts to a question.

It can easily misread certain signs that could indicate something other than lying. Someone reading the polygraph would have to "translate" the body signs.

In short, it's subject to more than just simple machine error. It's also subject to human error.

Furthermore, there are ways for trained individuals to bypass a polygraph.

Could this or could this not affect your hypothesis that anytime the polygraph is used, it will always allow 80 out of 100 people, 100% of the time, be caught in lying? The polygraph is not a mind-reader. It's a body-reader. And some can control their bodies better than others, and some can read bodies worse than others. It's as simple as that.

Did you see my post about the off topic matter in the dungeon banished thread?
Now who's getting off topic?

[Skeptic Ginger]

I cannot reply to either of you because I don't for the life of me get your issues.

Are you talking about testing one person or the average results of 100 tests?

Hello.

[Lonewulf]

Lol.

Okay, I guess there isn't anything to discuss then.

Good day.

[Skeptic Ginger]

From the cited source:Estimate of Accuracy Notwithstanding the limitations of the quality of the empirical research and the limited ability to generalize to real-world settings, we conclude that in populations of examinees such as those represented in the polygraph research literature, untrained in countermeasures, specific-incident polygraph tests for event-specific investigations can discriminate lying from truth telling at rates well above chance, though well below perfection.

[CFLarsen]

I cannot reply to either of you because I don't for the life of me get your issues.

Are you talking about testing one person or the average results of 100 tests?

Hello.

What difference does it make?

If you can't tell if one person lies or not, how can you say anything about how the polygraph does on average?

You can't!

That's why your explanation isn't valid. You can't point to an average performance, unless you can tell if it works at all.

[CFLarsen]

From the cited source:

Okie doke.

Are you willing to present a paper at TAM, where you present the scientific evidence that polygraphs work?

[Skeptic Ginger]

Claus, why would I bother? However, if you have an argument with the logic, then spell it out. If you test 100 people and the results are expected to be correct 80% of the time then you can draw a conclusion about the group that you cannot draw about an individual.

The idea a test must be 100% accurate to be useful will be news to the medical community. Care to present your arguments why we should not use a test if it isn't 100% accurate?

[CFLarsen]

Claus, why would I bother?

Are you serious??

You believe you have scientific evidence that polygraphs work. We cannot let that opportunity pass us by, as skeptics. That's precisely what the meetings are for: To examine such claims and evaluate the evidence.

You think you can attend the Amazing Meeting - the biggest conference for skepticism and critical thinking in the world - and not be challenged because of your claim that polygraphs work?

What do you think being a skeptic is?

However, if you have an argument with the logic, then spell it out. If you test 100 people and the results are expected to be correct 80% of the time then you can draw a conclusion about the group that you cannot draw about an individual.

How do you know it is correct any of the time?

The idea a test must be 100% accurate to be useful will be news to the medical community. Care to present your arguments why we should not use a test if it isn't 100% accurate?

Whoa, whoa. Who said anything about 100%?

[Skeptic Ginger]

For pete's sake, this is a really annoying discussion.

What percent accuracy do you want? What is your estimate? What's your evidence?

Take that then apply it to the group of 100 tests. What do you get?

[Lonewulf]

From Wikipedia, here: http://en.wikipedia.org/wiki/Polygraph_test#2003_National_Academy_of_Sciences_R eport

The accuracy of the polygraph has been contested almost since the introduction of the device. In 2003, the National Academy of Sciences (NAS) issued a report entitled “The Polygraph and Lie Detection”. The NAS found that the majority of polygraph research was of low quality. After culling through the numerous studies of the accuracy of polygraph detection the NAS identified 57 that had “sufficient scientific rigor”. These studies concluded that a polygraph test regarding a specific incident can discern the truth at “a level greater than chance, yet short of perfection”. The report also concluded that this level of accuracy was probably overstated and the levels of accuracy shown in these studies "are almost certainly higher than actual polygraph accuracy of specific-incident testing in the field.” [1]

The report: http://www.nap.edu/openbook.php?isbn=0309084369

When polygraphs are used as a screening tool (in national security matters and for law enforcement agencies for example) the level of accuracy drops to such a level that “Its accuracy in distinguishing actual or potential security violators from innocent test takers is insufficient to justify reliance on its use in employee security screening in federal agencies.” In fact, the NAS extrapolated that if the test were sensitive enough to detect 80% of spies (a level of accuracy which it did not assume), in a hypothetical polygraph screening of 10,000 employees including 10 spies, 8 spies and 1,598 non-spies would fail the test. Thus, roughly 99.6 percent of positives (those failing the test) would be false positives. The NAS concluded that the polygraph “…may have some utility” [2] but that there is "little basis for the expectation that a polygraph test could have extremely high accuracy."[3]

The NAS conclusions paralleled those of the earlier United States Congress Office of Technology Assessment report "Scientific Validity of Polygraph Testing: A Research Review and Evaluation”.[4]

Certainly nothing that I would want to use in any scientific study.

[Skeptic Ginger]

But this discussion was never about using polygraphs to weed out spies. It was about drawing a conclusion about 100 tests grouped together.

[Lonewulf]

But this discussion was never about using polygraphs to weed out spies. It was about drawing a conclusion about 100 tests grouped together.

So this doesn't factor into that?:

roughly 99.6 percent of positives (those failing the test) would be false positives.

That would mean that 99.6 percent your positives of "real atheists" could very well be false positives.

Does this not register? It really isn't that awful for you to admit you were wrong, for once. I certainly wouldn't hold it against you.

How is 99.6 percent of false positives not relevant to a case where you're trying to find "fake" atheists from "real" atheists?

If 100 of them showed up as liars, 99.6 of them could easily be false positives. Is this not significant? The only real "fake" atheist would be .4 of a person! You're talking about atheist legs here, not even above the hips! :D

[CFLarsen]

For pete's sake, this is a really annoying discussion.

What percent accuracy do you want? What is your estimate? What's your evidence?

Take that then apply it to the group of 100 tests. What do you get?

How do you know it is correct any of the time?

Will you present a paper at TAM?

[drkitten]

That would mean that 99.6 percent your positives of "real atheists" could very well be false positives.

Highly unlikely. The stats just don't bear this out.


How is 99.6 percent of false positives not relevant to a case where you're trying to find "fake" atheists from "real" atheists?

Because we have a priori (by assumption) knowledge of the percentages of "real spies" in the case under discussion. Specifically, by assumption, 0.1% of the "employees" are "spies" in the case under discussion. Another way of looking at it is that in the pool of people who fail the polygraph examination, the percentage of spies is four times higher than in the population at large, which is quite a substantial lincrease.


If 100 of them showed up as liars, 99.6 of them could easily be false positives.

Not under the circumstances of the test as described.

By assumption, under the test you (and the NAS) described, the polygraph is about 80% accurate, which means that in a pool of 100 genuine atheists, you would expect about 20 of them to be (incorrectly) identified as liars. The form of this distribution is well-understood. For example, the chance of getting between 15-25 (incorrect) liars is about 65%. The chance of getting betweel 10-30 (incorrect) liars is about 95%. The chance of getting between 5-35 (incorrect) liars is about 99%. Alternatively, the chance of getting 31 or more "false positives" is less than 1%.

The chance of getting 50 or more "false positives" is sufficiently small that it would qualify for Randi's million. The chance of getting 99.6 "false positives" is sufficiently small as to make Randi's million a sure thing.

The difference? The size of the subject pool, and the rarity of the syndrome under study. With 10000 truthful subjects, I will probably find at least 2000 false liars by chance alone, but that would still dwarf something of which there are only ten to find in the pool. But with 100 truthful subjects, I will probably only find about 20 false liars --- and so if I find fifty liars, I can be confident that about thirty of them are probably genuine.

(I should point out that this is a well-studied problem in biostatistics. There's nothing "woo" about tests with less than perfect sensitivity and accuracy, they're a simple fact of life. Which is why it's so difficult to do large-scale testing for rare diseases. Some of the best work on this has been done by the various Armed Forces, when they're looking for rare tropical diseases in returning soldiers....)


Is this not significant?

Not in the slightest.

The NAS report is talking about large-scale screening, and identifies, correctly, that any detection device with only moderate accuracy is not very useful for such large-scale screening. Scepticgirl is talking about event-specific, small-scale analysis. Different problems, different issues, different solutions, and different requirements.

[3point14]

But this discussion was never about using polygraphs to weed out spies. It was about drawing a conclusion about 100 tests grouped together.

My recommendation:

Stop trying to explain statistical analysis of experimental results to people who don't understand statistical analysis.

Sometimes it is better to throw in the towel to stop yourself getting a headache.

[Lonewulf]

Ah yes, the "those ignorant people!" argument and "we can't learn them!"

Yeah, I'm unsubscribing from this thread. Doubt it'll go anywhere.

[Aepervius]

The only problem I see here is that you are all speaking about lie detection, statistics, but all polygraph does is *at best* try to detect "emotional stress" as shown by the various measurement of bodily function, in comparison to a baseline. Sometimes , but not always, a lie will generate (for an unaware/untrained persons) a fluctuation above the baseline (that is the sad theory). But so will many other stuff. And since the baseline is made by an operator NOT on valid scientific measurement or whatnot, but more or less on gut feeling, this make it already doubtful.

So it does not matter if it detects 80% truth, if along the way it detect 100% emotional stress, and thus loaded question , and question with an emotional baggage. You would have to first separate the emotional response to the lie, and for such question this would darn be impossible. Your polygraph test would show that on the question "did you rape your neighbors child multiple time ?" most people would fail the polygraph test.

[ben m]

How do you know it is correct any of the time?


CFLarsen, you need to step back and think about this. Statistical tests can tell you how well an imperfect test works. We do it all the time:

1) A self-proclaimed psychic says "I can psychically read Zener cards while blindfolded." We draw 100 known Zener cards and as the psychic to guess them. Then we compare each guess to the card. At the end of the test we can say "The psychic got 70% of these guesses right". In the future, if the psychic guesses at an unknown Zener card, we make an inference: "The psychic has a 70% chance of getting this guess right."

2) A nuclear-physics experiment has a detector which makes "blips" in response to neutrons and to alpha particles. Usually neutrons make sharper blips and alphas make broader blips. We expose the detector to a neutron beam and find that 70% of the blips are sharp; then we expose it to an alpha beam and find that 70% of the blips are broad. Later on, when we see an unknown blip in the detector, we check whether it was sharp or broad, and make a statistical inference (with appropriate error bars) about whether that blip came from a neutron or an alpha.

(This does *not* tell you how many false-positives or false-negatives this detector produces: that's a function both of the detector acceptance and rejection (as we call them) and of the actual neutron/alpha ratio encountered in the experiment.)

3) We take 100 people and make them play a psych-experiment-ish game. "Test subject #1, you are now on the Red team. You need to steal a flag from the Blue side; first, sneak past the Blue lie-detector by convincing it that you are Blue, then come back and present yourself to the Red lie-detector." Or whatever. At the end of the experiment, you have 100 examples of people lying (as the experimenter, you *know they're lying*) and 100 examples of people telling the truth (and, as the experimenter, you *know* they're telling the truth.) You can tally up that "70% of liars were correctly identified as liars by the machine", and "25% of truth-tellers were misidentified as liars".

It's just like any other test. You need a "calibration" phase where you provide the machine with known quantities (known neutrons, known Zener cards, known lies), and check how often its reports are correct.

So, a lie-detector calibration experiment is a perfectly reasonable thing. Your general objection is mistaken, and if you think about it a bit you ought to withdraw it. You may still (rightly) have a specific objection: "Has an appropriate lie-detector-evaluation actually been done? And what were the results?" But skepticgirl's links are a perfectly acceptable attempt to answer that objection. If you want to fisk the detailed experimental design, please do so.

[drkitten]

The only problem I see here is that you are all speaking about lie detection, statistics, but all polygraph does is *at best* try to detect "emotional stress" as shown by the various measurement of bodily function, in comparison to a baseline. Sometimes , but not always, a lie will generate (for an unaware/untrained persons) a fluctuation above the baseline (that is the sad theory). But so will many other stuff. And since the baseline is made by an operator NOT on valid scientific measurement or whatnot, but more or less on gut feeling, this make it already doubtful.

I'm afraid that this isn't correct. The baseline is made by asking the subject a number of other questions, including questions that are known to be stressful in and of themselves (as well as questions that are known to not generally be stressful), and calculated on a per-subject basis.



So it does not matter if it detects 80% truth, if along the way it detect 100% emotional stress, and thus loaded question , and question with an emotional baggage. You would have to first separate the emotional response to the lie, and for such question this would darn be impossible.

Not at all. It's standard practice.

Your polygraph test would show that on the question "did you rape your neighbors child multiple time ?" most people would fail the polygraph test.

One doesn't "fail" or "pass" a polygraph test on individual questions. In this regard, the term "lie detector" is misleading; the results of polygraph tests give the overall probability that the subject is attempting deception, not simple detection of individual lies. That's one reason why the tests are so long, so that the same question (or very subtle variations) can be asked over and over again --- to lie to one, you will have to lie to all (and therefore you will display an overall pattern of deception). (It also works to reduce the emotional response --- you may be shocked and offended the first time you are asked about raping your neighbor's child, but such shock and offense wears off relatively quickly if you are innocent; much of the emotional stress will vanish by the end of the test, simply because by that time you're simply responding truthfully to a rather stupid question.

The rest of it is simply empirical science. It is in fact relatively easy to set up a controlled test where the truth or falsity of the statements are known to the overall investigators (but not to the individual polygraph operators); Mythbusters has done such a test (and the polygraph worked). In the Mythbusters experiment, Kari, Tore, and Grant were handed sealed envelopes (at random) in which two of them were instructed to go into a room and steal a wallet. Kari was told to go into a room and look around but not touch anything. The polygraph operator was told that a wallet had been stolen and the three suspects were our three friends.

In this case, "we" (the viewers) know the truth; if the polygraph operator nailed Kari but let the others walk, we would know he had missed, and conversely, if he nailed the guys but let Kari walk, he hit. As it happened, he hit. Not that impressive in a sample size of three, but this is TV, not science.

If you want the science, check out the opening citation (http://www.nap.edu/openbook.php?record_id=10420&page=123). They cite 52 different studies and present ROC curves for all 52. If you've never seen such curves before, they may be difficult to interpret, but the curves themselves are fairly standard (and rather impressive).

"Chance" performance would be the diagonal line from lower left to upper right. No study came close to doing that poorly. All studies had a much higher chance of identifying a deceptive person as deceptive than a non-deceptive one, at any meaningful threshhold.

For example, say that you will accept a 10% "false positive" rate -- a fairly conservative one. In practice, this means that you look at the "baseline" (there's that word again) that 90% of the known truthful people never crossed. Now, the question becomes : how many of the known liars crossed that baseline (and would be detected as liars)?

In the worst-performing study, about 40% of the liars would be caught by a test that caught only 10% false positives. In the best-performing one, 100% of the liars were caught at that level.

So out of a group of 200 people, half liars, we would expect (from a high-performing case) to get about 95 of the 100 liars, and 10 of the truth-tellers; of the 105 people who failed the test, better than 90% were in fact liars. That's pretty good. (Actually, from the best performing case, we'd get all the liars, but ceiling effects enter here and you can't trust that particular number to hold up.)

[CFLarsen]

So, a lie-detector calibration experiment is a perfectly reasonable thing. Your general objection is mistaken, and if you think about it a bit you ought to withdraw it. You may still (rightly) have a specific objection: "Has an appropriate lie-detector-evaluation actually been done? And what were the results?" But skepticgirl's links are a perfectly acceptable attempt to answer that objection. If you want to fisk the detailed experimental design, please do so.

I understand the concept fine. However, your third example is countered in the study skeptigirl linked to:

Realism of Evidence
The research on polygraph accuracy fails in important ways to reflect critical aspects of field polygraph testing, even for specific-incident investigation. In the laboratory studies focused on specific incidents using mock crimes, the consequences associated with lying or being judged deceptive almost never mirror the seriousness of those in real-world settings in which the polygraph is used. Polygraph practitioners claim that such studies underestimate the accuracy of the polygraph for motivated examinees, but we have found neither a compelling theoretical rationale nor a clear base of empirical evidence to support this claim; in our judgment, these studies overestimate accuracy. Virtually all the observational field studies of the polygraph have been focused on specific incidents and have been plagued by measurement biases that favor over-estimation of accuracy, such as examiner contamination, as well as biases created by the lack of a clear and independent measure of truth.
From skeptigirl's link (http://www.nap.edu/openbook.php?record_id=10420&page=213)

Probably the biggest problem with polygraphs is that we cannot distinguish between lying and being emotionally distressed for other reasons.

Polygraphs have all the characteristics of a woo claim: The theory behind it is very weak, the evidence is very weak, there is no standardization, there is no scientific progress, there is no accumulated knowledge or accumulated evidence in favor of it.

skeptigirl claims otherwise. I want to see her defend her position at TAM. As skeptics, we simply cannot sit by and let such a claim go unchallenged.

[drkitten]

Polygraphs have all the characteristics of a woo claim: The theory behind it is very weak, the evidence is very weak, there is no standardization, there is no scientific progress, there is no accumulated knowledge or accumulated evidence in favor of it.

They have all the characteristics of a woo claim except for one. Unlike woo claims, there is a solid body of peer-reviewed evidence that demonstrates that their performance is substantially better than chance.


Neither the lack of standardization, of progress, or of accumulated knowledge are signs that a claim is unfounded, only that it is poorly understood. This is something that the NAS makes totally clear in their report.

Despite this, they are very firm that under proper conditions, polygraphs are effective. From the executive summary,


We conclude that in populations of examinees such as those represented in the polygraph research literature, untrained in countermeasures, specific-incident polygraph tests can discriminate lying from truth telling at rates well above chance (emphasis in original)

They are very concerned about the use of polygraphs for security screening, but that's (as has been pointed out) entirely different.

It is a misrepresentation verging on a lie to claim that the evidence that a specific-incident polygraph test can distriminate lying from truth telling at rates substantially better than chance is weak.

[CFLarsen]

They have all the characteristics of a woo claim except for one. Unlike woo claims, there is a solid body of peer-reviewed evidence that demonstrates that their performance is substantially better than chance.

The quote you provided does not say "substantially".

Do you think that skeptics like Randi, Shermer and Bob Carroll should change their minds about polygraphs?

Neither the lack of standardization, of progress, or of accumulated knowledge are signs that a claim is unfounded, only that it is poorly understood. This is something that the NAS makes totally clear in their report.

We are not talking about a new method here. Polygraphs are about 80 years old, but we don't see a heightened understanding of what it is. On the contrary, the more we learn about the psychology and physiology of lying, the less credible are polygraphs.

Despite this, they are very firm that under proper conditions, polygraphs are effective. From the executive summary,

They are very concerned about the use of polygraphs for security screening, but that's (as has been pointed out) entirely different.

It is a misrepresentation verging on a lie to claim that the evidence that a specific-incident polygraph test can distriminate lying from truth telling at rates substantially better than chance is weak.

It is absolutely neither misrepresentation or a lie. You, OTOH, misrepresented the conclusion by deliberately leaving out two the preceding sentence, and the following sentence:

Notwithstanding the limitations of the quality of the empirical research and the limited ability to generalize to real-world settings, we conclude that in populations of examinees such as those represented in the polygraph research literature, untrained in countermeasures, specific-incident polygraph tests can discriminate lying from truth telling at rates well above chance, though well below perfection.

The problem is that you can't know in real life if your population has been trained in countermeasures.

[drkitten]

The quote you provided does not say "substantially".

That is correct. It says "well."


Do you think that skeptics like Randi, Shermer and Bob Carroll should change their minds about polygraphs?

I do not have personal knowledge of their beliefs. To the extent that they believe that polygraphs cannot distinguish lying from truth telling, then yes, they should change their minds.





We are not talking about a new method here. Polygraphs are about 80 years old, but we don't see a heightened understanding of what it is. [/QQUOTE]

Which is not relevant. We didn't understand the mechanism of genetics for 80 years after Darwin (with the Nobel-winning work of Watson and Crick), but lack of understanding of the mechanism did not prevent us from observing the reality.

For that matter, we knew that the orbit of Mercury did not match Newton's predictions for 80 years before Einstein solved the gravity problem.



It is absolutely neither misrepresentation or a lie. You, OTOH, misrepresented the conclusion by deliberately leaving out two the preceding sentence, and the following sentence:

The only misrepresentation is on your part.

if your claim is that a technology must be "perfect" to be not be woo -- well, then you've just put both penicillin and space flight into the woo category. If you claim that only experiments done under field conditions can demonstrate the viability of a theory, then you've put both biological evolution and electroweak unification in to the woo category. If you insist that only a fully-developed technology is not woo --- well, I'm not sure what areas of technology cannot be improved. Perhaps the screwdriver.

Despite the fact that the technology works much better in the lab than in the field, the simple fact that the technology works at all and can be reproduced at will makes it "not woo."





The problem is that you can't know in real life if your population has been trained in countermeasures.

[CFLarsen]

That is correct. It says "well."

Why did you overstate the effectiveness of the polygraph?

Which is not relevant. We didn't understand the mechanism of genetics for 80 years after Darwin (with the Nobel-winning work of Watson and Crick), but lack of understanding of the mechanism did not prevent us from observing the reality.

For that matter, we knew that the orbit of Mercury did not match Newton's predictions for 80 years before Einstein solved the gravity problem.

Nonsense. The basic premise of the polygraph - that it says "Lie" whenever you are stressed - is a flawed premise.

The only misrepresentation is on your part.

And yet, you said "substantial".

Did you know that:

2003 National Academy of Sciences Report

The accuracy of the polygraph has been contested almost since the introduction of the device. In 2003, the National Academy of Sciences (NAS) issued a report entitled “The Polygraph and Lie Detection”. The NAS found that the majority of polygraph research was of low quality. After culling through the numerous studies of the accuracy of polygraph detection the NAS identified 57 that had “sufficient scientific rigor”. These studies concluded that a polygraph test regarding a specific incident can discern the truth at “a level greater than chance, yet short of perfection”. The report also concluded that this level of accuracy was probably overstated and the levels of accuracy shown in these studies "are almost certainly higher than actual polygraph accuracy of specific-incident testing in the field.”
Source (http://en.wikipedia.org/wiki/Polygraph)

?

if your claim is that a technology must be "perfect" to be not be woo

It isn't.

If you claim that only experiments done under field conditions can demonstrate the viability of a theory

I don't.

If you insist that only a fully-developed technology is not woo

I don't.

Despite the fact that the technology works much better in the lab than in the field, the simple fact that the technology works at all and can be reproduced at will makes it "not woo."

Can't agree there. The polygraph cannot be used to detect whether people lie or not. Simple as that.

I do not have personal knowledge of their beliefs. To the extent that they believe that polygraphs cannot distinguish lying from truth telling, then yes, they should change their minds.

Argue that at TAM. Write an article and send it to Skeptic Magazine or SkepticReport.

[drkitten]

Write an article and send it to Skeptic Magazine or SkepticReport.

Not a chance. I don't trust the editorial integrity of the SkepticReport editors. I think they're too strongly biased to give fair reading to articles, and they don't have the necessary background to understand the statistics.

[CFLarsen]

Not a chance. I don't trust the editorial integrity of the SkepticReport editors. I think they're too strongly biased to give fair reading to articles, and they don't have the necessary background to understand the statistics.

You don't write for the editor - singularis. You write for the audience.

I'll publish whatever you write as it is, completely unedited.

What about Skeptic Magazine, or TAM?

Why did you overstate the effectiveness of the polygraph?

Did you know that the report also:

concluded that this level of accuracy was probably overstated

?

[drkitten]

Y
Did you know that the report also:


concluded that this level of accuracy was probably overstated
?
?

And this is why I won't write for SkepticReport. The editors cherry-pick and misquote, out of context, nearly as badly as creationists do.

What, in context, is 'this level of accuracy"?

Your prior quotation, from Wikipedia, suggests that:


After culling through the numerous studies of the accuracy of polygraph detection the NAS identified 57 that had “sufficient scientific rigor”. These studies concluded that a polygraph test regarding a specific incident can discern the truth at “a level greater than chance, yet short of perfection”. The report also concluded that this level of accuracy was probably overstated and the levels of accuracy shown in these studies "are almost certainly higher than actual polygraph accuracy of specific-incident testing in the field.”

This misrepresents the NAS report at several levels.

First, "the studies" did not conclude that polygraphs could discern the truth at "a level greater than chance, yet short of perfection." That conclusion, as well as the quotation, is NAS's. (Report, p.4) Individual studies, of course, have their own individual estimates which vary widely as are reported in the appendix.

In that same paragraph, they compare the use of polygraph for two completely different purposes --- the use as a specific-incident investigation tool and the use as a screening tool. The report stated (p. 4) that "because actual screening applications involve considerably more ambiguity for the examinee and in determining truth than arises in specific-incident studies, polygraph accuracy for screening purposes is almost certainly lower than can be achieved by specific-incident polygraph tests in the field."

That says nothing about the accuracy of the polygraph per se, merely that the accuracy to be expected in screening situations is lower than that in specific-incident situations. It's an apples to oranges comparison, with us having a priori reason to believe that oranges are harder. To make the fact that oranges are believed to be harder into an indictment of apples is misrepresentation.

So, in direct answer to your question, I did not know that the report claimed that better-than-chance accuracy for polygraph testing is probably overstated, because it simply did not make that claim.

The closest to this is probably the section on p. 128 :


Theory and basic research give no clear guidance about whether laboratory conditions underestimate or overestimate the accuracy that can be expected in realistic settings.

Available data are inadequate to test these hypotheses. [...]

Evidence from Medical Diagnostic Testing. Substantial experience with clinical diagnostic and screening tests suggests that laboratory models, as well as observational field studies of the type found in the polygraph literature, are likely to overstate true polygraph accuracy.Much information has been obtained by comparing observed accuracy when clinical medical tests are evaluated during development with subsequent accuracy when they become accepted and are widely applied in the field. An important lesson is that medical tests seldom perform as well in general field use as their performance in initial evaluations seems to promise (Ransohoff and Feinstein, 1978; Nierenberg and Feinstein, 1988; Reid, Lachs, and Feinstein, 1995; Fletcher, Fletcher, and Wagner, 1996; Lijmer et al., 1999).

The reasons for the falloff from laboratory and field research settings to performance in general field use are fairly well understood.

In other words, the problem is with the specific numeric claims of accuracy, not with the "better than chance" and is to be expected in any transition from laboratory to field.

This is followed up (p. 129) by

In view of the above issues, we believe that the range of accuracy indexes (A) estimated from the scientifically acceptable laboratory and field studies, with a midrange between 0.81 and 0.91, most likely over-states true polygraph accuracy in field settings involving specific-incident investigations.

Thus, what is specifically overstated are the estimates (from 0.81 to 0.91), not the simple better-than-chance accuracy.

They specifically reiterate this point in their conclusion. After discussing (p148) a number of problems with measuring accuracy, including the overestimation problem to which they specifically mention (p. 149: "the accuracy index most likely overestimates performance in realistic field situations due to technical biases in field research designs, the increased variability created by the lack of control of test administration and interpretation in the field, the artificiality of laboratory settings, and possible publication bias.") they nevertheless continue (p. 149) with

Despite these caveats, the empirical data clearly indicate that for several populations of naïve examinees not trained in countermeasures, polygraph tests for event-specific investigation detect deception at rates well above those expected from random guessing. Test performance is far below perfection and highly variable across situations. The studies report accuracy levels comparable to various diagnostic tests used in medicine.

The study goes on to warn
We note, however, that the performance of medical diagnostic tests in widespread field applications generally degrades relative to their performance in validation studies, and this result can also be expected for polygraph testing
but this fact obviously has not reduced the usefulness of medical diagnostic tests "in widespread field applications" despite the marginally degraded performance accuracy.

It is therefore fair to say that the NAS is largely positive about the ability of polygraphs to distinguish lying from truth-telling under laboratory conditions, with naive subjects, in specific-incident investigations. Indeed, the NAS explicitly makes such a statement at least twice, on page 4 and page 149.

There is no statement in the entire body of the NAS report that suggest that, under such conditions, polygraph testing is no more accurate than chance, or even that polygraph testing is not significantly more accurate than chance. They do suggest that some of the specific claims of laboratory accuracy may not be reproducible under field conditions, but this is neither surprising nor does it make the technology invalid.

I stand by my writing. You, and Wikipedia, have misrepresented the content of the NAS report in a way to make it much less positive about the actual obtainable accuracy of polygraph testing. Given this history of editorial misrepresentation, there is no possible way that I would write for your journal.

[Skeptic Ginger]

Oh thank the flying spaghetti monsters some rational people have joined the discussion.

Lonewulf, 3point did not call you ignorant. But you are uninformed or uneducated about some of the principles involved in this discussion. As is Aepe. (I'll get to Claus in the next post).

---

First to Aepe (though I am repeating what has already been said, sometimes it helps to hear things in more than one way), we use many indirect measurements to tell us about something else. For example, I might measure my body temperature to tell me if I have an infection. I am not measuring infection, I am measuring body temperature.

I do need to validate that body temperature is a measure of infection. But once I do that, I may continue to measure body temperature and know that it is a reasonable indicator of infection.

So measuring emotional responses, once correlated with truthfulness, is a valid means of measuring truthfulness. The question becomes, not are you measuring truthfulness, but are the emotional reactions you are measuring a good indicator of truthfulness. While there is great variability in the studies of whether the emotional responses you measure on a polygraph are an indicator of truthfulness, the results are better than chance.

---

Lonewulf, it matters greatly what you use a test for when deciding if that test is useful. I screen people for HIV. The HIV antibody test is not the most accurate test in the world. I'll get to what we do about that next, but I need to discuss something else first.

Now, take the treatment for HIV. Is it very risky? Yes. Is not treating HIV very risky? Yes. So can I afford false positives and is there a better alternative? That also has to be considered if I am deciding to use an inaccurate test.

If I can't afford false positives the test may not be useful to determine who to treat, but it may still be useful to determine who to do more testing on. So even though the HIV antibody test is fairly inaccurate, it is still very useful to screen people and do further, more expensive tests on those who test positive.

In the case of the inaccurate polygraph, sending someone who is innocent to jail or firing them from their job and labeling them a spy is not acceptable. And there are alternative means of accomplishing your goals. So the inaccurate polygraph is not useful in court nor in weeding out spies. But that doesn't mean the polygraph, because it is inaccurate is totally useless.

Now suppose I wanted to use the inaccurate HIV test, not to determine who to treat and who not to treat, or who to test further, but to determine the rate of HIV in a group or the trend upward or downward of new infections? I can take the rate of false positives and do some calculations first.

In a low risk population, just due to statistical factors alone, more of the positives will be false. In a high risk population just due to statistical factors alone, more of the positives will represent true positives. Take this one step further to see the reason. In a population where everyone is infected with HIV, all of the positives will be true positives. And in a population where no one has HIV, all of the positives will be false positives.

With that background information, I can now test a population and analyze the results even with an inaccurate test.

Say I have a low risk population and a test that is only accurate 50% of the time, I test the group and get 2% positives. I can conclude from that, 1% of the population is infected. I can draw a conclusion about the group which I cannot draw about the individual. An inaccurate test is still very useful, depending on what you are using it for.

--

Now, with that said, I do agree that the polygraph is not as reliable as one would think. It did appear however, that a lot of the inaccuracy depended on the skill of the person tested in thwarting the results. That would make it even less accurate in a court or for weeding out spies since those would be cases where one would expect more people to be purposefully deceptive.

We would need a lot more information about the circumstances of the urban mythical 100 atheist tests to interpret the polygraph results. And since there is no evidence the urban myth is true, it is unlikely we can take this discussion on the polygraph results any further in that thread.

[drkitten]

It is therefore fair to say that the NAS is largely positive about the ability of polygraphs to distinguish lying from truth-telling under laboratory conditions, with naive subjects, in specific-incident investigations. Indeed, the NAS explicitly makes such a statement at least twice, on page 4 and page 149.

Goodness. I just found a third statement. On p. 178, the report reads (emphasis mine).


The available evidence indicates that in the context of specific-incident investigation and with inexperienced examinees untrained in countermeasures, polygraph tests as currently used have value in distinguishing truthful from deceptive individuals. However, they are far from perfect in that context, and important unanswered questions remain about polygraph accuracy in other important contexts. No alternative techniques are available that perform better, though some show promise for the long term. The limited evidence on screening polygraphs suggests that their accuracy in field use is likely to be somewhat lower than that of specific-incident polygraphs.

There is no suggestion that this "value" is in any way overstated, just the standard note that screening is harder than incident investigation.

In fact, they are very positive about polygraph use in a criminal investigation setting. (p 184)


Suppose that in a criminal investigation the polygraph is used on suspects who, on other grounds, are estimated to have a 50 percent chance of being guilty. For a test with A = 0.80 and a sensitivity of 50 percent, the false positive index is 0.23 and the positive predictive value is 81 percent. That means that someone identified by this polygraph protocol as deceptive has an 81 percent chance of being so, instead of the 0.4 percent (1 in 250) chance of being so if the same test is used for screening a population with a base rate of 1 in 1,000.

Note that the A value proposed, 0.80, is in-line with the estimates from the prior sections; they didn't just pull it out of thin air. But this "contrast[s] sharply" with the numbers from the screening problem : "a test that may look attractive for identifying deceptive individuals in a population with a base rate above 10 percent looks very much less attractive for screening a population with a very low base rate of deception. It will create a very large pool of suspect individuals, within which the probability of any specific individual being deceptive is less than 1 percent—and even so, it may not catch all the target individuals in the net. To put this another way, if the polygraph identifies 100 people as indicating deception, but only 1 of them is actually deceptive, the odds that any of these identified examinees is attempting to deceive are quite low, and it would take strong and compelling evidence for a decision maker to conclude on the basis of the test that this particular examinee is that 1 in 100." (p. 184).



Similarly, the report notes on p. 201 that "Although polygraphs clearly have utility in some settings, courts have been unwilling to conclude that utility denotes validity. The value of the test for law enforcement and employee screening is an amalgam of utility and validity, and the two are not easily separated." (emphasis mine.) As the report discusses in some detail, legal admissibility and scientific validity are separate (although related) concepts, and you can't simply say that something is invalid because it is inadmissible.

Finally, on p. 214, they present the ultimate statement of their finding on accuracy(emphasis in original).

[i]Notwithstanding the limitations of the quality of the empirical research and the limited ability to generalize to real-world settings, we conclude that in populations of examinees such as those represented in the polygraph research literature, untrained in countermeasures, specific-incident polygraph tests for event-specific investigations can discriminate lying from truth telling at rates well above chance, though well below perfection.


Given that, at this point, they have formally recognized that polygraphs work in lab conditions something like eight times in the report, I see no reason for you -- or for anyone -- to dispute the basic fact that they work (in lab conditions).

Which makes them decidedly not woo.

[mayday]

Hypothetically, say you asked 100 theists if they really believed in god and they all answered yes and the results showed they were all lying. If the polygraph gives valid results 80% of the time (without trying to fool it the report said the results were valid close to 100% of the time but say it was only 80%), then 80 theists would be lying. You could not tell which 80 were lying and which 20 were not lying but you could conclude 80 were lying based on the average validity of the polygraph results.

I really don't understand why the two of you don't understand the scientific concept of sensitivity and specificity.

It's an old cynic's trick called [I]raising the bar.[I]

No one will be able to provide a good enough explanation or amount of evidence.

[Skeptic Ginger]

Drkitten has done a better job addressing Claus that I could hope to.

I asked you Claus, to give your estimate of the accuracy of a polygraph. You can even describe the circumstances. Then we can get to the issue of how to interpret the mythical 100 atheists tests.

So with people trying to deceive the polygraph, the reliability appears abysmal.

With a skilled test administrator and a population not trying to fool the test, the results are better than chance and perhaps even as good as 80%. I am not going to attempt to offer an opinion as to how high that number is except to say the range is wide according to the extensive analysis by the GAO source I cited.

[drkitten]

It's an old cynic's trick called [I]raising the bar.[I]

As opposed to the old woo's trick called lying through your teeth?



No one will be able to provide a good enough explanation or amount of evidence.

I suspect that no one will even be able to provide a primary source indicating that the alleged experiment with atheists was done. Given the highly inethical nature of the experiment (the experiment as described is so far outside the parameters of a valid polygraph experiment it couldn't even telephone validity), I suspect it was either manufactured out of whole cloth (more likely), or a simple analysis of the testing organization's credentials will show that they have no relevant expertise.

[Skeptic Ginger]

It's an old cynic's trick called [I]raising the bar.[I]

No one will be able to provide a good enough explanation or amount of evidence.I purposefully asked Claus to take this to a new thread instead of hijacking yours. Do you think you could give him the same courtesy and keep the 100 atheists discussion in your thread?

[drkitten]

I asked you Claus, to give your estimate of the accuracy of a polygraph. You can even describe the circumstances.

Let's be fair. As posed, the question is not answerable. I can easily design a polygraph system that will have a 0% false negative rate -- i.e. it correctly detects ALL attempts at deception. It's a face-down piece of paper with "he's lying" written on one side, which I turn over at the end of the test. (Oddly enough, although it's got 100% sensitivity for lying, its ability to detect gender is only about 50%....)

Needless to say, under such circumstances, it would be very easy for me to get a group of 100 atheists and have them all test as liars. But not especially convincing.

In order to specify accuracy meaningfully, you need at least two estimates, one for the sensitivity and one for the specificity.

[Skeptic Ginger]

It isn't answerable as snipped. Perhaps I wasn't clear, I was asking for any hypothetical situation he cared to give that involved interpreting the results of a group of polygraph answers as opposed to a single answer. I am talking about what a polygraph can tell you about a group of tests and Claus is talking about what a polygraph can tell you about a single test. I agree with him on the single test. But his one track mind has stopped him from ever getting to the group of tests question.

I just want him to get to the same conversation as I'm having with whichever hypothetical numbers he wants to use. It's like pulling teeth to get him off his single minded discussions.

[Jimbo07]

Is the validity of the polygraph really the point?


With a skilled test administrator

*cough* interview theater *cough*

[Skeptic Ginger]

What's your point, Jim?

Let me sum up the point of the thread for you. I made a statement in another thread that even though you cannot use a polygraph result to determine if an individual is deceptive, you can use it to draw a conclusion about a group of people being tested. If the results are that 100 people are being deceptive per the results of the polygraph, and the polygraph is accurate 80% of the time, then you can say that at least 80% of those people are likely being deceptive.

I did not declare what the actual percentage of valid results were because we don't have the details of the 100 tests by which to determine that including knowledge of the skills of the test administrator. I used 80% hypothetically and provided a source which did a very thorough scientific meta-analysis (GAO report to Congress to determine the usefulness of using the polygraph to weed out spies). That source found a range of reliability from very high to very low. The intent of the person being tested to deceive the tester greatly decreased the reliability of the results among other factors.

For whatever reason, Lonewulf and Claus didn't get the concept of using a test that was 80% reliable to draw a conclusion about 80% of 100 test results. And I don't get the problem. If Claus or Lonewulf wanted to argue that 50% was a better average estimate, I wouldn't argue. We can't say unless we know more about the 100 tests. But to say, you cannot use any figure to draw a conclusion about the group test results because an individual test result is not reliable is not a correct statement. Claus may be trying to make the case that the polygraph is never better than chance. I don't think he has provided any source supporting that claim.

[CFLarsen]

And this is why I won't write for SkepticReport. The editors cherry-pick and misquote, out of context, nearly as badly as creationists do.

I don't think you read what I said. I specifically said that I would publish what you wrote, complete unedited.

What about Skeptic Magazine? TAM?

This misrepresents the NAS report at several levels.

Fine. Write that in your article/presentation.

Drkitten has done a better job addressing Claus that I could hope to.

I asked you Claus, to give your estimate of the accuracy of a polygraph. You can even describe the circumstances. Then we can get to the issue of how to interpret the mythical 100 atheists tests.

So with people trying to deceive the polygraph, the reliability appears abysmal.

With a skilled test administrator and a population not trying to fool the test, the results are better than chance and perhaps even as good as 80%. I am not going to attempt to offer an opinion as to how high that number is except to say the range is wide according to the extensive analysis by the GAO source I cited.

How do you know the polygraph is correct any of the time?

Will you present a paper at TAM?

[.13.]

CFLarsen, you need to step back and think about this. Statistical tests can tell you how well an imperfect test works. We do it all the time:

1) A self-proclaimed psychic says "I can psychically read Zener cards while blindfolded." We draw 100 known Zener cards and as the psychic to guess them. Then we compare each guess to the card. At the end of the test we can say "The psychic got 70% of these guesses right". In the future, if the psychic guesses at an unknown Zener card, we make an inference: "The psychic has a 70% chance of getting this guess right."

Really?

[Cuddles]

Really?

yes.

[Philip]

Sorry, skeptigirl and drkitten, if the polygraph is not 100% effective in detecting lies by individuals or if you're unwilling to write an article about the polygraph for Skeptic Magazine or make a presentation about it at TAM, your entire argument is worthless and the polygraph is useless. :rolleyes:

[CFLarsen]

I did not declare what the actual percentage of valid results were because we don't have the details of the 100 tests by which to determine that including knowledge of the skills of the test administrator. I used 80% hypothetically and provided a source which did a very thorough scientific meta-analysis (GAO report to Congress to determine the usefulness of using the polygraph to weed out spies). That source found a range of reliability from very high to very low. The intent of the person being tested to deceive the tester greatly decreased the reliability of the results among other factors.

This is what you said:

I did but if you didn't catch it, here it is again, the error rate in polygraphs would not be 100%, in fact it would be close to zero. Polygraphs are not reliable enough to convict people in a court of law, nor should they be used to fire people. But if you asked 100 atheists if they believed in god and you got 100 results indicating they were lying, then most of them would be lying. Only a few wouldn't be.

Please provide the scientific evidence that the error rate in polygraphs is close to zero.

This is what this thread is entirely about.

[.13.]

yes.

How can you do that?

[drkitten]

I don't think you read what I said. I specifically said that I would publish what you wrote, complete unedited.

You did not, however, promise that you would not later distort, misrepresent, or lie about what I wrote.

And even if you made that promise, I have no reason to believe that you would keep it.


How do you know the polygraph is correct any of the time?

Any of the time? Be serious.

We know it's been correct at least three times, because it correctly identified Kari as telling the truth, and Tore and Grant as lying.

[Jimbo07]

What's your point, Jim?

Let me sum up the point of the thread for you. I made a statement in another thread that even though you cannot use a polygraph result to determine if an individual is deceptive, you can use it to draw a conclusion about a group of people being tested. If the results are that 100 people are being deceptive per the results of the polygraph, and the polygraph is accurate 80% of the time, then you can say that at least 80% of those people are likely being deceptive.

My point is that this is an interesting discussion. This isn't what polygraphs are typically used for, nor to my (albeit limited) understanding, how they're used.


(GAO report to Congress to determine the usefulness of using the polygraph to weed out spies). That source found a range of reliability from very high to very low.

The problem with elected leaders in democracies is that they may be inexpert in things other than how to get elected. Therefore, various branches have to come up with explanations for or against the (possibly random) ideas of those elected officials.

I have no idea how they might be used in a National Security setting. However, in law enforcement I believe that the effectiveness of the machine itself is somewhat irrelevant. I've been watching this thread, hoping that someone with expertise in law enforcement will show up to either confirm or deny this. There are many more factors at play than someone either simply trying to deceive the machine or not. What about those who believe they have done nothing wrong (apparently common)? Interviewing is both a science and an art.

Interestingly, around here, it's the police forces who are using polygraphs as part of their screening processes. I'd bet you a dollar (Canadian currency, now ;)) that the examiners are more interested in watching the face of the subject than the needle on the kerjigger.


Claus may be trying to make the case that the polygraph is never better than chance. I don't think he has provided any source supporting that claim.

I agree. I also think the claims for evidence at TAM are kinda weird. Is this some kind of controversial debate going on in the States right now? Is there a plan to take incompetent mouth-breathers off the street, put "lie detectors" in their hands and conduct national security screenings? I guess I could make a joke about not being surprised... ;)

[Big Les]

No, the demands to present a TAM paper/write a skepticreport article are all rather unusual debate tactics that Claus uses in general. Some sort of implied appeal to authority - that the true big name sceptics would laugh the arguer out of the room I suppose. You'd have to ask him.

[remirol]

No, the demands to present a TAM paper/write a skepticreport article are all rather unusual debate tactics that Claus uses in general. Some sort of implied appeal to authority - that the true big name sceptics would laugh the arguer out of the room I suppose. You'd have to ask him.

I've seen a variant of it that could be called the "gambler's proof": where you ask someone to bet a very large amount of money about the truth/validity of a given issue. Generally, the attacker sets the amount so high such that he can be confident the opposing party either doesn't have that much money available, or won't be willing to risk it on something so (comparably) trivial. It also requires that the validity of the issue in question cannot be determined easily, such that neither party can be certain of the truth prior to accepting the bet.

(Example: "If you're so sure the Patriots will win the Super Bowl, you should be willing to bet your house on it. Let's bet your house against mine, OK?")

In some instances, it can be used effectively to demonstrate the depth of someone's conviction in their stated ideas, but has no bearing whatsoever on determining the validity of the ideas themselves. It is generally a tactic used to attempt to stop discussion of a topic and force a concession to the attacker's viewpoint.

The requests to write a paper and/or present at TAM in this thread are similar.

[CFLarsen]

You did not, however, promise that you would not later distort, misrepresent, or lie about what I wrote.

And even if you made that promise, I have no reason to believe that you would keep it.

That would only be evidence of my dishonesty. What have you got to lose?

What about Skeptic Magazine? TAM?

You keep evading this.

Any of the time? Be serious.

We know it's been correct at least three times, because it correctly identified Kari as telling the truth, and Tore and Grant as lying.

Because we know from other sources that they did lie, yes.

I agree. I also think the claims for evidence at TAM are kinda weird. Is this some kind of controversial debate going on in the States right now? Is there a plan to take incompetent mouth-breathers off the street, put "lie detectors" in their hands and conduct national security screenings? I guess I could make a joke about not being surprised... ;)

No, the demands to present a TAM paper/write a skepticreport article are all rather unusual debate tactics that Claus uses in general. Some sort of implied appeal to authority - that the true big name sceptics would laugh the arguer out of the room I suppose. You'd have to ask him.

It's perfectly simple: That's where the world's best skeptics are. It's a unique opportunity to really have skeptics change their minds, based on scientific evidence. If the evidence is there, of course.

As you can see, both Shermer (http://www.skeptic.com/eskeptic/07-10-03.html#video) and Bob Carroll (http://skepdic.com/polygrap.html) argue that polygraphs don't work. Randi (http://www.randi.org/jr/042602.html) does too, of course.

They will certainly be very interested in the evidence. As will the rest of the audience.

[drkitten]

Because we know from other sources that they did lie, yes.

Exactly. And that's how the various studies cited in the NAS report determined the accuracy and ROC curves for their polygraphs; they compared the results of the polygraph examination to the ground truths that they knew from other sources. (And, yes, they know about things like "blinding""; the polygraph examiner can't be the one who knows the ground truths.)

This is hardly rocket science; it's standard scientific practice and has been for literally decades. It's also how Randi knows that dowsers don't work -- he knows where the water is and he compares it to where the dowsers tell him that it is.

The difference is, in Randi's studies, the dowsers' performance is generally within the chance expectations --- it has never been beyond the rather stringent alpha cutoff of 0.001. The polygraph tests appear to be well beyond the threshhold of accuracy established by chance.

[rwguinn]

skeptigirl and drkitten (and you other supporters of statistical accuracy)

This is all very interesting, and a nice laboratory trick, but Polygraph testing is not done on groups--it is done on individuals.
A single false positive is one too many.

Our system on laws is based on the theory (but not always the practice) that is is better to allow 100 guilty to go free than to imprison 1 innocent.
Polygraph testing has just the opposite effect.

Statistical analysis is wonderful stuff, used properly. Statistically, the probability of you getting injured by a drunk driver after 2:00 AM is .0001 (a number I made up), or 1/10000. Good odds, yes? Unless you happen to be that 1.

[Jimbo07]

It's perfectly simple: That's where the world's best skeptics are. It's a unique opportunity to really have skeptics change their minds, based on scientific evidence. If the evidence is there, of course.

As you can see, both Shermer (http://www.skeptic.com/eskeptic/07-10-03.html#video) and Bob Carroll (http://skepdic.com/polygrap.html) argue that polygraphs don't work. Randi (http://www.randi.org/jr/042602.html) does too, of course.

They will certainly be very interested in the evidence. As will the rest of the audience.

Change their minds about what?

Unfortunately, I can't view videos at this time. I'm not sure what position they are arguing, per se. Are they claiming that polygraphs are not useful in law-enforcement? Are they claiming that they are not magic lie detectors?

I guess I'm just coming too late to the issue. I'm still hoping that someone with a law enforcement background will show up and either confirm or deny my (what must seem like a random) claim that: polygraphs are a law-enforcement interview tool (or even technique), not magic lie detectors.

[Aoidoi]

You know, it wouldn't suprise me if the Polygraph does work better than chance at determining a 50/50 proposition under some circumstances. My main problem with it is that it's practically the opposite of double blind. It all comes down to an operator making the call. How does one eliminate operator bias in this situation?

In your (apparently apocryphal, wish I'd realized that earlier cause I was thinking people were actually supporting that factoid as real) 100 athiest test, would your operator be an athiest or a theist? How could you tell, ask them? They could be lying. ;)

Run multiple runs with multiple operators of different backgrounds and compare results? What if the operator is racist? Or sexist? Or just likes to say every prime number candidate is lying because he can? While I can see a polygraph being a potentially effective tool for interrogation (better than electrodes on the naughty bits, at least), it seems like the entire premise is baised on perfectly rational and unbiased operators. There ain't no such animal.

Operator issues aside, you can certainly draw statistical conclusions from imperfect tests. The problem is I don't believe that polygraphs have a known error rate for beliefs. They don't detect "lies" they detect when someone is stressed about something. As they pointed out in mythbusters, there has to be a reason for there to be stress for it to work, having actually stolen something or having a punishment/reward for fooling the test. How can you seperate out "I'm lying about my belief" from "I was beaten as a child for not going to church" or "I was molested by a priest?"

If you asked 100 people who say they're heterosexual if they're lying about being straight, would you trust the answers? Sexuality, religion, politics... these are things that people get emotional about.

Or I could totally be blowing smoke, guess you'll have to hook me up to a machine to find out. I'd like my polygraph operator to be Natalie Portman, please. You won't get good results, but I'll enjoy it. ;)

[skepHick]

I've seen a variant of it that could be called the "gambler's proof": where you ask someone to bet a very large amount of money about the truth/validity of a given issue. Generally, the attacker sets the amount so high such that he can be confident the opposing party either doesn't have that much money available, or won't be willing to risk it on something so (comparably) trivial. It also requires that the validity of the issue in question cannot be determined easily, such that neither party can be certain of the truth prior to accepting the bet.

(Example: "If you're so sure the Patriots will win the Super Bowl, you should be willing to bet your house on it. Let's bet your house against mine, OK?")

In some instances, it can be used effectively to demonstrate the depth of someone's conviction in their stated ideas, but has no bearing whatsoever on determining the validity of the ideas themselves. It is generally a tactic used to attempt to stop discussion of a topic and force a concession to the attacker's viewpoint.

The requests to write a paper and/or present at TAM in this thread are similar.

I interpret it this way, as well. An extreme variant of the old "put up or shut up." It certainly doesn't promote discussion. Someone's willingness, or lack thereof, to write an article or make a presentation at TAM doesn't speak to the merits of their position. This is a discussion forum, for Pete's sake. There are acceptable levels of evidence that one could provide here that are well below doing either of these things. Especially the TAM presentation. Even if someone is willing and able to go to that extreme, and even if their position is deemed valid, that doesn't mean it will be accepted as a topic. So what would that prove, exactly?

Oh, and the Pats are SO going to win the Super Bowl! :D

[drkitten]

You know, it wouldn't suprise me if the Polygraph does work better than chance at determining a 50/50 proposition under some circumstances. My main problem with it is that it's practically the opposite of double blind. It all comes down to an operator making the call. How does one eliminate operator bias in this situation?

Well, one standard way is to have as much of the data processed by computer (objectively) as possible. If the machine shows that on standardized data set alpha-niner-four, it was possible to draw a separating hyperplane between the truthful and non-truthful candidates' measurements, and that my measurement is on the lying side of that hyperplane, it's hard to attribute that to bias. We may argue about whether alpha-niner-four is a representative control group, but there's not much to argue about involving the categorization itself.

Another way is simply to look at the evidence upon which the operator makes his/her decision and see if it appears reasonable. If the operator says "you're lying," that doesn't mean much. But if the operator says "your skin temperature says you're lying," then there should be a visible difference in the graphs, specifically in the one for skin temperature. If the operator gives an opinion, but doesn't have a basis for the opinion, then one could rightly disregard that particular opinion -- but that's hardly unique to polygraphs. This problem has been around in "expert" testimony for centuries. When the doctor says that such-and-such a person died of heart disease and not pneumonia, you expect him to have citable medical reasons for her opinion.


Operator issues aside, you can certainly draw statistical conclusions from imperfect tests. The problem is I don't believe that polygraphs have a known error rate for beliefs. They don't detect "lies" they detect when someone is stressed about something. As they pointed out in mythbusters, there has to be a reason for there to be stress for it to work, having actually stolen something or having a punishment/reward for fooling the test. How can you seperate out "I'm lying about my belief" from "I was beaten as a child for not going to church" or "I was molested by a priest?"

You don't need to, unless you're suggesting that so many people were molested by priests that they constitute a significant minority of the target population, on the same order of magnitude as the detection threshhold. In some cases, this may be the case (e.g. if you're doing this experiment at a "survivors of priestly molestation" convention). But otherwise, any reasonable estimate of the probability of this would put it down in the "lost in the noise" section of the spectrum.



If you asked 100 people who say they're heterosexual if they're lying about being straight, would you trust the answers?

The first and only time? Possibly not. The thirtieth time over the course of two hours? Quite probably. As I said earlier, the emotional response from being asked essentially the same nosy question over and over by the same person bleaches quite quickly.

[NeilC]

skeptigirl and drkitten (and you other supporters of statistical accuracy)

This is all very interesting, and a nice laboratory trick, but Polygraph testing is not done on groups--it is done on individuals.
A single false positive is one too many.

Our system on laws is based on the theory (but not always the practice) that is is better to allow 100 guilty to go free than to imprison 1 innocent.
Polygraph testing has just the opposite effect.

Statistical analysis is wonderful stuff, used properly. Statistically, the probability of you getting injured by a drunk driver after 2:00 AM is .0001 (a number I made up), or 1/10000. Good odds, yes? Unless you happen to be that 1.

But we're not talking about the legal use of it. In fact everyone seems to agree it shouldn't be used to convict people.

And the supposed experiment that spawned this debate was (well wasn't in fact) done on a group. So if you could demonstrate a reasonable but far from perfect accuracy to the polygraph then the results would have some statistical significance.

[CFLarsen]

I've seen a variant of it that could be called the "gambler's proof": where you ask someone to bet a very large amount of money about the truth/validity of a given issue. Generally, the attacker sets the amount so high such that he can be confident the opposing party either doesn't have that much money available, or won't be willing to risk it on something so (comparably) trivial. It also requires that the validity of the issue in question cannot be determined easily, such that neither party can be certain of the truth prior to accepting the bet.

(Example: "If you're so sure the Patriots will win the Super Bowl, you should be willing to bet your house on it. Let's bet your house against mine, OK?")

In some instances, it can be used effectively to demonstrate the depth of someone's conviction in their stated ideas, but has no bearing whatsoever on determining the validity of the ideas themselves. It is generally a tactic used to attempt to stop discussion of a topic and force a concession to the attacker's viewpoint.

The requests to write a paper and/or present at TAM in this thread are similar.

That is a preposterous idea.

First, there is no money involved. Nobody will lose anything. Second, this is not an attempt to stop a discussion, quite contrary: The thread is opened specifically to get the evidence out in the open, to a much bigger audience.

Exactly. And that's how the various studies cited in the NAS report determined the accuracy and ROC curves for their polygraphs; they compared the results of the polygraph examination to the ground truths that they knew from other sources. (And, yes, they know about things like "blinding""; the polygraph examiner can't be the one who knows the ground truths.)

This is hardly rocket science; it's standard scientific practice and has been for literally decades. It's also how Randi knows that dowsers don't work -- he knows where the water is and he compares it to where the dowsers tell him that it is.

The difference is, in Randi's studies, the dowsers' performance is generally within the chance expectations --- it has never been beyond the rather stringent alpha cutoff of 0.001. The polygraph tests appear to be well beyond the threshhold of accuracy established by chance.

Save it for your article/presentation.

What have you got to lose by submitting an article to SkepticReport?

What about Skeptic Magazine? TAM?

You keep evading this.

skeptigirl and drkitten (and you other supporters of statistical accuracy)

This is all very interesting, and a nice laboratory trick, but Polygraph testing is not done on groups--it is done on individuals.
A single false positive is one too many.

Our system on laws is based on the theory (but not always the practice) that is is better to allow 100 guilty to go free than to imprison 1 innocent.
Polygraph testing has just the opposite effect.

Statistical analysis is wonderful stuff, used properly. Statistically, the probability of you getting injured by a drunk driver after 2:00 AM is .0001 (a number I made up), or 1/10000. Good odds, yes? Unless you happen to be that 1.

Exactly. Calling it a "laboratory trick" is very spot on.

Change their minds about what?

Unfortunately, I can't view videos at this time. I'm not sure what position they are arguing, per se. Are they claiming that polygraphs are not useful in law-enforcement? Are they claiming that they are not magic lie detectors?

I guess I'm just coming too late to the issue. I'm still hoping that someone with a law enforcement background will show up and either confirm or deny my (what must seem like a random) claim that: polygraphs are a law-enforcement interview tool (or even technique), not magic lie detectors.

Did you read the links?

I interpret it this way, as well. A extreme variant of the old "put up or shut up." It certainly doesn't promote discussion. Someone's willingness, or lack thereof, to write and article or make a presentation at TAM doesn't speak to the merits of their position. This is a discussion forum, for Pete's sake. There are acceptable levels of evidence that one could provide here that are well below doing either of these things. Especially the TAM presentation. Even if someone is willing and able to go to that extreme, and even if their position is deemed valid, that doesn't mean it will be accepted as a topic. So what would that prove, exactly?

Oh, and the Pats are SO going to win the Super Bowl! :D

It is precisely why I want to take it outside this forum: It has gone way beyond a discussion. We are talking about skeptics claiming evidence that lie detectors works.

That's a whole different ballgame.

[drkitten]

This is all very interesting, and a nice laboratory trick, but Polygraph testing is not done on groups--it is done on individuals.
A single false positive is one too many.

Huh? How.... "interesting."

Surely that depends on the actions you take w.r.t. the false positive, doesn't it?

As a simple example, if you're using a polygraph to investigate a specific criminal incident with small group of suspects, the results can be used to eliiminate (tentatively) a number of people who are likely to be innocent, so that you can concentrate the police resources more effectively. One robbery, ten suspects, eight of whom "pass" the polygraph test and are thus cleared. The cops still don't have enough evidence to convict anyone -- but they now only have two suspects to investigate in detail instead of ten, so they can use only 20% of the manpower and time. The alternative to the polygraph investigation would be to investigate all ten in detail -- in this case, the "false positive" will be subjected to exactly the same investigation that he would have been in any case. However, eight people have been spared that nuisance --- and eight cops are free to work on other crimes.

You might as well complain that blood typing should not be used, because of the possibility of spurious matches. If we know the robber's blood is type B negative, we still don't know who did it. Maybe there are three people with B negative blood in the suspect pool. But we do know it wasn't me (I'm O positive), and that's a step forward, yes?



Our system on laws is based on the theory (but not always the practice) that is is better to allow 100 guilty to go free than to imprison 1 innocent.
Polygraph testing has just the opposite effect.

Which is why people aren't imprisoned on polygraph evidence alone. Indeed, since polygraphs aren't usually admissible in (US) courts (by fiat), polygraph testing cannot be used even as part of a case to imprison someone.

[NeilC]

CFLarson: this whole TAM thing is irrelevant and just makes the discussion irritating and increasingly boring to follow. Why not drop it? No-one is evading anything. No-one is bound by you or anyone else to present anything at TAM.

If people have evidence or interpretations of evidence then they can "present" it here and we can discuss it. Hopefully like adults.

[rwguinn]

But we're not talking about the legal use of it. In fact everyone seems to agree it shouldn't be used to convict people.

And the supposed experiment that spawned this debate was (well wasn't in fact) done on a group. So if you could demonstrate a reasonable but far from perfect accuracy to the polygraph then the results would have some statistical significance.
I guess that is the point:
It's a nice lab trick, but from a practical standpoint, not particularly useful. The problem is that once people get it in their minds that it is 80%, or 90% (or whatever you want to go with) accurate, those numbers will be used to convince juries/employers/investigators that the value is as valid for individuals as for groups.
Therein lies the danger...

[drkitten]

What have you got to lose by submitting an article to SkepticReport?

My time and trouble, combined with the high probability that you will misrepresent what I write in such a way as to attribute to me opinions that I do not hold, to the general detriment of "the truth." Basically, the same reason that Dawkins doesn't write guest editorials to the Journal of Creation Science. He's got more important calls on his time.

I do write on this subject (well, I have not written on polygraphs specifically, but on the general problem of the forensics of moderately accurate information), and at length; indeed, I hope to have a book on this subject out in the next several months. I see no reason to give valuable "previously unpublished" that I can use in a professional setting to a cat-box liner run by an editor of questionable integrity and with an overinflated ego. (And if you think SkepticReport is anywhere near as useful or as valuable a publication medium to me as the Journal of Forensic Sciences, that's proof enough of ego.)

Make it worth my while. Explain why SR should trump JFS.

[Jimbo07]

Did you read the links?


This?


On the other hand, it is possible that one of the main reasons so many government, law enforcement and private sector employers want to use polygraphs is that they think the test will frighten away liars and cheats who are seeking jobs, or it will frighten confessions out of those accused of wrongdoing. In other words, the users of the machine don't really believe it can detect lies, but they know that the people they administer it to think the machine can catch them in a lie. So, the result is the same as if the test really worked: they don't hire the liar/cheat and they catch the dishonest employee.

Ding! I feel justified that at least my assertions aren't coming totally out of my rear end (I know they aren't, but now you have some evidence, too). The polygraph is not the only technique that law-enforcement officials use to trip up interviewees. It is one tool. There have been interviewees charged using techniques much less extreme (read: expensive, in operator time, etc.).

A really neat psychological experiment would be to see which interview techniques work best. As I suggested earlier, interviews are highly subjective. This would add science, but it's also an art.

The rest of that article, btw, looks like a rant about American politics. The FBI failed to learn this, pharma companies can do that, etc. :( I'm back to asking, what is being opposed, here?

[Jekyll]

The first and only time? Possibly not. The thirtieth time over the course of two hours? Quite probably. As I said earlier, the emotional response from being asked essentially the same nosy question over and over by the same person bleaches quite quickly.

Shouldn't the same be true of lying in response to a nosy question?
I can see hew generating and maintaining a consistent lie on the fly could be more stressful than telling the truth, but we're just talking about repeating a single answer in response to a question.

[drkitten]

I guess that is the point:
It's a nice lab trick, but from a practical standpoint, not particularly useful. The problem is that once people get it in their minds that it is 80%, or 90% (or whatever you want to go with) accurate, those numbers will be used to convince juries/employers/investigators that the value is as valid for individuals as for groups.
Therein lies the danger...

Why single polygraph testing out, though?

Every forensic technology has this problem. Even ballistics and toolmarks. Telling me that you can distinguish size 8 from size 9 shoes with 100% accuracy is very important, and telling me that there is a 100% chance that the burglar wears size 9 is very important. And that's much better than polygraph testing will ever be able to offer in terms of confidence....

So what am I supposed to do when the DA steps from "we know that the burglar wears size 9 shoes, and we know that rwguinn wears size 9 shoes" to "rwguinn must be the burglar"?

Are you going to blame the toolmarks guy, or the incompetent DA?

[Cardelitre]

I guess that is the point:
It's a nice lab trick, but from a practical standpoint, not particularly useful. The problem is that once people get it in their minds that it is 80%, or 90% (or whatever you want to go with) accurate, those numbers will be used to convince juries/employers/investigators that the value is as valid for individuals as for groups.
Therein lies the danger...
This looks a lot like an argumentum ad consequentiam (http://en.wikipedia.org/wiki/Appeal_to_consequences)...

[drkitten]

Shouldn't the same be true of lying in response to a nosy question?

Perhaps it should be, but it isn't. That's an empirical question, and, indeed, one of the key questions underlying "do polygraphs work."

Since they demonstrably work, the answer appears to be "no, the same isn't true for lying in response to a nosy question." For evidence, see the bibliography of the NAS report we've been discussing.

[CFLarsen]

CFLarson: this whole TAM thing is irrelevant and just makes the discussion irritating and increasingly boring to follow. Why not drop it? No-one is evading anything. No-one is bound by you or anyone else to present anything at TAM.

If people have evidence or interpretations of evidence then they can "present" it here and we can discuss it. Hopefully like adults.

No, it's very relevant. Remember that we are skeptics - and skeptics must follow the evidence, regardless if it means that we would have to change our minds.

That's what skepticism is all about.

I guess that is the point:
It's a nice lab trick, but from a practical standpoint, not particularly useful. The problem is that once people get it in their minds that it is 80%, or 90% (or whatever you want to go with) accurate, those numbers will be used to convince juries/employers/investigators that the value is as valid for individuals as for groups.
Therein lies the danger...

There is also the opposite to consider: If the evidence really is strong enough (as skeptigirl claims), then we cannot refrain from using it.

A tool which will give us a very high reliability - or in drkitten's case, substantially above chance - on detecting lies?

Sorry, that's too valuable not to use.

My time and trouble, combined with the high probability that you will misrepresent what I write in such a way as to attribute to me opinions that I do not hold, to the general detriment of "the truth." Basically, the same reason that Dawkins doesn't write guest editorials to the Journal of Creation Science. He's got more important calls on his time.

I do write on this subject (well, I have not written on polygraphs specifically, but on the general problem of the forensics of moderately accurate information), and at length; indeed, I hope to have a book on this subject out in the next several months. I see no reason to give valuable "previously unpublished" that I can use in a professional setting to a cat-box liner run by an editor of questionable integrity and with an overinflated ego. (And if you think SkepticReport is anywhere near as useful or as valuable a publication medium to me as the Journal of Forensic Sciences, that's proof enough of ego.)

Make it worth my while. Explain why SR should trump JFS.

If you don't want to write for SkepticReport, what about Skeptic Magazine? Or present it at TAM?

This?

Ding! I feel justified that at least my assertions aren't coming totally out of my rear end (I know they aren't, but now you have some evidence, too). The polygraph is not the only technique that law-enforcement officials use to trip up interviewees. It is one tool. There have been interviewees charged using techniques much less extreme (read: expensive, in operator time, etc.).

A really neat psychological experiment would be to see which interview techniques work best. As I suggested earlier, interviews are highly subjective. This would add science, but it's also an art.

The rest of that article, btw, looks like a rant about American politics. The FBI failed to learn this, pharma companies can do that, etc. :( I'm back to asking, what is being opposed, here?

What about the rest of the links?

[Belz...]

Hypothetically, say you asked 100 theists if they really believed in god and they all answered yes and the results showed they were all lying. If the polygraph gives valid results 80% of the time (without trying to fool it the report said the results were valid close to 100% of the time but say it was only 80%), then 80 theists would be lying. You could not tell which 80 were lying and which 20 were not lying but you could conclude 80 were lying based on the average validity of the polygraph results.

I really don't understand why the two of you don't understand the scientific concept of sensitivity and specificity.

I was under the impression that the polygraph didn't detect "lies" but, specifically, body functions. If someone's very nervous or very calm, doesn't that kinda defeat the whole thing ?

[Belz...]

I cannot reply to either of you because I don't for the life of me get your issues.

Are you talking about testing one person or the average results of 100 tests?

Hello.

100 flawed tests average wrong.

ETA: Never mind, I get the statistics thingy...

[rwguinn]

Why single polygraph testing out, though?

Every forensic technology has this problem. Even ballistics and toolmarks. Telling me that you can distinguish size 8 from size 9 shoes with 100% accuracy is very important, and telling me that there is a 100% chance that the burglar wears size 9 is very important. And that's much better than polygraph testing will ever be able to offer in terms of confidence....

So what am I supposed to do when the DA steps from "we know that the burglar wears size 9 shoes, and we know that rwguinn wears size 9 shoes" to "rwguinn must be the burglar"?

Are you going to blame the toolmarks guy, or the incompetent DA?
Well--
In your example, you cannot even rule out people with smaller feet (or to a certain extent, larger feet) because a small foot can go inside a big shoe, and a bigger foot can be somehat squeezed into a smaller shoe. So you neefd more data.
In acourt of law, there will be other evidence--not just shoe size but sole pattern, my fingerprints on the window, i was seen in the area at about that time, nobody could support my contention as to my whereabouts, mud matching the location, pawnbroker saying I sold him the jewells,etc--all of which make the shoe size very insignificant. A polygraph would require the same level of proof--supporting evidence, and probable cause to direct an investigation.


and as far as argumentum ad consequentiam (http://en.wikipedia.org/wiki/Appeal_to_consequences) goes--in this case I don't believeit is applicable. My argument is not a logical falacy, it is just an evaluation of human nature and greed. From my reading, and from what I know, people have lost jobs and had careers derailed --all from a single polygraph test.

[Rob Lister]

As a simple example, if you're using a polygraph to investigate a specific criminal incident with small group of suspects, the results can be used to eliiminate (tentatively) a number of people who are likely to be innocent, so that you can concentrate the police resources more effectively. One robbery, ten suspects, eight of whom "pass" the polygraph test and are thus cleared. The cops still don't have enough evidence to convict anyone -- but they now only have two suspects to investigate in detail instead of ten, so they can use only 20% of the manpower and time. The alternative to the polygraph investigation would be to investigate all ten in detail -- in this case, the "false positive" will be subjected to exactly the same investigation that he would have been in any case. However, eight people have been spared that nuisance --- and eight cops are free to work on other crimes.


More to the point, the one guilty person would likely confess. As a "lie detector", the lie detector has no value in terms of admissible evidence, some value in terms of culling (as you alluded), and a great deal of value in terms of getting a confession. That value greatly overshadows all others.

This is off-topic but the primary use of lie-detectors by police agencies (as opposed to other 'agencies') is one of interrogation. It is a tremendously effective intimidation tactic, given a skilled operator [interrogator] and a naive subject. Often, that's all you need. Additionally, it is a form of interrogation that is completely free of attorney representation and, even though the results are inadmissible in court, the things said during the test are completely admissible. Often that's all you need. It reminds me of the case of the run-away bride (do you remember it) wherein they asked the boy friend to take a lie detector test. He agreed under the condition that it would be video-taped and he could have his lawyer present. They, of course, declined...bitching all the way.

[Jimbo07]

What about the rest of the links?

The first was a video, which I can't watch right now. I've read the second. Like DrK, I'm coming up on a barrier of time I want to invest. Please just answer me.

What is the issure here? What is being contested? Polygraphs only? Polygraph interviews? Abuse of authority?

ETA: Thank you, Rob Lister.

[drkitten]

I was under the impression that the polygraph didn't detect "lies" but, specifically, body functions. If someone's very nervous or very calm, doesn't that kinda defeat the whole thing ?

No. It doesn't respond directly to body functions, but to changes in body functions. So if you're very nervous at the start, but you get even more nervous when asked a question related to the incident in question, you will be flagged. If you're very nervous at the start, but are uniformly nervous throughout the interview, you will probably pass.

Now, you're right that someone with superhuman "nerves of steel" could almost certainly pass a polygraph by simply willing his body to display no signs of stress whatsoever. Such people appear to be relatively rare in the general population, as do the "nervous Nellies." As to whether or not you can be trained to become such a superman --- that was one of the issues that the NAS was concerned about.

[drkitten]

If you don't want to write for SkepticReport, what about Skeptic Magazine? Or present it at TAM?

What about JFS? Why is it worse to present the material in a specialist journal with higher readership and greater impact?

[drkitten]

Well--
In your example, you cannot even rule out people with smaller feet (or to a certain extent, larger feet) because a small foot can go inside a big shoe, and a bigger foot can be somehat squeezed into a smaller shoe. So you neefd more data.
In acourt of law, there will be other evidence--not just shoe size but sole pattern, my fingerprints on the window, i was seen in the area at about that time, nobody could support my contention as to my whereabouts, mud matching the location, pawnbroker saying I sold him the jewells,etc--all of which make the shoe size very insignificant. A polygraph would require the same level of proof--supporting evidence, and probable cause to direct an investigation.

Again, so why single polygraph testing out? As you point out, there are flaws with the DA's argument. Given that I specifically termed him "incompetent," I recognize that.

You seem to feel that the same defense attorney who would fight the footprint report tooth and nail would somehow roll over and play dead when presented with a polygraph report. Or perhaps you feel that someone would be more likely to be fired over a pollygraph report than over footprint report. In either case, I don't see why you attribute polygraph reports with these magical talisman-like credibility properties.

Any evidence can be misused, which is precisely why it is so important to know the actual value of the evidence....

[Belz...]

Any of the time? Be serious.

We know it's been correct at least three times, because it correctly identified Kari as telling the truth, and Tore and Grant as lying.

Besides, if it was incorrect ALL of the time I'd be easy to correct the situation, no ? :D

[Belz...]

No. It doesn't respond directly to body functions, but to changes in body functions. So if you're very nervous at the start, but you get even more nervous when asked a question related to the incident in question, you will be flagged. If you're very nervous at the start, but are uniformly nervous throughout the interview, you will probably pass.

Now, you're right that someone with superhuman "nerves of steel" could almost certainly pass a polygraph by simply willing his body to display no signs of stress whatsoever. Such people appear to be relatively rare in the general population, as do the "nervous Nellies." As to whether or not you can be trained to become such a superman --- that was one of the issues that the NAS was concerned about.

I was already aware of the "changes in body functions". I worded it badly. Thanks. That answers my question neatly.

[CFLarsen]

The first was a video, which I can't watch right now. I've read the second. Like DrK, I'm coming up on a barrier of time I want to invest. Please just answer me.

What is the issure here? What is being contested? Polygraphs only? Polygraph interviews? Abuse of authority?

See post #51.

You seem to have plenty of interest.

What about JFS? Why is it worse to present the material in a specialist journal with higher readership and greater impact?

It isn't a question of publishing your argument in other publications. It's a question of publishing it in skeptical publications.

You're a skeptic, right? You know that other skeptics disagree with you on this. You think that they should be swayed by the evidence you point to.

Why won't you argue your case in front of your skeptic peers?

[drkitten]

Why won't you argue your case in front of your skeptic peers?

Which part of "it does not appear to be a worthwhile investment of my time and trouble" did you miss the first time I posted it?

[CFLarsen]

No. It doesn't respond directly to body functions, but to changes in body functions. So if you're very nervous at the start, but you get even more nervous when asked a question related to the incident in question, you will be flagged. If you're very nervous at the start, but are uniformly nervous throughout the interview, you will probably pass.

Wrong.

A polygraph is an instrument that simultaneously records changes in physiological processes such as heartbeat, blood pressure, respiration and electrical resistance (galvanic skin response or GSR). The polygraph is used as a lie detector by police departments, the FBI, the CIA, federal and state governments, and numerous private agencies. The underlying theory of the polygraph is that when people lie they also get measurably nervous about lying. The heartbeat increases, blood pressure goes up, breathing rhythms change, perspiration increases, etc. A baseline for these physiological characteristics is established by asking the subject questions whose answers the investigator knows. Deviation from the baseline for truthfulness is taken as sign of lying.
Source (http://skepdic.com/polygrap.html)

[drkitten]

Wrong.

You've now crossed the border from misrepresentation into outright lying.

Your quotation:


A polygraph is an instrument that simultaneously records changes in physiological processes such as heartbeat, blood pressure, respiration and electrical resistance (galvanic skin response or GSR). The polygraph is used as a lie detector by police departments, the FBI, the CIA, federal and state governments, and numerous private agencies. The underlying theory of the polygraph is that when people lie they also get measurably nervous about lying. The heartbeat increases, blood pressure goes up, breathing rhythms change, perspiration increases, etc. A baseline for these physiological characteristics is established by asking the subject questions whose answers the investigator knows. Deviation from the baseline for truthfulness is taken as sign of lying.

Note the italicized bit, which directly support what I said.

The instrument responds to "deviation from the baseline," i.e. to changes in body functions, not to body functions themselves.

A person with high signs of stress generally (e.g. "heartbeat, blood pressure, respiration and electrical resistance") will establish a high baseline. A person who is "uniformly nervous throughout the interview" will not vary from that baseline, and therefore will have no detectable "deviations." Absent such deviations, there are no signs of lying, and the individual will pass.

A person who is "very nervous at the start, but [who gets] even more nervous when asked a question related to the incident in question" will display the noted deviations, and be identified as lying.

I stand by what I wrote -- and list this as another reason on my part not to touch your nasty little vanity journal.

[Jimbo07]

See post #51.

This?


Please provide the scientific evidence that the error rate in polygraphs is close to zero.

For what purpose? How 'close' to zero? Why don't you think it's been done already? What would presenting at TAM resolve? Would anyone there really care (honest question, I haven't been)? etc.

This is what this thread is entirely about.

It's entirely about a statement that raises more questions than it answers?


You seem to have plenty of interest.


I do. I don't have time to read the secondary links of someone who won't tell me their own position. I will credit you with mine:

I've been to a couple ride-alongs/interviews. I have not been to a polygraph interview. I've been exposed to just enough to wonder if what people think they know about law enforcement and what they actually know are two different things. In fact, this can probably be generalized to any complex discipline which has evolved over time (biology? physics? ;)). This was all triggered by a personal connection of mine.


It isn't a question of publishing your argument in other publications. It's a question of publishing it in skeptical publications.

Wow. I've often wondered if popular skepticism is somehow removed from science... :rolleyes:

[Cardelitre]

and as far as argumentum ad consequentiam (http://en.wikipedia.org/wiki/Appeal_to_consequences) goes--in this case I don't believeit is applicable. My argument is not a logical falacy, it is just an evaluation of human nature and greed. From my reading, and from what I know, people have lost jobs and had careers derailed --all from a single polygraph test.
Well, so what? Human nature and greed or people losing their jobs has no influence on the hypothesis that polygraphs can dicriminate lies from truth in a tested sample with an efficiency significantly better than pure chance (or not, for all I know).

[remirol]

I've seen a variant of it that could be called the "gambler's proof": where you ask someone to bet a very large amount of money about the truth/validity of a given issue. Generally, the attacker sets the amount so high such that he can be confident the opposing party either doesn't have that much money available, or won't be willing to risk it on something so (comparably) trivial. It also requires that the validity of the issue in question cannot be determined easily, such that neither party can be certain of the truth prior to accepting the bet.

(Example: "If you're so sure the Patriots will win the Super Bowl, you should be willing to bet your house on it. Let's bet your house against mine, OK?")

In some instances, it can be used effectively to demonstrate the depth of someone's conviction in their stated ideas, but has no bearing whatsoever on determining the validity of the ideas themselves. It is generally a tactic used to attempt to stop discussion of a topic and force a concession to the attacker's viewpoint.

The requests to write a paper and/or present at TAM in this thread are similar.

That is a preposterous idea.

I agree, the tactic is completely preposterous, as is the idea that someone who claims to be a skeptic would resort to a "shout down" approach like the one I described above. So stop using the tactic, and start discussing the issue like a rational adult.

For what it's worth, I agree with at least one premise of this discussion: I do not believe polygraphs are useful for determining whether a single individual is lying. Whether polygraphs are useful for determining whether some part of a _group_ of people are lying is a different claim altogether.

Second, this is not an attempt to stop a discussion, quite contrary: The thread is opened specifically to get the evidence out in the open, to a much bigger audience.

I do not believe you. I think you are deliberately attempting to push a scenario and standard of proof that involves far more risk and effort than this issue and discussion warrant, for the express purpose of forcing skeptigirl to concede your point of view here on this forum _OR_ decline to present -- at which point you will bring up the declination at every opportunity as "evidence" to support your side.

Argue the issue itself, (if you can). But please stop the blather about her presenting at TAM; it's completely irrelevant to the issue itself.

We are talking about skeptics claiming evidence that lie detectors works.

Your statement is vague. Please quote the claim made that you object to.

[CFLarsen]

You've now crossed the border from misrepresentation into outright lying.

Your quotation:



Note the italicized bit, which directly support what I said.

The instrument responds to "deviation from the baseline," i.e. to changes in body functions, not to body functions themselves.

A person with high signs of stress generally (e.g. "heartbeat, blood pressure, respiration and electrical resistance") will establish a high baseline. A person who is "uniformly nervous throughout the interview" will not vary from that baseline, and therefore will have no detectable "deviations." Absent such deviations, there are no signs of lying, and the individual will pass.

A person who is "very nervous at the start, but [who gets] even more nervous when asked a question related to the incident in question" will display the noted deviations, and be identified as lying.

I stand by what I wrote -- and list this as another reason on my part not to touch your nasty little vanity journal.

I'm not lying. You miss the point of such questions entirely. The "easy" questions are asked to put the subject at ease so a baseline can be established:

The Questioning Procedures

1. First, you are "put at ease" by the administrator, and he or she talks to you about all the questions you already answered on a questionnaire or in the pre-interview. They put you at ease and tell you that it is completely normal (because it is) for some things to “suddenly” pop in your head about things you might have forgotten to mention in the pre-questionnaire/interview.
How to cheat a polygraph test (http://www.wikihow.com/Cheat-a-Polygraph-Test-(Lie-Detector))

I sincerely doubt that you have much knowledge about polygraphs at all.

This?

For what purpose? How 'close' to zero? Why don't you think it's been done already? What would presenting at TAM resolve? Would anyone there really care (honest question, I haven't been)? etc.

Because I have yet to see all that many skeptics agree with skeptigirl. I certainly haven't seen evidence that the polygraph is that accurate as she claims.

It's entirely about a statement that raises more questions than it answers?

If skeptigirl is right, skeptics are wrong. That's pretty important.

I do. I don't have time to read the secondary links of someone who won't tell me their own position.

If you won't read the links, you won't be convinced, that's for sure.

I will credit you with mine:

I've been to a couple ride-alongs/interviews. I have not been to a polygraph interview. I've been exposed to just enough to wonder if what people think they know about law enforcement and what they actually know are two different things. In fact, this can probably be generalized to any complex discipline which has evolved over time (biology? physics? ;)). This was all triggered by a personal connection of mine.

Thank you for that. It doesn't constitute scientific evidence, though.

Wow. I've often wondered if popular skepticism is somehow removed from science... :rolleyes:

It isn't. But this subject has been discussed by skeptics as well, also in their publications, by the leading skeptics. It's an example of pseudoscience, which definitely is a subject skeptics are interested in.

[rwguinn]

I'm not lying. You miss the point of such questions entirely. The "easy" questions are asked to put the subject at ease so a baseline can be established:
yes, you are.
Either that, or you best stop trying to discuss things in the English language, because your comprehension is worse than a US educated 6th grader.Those two snips, dr kittens and your quotation said exactly the same thing.




I sincerely doubt that you have much knowledge about polygraphs at all.



Because I have yet to see all that many skeptics agree with skeptigirl. I certainly haven't seen evidence that the polygraph is that accurate as she claims.



If skeptigirl is right, skeptics are wrong. That's pretty important.



If you won't read the links, you won't be convinced, that's for sure.



Thank you for that. It doesn't constitute scientific evidence, though.



It isn't. But this subject has been discussed by skeptics as well, also in their publications, by the leading skeptics. It's an example of pseudoscience, which definitely is a subject skeptics are interested in.
After all that, I agree--You're not lying. You are either trying to justify an untennable position, or you cannot read and comprehend English--or you are doing the typical CFL troll, which would get anybody else suspended.

[Cardelitre]

It isn't. But this subject has been discussed by skeptics as well, also in their publications, by the leading skeptics. It's an example of pseudoscience, which definitely is a subject skeptics are interested in.
Just a little question, since I am not particularly familiar with the skeptic-specific litterature, have the leading skeptics (or anyone else) performed an experiment that demonstrated that the polygraphs are not better than chance in determining the proportion of liars in a sample population? Is this published somewhere, I would be curious to see what kind of test led them to this conclusion?

[LTC8K6]

Does a polygraph work if the examiner cannot see the subject?

[drkitten]

You miss the point of such questions entirely. The "easy" questions are asked to put the subject at ease so a baseline can be established:

More misrepresentation. Nowhere in the quoted paragraph is the phrase "easy questions" or "put the subject at ease." mentioned. What is mentioned is the necessity of establishing a baseline.

Wikihow isn't a very good source for "how to beat a polygraph," by the way. You'll notice that their advice is largely the same advice that Tore and Grant tried to use (and failed). I have no real problem dismissing it under the "don't take diet advice from fat people" rule. But even Wikihow says that you're lying.

If it was necessary to be put "at ease" in order to work, then truly nervous people couldn't be examined. But that's not what Wikihow says.

The polygraph takes your baseline, which is your base physiological response when you are nervous. Even if you were not nervous, the baseline measures that as well. It matters not where the baseline is on the graph. What matters, is how that base line changes as the questions are asked. So, even if you are nervous as a rabbit with the baseline, you still will likely show an anticipation/relief response when asked a question you have lied about, or are holding back information about.






I sincerely doubt that you have much knowledge about polygraphs at all.

Shrug. I've lectured on them, professionally. I've reviewed papers on them. I don't need to have that much knowledge to spot misstatements, errors, and misrepresentations.

[Jimbo07]

If you won't read the links, you won't be convinced, that's for sure.

Of what? An abstract is a guide for further reading. Give me an abstract. State a position. Anybody!


It doesn't constitute scientific evidence, though.

Of what? Scientific evidence of my personal motivation? :boggled:

I'm sorry. I've followed both these threads and finally decided to come in and ask what the hubbub was about. I've stated why I would even bother reading a thread on this subject. I have a certain understanding, and am willing to admit to errors. However, I have no idea what the central contention is.

Is it: polygraphs are not 100% accurate? Polygraphs should never be used? Some have suggested that it's about the efficacy of individual vs. group testing.

Note to remirol: I'll actually disagree. With what I've posted regarding interview tactics, I'd have assumed that polygraph interviews were more effective in individual cases. It's neat to see the group thing...

[remirol]

Note to remirol: I'll actually disagree. With what I've posted regarding interview tactics, I'd have assumed that polygraph interviews were more effective in individual cases. It's neat to see the group thing...

In this case, I'd ask: is it that the polygraph is effective, or that the interviewer is skilled at 'breaking' naive people and inducing a confession?

Would that same (hypothetical) interviewer be capable of a similar success rate by simply using a machine that appeared to be a polygraph, but instead of actually reading the interviewee's physiological responses, had its needles "jump" every time the interviewer stepped on a button on the floor?

(Has anyone ever designed a test similar to the above?)

[LTC8K6]

We know it's been correct at least three times, because it correctly identified Kari as telling the truth, and Tore and Grant as lying.

What was correct? The machine or the operator?

[drkitten]

Does a polygraph work if the examiner cannot see the subject?

Studies have shown so, yes. From the FAS report:


The experimenters also used objective measurements of all physiological response measures with the aid of computers and persons who had no knowledge of the field evaluations or treatments administered. The experimenters used the decisions made by the independent blind evaluator to assess the validity of the polygraph test. This was, however, equivalent to using the polygraph examiner’s decision, because the independent rater and the examiner agreed on 100 percent of their decisions.

[Aoidoi]

Well, one standard way is to have as much of the data processed by computer (objectively) as possible. If the machine shows that on standardized data set alpha-niner-four, it was possible to draw a separating hyperplane between the truthful and non-truthful candidates' measurements, and that my measurement is on the lying side of that hyperplane, it's hard to attribute that to bias. We may argue about whether alpha-niner-four is a representative control group, but there's not much to argue about involving the categorization itself.

Another way is simply to look at the evidence upon which the operator makes his/her decision and see if it appears reasonable. If the operator says "you're lying," that doesn't mean much. But if the operator says "your skin temperature says you're lying," then there should be a visible difference in the graphs, specifically in the one for skin temperature. If the operator gives an opinion, but doesn't have a basis for the opinion, then one could rightly disregard that particular opinion -- but that's hardly unique to polygraphs. This problem has been around in "expert" testimony for centuries. When the doctor says that such-and-such a person died of heart disease and not pneumonia, you expect him to have citable medical reasons for her opinion. I'm clearly not an expert... but is it possible for other trained polygraphers (er... whatever the word is for that) to go back over the data from a session and come up with the same answers as the operator involved? Expert testimony in court is regularly impeached by other experts based on the data (or in Law & Order based on wild theories), but I'm not sure how practical that is for polygraphs. Seems like one guy can say "look, his heartbeat increased" and the other say "skin temperature remained the same" and a the operator can say "that SOB farted and damn near killed me."

Well, you get the idea, anyway. ;)

How consistent is the interpretation of results? Seems like there's always going to be that bias problem if there's enough data that the operator can just pick one that agrees with him as justification.

I guess my issue is that I don't really see how interpreting polygraph data is any different from, say, a psychologist saying that based on his professional opinion the subject is being truthful or lying. They can point to all sorts of physical cues (not meeting eyes, fidgeting, posture, etc) and claim that this conveys more information, but it is largely dependent on the individual's perceptions.

I suppose basically I'm worried about potential lack of reproducibility, lack of effective documentation, and concern over how one determines expertise.

You don't need to, unless you're suggesting that so many people were molested by priests that they constitute a significant minority of the target population, on the same order of magnitude as the detection threshhold. In some cases, this may be the case (e.g. if you're doing this experiment at a "survivors of priestly molestation" convention). But otherwise, any reasonable estimate of the probability of this would put it down in the "lost in the noise" section of the spectrum.But during a large scale test you'd have to have some way to determine all the factors involved in the noise. It's unlikely that in a random sample you'll get a whole host of the same reason, but I'm sure there's more reasons to be worried/erratic/freaked out/whatever about something as dramatic as religion than I can come up with off the top of my head. I've seen people do things for religious reasons I can't even explain having seen them, I have no way of determining percentages of populations that will have strange reactions to things.


The first and only time? Possibly not. The thirtieth time over the course of two hours? Quite probably. As I said earlier, the emotional response from being asked essentially the same nosy question over and over by the same person bleaches quite quickly.I'm rather intrigued by this theory. Seems to me that I know lots of people who would get progressively more upset by being asked the same (or similar) questions over and over. I know I generally get pissed having to repeatedly explain myself on subjects on which I am quite certain I am telling the truth. Heck, just having to repeat myself at family gatherings because half the people aren't paying attention annoys me. :D

[drkitten]

I'm clearly not an expert... but is it possible for other trained polygraphers (er... whatever the word is for that) to go back over the data from a session and come up with the same answers as the operator involved?

See my immediately previous post. It doesn't even need to be "other trained polygraphers."

[LTC8K6]

I don't think that meets the requirement, drkitten.

[drkitten]

Just a little question, since I am not particularly familiar with the skeptic-specific litterature, have the leading skeptics (or anyone else) performed an experiment that demonstrated that the polygraphs are not better than chance in determining the proportion of liars in a sample population? Is this published somewhere, I would be curious to see what kind of test led them to this conclusion?

I have seen no such far-reaching paper published, although I'm sure that CFL will be happy to provide buckets of them upon demand. Unless, of course, he woudl prefer to admit that he holds his opinion that they don't work based on no supporting evidence whatsoever.

I have seen a number of papers published that indicate that a type X polygraph, under circumstances Y, as operated by Z, is not accurate. But that, of course, isn't an indictment of the entire field, any more than an observation that vitamin C cannot be used to treat cancer means that it's not a good for anything (and for scurvy in particular). In fact, it's very difficult to prove that "polygraphs" as a whole are ineffective for partly the same reason that it's very difficult to prove that astrology is ineffective --- there are still variations of both that have not been subject to testing.

The primary difference is that there are no variants of astrology that reproducibly test positively, and there are many variants of polygraphy that do.

[drkitten]

I don't think that meets the requirement, drkitten.

How about this quotation, from the same study?


An objective quantitative analysis for each physiological measure was employed to determine if each was effective in discriminating between guilty and innocent. Most of the measures yielded significant discriminations, with the exception of a few of the cardiovascular measures.

or from another study


Some of the latest work of the Utah laboratory explores the use of computers in the analysis of polygraph recordings. Kircher (91a) compared the accuracy of a computer decisionmaking process to the accuracy of assessments of a field examiner. The computerized analysis cannot be included in the statistical analysis of this technical memorandum, because it is not presently a field scoring method, but the decisions of an independent evaluator who was used can be. This mock crime study followed the basic procedures of Podlesny and Raskin (127) with 100 subjects from the community. The accuracy of the original examiner was not reported though the results of an independent evaluator were. The independent evaluator, who numerically scored the charts blindly, correctly diagnosed 87 percent of the subjects; misdiagnosed 6 percent; and made a judgment of inconclusive on 7 percent. The six errors were evenly divided between three false negatives and three false positives. In comparison, different computer decision models, on the average, correctly identified 84.9 percent of subjects, misidentified 7.85 percent, and placed 7.2 percent in an inconclusive category.

[LTC8K6]

It's interesting, but I meant on the fly.

Does the fact that the subject can see the examiner have an effect, for example.

Are you more likely to have a reaction when you can see the examiner looking right at you?

[drkitten]

It's interesting, but I meant on the fly.

Does the fact that the subject can see the examiner have an effect, for example.

Are you more likely to have a reaction when you can see the examiner looking right at you?

Ah. That I don't know. I do know that subjects hooked up to "dummy" polygraph machines (machines that they "know" aren't working or are turned off) display less response than subjects hooked up to machines they know are working. So I would suspect the answer is yes, but have no data to support it.

[CFLarsen]

Just a little question, since I am not particularly familiar with the skeptic-specific litterature, have the leading skeptics (or anyone else) performed an experiment that demonstrated that the polygraphs are not better than chance in determining the proportion of liars in a sample population? Is this published somewhere, I would be curious to see what kind of test led them to this conclusion?

I'm not sure where you are going with this. Why is that important?

Your statement is vague. Please quote the claim made that you object to.

See post #51.

Does a polygraph work if the examiner cannot see the subject?

The polygraph is a tool that "works" by intimidation: The subject is put under strain from the technobabble and pseudoscience: "We can see that you are lying - so confess!"

Of course, if the polygraph really could tell if people lie or not, there would not be any need for any confession....

Shrug. I've lectured on them, professionally. I've reviewed papers on them. I don't need to have that much knowledge to spot misstatements, errors, and misrepresentations.

In which case your article/presentation would be regarded with much more interest.

What about Skeptic Magazine? TAM?

I'm sorry. I've followed both these threads and finally decided to come in and ask what the hubbub was about. I've stated why I would even bother reading a thread on this subject. I have a certain understanding, and am willing to admit to errors. However, I have no idea what the central contention is.

Is it: polygraphs are not 100% accurate? Polygraphs should never be used? Some have suggested that it's about the efficacy of individual vs. group testing.

Polygraphs are pseudoscience. They cannot be regarded as reliable tools to tell whether people lie or not.

drkitten and skeptigirl in particular claim otherwise.

[Cardelitre]

I'm not sure where you are going with this. Why is that important?
Why is that important? What a strange question...

If the leading skeptics as you call them concluded that polygraphs don't perform better than chance to detect the proportion of liars in a sample population, I guess that this is not just wishfull thinking. They must have based their opinion on some experiments demonstrating that point in a way or another, that's what skeptics do right?

At least that's what they do when addressing something like radiesthesia or telepathy or whatever, as far as I know.

So I would be curious to see one of these experiment reports, if there are?

[Jimbo07]

Polygraphs are pseudoscience. They cannot be regarded as reliable tools to tell whether people lie or not.

drkitten and skeptigirl in particular claim otherwise.

Thank you. You seem to have summarized it in two points:

i) Polygraphs are pseudoscience: My personal preference would be to say that polygraph interviews are an art. Art can generate reproducable results, and those can be studied... scientifically, I suppose.

ii) They cannot be regarded as reliable tools to tell whether people lie or not: is very difficult, IMO. No piece of evidence is a 100% guarantee that someone is, or is not, lying. However, I think you are being a little bit obstinate in rejecting their claims that there is some effect.

Would you call for a ban on the polygraph interview? Do you think it's a law-enforcement strong-arm tactic? Has the idea of the polygraph as some sort of magic lie detector become a strawman for any objections?

I would agree if everyone was saying, "the polygraph interview is an effective law-enforcement tool." Everything else, I'm regarding as educational!

[CFLarsen]

Why is that important? What a strange question...

If the leading skeptics as you call them concluded that polygraphs don't perform better than chance to detect the proportion of liars in a sample population, I guess that this is not just wishfull thinking. They must have based their opinion on some experiments demonstrating that point in a way or another, that's what skeptics do right?

At least that's what they do when addressing something like radiesthesia or telepathy or whatever, as far as I know.

So I would be curious to see one of these experiment reports, if there are?

Of course they base their opinions on experiments.

But why do they have to have performed the experiments themselves?

[verisimilidude]

Why is it after reading this thread I am not able to come to any conclusion about the reliability of this device?

Judging from this thread, however, it's obvious that no consensus exists regarding the reliability of these machines. That's probably why their findings are not admissible as evidence in most states unless otherwise agreed to by both parties. And believe me, it doesn't take much validation for any "scientific" technique, method or device to be considered reliable and therefore admissible in court.

Should something which is at best partly woo be considered totally woo?

[CFLarsen]

Thank you. You seem to have summarized it in two points:

i) Polygraphs are pseudoscience: My personal preference would be to say that polygraph interviews are an art. Art can generate reproducable results, and those can be studied... scientifically, I suppose.

ii) They cannot be regarded as reliable tools to tell whether people lie or not: is very difficult, IMO. No piece of evidence is a 100% guarantee that someone is, or is not, lying. However, I think you are being a little bit obstinate in rejecting their claims that there is some effect.

Would you call for a ban on the polygraph interview? Do you think it's a law-enforcement strong-arm tactic? Has the idea of the polygraph as some sort of magic lie detector become a strawman for any objections?

I would agree if everyone was saying, "the polygraph interview is an effective law-enforcement tool." Everything else, I'm regarding as educational!

The reason why polygraphs reportedly "work" is not because you can tell from the readings that people lie. You can use polygraphs to intimidate those who don't know that it is bull.

But that doesn't mean they should be used. We don't use pseudoscience, we shouldn't encourage it.

If your suspect was superstitious, would you threaten to cast a curse, if he didn't confess?

[Cardelitre]

Of course they base their opinions on experiments.

But why do they have to have performed the experiments themselves?
I didn't particularly mean that they had to do it themselves, maybe I could have been more explicit sorry. They can of course base their opinion on other experiments that already exist.

So what are the experiments that led them to conclude that there is no basis in the principle of polygraphs exactly? Do you know the reference of one that conclusively demonstrates this?

Until now, all I have seen is the meta-analysis of the NAS, which concludes that the results, although very variable depending on the conditions, are still significantly more efficient than chance alone.

[CFLarsen]

I didn't particularly mean that they had to do it themselves, maybe I could have been more explicit sorry. They can of course base their opinion on other experiments that already exist.

Gotcha.

So what are the experiments that led them to conclude that there is no basis in the principle of polygraphs exactly? Do you know the reference of one that conclusively demonstrates this?

No single experiment conclusively demonstrates anything. You can start with Bob's article on SkepDic.

Until now, all I have seen is the meta-analysis of the NAS, which concludes that the results, although very variable depending on the conditions, are still significantly more efficient than chance alone.

Not quite. The problem lies precisely in the many different conditions. Some seem to work some of the time under some conditions, at other times not.

You would think that after this many years, we would see some strenghtening of the evidence - some standardization of method that was universally accepted by the scientific community.

We don't. The evidence isn't getting stronger, no standardized method is emerging, and the more we know of the basic premise, the less reason we have to believe it works.

It's quite similar to homeopathy.

[Cardelitre]

No single experiment conclusively demonstrates anything.
That's right. So it means that there must be many experiments showing the same negative results I guess, skeptics wouldn’t conclude on only one experiment. Is there one that you think is more representative? I'm afraid I don't have the time to review all the literature on this subject...
You can start with Bob's article on SkepDic.
I found this article (http://www.skepdic.com/polygrap.html) on SkepDic, but no reference to any Bob. Is that the one you are talking about?
Anyway it gives an interesting point of view on the polygraph and I agree on many points. But I couldn't find any reference to the type of experiments I was talking about, maybe I missed it. Would you know if this has been done, it must have, right?
Not quite. The problem lies precisely in the many different conditions. Some seem to work some of the time under some conditions, at other times not.
Yes, as I understand it, it works at different degrees depending on the conditions. But it still works significantly better than chance, even if it is not much better.
You would think that after this many years, we would see some strenghtening of the evidence - some standardization of method that was universally accepted by the scientific community.

We don't. The evidence isn't getting stronger, no standardized method is emerging, and the more we know of the basic premise, the less reason we have to believe it works.
The fact that the method is not improved or standardized is not a direct argument against the possible efficiency of the polygraph. It only proves that no one had the will, opportunity or fund to do that, or that the way it is currently used (more as a dissuasive threat) is considered sufficient, or whatever. It is not an evidence against the efficiency if the polygraph.
It's quite similar to homeopathy.
Yes, and those experiments have been conducted with homeopathy countless times and demonstrated that homeopathy is clearly unable to do anything better than placebo.
There should be somewhere a study of the same type for polygraphs right? This would surely help to shed some light on this...

[TV's Frank]

Polygraphs are pseudoscience. They cannot be regarded as reliable tools to tell whether people lie or not.

I would just like to ask a clarifying question, that hopefully will improve the discussion. What exactly do you mean by "reliable"? Better than chance? 100% always catching the liar and never generating false positives? Somewhere in between? I think something specific here would be nice.

Also, under what conditions? For example, in this thread, I have seen a discussion of two separate real-world operating conditions: finding a crook in a small sample of suspected bad guys, and sifting through many people in an attempt to ferret out spies. The cited articles suggest that these two scenarios have different rates of "reliability".

[CFLarsen]

That's right. So it means that there must be many experiments showing the same negative results I guess, skeptics wouldn’t conclude on only one experiment. Is there one that you think is more representative? I'm afraid I don't have the time to review all the literature on this subject...

No, not particularly.

I found this article (http://www.skepdic.com/polygrap.html) on SkepDic, but no reference to any Bob. Is that the one you are talking about?

Bob Carroll. The owner of the site.

Anyway it gives an interesting point of view on the polygraph and I agree on many points. But I couldn't find any reference to the type of experiments I was talking about, maybe I missed it. Would you know if this has been done, it must have, right?

As you can see in the references, Bob refers to e.g. the NAS report.

Bob does his homework.

Yes, as I understand it, it works at different degrees depending on the conditions. But it still works significantly better than chance, even if it is not much better.

The fact that the method is not improved or standardized is not a direct argument against the possible efficiency of the polygraph. It only proves that no one had the will, opportunity or fund to do that, or that the way it is currently used (more as a dissuasive threat) is considered sufficient, or whatever. It is not an evidence against the efficiency if the polygraph.

It isn't as if the polygraph is a fringe technology: It is widely used in many situations in the US, both government and the private sector.

Yes, and those experiments have been conducted with homeopathy countless times and demonstrated that homeopathy is clearly unable to do anything better than placebo.
There should be somewhere a study of the same type for polygraphs right? This would surely help to shed some light on this...

No, not clearly: Some studies show that homeopathy works - only they are small, and/or poorly designed and conducted.

I would just like to ask a clarifying question, that hopefully will improve the discussion. What exactly do you mean by "reliable"? Better than chance? 100% always catching the liar and never generating false positives? Somewhere in between? I think something specific here would be nice.

Also, under what conditions? For example, in this thread, I have seen a discussion of two separate real-world operating conditions: finding a crook in a small sample of suspected bad guys, and sifting through many people in an attempt to ferret out spies. The cited articles suggest that these two scenarios have different rates of "reliability".

That's precisely it: When you find a crook, you find a spy or you test a job applicant, you are talking about the exact same goal: To find out if they lie or not. Yet, we find very different results.

That underlines the unreliability of polygraphs - which is a better way to say it: Polygraphs are unreliable.

[TV's Frank]

That's precisely it: When you find a crook, you find a spy or you test a job applicant, you are talking about the exact same goal: To find out if they lie or not. Yet, we find very different results.

That underlines the unreliability of polygraphs - which is a better way to say it: Polygraphs are unreliable.

OK, given all that, what do you mean by "reliable"? Anything better than chance? 100%? What?

[CFLarsen]

OK, given all that, what do you mean by "reliable"? Anything better than chance? 100%? What?

I told you: It's a better way to say they are unreliable.

[Cardelitre]

No, not particularly.So you can't think of any study showing that the polygraphs can't perform better than chance to detect liars in a sample population? That's too bad... I wonder how the skeptics that you mentioned where able to draw that conclusion without the necessary evidence then...
Bob Carroll. The owner of the site.Oh, Bob is for Robert, ok. I never heard of the guy before, so I had a hard time guessing who you where talking about...
As you can see in the references, Bob refers to e.g. the NAS report.
Yes, and the NAS report shows that the polygraphs are apparently able to perform better than chance when discriminating liars in a sample population. It seems that these results show actually the contrary to what you claim and what I was asking about...
It isn't as if the polygraph is a fringe technology: It is widely used in many situations in the US, both government and the private sector.If you say so. Still, it is not evidence against the possible efficiency of the polygraphs, merely a hint.
No, not clearly: Some studies show that homeopathy works - only they are small, and/or poorly designed and conducted.
Well, do we agree that the large majority of the studies on this subject show that homeopathy doesn't work better than placebo, and that it is the most common conclusion of the meta analysis on this subject?
So the question is: is there a study showing the same kind of negative results for polygraphs? I'm starting to think that there is none, or if there is, small and/or poorly designed. But I could be wrong; I didn't personally extensively review the literature about this...

[TV's Frank]

I told you: It's a better way to say they are unreliable.

I don't get it! If you say, "polygraph is unreliable", do you mean "polygraph performs no better than chance"? Or something else?

[CFLarsen]

So you can't think of any study showing that the polygraphs can't perform better than chance to detect liars in a sample population? That's too bad... I wonder how the skeptics that you mentioned where able to draw that conclusion without the necessary evidence then...

No, that's not what I said, and it's not what you asked me. You asked me if I could find one experiment that is more representative.

Oh, Bob is for Robert, ok. I never heard of the guy before, so I had a hard time guessing who you where talking about...

My bad, then. I thought you knew about Bob. Bob is an institution in skepticism.

Yes, and the NAS report shows that the polygraphs are apparently able to perform better than chance when discriminating liars in a sample population. It seems that these results show actually the contrary to what you claim and what I was asking about...

No, it doesn't. It says that in some situations under some conditions, they sometimes perform better than change. What we have is a "Charlie Brown" scenario: Wishy-washy.

If you say so. Still, it is not evidence against the possible efficiency of the polygraphs, merely a hint.

It isn't up to skeptics to prove that polygraphs don't work. It is up to proponents of the polygraph to prove that they work.

Well, do we agree that the large majority of the studies on this subject show that homeopathy doesn't work better than placebo, and that it is the most common conclusion of the meta analysis on this subject?
So the question is: is there a study showing the same kind of negative results for polygraphs? I'm starting to think that there is none, or if there is, small and/or poorly designed. But I could be wrong; I didn't personally extensively review the literature about this...

If you point to meta-analyses, why do you insist on one single study when it comes to polygraphs?

That is totally inconsistent of you.

I don't get it! If you say, "polygraph is unreliable", do you mean "polygraph performs no better than chance"? Or something else?

I mean that they are unreliable.

[TV's Frank]

I mean that they are unreliable.

Okay...but I want to know that you mean by "unreliable"! I want to make sure that when you say "unreliable" it means the same thing that it means to me. Let's play a game. I would like you to replace "polygraph is unreliable" with "polygraph is __________ ________ ______", where the blanks indicated a longer, more detailed description of your opinion of polygraph, specifically what you mean by "unreliable".

[David Wong]

I mean that they are unreliable.

You wouldn't tolerate this kind of evasion from anyone else. Why do you tolerate it from yourself?

You've been asked to clarify the statement. Wouldn't the rational, mature thing to do be to clarify it? Or state why it can't be clarified?

To snidely repeat the same statement, intentionally making no effort to advance the point or aid the other person's understanding of the issue, is not the sign of someone interested in a rational discussion. It's the sign of someone childishly trying to "win."

[Myshkin]

I'm confused. CFLarsen, do you dispute that a measurable phenomenon exists? Or simply that the phenomenon is too slight and variable across different conditions to be of any value?

http://fas.org/sgp/othergov/polygraph/ota/index.html
http://www.nap.edu/openbook.php?isbn=0309084369

The meta-analyses seem to demonstrate that, more often than would be expected by chance, the results of lab experiments deviate from what would be expected by chance. This is the meta-result, so to speak. Does that not suggest that there is a measurable phenomenon occurring? The experiments were far from perfect, but many appeared to have adequate control and blinding.

Whether the difference in lying and truthful populations can be used to address any specific question, especially across the myriad cases and conditions encountered in the real world...is a great question but not the one I'm asking here.

If there really is a net difference, statistically deviating from what would be expected by chance, it would indicate that the polygraph correctly categorize liars and truthers at a rate greater than simple guesswork, given a sufficiently large sample size. And that would be an interesting phenomenon.

[CFLarsen]

Okay...but I want to know that you mean by "unreliable"! I want to make sure that when you say "unreliable" it means the same thing that it means to me. Let's play a game. I would like you to replace "polygraph is unreliable" with "polygraph is __________ ________ ______", where the blanks indicated a longer, more detailed description of your opinion of polygraph, specifically what you mean by "unreliable".

No, let's not play games. Let's just acknowledge that I think that polygraphs are unreliable.

Do you think I have any reason to think otherwise?

You wouldn't tolerate this kind of evasion from anyone else. Why do you tolerate it from yourself?

You've been asked to clarify the statement. Wouldn't the rational, mature thing to do be to clarify it? Or state why it can't be clarified?

To snidely repeat the same statement, intentionally making no effort to advance the point or aid the other person's understanding of the issue, is not the sign of someone interested in a rational discussion. It's the sign of someone childishly trying to "win."

There's no evasion.

I think polygraphs are unreliable. I have extensively explained what my stance is wrt to polygraphs. The scientific evidence that they work is simply not compelling. Not only do major skeptics agree with this, the scientific community is also with me on this one.

That's not childish. At all.

[Jimbo07]

The reason why polygraphs reportedly "work" is not because you can tell from the readings that people lie. You can use polygraphs to intimidate those who don't know that it is bull.

That's a bit of an oversimplification, but at least it's related to what I was saying.


But that doesn't mean they should be used. We don't use pseudoscience, we shouldn't encourage it.

If your suspect was superstitious, would you threaten to cast a curse, if he didn't confess?

Do you mean to say that police do not lie and exaggerate in interviews? Or are you saying they should be discouraged from doing so?

[TV's Frank]

Personally, I'm not surprised that the "sifting for spies" doesn't perform as well as "catching the crook". In a police investigation, I image the questions are more detailed, direct, and emotional (i.e. "did you stab mrs. jameson in the neck?") . When sifting, you have to keep to generalities (i.e. "have you ever been contacted by an agent of a foreign government?") One might find it easier to lie about the latter case.

Mind you, this is all made up and probably wrong.

As a possibly related and slightly amusing anecdote, I was once the subject of a polygraph test by the NSA (it was for a job). For those of you wondering about observer bias, the questioner told me that the polygraph results were examined by three independent examiners, who were not present during the questioning. The questioner simply adminstered the test and recorded spurius results (such as a cough). A failure at the examination would not automatically disqualify you (they were away of the high false-positive rate), but simply "flagged" you for more detailed scrutiny.

Anyway, I came pretty close to passing out during the test. Sitting in a comfy chair, completely aware of every single heartbeat and breath, and answering inane questions (like "were you born in milwaukee" and "have you ever been a member of a group that advocated the violent overthrow of the united states government?"...they even told me what all the questions were before the test even began!) over and over and over for more an hour was enough to drive me slowly crazy. The questioner said that some folks start hallucinating halfway through it. Awesome.

Anybody else have fun polygraph stories?

Edit: Oh yeah, I passed ;)

[CFLarsen]

I'm confused. CFLarsen, do you dispute that a measurable phenomenon exists? Or simply that the phenomenon is too slight and variable across different conditions to be of any value?

http://fas.org/sgp/othergov/polygraph/ota/index.html
http://www.nap.edu/openbook.php?isbn=0309084369

The meta-analyses seem to demonstrate that, more often than would be expected by chance, the results of lab experiments deviate from what would be expected by chance. This is the meta-result, so to speak. Does that not suggest that there is a measurable phenomenon occurring? The experiments were far from perfect, but many appeared to have adequate control and blinding.

Whether the difference in lying and truthful populations can be used to address any specific question, especially across the myriad cases and conditions encountered in the real world...is a great question but not the one I'm asking here.

If there really is a net difference, statistically deviating from what would be expected by chance, it would indicate that the polygraph correctly categorize liars and truthers at a rate greater than simple guesswork, given a sufficiently large sample size. And that would be an interesting phenomenon.

It sure would. And if the scientific evidence showed that polygraphs could tell us who lied and who told the truth, it would be a tremendous tool to fight any kind of deceit, be it when screening government employees, employees in private companies, possible spies or people suspected of crimes.

That's why it is so important for proponents of the polygraph to state their case in a forum of skeptics. If skeptics are not able to distinguish between pseudoscience and real science, who is?

Oddly enough, it seems to be very hard to get proponents of the polygraph to step up and defend their - purportedly scientifically supported - claim that the polygraph can tell if people lie or not.

Now....why is that?

[TV's Frank]

No, let's not play games. Let's just acknowledge that I think that polygraphs are unreliable.

Do you think I have any reason to think otherwise?



There's no evasion.

I think polygraphs are unreliable. I have extensively explained what my stance is wrt to polygraphs. The scientific evidence that they work is simply not compelling. Not only do major skeptics agree with this, the scientific community is also with me on this one.

That's not childish. At all.

Dude, I am aware of the fact that you think polygraphs are unreliable. I know what your stance is. Please describe what you mean by "unreliable". Do you mean "performs no better than chance"? Or something else? I really don't understand why you're fighting this so much. I might even agree with you, if you would just tell me! I just want to know, so we have a common ground! Please!

[Myshkin]

Originally Posted by CFLarsen
The reason why polygraphs reportedly "work" is not because you can tell from the readings that people lie. You can use polygraphs to intimidate those who don't know that it is bull.

But this isn't what was being measured in the many lab experiments, mostly with faux crimes albeit. They were looking at the polygraph, blind to the truth-state of the participant.

[CFLarsen]

Personally, I'm not surprised that the "sifting for spies" doesn't perform as well as "catching the crook". In a police investigation, I image the questions are more detailed, direct, and emotional (i.e. "did you stab mrs. jameson in the neck?") . When sifting, you have to keep to generalities (i.e. "have you ever been contacted by an agent of a foreign government?") One might find it easier to lie about the latter case.

Mind you, this is all made up and probably wrong.

As a possibly related and slightly amusing anecdote, I was once the subject of a polygraph test by the NSA (it was for a job). For those of you wondering about observer bias, the questioner told me that the polygraph results were examined by three independent examiners, who were not present during the questioning. The questioner simply adminstered the test and recorded spurius results (such as a cough). A failure at the examination would not automatically disqualify you (they were away of the high false-positive rate), but simply "flagged" you for more detailed scrutiny.

Anyway, I came pretty close to passing out during the test. Sitting in a comfy chair, completely aware of every single heartbeat and breath, and answering inane questions (like "were you born in milwaukee" and "have you ever been a member of a group that advocated the violent overthrow of the united states government?"...they even told me what all the questions were before the test even began!) over and over and over for more an hour was enough to drive me slowly crazy. The questioner said that some folks start hallucinating halfway through it. Awesome.

Anybody else have fun polygraph stories?

Edit: Oh yeah, I passed ;)

You hunt for spies because the damage they can do is a hell of a lot more serious than the odd homicide. We are not talking about offing the odd enemy, we are talking about jeopardizing whole nations.

If anything, the screening for spies would be much more tough than any other scenario I could think of.

That's a bit of an oversimplification, but at least it's related to what I was saying.

Unfortunately, it isn't an oversimplification. Polygraphs are solely methods of intimidation, to be used on the uninitiated.

Let's say that polygraphs were generally applied around the world. Those who did their homework could pass tests more easily. The result would be that the less knowledgable about cheating would not get the job.

Using the polygraph in general would mean that those who were able to cheat it, would get the most trusted positions.

Thereby totally undermining the purpose of the polygraph.

Do you mean to say that police do not lie and exaggerate in interviews? Or are you saying they should be discouraged from doing so?

They cannot lie and exaggerate to impede justice.

If your suspect was superstitious, would you threaten to cast a curse, if he didn't confess?

[Rodney]

Any of the time? Be serious.

We know it's been correct at least three times, because it correctly identified Kari as telling the truth, and Tore and Grant as lying.
More importantly, it identified Harold Nicholson, a CIA employee later convicted of spying for Russia, as lying. See http://forums.randi.org/showthread.php?postid=2728166#post2728166

[CFLarsen]

Dude, I am aware of the fact that you think polygraphs are unreliable. I know what your stance is. Please describe what you mean by "unreliable". Do you mean "performs no better than chance"? Or something else? I really don't understand why you're fighting this so much. I might even agree with you, if you would just tell me! I just want to know, so we have a common ground! Please!

I'm not fighting at all. What is it about the word "unreliable" you don't understand?

But this isn't what was being measured in the many lab experiments, mostly with faux crimes albeit. They were looking at the polygraph, blind to the truth-state of the participant.

That's the point: Why would you be truly afraid to lie about a faux crime?

Think about it: If you want to argue that you can be truly afraid to lie about a faux crime, then you also admit that the sentiments recorded by the polygraph aren't real.

[TV's Frank]

You hunt for spies because the damage they can do is a hell of a lot more serious than the odd homicide. We are not talking about offing the odd enemy, we are talking about jeopardizing whole nations.

If anything, the screening for spies would be much more tough than any other scenario I could think of.



While only an anecdote, my personal experience related above suggests otherwise.

[Myshkin]

That's why it is so important for proponents of the polygraph to state their case in a forum of skeptics. If skeptics are not able to distinguish between pseudoscience and real science, who is?

Oddly enough, it seems to be very hard to get proponents of the polygraph to step up and defend their - purportedly scientifically supported - claim that the polygraph can tell if people lie or not.

Whether a polygraph "can tell if people lie or not" is the larger question. The simpler question, and maybe the best starting point to separate science from pseudoscience is "are the populations of polygraph metrics from lying and truthful respondents two statistically distinct populations (i.e. with differing mean and/or variance in metrics), or are they one?" What do you say?

[TV's Frank]

I'm not fighting at all. What is it about the word "unreliable" you don't understand?

I know the dictionary definition of the word "unreliable". But say a claimed psychic were to come to you and say, "I can reliably predict the future". You might say, "What do you mean? Do you mean 70% accurate? 20%? Better than chance? Make a specific claim so we can test it!" Are you saying that polygraph performs no better than chance? Yes? No? Something else?

[Cardelitre]

No, that's not what I said, and it's not what you asked me. You asked me if I could find one experiment that is more representative.
Very well. Then, maybe you could point me to any study of this type, even one that you don't find particularly representative?
My bad, then. I thought you knew about Bob. Bob is an institution in skepticism. In the US maybe, but the US is not the rest of the world.
No, it doesn't. It says that in some situations under some conditions, they sometimes perform better than change. What we have is a "Charlie Brown" scenario: Wishy-washy.
Strangely, that’s not what I could read in this same report. I should quote here this post (http://forums.randi.org/showpost.php?p=3296842&postcount=36) of drkitten that apparently you forgot to address:
Theory and basic research give no clear guidance about whether laboratory conditions underestimate or overestimate the accuracy that can be expected in realistic settings.

Available data are inadequate to test these hypotheses. [...]

Evidence from Medical Diagnostic Testing. Substantial experience with clinical diagnostic and screening tests suggests that laboratory models, as well as observational field studies of the type found in the polygraph literature, are likely to overstate true polygraph accuracy. Much information has been obtained by comparing observed accuracy when clinical medical tests are evaluated during development with subsequent accuracy when they become accepted and are widely applied in the field. An important lesson is that medical tests seldom perform as well in general field use as their performance in initial evaluations seems to promise (Ransohoff and Feinstein, 1978; Nierenberg and Feinstein, 1988; Reid, Lachs, and Feinstein, 1995; Fletcher, Fletcher, and Wagner, 1996; Lijmer et al., 1999).

The reasons for the falloff from laboratory and field research settings to performance in general field use are fairly well understood.
In other words, the problem is with the specific numeric claims of accuracy, not with the "better than chance" and is to be expected in any transition from laboratory to field.

As well as:
In view of the above issues, we believe that the range of accuracy indexes (A) estimated from the scientifically acceptable laboratory and field studies, with a midrange between 0.81 and 0.91, most likely over-states true polygraph accuracy in field settings involving specific-incident investigations.
Thus, what is specifically overstated are the estimates (from 0.81 to 0.91), not the simple better-than-chance accuracy.
Or also:
The available evidence indicates that in the context of specific-incident investigation and with inexperienced examinees untrained in countermeasures, polygraph tests as currently used have value in distinguishing truthful from deceptive individuals. However, they are far from perfect in that context, and important unanswered questions remain about polygraph accuracy in other important contexts. No alternative techniques are available that perform better, though some show promise for the long term. The limited evidence on screening polygraphs suggests that their accuracy in field use is likely to be somewhat lower than that of specific-incident polygraphs.
There is no suggestion that this "value" is in any way overstated, just the standard note that screening is harder than incident investigation.

[...]

Finally, on p. 214, they present the ultimate statement of their finding on accuracy(emphasis in original).

Notwithstanding the limitations of the quality of the empirical research and the limited ability to generalize to real-world settings, we conclude that in populations of examinees such as those represented in the polygraph research literature, untrained in countermeasures, specific-incident polygraph tests for event-specific investigations can discriminate lying from truth telling at rates well above chance, though well below perfection.

Given that, at this point, they have formally recognized that polygraphs work in lab conditions something like eight times in the report, I see no reason for you -- or for anyone -- to dispute the basic fact that they work (in lab conditions).

This looks to me very far from the summary you do that "in some situations under some conditions, they sometimes perform better than chance."
It isn't up to skeptics to prove that polygraphs don't work. It is up to proponents of the polygraph to prove that they work.
Yes, you are perfectly right. And it seems clear to me that the NAS report did just that. Now some skeptics make the claim that this is in fact pseudoscience, but still don't see any study (or meta-analyse) disproving those previous results. Why?
If you point to meta-analyses, why do you insist on one single study when it comes to polygraphs?

That is totally inconsistent of you.
I'm truly sorry if I looked inconsistent to you, I really try to be as clear as I can. So let me please rephrase: could you present a single study or multiple studies and/or meta-analyses that show that polygraphs are unable to detect liars in a sample population better than chance? Any of those will do, for the sake of discussion...

[David Wong]

I'm not fighting at all. What is it about the word "unreliable" you don't understand?


"I'm not fighting at all. Now let me ask you this nasty, insulting, sarcastic question."

Observing this thread has been fascinating because it's a great demonstration of how unprofitable discussions are when one party is in "battle mode." There is no sharing of information; just ground to be seized. Every piece of information, every phrasing of every sentence, is thought of in terms of winning the fight and nothing more.

He has that seige mentality some people get when they argue; they won't clarify what they mean because they're afraid any information they give will be used against them by the enemy. Information is nothing more than ammunition.

You look at the snide tone, the "SUBMIT A PAPER TO MY SITE OR ADMIT YOU'RE WRONG" bullying, the sarcasm... it's not about advancing anyone's understanding. He may have a point but he's advancing it using the tactics of a right-wing talk radio host.

Maybe it's just his personality, I don't know.

[Myshkin]

That's the point: Why would you be truly afraid to lie about a faux crime?

Think about it: If you want to argue that you can be truly afraid to lie about a faux crime, then you also admit that the sentiments recorded by the polygraph aren't real.

I don't know. That's what is so interesting. There was a treatment randomly applied - you lie or you don't. More often than would be expected by chance, the respondents were correctly identified as either having lied or not. That indicates a that something differed in the polygraph readings of liars and truth-tellers. If the studies were valid, and the meta-analysis valid, then it appears that liars and truth-tellers are having measurably different polygraphs...on average.

It is possible that most of the studies were had directional bias in their design, but in looking briefly at some of them, I can't see anything obvious.

I am assuming, then, that you think that the design, implementation, and analysis of the studies or the meta-analysis were directionally biased. Is that right?

[Zalbik]

Not only do major skeptics agree with this, the scientific community is also with me on this one.
That's not childish. At all.

CFLarsen, can you provide some scientific reports that conclude that polygraph tests do not perform significantly better than chance?

I found the report that skeptigirl provided very surprising.

I had thought polygraphs were pretty much debunked as any sort of detection tool and were only really used due to historical woo-like belief in their accuracy.

I'm a bit floored that the report indicates that they work "well" beyond random chance...

I'd like to see any studies that debunk this.

[Cardelitre]

I'm a bit floored that the report indicates that they work "well" beyond random chance...

I'd like to see any studies that debunk this.
Yes, me too. But this is unexpectedly harder to obtain than what I imagined...

[Jimbo07]

Unfortunately, it isn't an oversimplification. Polygraphs are solely methods of intimidation, to be used on the uninitiated.

It depends. I think this thread has enough information available to indicate that this assertion is probably somewhat incorrect.


Let's say that polygraphs were generally applied around the world.


Will they? Have they? I wasn't aware that this was a problem. That's why I asked if there was some big political thing afoot in the States: We gotsta make all them thar Mexicans take a lie detector! (sorry :o)


Using the polygraph in general would mean that those who were able to cheat it, would get the most trusted positions.

Anyone who allowed someone into a most trusted position, based solely on a polygraph, even if effective, would deserve what they got...


They cannot lie and exaggerate to impede justice.


How common, exactly, do you think that is?


If your suspect was superstitious, would you threaten to cast a curse, if he didn't confess?

I'm not a police officer, and I refuse to ask one for you, to avoid confirming a suspicion that we internet debaters are morons. ;)

[CFLarsen]

While only an anecdote, my personal experience related above suggests otherwise.

I think your anecdote says a hell of a lot about how polygraphs are used.

Whether a polygraph "can tell if people lie or not" is the larger question. The simpler question, and maybe the best starting point to separate science from pseudoscience is "are the populations of polygraph metrics from lying and truthful respondents two statistically distinct populations (i.e. with differing mean and/or variance in metrics), or are they one?" What do you say?

What is the difference?

I know the dictionary definition of the word "unreliable". But say a claimed psychic were to come to you and say, "I can reliably predict the future". You might say, "What do you mean? Do you mean 70% accurate? 20%? Better than chance? Make a specific claim so we can test it!" Are you saying that polygraph performs no better than chance? Yes? No? Something else?

They are unreliable because they work by intimidation. Similarly, torture doesn't work to extract truthful information either.

The technology isn't gauging whether you lie or not. It intimidates you to become confused so you trip up.

Very well. Then, maybe you could point me to any study of this type, even one that you don't find particularly representative?

Where are you going with this?

In the US maybe, but the US is not the rest of the world.

Not just in the US.

Strangely, that’s not what I could read in this same report. I should quote here this post (http://forums.randi.org/showpost.php?p=3296842&postcount=36) of drkitten that apparently you forgot to address:

No, I didn't "forget" to address it.

This looks to me very far from the summary you do that "in some situations under some conditions, they sometimes perform better than chance."

Yes, you are perfectly right. And it seems clear to me that the NAS report did just that. Now some skeptics make the claim that this is in fact pseudoscience, but still don't see any study (or meta-analyse) disproving those previous results. Why?

Oh, it isn't just "some skeptics". The scientific community is still unconvinced.

Why?

I'm truly sorry if I looked inconsistent to you, I really try to be as clear as I can. So let me please rephrase: could you present a single study or multiple studies and/or meta-analyses that show that polygraphs are unable to detect liars in a sample population better than chance? Any of those will do, for the sake of discussion...

That is inconsistent of you to ask for a single study.

"I'm not fighting at all. Now let me ask you this nasty, insulting, sarcastic question."

Observing this thread has been fascinating because it's a great demonstration of how unprofitable discussions are when one party is in "battle mode." There is no sharing of information; just ground to be seized. Every piece of information, every phrasing of every sentence, is thought of in terms of winning the fight and nothing more.

He has that seige mentality some people get when they argue; they won't clarify what they mean because they're afraid any information they give will be used against them by the enemy. Information is nothing more than ammunition.

You look at the snide tone, the "SUBMIT A PAPER TO MY SITE OR ADMIT YOU'RE WRONG" bullying, the sarcasm... it's not about advancing anyone's understanding. He may have a point but he's advancing it using the tactics of a right-wing talk radio host.

Maybe it's just his personality, I don't know.

It is very much advancing understanding. We don't see an acceptance of the polygraph from the scientific community, but two skeptics suddenly say that polygraphs work - and that there is scientific evidence.

That is exactly what skeptics are looking for. That's why I would like to see skeptigirl and drkitten submit their articles/presentations to a larger skeptical audience.

I don't know. That's what is so interesting. There was a treatment randomly applied - you lie or you don't. More often than would be expected by chance, the respondents were correctly identified as either having lied or not. That indicates a that something differed in the polygraph readings of liars and truth-tellers. If the studies were valid, and the meta-analysis valid, then it appears that liars and truth-tellers are having measurably different polygraphs...on average.

It is possible that most of the studies were had directional bias in their design, but in looking briefly at some of them, I can't see anything obvious.

I am assuming, then, that you think that the design, implementation, and analysis of the studies or the meta-analysis were directionally biased. Is that right?

You don't know? Do you think that you are equally afraid from admitting a faux lie than a real one?

CFLarsen, can you provide some scientific reports that conclude that polygraph tests do not perform significantly better than chance?

I found the report that skeptigirl provided very surprising.

I had thought polygraphs were pretty much debunked as any sort of detection tool and were only really used due to historical woo-like belief in their accuracy.

I'm a bit floored that the report indicates that they work "well" beyond random chance...

I'd like to see any studies that debunk this.

First, I'd like to see the scientific community accept this claimed conclusion that polygraphs work.

It depends. I think this thread has enough information available to indicate that this assertion is probably somewhat incorrect.

Then, let's put it out to a larger skeptical audience.

Are there any downsides to that?

Anyone who allowed someone into a most trusted position, based solely on a polygraph, even if effective, would deserve what they got...

Even if a polygraph is only one part of the metods, it still adds up.

How common, exactly, do you think that is?

I don't know. We do see cases where police brutality has been used to coerce a confession and result in injust convictions.

I'm not a police officer, and I refuse to ask one for you, to avoid confirming a suspicion that we internet debaters are morons. ;)

Why is that so different from using a polygraph? Polygraphs "work" because people believe they do.

[NeilC]

"I'm not fighting at all. Now let me ask you this nasty, insulting, sarcastic question."

Observing this thread has been fascinating because it's a great demonstration of how unprofitable discussions are when one party is in "battle mode." There is no sharing of information; just ground to be seized. Every piece of information, every phrasing of every sentence, is thought of in terms of winning the fight and nothing more.

He has that seige mentality some people get when they argue; they won't clarify what they mean because they're afraid any information they give will be used against them by the enemy. Information is nothing more than ammunition.

You look at the snide tone, the "SUBMIT A PAPER TO MY SITE OR ADMIT YOU'RE WRONG" bullying, the sarcasm... it's not about advancing anyone's understanding. He may have a point but he's advancing it using the tactics of a right-wing talk radio host.

Maybe it's just his personality, I don't know.

Totally. This has to be one of the most tedious discussions on a subject that is potentially interesting that I've ever read - and mostly because CFLarson is acting as you describe. No wonder skeptics get a bad name.

[CFLarsen]

Totally. This has to be one of the most tedious discussions on a subject that is potentially interesting that I've ever read - and mostly because CFLarson is acting as you describe. No wonder skeptics get a bad name.

Then focus on the argument, not the arguer.

[NeilC]

I'd like to but your ridiculous manner of arguing (not discussing) in this thread makes it very difficult.

[CFLarsen]

I'd like to but your ridiculous manner of arguing (not discussing) in this thread makes it very difficult.

I'm sure you are able to separate the argument from the arguer.

[Cardelitre]

Where are you going with this?
As I explained this maybe 4-5 times already, I just would like to see data confirming that polygraphs don't perform better than chance, data that would oppose those gathered by the NAS report. What is so hard to understand here? Isn't this some basic skeptical methodology, looking at reliable experiments in order to forge his opinion?
Not just in the US.If you say so. Still, he is perfectly unknown where I live.
No, I didn't "forget" to address it.Maybe it was intentional then, or I am developing visual problems, because I just can’t see where you addressed these points in this thread, could you show me where you did exactly?
Oh, it isn't just "some skeptics". The scientific community is still unconvinced.
Are the scientists who produced the NAS report part of the scientific community? Because their conclusion is quite different on this issue…
Why?
Probably because they base their judgment on experiments showing that polygraphs don’t work. I would really like to see them also, so I can make my opinion on an informed basis. Your opinion seems to be already definitive, I guess you could point me to one of these experiments, then?
That is inconsistent of you to ask for a single study.
As I already said, you make perfect sense here. That’s why I now prefer to ask multiple study and/or meta-analyses supporting your claim. Would you be kind enough to cite me some?

[Cuddles]

I'm sure you are able to separate the argument from the arguer.

Now, if only you'd actually make an argument, there might be something for people to address.

[TV's Frank]

They are unreliable because they work by intimidation. Similarly, torture doesn't work to extract truthful information either.


I did not ask why or why not they are unreliable. Before we can even begin such a discussion, we must agree on what you mean by "unreliable".

Do you mean that polygraph performs no better than chance?

Are you admitting the polygraph DOES perform better than chance? But that any polygraph results that are better than chance are due to intimidation, and not any objective performance of the machine?

I have a feeling that the latter is what you are claiming. If so, your next post can go something like this:

"If we control for the effects of intimidation, the results of polygraph perform no better than chance. This position is supported by:

CITATION

CITATION

CITATION"

Please provide the citations.

[Lucky]

Polygraphs have all the characteristics of a woo claim: The theory behind it is very weak, the evidence is very weak, there is no standardization, there is no scientific progress, there is no accumulated knowledge or accumulated evidence in favor of it.
Please provide the scientific evidence that the error rate in polygraphs is close to zero.

This is what this thread is entirely about.
Calling it a "laboratory trick" is very spot on.
It's an example of pseudoscience, which definitely is a subject skeptics are interested in.
Polygraphs are pseudoscience. They cannot be regarded as reliable tools to tell whether people lie or not.
So:
Cervical smear tests are pseudoscience because they don't reliably detect all cancers and ignore the healthy condition.
Academic examinations are pseudoscience because they don't reliably grade candidates into discrete categories of knowledge and ability.
Post mortems are pseudoscience because they don't reliably distinguish between death by natural and unnatural causes.

Do you begin to see the logical fallacy?

Do you seriously believe that any test (forensic, medical, whatever) that doesn't have false positive and false negative rates close to zero is 'pseudoscience'?

If the evidence really is strong enough (as skeptigirl claims), then we cannot refrain from using it.
And if the scientific evidence showed that polygraphs could tell us who lied and who told the truth, it would be a tremendous tool to fight any kind of deceit, be it when screening government employees, employees in private companies, possible spies or people suspected of crimes.

That's why it is so important for proponents of the polygraph to state their case in a forum of skeptics. If skeptics are not able to distinguish between pseudoscience and real science, who is?

Oddly enough, it seems to be very hard to get proponents of the polygraph to step up and defend their - purportedly scientifically supported - claim that the polygraph can tell if people lie or not.
Throughout this thread you have consistently pointed to the lack of proven utility of the polygraph in real-world settings as a supposed rebuttal to the scientific validity of polygraphy as a concept. This is quite illegitimate, and in order to further your understanding of the subject (and the area of screening and testing in general), it is essential to separate the purely scientific questions from considerations of utility and public policy.

It is entirely possible for the scientific basis of the polygraph to be valid but unfortunately such that no possible refinement of the method can improve accuracy to a useful level – that is a common occurrence in many scientific areas.

p122 to 124 of the National Academies report (http://books.nap.edu/openbook.php?record_id=10420&page=R1) summarise the results from 52 datasets from laboratory studies (all the studies of sufficient quality from a literature search).

First, the data (and their errors of estimate; see Appendix H, Figure H-3 (http://books.nap.edu/openbook.php?record_id=10420&page=345)) clearly fall above the diagonal line, which represents chance accuracy. Thus, we conclude that features of polygraph charts and the judgments made from them are correlated with deception in a variety of controlled situations involving naïve examinees untrained in countermeasures: for such examinees and test contexts, the polygraph has an accuracy greater than chance. Random variation and biases in study design are highly implausible explanations for these results, and no formal integrative hypothesis test seems necessary to demonstrate this point.
Second, with few exceptions, the points fall well below the upper left-hand corner of the figure indicative of perfect accuracy. No formal hypothesis test is needed or appropriate to demonstrate that errors are not infrequent in polygraph testing.
Third, variability of accuracy across studies is high. This variation is likely due to a combination of several factors: “sampling variation,” that is, random fluctuation due to small sample sizes; differences in polygraph performance across testing conditions and populations of subjects; and the varying methodological strengths and weaknesses of these diverse studies. The degree of variation in results is striking.
...
It is important to emphasize that these data and their descriptive statistics represent the accuracy of polygraph tests under controlled laboratory conditions with naïve examinees untrained in countermeasures, when the consequences of being judged deceptive are not serious. We discuss below what accuracy might be under more realistic conditions.

http://forums.randi.org/imagehosting/thum_2550477e34c28029c.jpg (http://forums.randi.org/vbimghost.php?do=displayimg&imgid=10080)

Do you understand what the ROC figure is telling us? (If not, do a search on 'receiver operating'.) Considered solely as a test of whether the claimed effect is real, these results are extremely convincing. They are also well within the range of accuracy required in many real-world screening (though not diagnostic) tests. (If, for example, a pregnant woman has a routine serum screening test for fetal chromosomal anomaly, then the accuracy of that test is reasonably well represented by the better of the two ROC curves above.)

If Randi etc. are claiming that polygraphy 'does not work' in the scientific sense - that it cannot be shown to be a real, repeatable effect, independent of subjective, unblinded, post hoc rationalisation - then I am confident that they cannot back up that claim. On what evidence do they base their belief?

The polygraph is a tool that "works" by intimidation: The subject is put under strain from the technobabble and pseudoscience: "We can see that you are lying - so confess!"

Of course, if the polygraph really could tell if people lie or not, there would not be any need for any confession....
The reason why polygraphs reportedly "work" is not because you can tell from the readings that people lie. You can use polygraphs to intimidate those who don't know that it is bull.

But that doesn't mean they should be used. We don't use pseudoscience, we shouldn't encourage it.

If your suspect was superstitious, would you threaten to cast a curse, if he didn't confess?
Polygraphs are solely methods of intimidation, to be used on the uninitiated.
They are unreliable because they work by intimidation. Similarly, torture doesn't work to extract truthful information either.

The technology isn't gauging whether you lie or not. It intimidates you to become confused so you trip up.
...
Polygraphs "work" because people believe they do.
No, the above tests were performed under laboratory conditions, where the questions were of no real-world significance, subjects had no interest in the results, and testers did not know the answers. The results can have nothing whatever to do with the factors you suggest. This point is obviously of extreme importance in the context of claims from you (and, taking your word for it, Randi and Shermer) that the effect is non-existent and the subject matter is pseudoscience.

The evidence isn't getting stronger, no standardized method is emerging, and the more we know of the basic premise, the less reason we have to believe it works.

It's quite similar to homeopathy.
Note that all the evidence turned up by the report's literature search (though not of the highest quality) indicated a real effect. Almost the entire body of published data suggests that, in laboratory conditions, polygraph tests give results that are better than chance. How on earth can you (or anyone) suggest that this is in any way comparable to studies of homeopathy? (I would say it's more akin to intelligence testing, but that's another discussion.)

Now, the point is made throughout the report that successful laboratory tests may not translate to successful applications, and in the case of polygraphy there has been a striking lack of progress. The inevitable methodological problems (in particular, inability of the technique to cope with a trained suspect) quite likely cannot be solved. And it is certainly true and relevant that throughout the polygraph's history it has been the subject of wildly exaggerated claims of its efficacy (politically and financially motivated).

However, exaggerated claims, limited discriminatory power of the technique, inadequate theory, flawed studies and lack of proven utility do not equal pseudoscience. Polygraphy may not be particularly useful science, and (from the quality of some of the studies included in the NA report) it may not so far be very well conducted science, but it is most decidedly not pseudoscience. (Even if larger and better studies eventually found the effect to be small and unreliable, and based on a mistaken theory, it would be faulty science, not pseudoscience.) I hope you will take a sceptical look at the evidence and stop propagating this myth.

[Belz...]

No, let's not play games. Let's just acknowledge that I think that polygraphs are unreliable.

Do they perform better than chance, yes or no ?

[CFLarsen]

People,

You are missing the point.

You can, in fact, use polygraphs to tell if people lie or not. What you can't use polygraphs for is to tell if people lie or not by reading the output.

Readings from polygraphs do not show if people lie or not. Using polygraphs puts people in a state of stress so they get confused, so they mess up their confession. That's what interrogations also do: You ask the same questions, again and again, in order to find out if the suspect has his story straight. Using a polygraph, you merely introduce a technobabble factor, to impress the lesser informed.

A polygraph is a an intimidation tool.

I'll return to this:

The primary difference is that there are no variants of astrology that reproducibly test positively, and there are many variants of polygraphy that do.

Astrology does "work": You can, in fact, tell people's futures and reveal their lives from casting their horoscope. Not because the planets influence our lives, but because of cold reading.

If you want to argue that polygraphs work the way they are claimed to work, then:

Explain why the scientific community isn't convinced.

Explain what harm it would do to present the argument at TAM and/or write articles for skeptical/scientific journals.

[Jimbo07]

Is this a derail?

Isn't the idea of detecting a lie something of a philosophical difficulty, long before anyone considers using physical machines? My first point is that the real world does not seem to lend itself well to reduction to binary TRUE/FALSE statements. My second point is that human psychology is a factor. Some people honestly believe they have done nothing wrong. Others try to pull evasive tricks. Others (though I don't believe this is as common as feared) may be bullied into confessions (which raises another question: do some deserve to be bullied into confessing). My last point is in the nature of a lie itself. Related to the first point, do you need simple TRUE/FALSE statements to have a lie? What about lies of omission? What about a dishonesty so deep that it's hard to tell the line between truth and lies anymore?

So...

... are there some physical parameters which can be shown to correlate with times when, by consensus perhaps, we'd generally determine that someone is lying? Maybe. I don't know. Perhaps it is these phenomena that are being studied (and, from what is already being said in this thread, I think I'm being redundant). I believe that posters in this thread have sufficiently demonstrated (at least for my own taste), that there is something to an impartial reading of a polygraph result.

I think that somewhere there has been some sort of bogeyman strawman constructed regarding polygraph testing. It may arise from some American political situation (wherein every sorry slob is administering polygraph tests to their own cats and dogs). I don't know.

It's not pseudoscience, by any stretch of the imagination.

I still prefer to think of it as a black magic art! :D

[TV's Frank]

People,

You are missing the point.

You can, in fact, use polygraphs to tell if people lie or not. What you can't use polygraphs for is to tell if people lie or not by reading the output.


So if we control for the intimidation, and just read outputs, should polygraph perform the same as chance? Do you have any citations for this, because this experiment would be very compelling.

[Thanz]

You can, in fact, use polygraphs to tell if people lie or not. What you can't use polygraphs for is to tell if people lie or not by reading the output.

Readings from polygraphs do not show if people lie or not. Using polygraphs puts people in a state of stress so they get confused, so they mess up their confession. That's what interrogations also do: You ask the same questions, again and again, in order to find out if the suspect has his story straight. Using a polygraph, you merely introduce a technobabble factor, to impress the lesser informed.
Do you have anything to back this up? The portions of the NAS report quoted by Lucky, with the graph, are pretty clear indications that you are incorrect. Again - they emphasize that "It is important to emphasize that these data and their descriptive statistics represent the accuracy of polygraph tests under controlled laboratory conditions with naïve examinees untrained in countermeasures, when the consequences of being judged deceptive are not serious."

The consequences are not serious. Therefore, not the intimidating interrogation that you are putting forward. And, all of the data showed a better than chance detection of deception. That is strong indication that the theory behind the polygraph has some merit.

If you want to argue that polygraphs work the way they are claimed to work, then:

Explain why the scientific community isn't convinced.
Show me that the scientific community isn't convinced, as you claim. The NAS are part of the scientific community, and they seem convinced that there is some merit to them.

Explain what harm it would do to present the argument at TAM and/or write articles for skeptical/scientific journals.
Explain the benefit. THe NAS study has been published - and we can all already read it. Can you show us a rebuttal paper that refutes the NAS conclusions?

In short, evidence has been presented to you that shows ploygraphs perform better than chance at detecting deception. Do you have any evidence to the contrary?

[Lucky]

People,

You are missing the point.

You can, in fact, use polygraphs to tell if people lie or not. What you can't use polygraphs for is to tell if people lie or not by reading the output.

Readings from polygraphs do not show if people lie or not. Using polygraphs puts people in a state of stress so they get confused, so they mess up their confession. That's what interrogations also do: You ask the same questions, again and again, in order to find out if the suspect has his story straight. Using a polygraph, you merely introduce a technobabble factor, to impress the lesser informed.

A polygraph is a an intimidation tool.
No, CFLarsen, you are missing the point that the National Academies report (http://books.nap.edu/openbook.php?record_id=10420&page=R1) examines all the data (from a literature search by the authors) that has been produced under laboratory conditions, and this data clearly shows that polygraph tests in these studies performed well above chance levels.

This research data is from studies in which:
the questions were of no real-world significance
and
the results had no consequences for the subjects
and
the testers did not know the answers.

Therefore the results can have nothing whatever to do with technobabble, intimidation, fear or gullibility.

Note that I am making no claims about the utility of polygraphy as a forensic technique. My post here (http://forums.randi.org/showthread.php?postid=3302366#post3302366) explains all of this in detail.

[Belz...]

People,

You are missing the point.

You can, in fact, use polygraphs to tell if people lie or not. What you can't use polygraphs for is to tell if people lie or not by reading the output.

Readings from polygraphs do not show if people lie or not. Using polygraphs puts people in a state of stress so they get confused, so they mess up their confession. That's what interrogations also do: You ask the same questions, again and again, in order to find out if the suspect has his story straight. Using a polygraph, you merely introduce a technobabble factor, to impress the lesser informed.

A polygraph is a an intimidation tool.

I'll return to this:



Astrology does "work": You can, in fact, tell people's futures and reveal their lives from casting their horoscope. Not because the planets influence our lives, but because of cold reading.

If you want to argue that polygraphs work the way they are claimed to work, then:

Explain why the scientific community isn't convinced.

Explain what harm it would do to present the argument at TAM and/or write articles for skeptical/scientific journals.

Do they perform better than chance, yes or no ?

[drkitten]

You can, in fact, use polygraphs to tell if people lie or not.

Well, I'm glad you finally acknowledged what was obvious from reading the ROC curves.

What you can't use polygraphs for is to tell if people lie or not by reading the output.

This statement, however, continues to be incorrect, as a number of citations in this thread (taken from the NAS report) indicate. For example, Kircher's study : "The independent evaluator, who numerically scored the charts blindly, correctly diagnosed 87 percent of the subjects." in other words, just by reading the output, he was able to tell if people lie or not with 87 percent accuracy.

Given that he's documented as doing the impossible, I suggest your claims of impossibility are overstated.

Similarly, "different computer decision models, on the average, correctly identified 84.9 percent of subjects." Again, the "impossible" is apparently routine. (And, of course, computer technology is such that the computers couldn't even UNDERSTAND the human answers; the idea of a "lie" as we understand it is inexpressible to computers. But separating hyperplanes in a dataset -- now THAT they understand.)

Readings from polygraphs do not show if people lie or not.

Except that both people and computers can determine from those readings if people are lying, with better than 80% accuracy.

[Myshkin]

...What you can't use polygraphs for is to tell if people lie or not by reading the output.

Just to give this dead horse one more whack...the scientific data do not support this claim, and in fact the data generally support quite the opposite, with qualifications.

Why? I don't know. Maybe we are so conditioned to "always tell the truth" that, even when it is explicitly requested for the purposes of an experiment that you must lie, we get uncomfortable about it...as silly as that may be. People blush when they're embarrassed, sweat when they're frightened, and maybe have some subtle physical reactions when they lie.

Look at the data.

ETA - I see the rude and condescending tags have been removed. Thanks.

[digithead]

Now having read through this thread and since I do research into this area, I'd like to clear up some misconceptions...

Firstly, the polygraph itself measures what it measure accurately, i.e. physiological response to questions. What is controversial is the method in which questions are asked. There are two basic forms of polygraph test. The first, most widely used and certainly the most controversial is the Control Question Test. Please note in this context that "control" does not have the same meaning as a scientific control in that responses and questions are not standardizable in any fashion. The CQT was the form of polygraphy that the NAS reviewed.

What makes CQT polygraphy so controversial is that it is based on emotional response to questions rather than cognitive response. The NAS concluded that lie detector tests based on emotional response of a subject have no scientific basis. This emotional basis of polygraph lie detection makes it inherently flawed because the polygraph cannot accurately distinguish between the innocent but nervous and the guilty but nervous. In addition, little or no research has been performed on the effects of undiagnosed cardiovascular, endocrinological, and mental illnesses on so-called psychophysiological detection of deception but one can surmise that it's probably not in the direction of improved accuracy. In screening applications, this is especially important.

The reason that CQT polygraphy performs above chance but well below perfection for specific incidents is because specific incidents also involve cognitive responses from subjects which increases its accuracy, however minimal it might be. There is a cognitively based polygraph test called the Guilty Knowledge Test (GKT) or Concealed Information Test (CIT). In this type of test, a series of multiple choice questions are asked of a subject consisting where each question has an answer that only the guilty person would know. This test is based on cognitive response and does have a scientific basis in that it can be expected through orienting response and habituation that humans will have a common physiological reaction to what they know, i.e. cognition. Additionally, GKTs do not rely on conditioned response, emotions, or concerns which are all used in CQT polygraph tests to detect deception.

Additionally, polygraphy is related to known psychological phenomenon called the "bogus pipeline." If you hook someone up to a machine (any machine) and convince the person that it can detect lies, then you get people to confess. If people are aware of the bogus nature of the machine, it does not have this effect.

Lastly, CQT polygraph itself does not measure deceit rather it measures only when the sympathetic autonomic nervous system has been activated via emotional response. If a subject shows a response during the polygraph examination, then the inference is made that the subject is deceptive. It requires a logical leap to assume that the response is due solely to deception because this response can also be attributed to fear of failing (rather than fear of detection), simple test anxiety, anger, or many medical or mental conditions. Simply put, nature did not equip us with a "Pinocchio's nose."

For those that want some reading material, see the following:

Faigman, D.L., S.E. Fienberg and P.C. Stern. 2003. The Limits of the Polygraph. Issues in Science and Technology 20(1):40.

Iacono, W.G. 2001. Forensic "Lie Detection": Procedures without Scientific Basis. Journal of Forensic Psychology Practice 1(1):75-86.

Jones, E.E., and H. Sigall. 1971. The Bogus Pipeline: A New Paradigm for Measuring Affect and Attitude. Psychological Bulletin 76:349-64.

Lykken, D. 1998. A Tremor in the Blood: Uses and Abuses of the Lie Detector, 2nd Edition. Reading, MA: Perseus.

Patrick, C.J. and W.G. Iacono. 1991. Validity of the Control Question Polygraph Test: The Problem of Sampling Bias. Journal of Applied Psychology 76(2):229-38.

[David Wong]

If you read the thread more closely, you'll see that all of your points have already been raised and addressed.

[digithead]

If you read the thread more closely, you'll see that all of your points have already been raised and addressed.

I read the thread fine. Some have been raised and partially (and sometimes incorrectly) addressed. Much of what is known about the polygraph is rooted in myth...

I also gave citations for people to follow up on...

[Reaver]

Additionally, polygraphy is related to known psychological phenomenon called the "bogus pipeline." If you hook someone up to a machine (any machine) and convince the person that it can detect lies, then you get people to confess. If people are aware of the bogus nature of the machine, it does not have this effect.


This reminds me of an episode of 'Balls of Steel':
http://www.youtube.com/watch?v=t7b0s5teLbc

[drkitten]

What makes CQT polygraphy so controversial is that it is based on emotional response to questions rather than cognitive response. The NAS concluded that lie detector tests based on emotional response of a subject have no scientific basis.

If they concluded it, I certainly didn't find it in the report. They were quite harsh on the voice stress analyzer (p. 168) : "Overall, this research and the few controlled tests conducted over the past decade offer little or no scientific basis for the use of the computer voice stress analyzer or similar voice measurement instruments as an alternative to the polygraph for the detection of deception. The practical performance of voice stress analysis for detecting deception has not been impressive. It is possible that research conducted in high-stakes situations would give better results, but we have not found reports of the accuracy of voice stress analysis in such situations."
I find no such statements in the NAS report.

Quite the contrary.


Marston (1917), Larson (1922), and Landis and Gullette (1925) all found elevated autonomic (blood pressure) responses when individuals engaged in deception. Marston (1917) described the underlying psychological state as fear; other writers have conceived it as arousal or excitement. The idea that fear or arousal is closely associated with deception provides the broad underlying rationale for the relevant-irrelevant test format. Subsequent research has confirmed that the polygraph instrument measures physiological reactions that may be associated with an examinee’s stress, fear, guilt, anger, excitement, or anxiety about detection or with an examinee’s orienting response to information (see below) that is especially relevant to some forbidden act. (p. 72)

You went on to write.

This emotional basis of polygraph lie detection makes it inherently flawed because the polygraph cannot accurately distinguish between the innocent but nervous and the guilty but nervous.

This statement is simply false and has been adequately addresed upthread.

In addition, little or no research has been performed on the effects of undiagnosed cardiovascular, endocrinological, and mental illnesses on so-called psychophysiological detection of deception but one can surmise that it's probably not in the direction of improved accuracy. In screening applications, this is especially important.

But not to the simple question of whether on not polygraphs work. We already know that field conditions make the problem more difficult. That doesn't mean that the problem is unsolvable. This argument is an irrelevancy at best.



The reason that CQT polygraphy performs above chance but well below perfection for specific incidents is because specific incidents also involve cognitive responses from subjects which increases its accuracy, however minimal it might be.

Well, I'm glad that you have achieved such a firm theoretical understanding of the reason that polygraphy works, something that apparently eluded the NAS in their extensive 2003 report. Is this understanding universal, or it is only your theory, and one not shared by the wider community?

But beyond that particular bit of sarcasm --- you've just admitted that "CQT polygraphy performs above chance but well below perfection." If you're arguing that CQT polygraphy doesn't work, why offer an explanation of why it does?



Lastly, CQT polygraph itself does not measure deceit rather it measures only when the sympathetic autonomic nervous system has been activated via emotional response. If a subject shows a response during the polygraph examination, then the inference is made that the subject is deceptive. It requires a logical leap to assume that the response is due solely to deception because this response can also be attributed to fear of failing (rather than fear of detection), simple test anxiety, anger, or many medical or mental conditions.


Yes. Similarly, when you catch someone entering your window at 2am with a prybar and an empty bag and wearing a ski mask, then an inference is made that the person is a burglar instead of a drunk on his way to a costume party. When a person with a badge and handcuffs shows up at your door in response to your call to 911, an inference is made that the person is a policeman and not a prostitute with a certain specialized clientele.

I have no problem with making grounded inferences, even ones that I know are not 100% certain. In fact, I'm not sure how you could infer with certainty that the cop was in fact a cop. The simple fact is that genuine cops are more common than the alternative.

The question is not technological, but statistical. If you assert that, say, 1% of the population will have an undiagnosed condition that guarantees they will fail a polygraph examination -- well, that doesn't mean much when I'm examining Kari, Tore, and Grant to see who took the wallet. It means much more when I'm examining all 10,000 employees of General Dynamics to see if any of them are spies. That's one reason that polygraph screening is ineffective -- not because there's anything wrong with the polygraph, but that it's simply hard to identify anything that rare.

[digithead]

If they concluded it, I certainly didn't find it in the report. They were quite harsh on the voice stress analyzer (p. 168) : "Overall, this research and the few controlled tests conducted over the past decade offer little or no scientific basis for the use of the computer voice stress analyzer or similar voice measurement instruments as an alternative to the polygraph for the detection of deception. The practical performance of voice stress analysis for detecting deception has not been impressive. It is possible that research conducted in high-stakes situations would give better results, but we have not found reports of the accuracy of voice stress analysis in such situations."
I find no such statements in the NAS report.

That's from Chapter 6 "Alternative Techniques and Technologies" and irrelevant to this conversation...

You cite the beginning of Chapter 3 "The Scientific Basis for Polygraph Testing." Read the rest of it...

Pg. 78 of the NAS report:"Most comparison question testing formats face the difficult challenge of calibrating the emotional content of relevant and comparison questions to elicit the levels of response that are needed in order to correctly interpret the test results." http://www.nap.edu/openbook.php?record_id=10420&page=78

If they have to calibrate emotional content, then one can conclude that it is emotionally based...

Pg 83. "The physiological responses measured by the polygraph do not all reflect a single underlying process such as arousal. Similarly, arousing stimuli do not produce consistent responses across these physiological indicators or across individuals. This knowledge implies that there is considerable lack of correspondence between the physiological data the polygraph provides and the underlying constructs that polygraph examiners believe them to measure. On theoretical grounds, it is therefore probable that any standard transformation of polygraph outputs (that is, scoring method) will correspond imperfectly with an underlying psychological state such as arousal and that the degree of correspondence will vary considerably across individuals. Little is known from basic physiological research about whether there are certain types of individuals for whom detection of arousal from polygraph measures is likely to be especially accurate—or especially inaccurate."

They go on in chapter 3 to list all of the endogenous and exogenous threats to polygraph accuracy.

You went on to write.

This emotional basis of polygraph lie detection makes it inherently flawed because the polygraph cannot accurately distinguish between the innocent but nervous and the guilty but nervous.

This statement is simply false and has been adequately addresed upthread.

I disagree, pg. 101: Basic scientific knowledge of psychophysiology offers support for expecting polygraph testing to have some diagnostic value, at least among naive examinees. However, the science indicates that there is only limited correspondence between the physiological responses measured by the polygraph and the attendant psychological brain states believed to be associated with deception—in particular, that responses typically taken as indicating deception can have other causes.

Besides the NAS report, my opinion relies on a large bulk of research about the polygraph both pro and con. CQT polygraph is inherently unreliable.


In addition, little or no research has been performed on the effects of undiagnosed cardiovascular, endocrinological, and mental illnesses on so-called psychophysiological detection of deception but one can surmise that it's probably not in the direction of improved accuracy. In screening applications, this is especially important.

But not to the simple question of whether on not polygraphs work. We already know that field conditions make the problem more difficult. That doesn't mean that the problem is unsolvable. This argument is an irrelevancy at best.
I agree that the problem is not unsolvable but in my opinion CQT polygraph should be abandoned in favor of cognitively based tests that have internal, construct, and domain validity which the CQT does not have.

Additionally, it is imperative to research and understand the threats that undiagnosed conditions pose to any polygraph test, let alone the CQT.

Well, I'm glad that you have achieved such a firm theoretical understanding of the reason that polygraphy works, something that apparently eluded the NAS in their extensive 2003 report. Is this understanding universal, or it is only your theory, and one not shared by the wider community?

But beyond that particular bit of sarcasm --- you've just admitted that "CQT polygraphy performs above chance but well below perfection." If you're arguing that CQT polygraphy doesn't work, why offer an explanation of why it does?
Your sarcasm aside, this understanding is shared by most of the scientists in psychophysiological testing and its implementation.

My argument about CQT is that it a flawed procedure rife with errors that undermine its usefulness. I'm about to submit a journal article based on this, I'll PM you a copy when I finish it.

I explain why it "works" with specific incidents because it has a cognitive component (i.e., knowledge of the crime by the guilty party) which makes it more accurate.

The question is not technological, but statistical. If you assert that, say, 1% of the population will have an undiagnosed condition that guarantees they will fail a polygraph examination -- well, that doesn't mean much when I'm examining Kari, Tore, and Grant to see who took the wallet. It means much more when I'm examining all 10,000 employees of General Dynamics to see if any of them are spies. That's one reason that polygraph screening is ineffective -- not because there's anything wrong with the polygraph, but that it's simply hard to identify anything that rare.

The problem is that we don't know what conditions will unduly affect any polygraph test beyond rudimentary guesses. Most polygraphers will not test anyone if they have the common cold because they know it affects the test regardless of test type. If the common cold can affect the test, what does hypertension, hyper- or hypothyroidism, bipolar, or other illness do to it? How about general mood?

DrK, I'm not trying to be snarky. My research into CQT polygraph deals with its use in sex offender programs. Thirty-eight states rely on CQT to determine if sex offenders are complying with conditions of parole. Similar to the NAS conclusion that CQT polygraph is a threat to national security, I feel its use in post-conviction sex offender treatment poses a similar threat.

This is a matter of using an unscientific and nonstandardizable procedure to screen people for positions of authority and to monitor behavior. Shouldn't we expect more from it than "better than chance but well below perfection?"

Do me a favor, please read Dr. Lykken's (1998) book A Tremor in the Blood and let me know if your opinion still stands. The late Dr. Lykken was a well-respected psychologist from the University of Minnesota who did a ton of research on the polygraph. He was one the first to show that psychopaths have a lower fear arousal than normal people. It sparked his interest in the polygraph...

Regards...

[drkitten]

Your own citations disprove you.


I disagree, pg. 101:


Basic scientific knowledge of psychophysiology offers support for expecting polygraph testing to have some diagnostic value, at least among naive examinees. However, the science indicates that there is only limited correspondence between the physiological responses measured by the polygraph and the attendant psychological brain states believed to be associated with deception—in particular, that responses typically taken as indicating deception can have other causes.



You originally claimed that "The NAS concluded that lie detector tests based on emotional response of a subject have no scientific basis."

The actual words of the NAS report directly contradict this. They're not wildly enthusiastic about the technology, but they specifically acknowledge that it has a valid scientific basis (as well as their specific statements that it performs well).


Besides the NAS report, my opinion relies on a large bulk of research about the polygraph both pro and con. CQT polygraph is inherently unreliable.

Given how you've misrepresented the very quotation you offer in support, I see no reason to treat your representation of the "large bulk of research" with any confidence.

Especially since the "evidence" that you cite against the CQT polygraph was, by and large, also available to the NAS and was not sufficient to shift their opinion. The Lykken book that you recommend was, in fact, specifically cited by the NAS but they evidently found it entirely unpersuasive.


I explain why it "works" with specific incidents because it has a cognitive component (i.e., knowledge of the crime by the guilty party) which makes it more accurate.

Yeah. That is to say, it works, and you have a theory of why.

And then you turn around and tell me that it doesn't work. Mere sarcasm cannot convey my contempt for this particular rhetorical flourish.



The problem is that we don't know what conditions will unduly affect any polygraph test beyond rudimentary guesses.

Which does not make polygraphy unreliable or pseudoscience. You're making the merely "good" the enemy of the "best" here (a common failing among academics) -- just because we have the potential for technology that works better does not mean that the old technology should be abandoned; if that were the case, we would have had electric cars fifty years ago -- and we'd probably still rely on the railroads for most of our transportation, because the cars would neither have the range nor the cargo capacity to be useful. We still don't know how to calibrate antipsychotic medicines beyond rudimentary guesses, so I suppose we should stop using them. Heck, we can't even perform spam filtering successfully.


Case in point:

This is a matter of using an unscientific and nonstandardizable procedure to screen people for positions of authority and to monitor behavior.

It is neither unscientific nor nonstandardizable; the science is well documented, and there are arguably too many standards and protocols. There are certainly abuses possible, when (ill-informed) people expect more of a tool than the tool is capable of delivering. But that's not the tool's fault.

Shouldn't we expect more from it than "better than chance but well below perfection?"

Well, let me know when you get "perfection" out of your technology, then. You'll be the first, you know. I don't think we've got any other test -- forensic, criminological, or diagnostic -- that can achieve "perfection." But I'll be sure to point out that any level of success of your own technology renders it -- by your own admission -- "unscientific and nonstandardizable."

I'm sorry to be so harsh. But I certainly hope that your standards for treatment of previous research is more even-handed in your journal article, or alternatively, that I don't end up reviewing it. If I caught the statement "the NAS concluded that lie detector tests based on emotional response of a subject have no scientific basis" in your article, I would read no further.

[verisimilidude]

A cold can affect the test?

Have the physiological conditions of a cold which affect the test been identified? If so, are they unique to that condition?

Correct me if I'm wrong, but it appears that a number of variables, which cannot be reliably controlled, can affect the accuracy of the test. If so, it makes me wonder how its use can be justified in any context. Does anyone other than those looking for a simplistic solution to a complex problem (the feds, corps.,etc.) really advocate that an individual be sanctioned for "failing" this test?

[Skeptic Ginger]

I haven't had time to keep up with this thread but I intend to catch back up later. In the mean time, Claus, I don't need to do any more research on polygraphs, read the huge, detailed GAO report I linked to that you quoted in the OP. Criminy, how much research should a person need to post to support their conclusions. It contains a thorough review of many studies. In some studies, per the analysis I referred to initially, the majority of the results were consistent with the polygraph being at least reasonably reliable especially in a group of people new to it and not trying to deceive the test.

All of the sources you cited debunking polygraphs were looking at it from the standpoint of assessing one individual. We all agree, I believe, that it is not accurate enough to fire someone, label them a spy, or send someone to jail.

There are other uses such as tricking someone into confessing in a police interview.

The context of my hypothetical application to the thread on the 100 atheists had absolutely nothing to do with how one might actually use a polygraph. The only thing I would change about my original statement, "you could assume most of the people who were supposedly deceitful were lying", is adding a caveat "in the ideal testing circumstances". The urban myth claiming there actually were polygraph results asking 100 atheists if they didn't believe god existed was as far as I can tell completely made up. If you can make up the claim, I can add "ideal testing circumstances" to my entirely hypothetical statement.

I have said repeatedly the polygraph studies showed wide range of accuracy of results. The statistical assessment of 100 test results was a valid concept. I posted an extremely detailed analysis of the research on polygraphs.

You, Claus are fighting Cyrano's windmills. You seem to think I've made claims about polygraphs which in the context you place them in, I did not make. The conclusions in the GAO report seemed adequate enough for me. If you want to argue, then argue against the GAO report. I'm not aware I disagree with anything in it.

[Skeptic Ginger]

A cold can affect the test?

Have the physiological conditions of a cold which affect the test been identified? If so, are they unique to that condition? ...Yes, heart rate, skin temperature, respiratory rate can all be affected by a respiratory infection.

As for, does anyone advocate using the polygraph to sanction someone, I don't believe so in the skeptical community but it is used in businesses to do so.

[digithead]

Your own citations disprove you.

No, they don't...

You originally claimed that "The NAS concluded that lie detector tests based on emotional response of a subject have no scientific basis."

The actual words of the NAS report directly contradict this. They're not wildly enthusiastic about the technology, but they specifically acknowledge that it has a valid scientific basis (as well as their specific statements that it performs well).

Pg. 81 To the extent that these principles do not hold universally, an examiner’s rapport with the examinee, the desired understanding of the polygraph examination and questions, and the clinical skill in determining the person’s veracity (i.e., detection of deception from demeanor) are all important in distinguishing among individuals who have physiological responses not indicative of deception (e.g., anxiety or anger regarding relevant questions, insufficient emotionality about the comparison questions), those who have physiological responses indicative of relatively innocuous transgressions, and those who have physiological responses indicative of significant transgressions. These distinctions are made on the basis of clinical judgment, which, though sometimes accurate, does not stand on a good foundation of theory or empirical evidence. There is little basis for relying on the accuracy of clinical judgments, especially in individual cases, without such a foundation. http://www.nap.edu/openbook.php?record_id=10420&page=81
Empasis is mine. No, they don't acknowledge that it has a scientific basis. In fact, they say that it does not have any foundation in which to believe CQT polygraph could have any accuracy.

Given how you've misrepresented the very quotation you offer in support, I see no reason to treat your representation of the "large bulk of research" with any confidence.
I've misrepresented nothing...

Especially since the "evidence" that you cite against the CQT polygraph was, by and large, also available to the NAS and was not sufficient to shift their opinion. The Lykken book that you recommend was, in fact, specifically cited by the NAS but they evidently found it entirely unpersuasive.
No, they found Lykken's book to be very persuasive.

Additionally, Stephen E. Feinberg, one of the main authors of the NAS polygraph study in his testimony to Congress stated: "Let me just conclude by reminding you that polygraph testing rests on weak scientic underpinnings despite nearly a century of study. Much of the available evidence for judging its validity lacks scientic rigor, and our committee sifted that evidence and the report makes clear the limitations of the polygraph for the present context."

From Fienberg, S.E. and P.C. Stern. 2005. In Search of the Magic Lasso: The Truth About the Polygraph. Statistical Science 20(3):249–60.

Yeah. That is to say, it works, and you have a theory of why.

As I stated, when you have a specific incident, cognition comes into play regardless of the emotional basis of the comparison question. It is in this situation that the CQT becomes more like its relative the GKT...

And then you turn around and tell me that it doesn't work. Mere sarcasm cannot convey my contempt for this particular rhetorical flourish.

When I say it doesn't work, I mean that it doesn't work consistently. You have to separate out utility of the polygraph from its validity and accuracy. The NAS said that the CQT polygraph has utility in that it might deter spies and possibly catch a few but it has little validity and its accuracy cannot be reliably estimated. They actually refused to state an accuracy number. The one they use in their calculations (80%) is a hypothetical and they do not believe that CQT polygraph has very high accuracy.

Which does not make polygraphy unreliable or pseudoscience. You're making the merely "good" the enemy of the "best" here (a common failing among academics) -- just because we have the potential for technology that works better does not mean that the old technology should be abandoned; if that were the case, we would have had electric cars fifty years ago -- and we'd probably still rely on the railroads for most of our transportation, because the cars would neither have the range nor the cargo capacity to be useful. We still don't know how to calibrate antipsychotic medicines beyond rudimentary guesses, so I suppose we should stop using them. Heck, we can't even perform spam filtering successfully.
Except that we do have a better type of test, GKT, that does not have the same pitfalls as CQT. GKT also has been shown to have a sound theoretical basis (orienting response) that the NAS rejected as plausible for CQT.

It is neither unscientific nor nonstandardizable; the science is well documented, and there are arguably too many standards and protocols. There are certainly abuses possible, when (ill-informed) people expect more of a tool than the tool is capable of delivering. But that's not the tool's fault.
Uniformity of process is not the same as validation nor is it standardizable in a psychological sense. Every person in a CQT polygraph serves as their own baseline, hence it is an unstandardizable test. Again to quote the NAS, pg. 213: Despite efforts to create standardized polygraph testing procedures, each test with each individual has significant unique features. http://www.nap.edu/openbook.php?record_id=10420&page=213

That sounds like unstandardizable to me.

Well, let me know when you get "perfection" out of your technology, then. You'll be the first, you know. I don't think we've got any other test -- forensic, criminological, or diagnostic -- that can achieve "perfection." But I'll be sure to point out that any level of success of your own technology renders it -- by your own admission -- "unscientific and nonstandardizable."
I don't expect perfection but I expect it to have validity, reliablity, and accuracy, all of which the CQT polygraph does not have.

I'm sorry to be so harsh. But I certainly hope that your standards for treatment of previous research is more even-handed in your journal article, or alternatively, that I don't end up reviewing it. If I caught the statement "the NAS concluded that lie detector tests based on emotional response of a subject have no scientific basis" in your article, I would read no further.

That's your perogative. I'm submitting it to an offender treatment journal though, are you a reviewer for any of those?

You and I have read the same NAS report and have come to differing conclusions about its accuracy, validity, and utility. That's fine. I really don't understand how anyone can read the NAS report and come away thinking that CQT polygraph was useful, accurate or grounded in science but then nothing much surprises me anymore when it comes to human behavior.

However, you can't argue that the NAS concluded that reliance on CQT polygraph in screening applications is unwise and poses a threat to national security. If the CQT polygraph was so useful, why do you think they draw such a conclusion?

I'm all for future research and efforts to further ground polygraphy in science. But CQT polygraph to me is a dead end. Efforts should be made towards GKT polygraph.

Finally, given what you've read so far, which do you think is better, GKT or CQT?

DrKitten, regards...

[digithead]

Yes, heart rate, skin temperature, respiratory rate can all be affected by a respiratory infection.

As for, does anyone advocate using the polygraph to sanction someone, I don't believe so in the skeptical community but it is used in businesses to do so.

Private businesses and most non-Law Enforcement public agencies cannot use the polygraph or force an employee to take a polygraph under the 1988 Federal Polygraph Protection Act.

The FPPA excluded law enforcement and national security agencies. So these agencies can still use the polygraph on their employees and potential employees. The controvery of its use in the DOE and the Wen Ho Lee case predicated the NAS review.

Also, polygraph can be used to sanction some people. CQT polygraph is used in what is called the Containment Method, a form of Post Conviction Sex Offender Treatment. Thirty eight states use the CQT polygraph to screen sex offenders to see if they are complying with their treatment and supervision protocols. Failing a polygraph and refusing to take a polygraph are grounds for revocation.

[CFLarsen]

Except that both people and computers can determine from those readings if people are lying, with better than 80% accuracy.

Present that at TAM, and/or write an article for Skeptic Magazine.

The context of my hypothetical application to the thread on the 100 atheists had absolutely nothing to do with how one might actually use a polygraph. The only thing I would change about my original statement, "you could assume most of the people who were supposedly deceitful were lying", is adding a caveat "in the ideal testing circumstances". The urban myth claiming there actually were polygraph results asking 100 atheists if they didn't believe god existed was as far as I can tell completely made up. If you can make up the claim, I can add "ideal testing circumstances" to my entirely hypothetical statement.

I have said repeatedly the polygraph studies showed wide range of accuracy of results. The statistical assessment of 100 test results was a valid concept. I posted an extremely detailed analysis of the research on polygraphs.

You, Claus are fighting Cyrano's windmills. You seem to think I've made claims about polygraphs which in the context you place them in, I did not make. The conclusions in the GAO report seemed adequate enough for me. If you want to argue, then argue against the GAO report. I'm not aware I disagree with anything in it.

I think it is extremely important that you present a paper at TAM.

It won't have to be a huge presentation. Just a few slides, a brief paragraph or two, and some references. I suspect that the questions that invariably will follow might take some time. But that's what you can expect, when you make such a claim.

This is big, skeptigirl. Scientific evidence that polygraphs work! This is exactly what makes TAM so great!

You know what to expect. You have months to prepare. Should be a walk in the park.

Will you do it?

[Pythra]

CFL, if you disagree with skeptigirl and dr kitten's claims (as you clearly do), will you please refute them and stop this ridiculous and frustrating dance of cowardice?

Count me among the readers of this thread who are fed up with your argumentative "technique" and would like to see some evidence.

[athon]

CFL, if you disagree with skeptigirl and dr kitten's claims (as you clearly do), will you please refute them and stop this ridiculous and frustrating dance of cowardice?

Count me among the readers of this thread who are fed up with your argumentative "technique" and would like to see some evidence.

Probably best do what most people do here and just ignore Claus's silliness. He's like the mad old uncle off his med's who nobody really takes seriously anymore. His 'present at TAM' tactic has become kind of annoying, I admit, but just as pointless as most of his points.

That said, the thread has been nicely saved by the interesting discussion between Dr. Kitten and digithead. Thanks folks. I can't claim a lot of knowledge on the topic, but have formed the opinion that it could well offer a better than chance ability to detect whether somebody is stating something that they know is a lie, but the confidence in this outcome is not sufficient that any drastic action, such as conviction of a crime, could be balanced on it.

Athon

[CFLarsen]

Probably best do what most people do here and just ignore Claus's silliness. He's like the mad old uncle off his med's who nobody really takes seriously anymore. His 'present at TAM' tactic has become kind of annoying, I admit, but just as pointless as most of his points.

It is anything but pointless.

Do you attend TAM solely to hear what you already agree with? Wouldn't you be absolutely thrilled to hear evidence of something considered a pseudoscience?

If Gary Schwartz would attend TAM, would you not challenge his claims?

[articulett]

Because polygraph tests are based on stress responses in the body, it can be fooled. I don't know if someone has mentioned this yet, but I remember Shermer talking about how contracting the anal sphincter to create a false stress response during the unstressful questions and relaxing it during the stressful ones can make for inconclusive readings.

I think it does help to know whether someone is willing to take them. If you were not guilty of a loved ones disappearance, and you were eager for them to stop suspecting you so that the right person could be located, then you would probably be eager to clear yourself... and if you avoided doing so, I think it's a valid red flag--

I've heard that sociopaths can often fool polygraphs because they don't have the stress response to lying that most people do.

I've heard of more recent brain imaging techniques where a suspect is shown an item that should look familiar and register a certain way in the brain if he recognizes it... (various pix are shown... some involving things only the perpetrator would find memorable)--and the brain reacts similarly to all the pix if there is no recognition-- I think this will be an interesting tool... but I think this will always be more useful in clearing suspects rather than in convicting them. It can help focus a search, however.

[HarryKeogh]

He started this thread in an attempt to embarrass another member and thought it would be an easy effort but probably wasn't expecting so much data from various members to come out. He wasn't prepared to deal with it so instead he has gone off on this strange kick of repeatedly saying "present at TAM or write an article for Skeptic".

His style is slightly less annoying than a child saying "I know you are but what am I?"

I agree with Athon. There has been some good discussion in it though. It's a fascinating topic that deserves more than just a sarcastic "This is such a breakthrough! Present your evidence at TAM" reply.

[CFLarsen]

He started this thread in an attempt to embarrass another member and thought it would be an easy effort but probably wasn't expecting so much data from various members to come out. He wasn't prepared to deal with it so instead he has gone off on this strange kick of repeatedly saying "present at TAM or write an article for Skeptic".

His style is slightly less annoying than a child saying "I know you are but what am I?"

I agree with Athon. There has been some good discussion in it though. It's a fascinating topic that deserves more than just a sarcastic "This is such a breakthrough! Present your evidence at TAM" reply.

I'm not saying there should only be a presentation at TAM.

Don't you want to see this discussed at TAM? Don't you want to see it discussed in skeptic magazines?

[HarryKeogh]

I'm not saying there should only be a presentation at TAM.

Don't you want to see this discussed at TAM? Don't you want to see it discussed in skeptic magazines?

I'm already seeing it being discussed here. This is one of the world's largest skeptical message boards. I'm seeing bright people go back and forth presenting evidence. I'm not qualified to contribute but I now feel more knowledgeable about polygraphs than I did yesterday. And not due to your contribution of "present it at TAM" but to reading what Dr. Kitten, digithead and others are posting.

If you are so eager for a TAM presentation you should consider putting one together yourself. There are skeptics that think the polygraph may have some merit. You can address this and present your evidence to the contrary. You think people are making a false claim so counter it at TAM like Lancaster countered psychic claims at TAM. Will you present this at TAM?

[CFLarsen]

I'm already seeing it being discussed here. This is one of the world's largest skeptical message boards. I'm seeing bright people go back and forth presenting evidence. I'm not qualified to contribute but I now feel more knowledgeable about polygraphs than I did yesterday. And not due to your contribution of "present it at TAM" but to reading what Dr. Kitten, digithead and others are posting.

If you are so eager for a TAM presentation you should consider putting one together yourself. There are skeptics that think the polygraph may have some merit. You can address this and present your evidence to the contrary. You think people are making a false claim so counter it at TAM like Lancaster countered psychic claims at TAM. Will you present this at TAM?

First, let's get one thing straight. Polygraphs are considered pseudoscience. They have been for quite some time. Randi, Shermer, Wiseman, Carroll, they all agree. I do, too. It is not up to me to present the case as if it is something new that polygraphs are pseudoscience. That would be to fool the audience.

If someone disagrees, then it is up to them to present their evidence. Skeptigirl disagrees that polygraphs are not pseudoscience. If she attends TAM, she has an obligation to present her evidence.

If anyone wants to stand up at TAM and say "Polygraphs are not pseudoscience", I would love to hear it.

[HarryKeogh]

First, let's get one thing straight. Polygraphs are considered pseudoscience. They have been for quite some time. Randi, Shermer, Wiseman, Carroll, they all agree. I do, too.

If appeals to authority and repeating "present at TAM" are the best you can do then I'm not going to waste any more time with you. I'd rather keep reading the interesting points the other posters in this thread are making.

[.13.]

It is claimed that a polygraph can distinguish between a nervous response to a question and a lie. Anyone willing to explain how that can be done outside laboratory conditions when you don't know the correct answer to your question?

[CFLarsen]

If appeals to authority and repeating "present at TAM" are the best you can do then I'm not going to waste any more time with you. I'd rather keep reading the interesting points the other posters in this thread are making.

I'm not saying that this is the best "I" can do.

But really: Why is it so abhorrent to you that this issue is discussed at TAM? Why confine the discussion to this thread alone?

Why not throw it out to the top skeptics at the biggest skeptical assembly in the known universe?

It is claimed that a polygraph can distinguish between a nervous response to a question and a lie. Anyone willing to explain how that can be done outside laboratory conditions when you don't know the correct answer to your question?

That is one of the questions I will be asking some TAM attendees.

[NoZed Avenger]

I'm not saying there should only be a presentation at TAM.

Don't you want to see this discussed at TAM? Don't you want to see it discussed in skeptic magazines?

Then write an article yourself. Summarize the findings of the quoted reports and the points made here by those in the threads talking about the statistics and then write your response.

It would be twice as good -- you could present both sides.

I believe that you know the guy editing skeptic report, so you could even guarantee getting it online; and then present it at TAM. You have a few months to do it, and this topic is BIG. It is absolutely crucial that someone put a paper together at TAM. This isn't about who's burden of proof it is -- this is important. This is big. This is far too big to let the subject remain undiscussed in skeptic magazines and TAM, according to you. Will you step up?

Will you do it?

[articulett]

If appeals to authority and repeating "present at TAM" are the best you can do then I'm not going to waste any more time with you. I'd rather keep reading the interesting points the other posters in this thread are making.

Oh, he tried that same tactic (present at TAM) with me on another thread. And I agree with Athon and the majority opinion regarding Claus. He's transparent. What is ironic is that his arch enemy of late (The Atheist) has done the same straw man thread thing in the past in an effort to supposedly make a point and embarrass another poster.

To me, certain people seldom make sense--they just always seem to be trying to win points in some entirely different conversation than the one everybody else is having. The more you try to clarify the more you get sucked into the crazy.

It drives Claus crazy when I talk about him--especially since I have him on ignore. But he drives other people crazy and has said some vile things about me, so I think he deserves it. Besides, I do want to support the people who are having problems with him, so they don't wonder if it's them. The socially obtuse, never seem to recognize that they are the problem while everyone else wonders if something they said was misinterpreted (as noted in my sig.)

But this thread has lots of good info... so Claus does inspire good conversations even if he doesn't ever seem to really participate in them himself.

It is interesting to know what polygraphs do. Scientologists use "e-meters" for clearing which are similar to polygraphs except they only measure one portion of what lie detectors measure--galvanic skin responses...

Polygraphs aren't mind readers, and I doubt "belief" is measurable for multiple reasons (this thread sprang from Mayday's thread regarding 80% of atheists showing up as believers...) but they aren't woo-- they do measure things associated with stress -- they are in essence, "stress detectors"--
and they are becoming more refined and useful over time.

I think one way the "calibrate" lie detectors is to tell people to answer no to every question... and then they ask them if various dates are their birthday. When they say the actual birthday, and the person responds no, there is usually a reaction noted that can be compared to all the other times the person said no.

[drkitten]

When I say it doesn't work, I mean that it doesn't work consistently.

Yes. That not only means you're misrepresenting, but you're actively lying, since you know that you're misrepresenting.

You have to separate out utility of the polygraph from its validity and accuracy. The NAS said that the CQT polygraph has utility in that it might deter spies and possibly catch a few but it has little validity and its accuracy cannot be reliably estimated. They actually refused to state an accuracy number. The one they use in their calculations (80%) is a hypothetical and they do not believe that CQT polygraph has very high accuracy.

Except that they cite the accuracy numbers estimated from the metaanalysis in the range of 0.81 to 0.91 repeatedly throughout the report.

I think this qualifies as another lie.



You and I have read the same NAS report and have come to differing conclusions about its accuracy, validity, and utility. That's fine. I really don't understand how anyone can read the NAS report and come away thinking that CQT polygraph was useful, accurate or grounded in science

That's easy. Because the NAS cites compelling evidence that it is.

If you had asked me about polygraphs before I started researching them, I probably would have parrotted CFLarsen. The positive report that the NAS gave to the polygraph is one of the single most influential pieces of evidence that changed my mind.

I'm astonished that anyone can read a report where the NAS goes out of its way, not once, but about six times, including stressing in both the report's conclusion and its executive summary, that polygraph testing can distinguish lies from the truth with better than chance accuracy and conclude that they do not believe that polygraphs work.



However, you can't argue that the NAS concluded that reliance on CQT polygraph in screening applications is unwise and poses a threat to national security. If the CQT polygraph was so useful, why do you think they draw such a conclusion?

Because those are two independent questions; one is whether the technology works at all, and the other is whether the technology works robustly enough to be applicable to specific problems. The NAS clearly separates those questions and answers them "yes" and "no" respectively (where the specific problem is screening, of course).

That's no different than someone asking "if hybrid technology works, why don't we see hybrid heavy goods vehicles?" Just because something works or is useful does not mean it's universallly useful.


Finally, given what you've read so far, which do you think is better, GKT or CQT?

Better for what? I agree that GKT is almost certainly a more promising line of research and may produce better polygraphs in the future; it has a better theoretical model, and there are some claims of better performance in the lab. However, CQT has a much longer track record and a much longer and better-documented history of positive evidence behind it. CQT is much more mature technology and there are much clearer protocols defining how to use it.

If you need a polygraph today to investigate an incident that happened last week, I would recommend CQT -- for one thing, you will be able to find a CQT-trained polygraph operator in almost any major metropolitan area, and the equipment is immediately available. If you are trying to build a polygraph/forensics consultation firm, I would recommend a mixture, leaning initially to CQT as that's what the clientele will be expecting you to use (and will find more persuasive in your reports), and educate your clients about the advantages of GKT over time (as more data comes in). If you are trying to build a polygraph research lab, then GKT is a promising technology and you will be in a position to develop the protocols and track record you need within your lab.

It's the difference between research-quality products and commercial-quality products. iBM has designs (in the lab) for computer chips that can run at a zillion teraHertz and use no power whatsoever -- but those designs are produced on ordinary Intel-based computers, because even IBM doesn't trust the new technology enough to rely on the research machines to do any actual work. CQT is a functional technology with some serious limitations in application; GKT at this point is barely out of vaporware. I like it. It looks nice. It looks elegant. Ask me again in ten years if it looks useful.

[digithead]

Yes. That not only means you're misrepresenting, but you're actively lying, since you know that you're misrepresenting.
Sigh, ironically enough for this thread, I did not lie nor did I misrepresent. My view of the evidence is different than yours.

Except that they cite the accuracy numbers estimated from the metaanalysis in the range of 0.81 to 0.91 repeatedly throughout the report.

I think this qualifies as another lie.
Now who's misrepresenting? The NAS did not give a summary number of accuracy. They did not give one because they were concerned that it would appear to be an endorsement of the polygraph when they expressly wanted to avoid that (see Fienberg, et al 2005 Statistical Science article I cited earlier). They also believed that the variability present in the studies they did use to be nonrandom and that all of the studies they used probably overestimated real accuracy (Ibid).

That's easy. Because the NAS cites compelling evidence that it is.
I suggest that you reread the NAS report again, because the evidence in there is less than compelling.

To quote Fienberg et al (2005: 12) again: "It may be harmless if television fails to discriminate between science and science fiction, but it is dangerous when government does not know the difference." They go on to say that the science does not support the contention that the CQT polygraph could have any degree of high accuracy.

If you had asked me about polygraphs before I started researching them, I probably would have parrotted CFLarsen. The positive report that the NAS gave to the polygraph is one of the single most influential pieces of evidence that changed my mind.

If that's what counts as positive evidence for you, you have a very low threshold.

I'm astonished that anyone can read a report where the NAS goes out of its way, not once, but about six times, including stressing in both the report's conclusion and its executive summary, that polygraph testing can distinguish lies from the truth with better than chance accuracy and conclude that they do not believe that polygraphs work.

You have left out the most important part of their conclusion about CQT: That it is for specific incidents only. They also said that lab-based accuracy estimates overestimate accuracy for specific incidents and that estimates from field studies also overestimate accuracy because of sampling and measurement bias...

Once again, in specific incidents, the CQT format approaches its theoretically sound cousin, the GKT...

Because those are two independent questions; one is whether the technology works at all, and the other is whether the technology works robustly enough to be applicable to specific problems. The NAS clearly separates those questions and answers them "yes" and "no" respectively (where the specific problem is screening, of course).

We have a different definition of what "works" is...

That's no different than someone asking "if hybrid technology works, why don't we see hybrid heavy goods vehicles?" Just because something works or is useful does not mean it's universallly useful.

I agree.

Better for what? I agree that GKT is almost certainly a more promising line of research and may produce better polygraphs in the future; it has a better theoretical model, and there are some claims of better performance in the lab. However, CQT has a much longer track record and a much longer and better-documented history of positive evidence behind it. CQT is much more mature technology and there are much clearer protocols defining how to use it.

It already does produce "better results."

More mature? You need to do more research. The GKT was developed by Lykken in the late 50s as an outgrowth of similar research performed in the 40s; the CQT replaced the Relevant/Irrelevant test about the same time. The reason that CQT became more prevalent is that GKT requires more work on the part of the polygrapher, not because it was better.

And just because something has been around awhile doesn't make it better.

Clearer protocols? How do I begin? You do know that in CQT polygraph, it is expected that the subject will lie to a comparison question, right? So if the polygrapher picks the wrong comparison question (e.g. the person is truthful to it), then its results are wrong. How is this more mature and better than a test designed to avoid false positives?

If you need a polygraph today to investigate an incident that happened last week, I would recommend CQT -- for one thing, you will be able to find a CQT-trained polygraph operator in almost any major metropolitan area, and the equipment is immediately available. If you are trying to build a polygraph/forensics consultation firm, I would recommend a mixture, leaning initially to CQT as that's what the clientele will be expecting you to use (and will find more persuasive in your reports), and educate your clients about the advantages of GKT over time (as more data comes in). If you are trying to build a polygraph research lab, then GKT is a promising technology and you will be in a position to develop the protocols and track record you need within your lab.

I wholeheartedly disagree...

Plus there's the whole 1988 Federal Employee Polygraph Protection Act which bars its use in employment with exceptions for law enforcement and national security agencies.

It's the difference between research-quality products and commercial-quality products. iBM has designs (in the lab) for computer chips that can run at a zillion teraHertz and use no power whatsoever -- but those designs are produced on ordinary Intel-based computers, because even IBM doesn't trust the new technology enough to rely on the research machines to do any actual work. CQT is a functional technology with some serious limitations in application; GKT at this point is barely out of vaporware. I like it. It looks nice. It looks elegant. Ask me again in ten years if it looks useful.

Sigh, did you read the NAS part about the woeful state of CQT polygraph research?

Sigh, have you done any research into the state of GKT application? It's widely used in Japanese law enforcement.

As for usefulness of the GKT, see Ben-Shakhar and Elaad. 2003. The Validity of Psychophysiological Detection of Information With the Guilty Knowledge Test: A Meta-Analytic Review. Journal of Applied Psychology 88(1):131–151.

Since it's obvious that there's no way I'm going to change your mind, I'll give up on the dueling quotations/citations and implore you to continue to do research into the matter with the hope that maybe you'll actually change your mind...

Regards...

[Jimbo07]

Don't you want to see it discussed in skeptic magazines?

'nother slight derail...

Does anybody get their information about science from skeptic magazines?

[CFLarsen]

Then write an article yourself. Summarize the findings of the quoted reports and the points made here by those in the threads talking about the statistics and then write your response.

It would be twice as good -- you could present both sides.

I believe that you know the guy editing skeptic report, so you could even guarantee getting it online; and then present it at TAM. You have a few months to do it, and this topic is BIG. It is absolutely crucial that someone put a paper together at TAM. This isn't about who's burden of proof it is -- this is important. This is big. This is far too big to let the subject remain undiscussed in skeptic magazines and TAM, according to you. Will you step up?

Will you do it?

I'm not convinced that the polygraphs work.

That's easy. Because the NAS cites compelling evidence that it is.

If you had asked me about polygraphs before I started researching them, I probably would have parrotted CFLarsen. The positive report that the NAS gave to the polygraph is one of the single most influential pieces of evidence that changed my mind.

I'm astonished that anyone can read a report where the NAS goes out of its way, not once, but about six times, including stressing in both the report's conclusion and its executive summary, that polygraph testing can distinguish lies from the truth with better than chance accuracy and conclude that they do not believe that polygraphs work.

So how do you explain that this "compelling evidence" is not enough to persuade the scientific community?

'nother slight derail...

Does anybody get their information about science from skeptic magazines?

Some of it. I read something in some magazines and dig deeper in other areas.

I think that's pretty normal.

[drkitten]

So how do you explain that this "compelling evidence" is not enough to persuade the scientific community?

Easy. The "scientific community" is larger than you (and digithead). The much larger part of the scientific community, the part that includes the NAS and the funding agencies, is sufficiently convinced of the scientific merit of polygraphy to write reports strongly positive of how they work under controlled conditions, to continue to publish extensively reports of such experiments reporting positively on their accuracy, and perhaps more importantly to continue funding the incremental development of the technology at a substantial rate (see, for example, funding opportunity W911NF-06-R-0009).

In short, I dispute the claim that the scientific community at large is not persuaded. If it weren't, do you think that digithead would even have a mainstream journal to present his findings in?

[digithead]

Easy. The "scientific community" is larger than you (and digithead). The much larger part of the scientific community, the part that includes the NAS and the funding agencies, is sufficiently convinced of the scientific merit of polygraphy to write reports strongly positive of how they work under controlled conditions, to continue to publish extensively reports of such experiments reporting positively on their accuracy, and perhaps more importantly to continue funding the incremental development of the technology at a substantial rate (see, for example, funding opportunity W911NF-06-R-0009).

In short, I dispute the claim that the scientific community at large is not persuaded. If it weren't, do you think that digithead would even have a mainstream journal to present his findings in?

I don't why I'm responding but I'll give you one more quote from the main authors of the NAS study. From Feinberg et al (2005: 22):

At the outset, we explained the seemingly compelling desire for a device that can assist law enforcement and intelligence agencies to identify criminals, spies, and saboteurs when direct evidence is lacking. The polygraph has long been touted as such a device. In this article and in the NRC report on which it draws, we explain the limited scientific basis for its use, the deep uncertainty about its level of accuracy, and the fragility of the evidence supporting claims of accuracy in any realistic application.

How should society and the courts in particular react to such a situation? At a minimum they should be wary about the claimed validity of the polygraph and its alternatives for use in the myriad settings in which they are used or proposed for use. This is especially relevant to current forensic uses of the polygraph. We believe that the courts have been justifed in casting a skeptical eye on the relevance and suitability of polygraph test results as legal evidence. Generalizing from the available scientifc evidence to the circumstances of a particular polygraph examination is fraught with difficulty. Further, the courts should extend their reluctance to rely upon the polygraph to the many quasi-forensic uses that are emerging, such as in sex offender management programs (see the discussion in Faigman, Fienberg and Stern [4]). The courts and the legal system should not act
as if there is a scientific basis for many, if any, of these uses. They need to hear the truth about lie detection.

Bolding is mine. This doesn't seem positive to me...

[CFLarsen]

Easy. The "scientific community" is larger than you (and digithead). The much larger part of the scientific community, the part that includes the NAS and the funding agencies, is sufficiently convinced of the scientific merit of polygraphy to write reports strongly positive of how they work under controlled conditions, to continue to publish extensively reports of such experiments reporting positively on their accuracy, and perhaps more importantly to continue funding the incremental development of the technology at a substantial rate (see, for example, funding opportunity W911NF-06-R-0009).

In short, I dispute the claim that the scientific community at large is not persuaded. If it weren't, do you think that digithead would even have a mainstream journal to present his findings in?

What rate is that?

[Belz...]

A cold can affect the test?

Have the physiological conditions of a cold which affect the test been identified? If so, are they unique to that condition?

Correct me if I'm wrong, but it appears that a number of variables, which cannot be reliably controlled, can affect the accuracy of the test. If so, it makes me wonder how its use can be justified in any context. Does anyone other than those looking for a simplistic solution to a complex problem (the feds, corps.,etc.) really advocate that an individual be sanctioned for "failing" this test?

But wouldn't the presence of these affecting factors be taken into consideration during the control phase ?

[Belz...]

Present that at TAM, and/or write an article for Skeptic Magazine.



I think it is extremely important that you present a paper at TAM.

It won't have to be a huge presentation. Just a few slides, a brief paragraph or two, and some references. I suspect that the questions that invariably will follow might take some time. But that's what you can expect, when you make such a claim.

This is big, skeptigirl. Scientific evidence that polygraphs work! This is exactly what makes TAM so great!

You know what to expect. You have months to prepare. Should be a walk in the park.

Will you do it?

Say, Claus. Do polygraphs perform better than chance ?

[Belz...]

You have left out the most important part of their conclusion about CQT: That it is for specific incidents only.

Doesn't that kind of mean that it works ?

[drkitten]

What rate is that?

The exact sums are classified, as you might expect. However, the Department of Defense Polygraph Institute routinely gives grants to universities on the order of US $300,000 per year -- and is specificlaly focused, as you might expect, on polygraph technology. More broadly, the NSF/CMI has a specific program dedicated to the identification of non-verbal cues to deception (I could probably dig up the total fund for that, since NSF numbers are not classified, but I won't bother). I believe there there were several tens of millions of dollars of NIJ funding offered a few years back as well (2005?), but I can't find the solicitation as it is no longer active and they do not publish past solicitations as the NSF does.

[digithead]

Doesn't that kind of mean that it works ?

Not with any degree of reliability...

[drkitten]

Doesn't that kind of mean that it works ?

Not at all. You see, it's neither perfect nor universal, and therefore it doesn't work at all, under any circumstances.

You should see the state that digithead's house is in. When someone breaks a window, it means his house no longer works, and so he just comes in with a wrecking ball and knocks the rest over.

[CFLarsen]

The exact sums are classified, as you might expect. However, the Department of Defense Polygraph Institute routinely gives grants to universities on the order of US $300,000 per year -- and is specificlaly focused, as you might expect, on polygraph technology. More broadly, the NSF/CMI has a specific program dedicated to the identification of non-verbal cues to deception (I could probably dig up the total fund for that, since NSF numbers are not classified, but I won't bother). I believe there there were several tens of millions of dollars of NIJ funding offered a few years back as well (2005?), but I can't find the solicitation as it is no longer active and they do not publish past solicitations as the NSF does.

I just called your bluff, didn't I?

[digithead]

The exact sums are classified, as you might expect. However, the Department of Defense Polygraph Institute routinely gives grants to universities on the order of US $300,000 per year -- and is specificlaly focused, as you might expect, on polygraph technology. More broadly, the NSF/CMI has a specific program dedicated to the identification of non-verbal cues to deception (I could probably dig up the total fund for that, since NSF numbers are not classified, but I won't bother). I believe there there were several tens of millions of dollars of NIJ funding offered a few years back as well (2005?), but I can't find the solicitation as it is no longer active and they do not publish past solicitations as the NSF does.

They're no longer called the Department of Defense Polygraph Institute. They changed their name to the Defense Academy for Credibility Assessment. I wonder why?

And NIJ grants rarely go into the millions...

And you can find all of NIJ awards at http://www.ojp.usdoj.gov/nij/funding.htm#post going back to 1996...

In 2005, one was awarded for "Use of Polygraphs to Combat Violence Against Women", BOTEC Analysis Corporation, $324,877, 2005–WG–BX–0010...

In 1997, one was awarded for :Sex Offenders in the Community: The Value of Polygraphs", Kim English, Colorado Division of Criminal Justice, $252,231, 1997-LB-VX-001...

That's it, just two in 11 years for little under $600k, hardly several millions of dollars...

[digithead]

Not at all. You see, it's neither perfect nor universal, and therefore it doesn't work at all, under any circumstances.

You should see the state that digithead's house is in. When someone breaks a window, it means his house no longer works, and so he just comes in with a wrecking ball and knocks the rest over.

So I point out that the NAS authors do not share your view that their report was positive or that they (the authors) believe that polygraph has little scientific support so rather than discuss this in an adult fashion, you engage in childish characterizations of my position...

There's a reason why I don't post here all that often and it's because of the childish behavior of some of members. Thanks for reminding me...

[drkitten]

So I point out that the NAS authors do not share your view that their report was positive or that they (the authors) believe that polygraph has little scientific support so rather than discuss this in an adult fashion, you engage in childish characterizations of my position...

Yes. You tell lies, and I ridicule you for it.

[digithead]

Yes. You tell lies, and I ridicule you for it.

Sorry, I told no lies. The NAS study and its authors do not support your assertion that the CQT polygraph is scientifically based or accurate except that it sometimes guesses correctly in specific incidents but not enough to be useful...

Again, to quote Fienberg et al (2005:22) who are the main authors of the NAS study: The courts and the legal system should not act as if there is a scientific basis for many, if any, of these uses.

[CFLarsen]

Yes. You tell lies, and I ridicule you for it.

Lies?

Conclusions and Recommendations

We have reviewed the scientific evidence on the polygraph with the goal of assessing its validity for security uses, especially those involving the screening of substantial numbers of government employees. Overall, the evidence is scanty and scientifically weak. Our conclusions are necessarily based on the far from satisfactory body of evidence on polygraph accuracy, as well as basic knowledge about the physiological responses the polygraph measures. We separately present our conclusions about scientific knowledge on the validity of polygraph and other techniques of detecting deception, about policy for employee security screening in the context of the U.S. Department of Energy (DOE) laboratories, and about the future of detection and deterrence of deception, including a recommendation for research.
Source (http://www.nap.edu/openbook.php?record_id=10420&page=212)


It is a lie that the polygraph has little scientific support?

[digithead]

Lies?



It is a lie that the polygraph has little scientific support?

Looking back, I assume drkitten thinks I'm a liar because I made the Popperian mistake of saying "none" rather than "little" on a message board. I suppose at some point, evidence could arise that there is a scientific basis for emotionally-based lie detection. But then I suppose that evidence could also arise that dowsing or telepathy also have a scientific basis...

But in my paper I'm preparing, I said "The NRC (2003) found that the emotionally-based CQT polygraph lacked sufficient scientific validity and that the majority of the research conducted on the reliability and validity of CQT polygraph could not meet minimum standards for National Science Foundation and National Institute of Health funding. It also found that no sufficient estimate of polygraph accuracy could be determined beyond the appearance that the CQT polygraph seems to detect deception at rates greater than chance for incident-specific tests and that increases in the reliability and accuracy of CQT olygraph were unlikely. They also found no support for the use of CQT polygraph in screening situations. From these findings, the NRC (2003) concluded that overconfidence in the polygraph created a significant risk to national security. "

At least I got it correct there. Next time, I'll try to pass my declarative statements on this board through a falsifiablity test. I wonder what the error rate of that is?

[Lucky]

I'm not convinced that the polygraphs work.
You have been asked this question a number of times in this thread, and have not given a satisfactory answer:

What do you mean by 'work'?

If you are still claiming that the basic method (using blinded interpretation of the physiological measurements only) cannot be shown to produce significantly better than chance results in laboratory conditions, then that is plain wrong. There really is no debate here: the laboratory data and analysis in the NA report is conclusive on this point. Or are you saying that the accuracy rate is low enough that polygraphy is not (and perhaps never can be) safe and effective in real-world applications? If so, do you see that this is a completely different claim?

Reasonably enough, polygraphy is a matter of great public concern, especially in the US, and there's a lot of debate on the subject on a non-scientific level. Almost everything I have seen (on either side) that's not from scientists is of poor quality. Unfortunately, the above two questions – whether it is a sound theoretical concept, backed up by research data, and whether it produces results at anything like the levels claimed by the proponents (and marketers) - have got hopelessly mixed up.

A quick google turns up many references to organised campaigns to prevent the routine use of polygraphy in a wide variety of situations; for example this (https://antipolygraph.org/) and this (http://www.spse.org/Polygraph_letter_campaign.html) and this (http://www.lectlaw.com/files/emp28.htm).

Now, I believe that on the whole these campaigns exist for the right reasons – that the technology is not sufficiently accurate for the purposes it is put to, and that (partly due to deliberate misinformation) government agencies, employers etc. have a grossly exaggerated notion of its value – and in general I support them. However, we should recognise that there is also a political element to the anti-polygraphy movement – it has an evident left/liberal tendency:

"Polygraphy is intimidation by state power, a degradation of employee and citizen rights."

"The Defense Academy for Credibility Assessment (the erstwhile DoD Polygraph Institute) suppressed a study suggesting that innocent blacks are more likely to fail the polygraph than innocent whites."

"The ACLU has long favored protective legislation against indiscriminate "lie detector" testing in the American workplace, not only because it is unreliable, but also because it is an extreme invasion of privacy."

Nothing at all wrong with that, but it underlines the point that the campaigners are not primarily interested in the scientific issues (I'm sure the large majority of them have little or no understanding of them).

Polygraphy is one of many policy matters that require the public and its representatives to have a good grounding in the basic scientific issues, and the ability to disentangle them from the social/political ones. You (and many of us here) wish to educate the public in this kind of thinking, but do you acknowledge that we can only be effective by taking every opportunity to discuss and explain the relevant science?

No matter how important the problem in a social or political sense, we should not be trying to convince with slogans, journalistic manipulation and appeals to authority. We must never fudge the argument or ignore unwelcome evidence.

Vague statements like 'polygraphs don't work' are just adding to the confusion.

[TV's Frank]

But in my paper I'm preparing, I said "The NRC (2003) found that the emotionally-based CQT polygraph lacked sufficient scientific validity and that the majority of the research conducted on the reliability and validity of CQT polygraph could not meet minimum standards for National Science Foundation and National Institute of Health funding. It also found that no sufficient estimate of polygraph accuracy could be determined beyond the appearance that the CQT polygraph seems to detect deception at rates greater than chance for incident-specific tests and that increases in the reliability and accuracy of CQT olygraph were unlikely. They also found no support for the use of CQT polygraph in screening situations. From these findings, the NRC (2003) concluded that overconfidence in the polygraph created a significant risk to national security. "


Oh, oh, now I'm a co-author!

[TV's Frank]

Hey folks, I think we're all having slightly different arguments here, and I'm getting thoroughly confused. Even the NAS authors appear to contradict themselves in different parts of the same paper (which is, unfortunately, not uncommon).

This is a question for digithead (which I'm sure you've answered already):

Is there any situation/scenario in which CQT polygraph has a valid scientific basis, and repeatedly performs better than chance? If so, can you explain why/how? If not, then what explains the strong better-than-chance results in the NAS report discussed earlier?

Thanks for your patience! I'm glad this conversation has become productive again..

[Eos of the Eons]

I'm just concerned because the RCMP subject new recruits to their versions of psychiatric testing and polygraph testing. If neither is that reliable, then are they just getting stuck with recruits that are just the best at blowing hot air and great at lying?

[CFLarsen]

I'm just concerned because the RCMP subject new recruits to their versions of psychiatric testing and polygraph testing. If neither is that reliable, then are they just getting stuck with recruits that are just the best at blowing hot air and great at lying?

In a word: Yes.

But that's OK. "Polygraphs perform better than chance", so you have nothing to worry about. Nothing!!

It could be interesting to hear from those who are so impressed with the evidence just what they think the polygraphs can be used for.

[Zalbik]

I'm confused.

CFLarsen and digithread seem to be arguing that polygraphs are not practical for use in the real world given that they cannot determine conclusively whether someone is lying or not.

I don't think anyone is arguing against that. I don't see anyone here arguing that polygraphs should be used in a court of law, for national security, employee screenings etc.

The OP dealt with the use of the polygraph on a large population of people. The NAS report indicates that the polygraph works "better than chance" in determining deception.

Therefore, if a large group of people take a polygraph test, and 80% are shown to by lying...we should be able to conclude that more than half of them are in fact lying, even if we can't tell which ones. This makes the polygraph useless in any practical sense, but interesting in that it does perform better than random guessing.

What am I missing here? Do people disagree with the methodology of the NAS report? Are there other reports that refute it? What evidence have I missed?

[.13.]

Therefore, if a large group of people take a polygraph test, and 80% are shown to by lying...we should be able to conclude that more than half of them are in fact lying, even if we can't tell which ones. This makes the polygraph useless in any practical sense, but interesting in that it does perform better than random guessing.

The polygraph doesn't show that 80% of the subjects were lying. It shows that 80% of the subjects had a deviation from the baseline. To repeat and rephrase my earlier question which hasn't been answered yet (not directed particularly at you but for the proponents in general):

How can you tell the difference between a lie and a nervous reaction to a question when you don't know what the truthfull answer is?

[Zalbik]

The polygraph doesn't show that 80% of the subjects were lying. It shows that 80% of the subjects had a deviation from the baseline. To repeat and rephrase my earlier question which hasn't been answered yet (not directed particularly at you but for the proponents in general):

How can you tell the difference between a lie and a nervous reaction to a question when you don't know what the truthfull answer is?

Then why do both the NAS report and the Office of technology report (here (http://www.fas.org/sgp/othergov/polygraph/ota/)) indicate that for specific incident investigations the polygraph works better than pure chance.

The OP was provided in the context of a polygraph being used against 100 atheists asked the question "Do you believe there is a god?". The claim was that this resulted in over 80% of them failing the test.

Based on what we know of polygraphs, if this test were actually done appropriately with a competent examiner, we could safely conclude that more than half of those atheists did in fact believe in god.

Likely some of the 80% would be false positives, and some of the 20% would not have been correctly identified, but we could still conclude that more than half of those atheists do believe in god, even though we would be unable to tell specifically which of them did.

[digithead]

Therefore, if a large group of people take a polygraph test, and 80% are shown to by lying...we should be able to conclude that more than half of them are in fact lying, even if we can't tell which ones. This makes the polygraph useless in any practical sense, but interesting in that it does perform better than random guessing.

What am I missing here? Do people disagree with the methodology of the NAS report? Are there other reports that refute it? What evidence have I missed?

What you're missing about your conclusion is that you're ignoring the base rate of deception in the population you're testing. Within your hypothetical 80% deception indicated group, you could only conclude that more than half were lying if the base rate of deception was greater than 50%. If the base rate of deception is low, then the majority of your 80% deception indicated group would be false positives...

[digithead]

Hey folks, I think we're all having slightly different arguments here, and I'm getting thoroughly confused. Even the NAS authors appear to contradict themselves in different parts of the same paper (which is, unfortunately, not uncommon).

This is a question for digithead (which I'm sure you've answered already):

Is there any situation/scenario in which CQT polygraph has a valid scientific basis, and repeatedly performs better than chance? If so, can you explain why/how? If not, then what explains the strong better-than-chance results in the NAS report discussed earlier?

Thanks for your patience! I'm glad this conversation has become productive again..
No, I don't think that the CQT polygraph has a valid scientific basis but that does not mean it has no utility. It can scare people into confessing things that they otherwise would not admit to. It also can scare people into false confessions...

What we're really talking about here is that the CQT polygraph is simply an interrogation technique rather than any test with diagnostic validity. It has three phases. The first is the pre-test where the polygrapher tries to convince the subject that the method works (i.e., make them fearful). In the second phase, the actual test is given. Lastly, based on what the polygrapher observed both from the physiological responses recorded by the polygraph and the subject's demeanor within the test, the polygrapher further interrogates the subject with such questions as "you're having a little difficulty with question x, is there anything you'd like to get off your chest?" or they let the subject go. A decision of deception is rendered from both physiological responses interpreted by the machine and the demeanor of the subject interpreted by the polygrapher. Anyone else see the problem with that?

As for the NAS concluding that the CQT was better than chance for specific incidents it is because in specific incidents the CQT has a cognitive component to it. That is, questions in the test can be formulated for specific features of the incident which only the guilty party would have knowledge of, ergo cognition increases its ability to detect deception. In these instances, the CQT becomes more like its cousin, the guilty knowledge test (GKT) which is based solely on cognitive response...

Lastly, the studies the NAS relied on were either mock crime lab experiments done on college students or they were observational field studies. The mock experiments suffer from real world validity in that they cannot match the threat of actually being accused of a crime. Observational field studies suffer from selection bias in that most subjects who undergo polygraph testing in a criminal investigation have been identified through other means as potential suspects, increasing the likelihood that they're guilty. If you knew that 80-90% of the people you were going to test were actually guilty, how hard do you think it would be to identify the guilty just through good interrogation alone?

[.13.]

Then why do both the NAS report and the Office of technology report (here (http://www.fas.org/sgp/othergov/polygraph/ota/)) indicate that for specific incident investigations the polygraph works better than pure chance.

The OP was provided in the context of a polygraph being used against 100 atheists asked the question "Do you believe there is a god?". The claim was that this resulted in over 80% of them failing the test.

Based on what we know of polygraphs, if this test were actually done appropriately with a competent examiner, we could safely conclude that more than half of those atheists did in fact believe in god.

Likely some of the 80% would be false positives, and some of the 20% would not have been correctly identified, but we could still conclude that more than half of those atheists do believe in god, even though we would be unable to tell specifically which of them did.

Digithead has a point in post #218 about deception rates.

I'm not disputing that you can correlate deviation from the baseline and deception with better than chance accuracy in a laboratory setting when you know what the truthfull answers are.

What I'm asking is how do you distinguish between a nervous reaction and deception in a real life situation when you don't know the truthfull answers?

[Jimbo07]

I'm just concerned because the RCMP subject new recruits to their versions of psychiatric testing... are they just getting stuck with recruits that are just the best at blowing hot air and great at lying?

In a word: Yes.


wow...

[Belz...]

In a word: Yes.

But that's OK. "Polygraphs perform better than chance", so you have nothing to worry about. Nothing!!

It could be interesting to hear from those who are so impressed with the evidence just what they think the polygraphs can be used for.

So, Claus. Do polygraphs perform better than chance ?

[Cuddles]

I'm not disputing that you can correlate deviation from the baseline and deception with better than chance accuracy in a laboratory setting when you know what the truthfull answers are.

What I'm asking is how do you distinguish between a nervous reaction and deception in a real life situation when you don't know the truthfull answers?

I really don't understand why you keep asking this. If something gets 70% correct when you know what the answers are, what makes you think it would be any different when you don't. To go back to your psychic example, if someone can guess which way up a coin will land 70% of the time under certain conditions that means that in the future, under the same conditions, if they guess a coin toss they have a 70% chance of getting it right. That's what 70% accuracy means.

To apply it to this topic, why does it matter if you know they are lying or not? You ask someone some questions and they answer, and you decide whether they are lying or not without you knowing what the answer actually is. Afterwards, you go back and check the actual answers. It turns out that out of 10 answers you said were lies, 7 actually were, ie. you were 70% accurate. Now go and ask them another question. If your machine says they are lying, what is the chance it is right? 70%. Whether you know the real answer or not.

[Lucky]

What you're missing about your conclusion is that you're ignoring the base rate of deception in the population you're testing. Within your hypothetical 80% deception indicated group, you could only conclude that more than half were lying if the base rate of deception was greater than 50%. If the base rate of deception is low, then the majority of your 80% deception indicated group would be false positives...

The bolded statement would only be true if we assume a priori that the polygraph doesn't work - i.e. it provides no information about the actual individuals being tested - which is what we are meant to be investigating. It is equivalent to saying: IF the real deception rate in the tested subjects is low AND the polygraph's positive rate is high THEN most of the positives are false - a completely contentless statement.

Continuing with the hypothetical example: I've forgotten what detection and false positive rates we're assuming (and can't be bothered trawling the entire thread to find out). Let's be conservative and say 70% detection and 10% false positive. We want to know the proportion of liars in the group that has tested positive. You are right that we also need to know the proportion of liars in the population under consideration – say 20%. Then:

Of 100 representative subjects:
20 are liars
80 are truthful

Of the 20 liars:
70% = 14 will be correctly detected
6 will not be detected

Of the 80 truthful subjects:
10% = 8 will be incorrectly 'detected'
72 will be found truthful

Summary:
True Negatives = 72
True Positives = 14
False Negatives = 6
False Positives = 8

Therefore, of the group that tests positive, 8 out of 22 i.e. 36% are false positives (innocent test failures) and not 'the majority'.

Imagine you run this test with 100 subjects every day for several years, with an average of 22 positives per day, and on one particular day 80 of the subjects test positive. That would certainly be anomalous.

So, reviewing the assumptions:
1) The detection rate (70%),
2) The false positive rate (10%),
3) The proportion of liars in the test's base population (20%),
4) That the group that scored 80% positive is representative of that population,

plainly, at least one of the above is wrong, but the numbers themselves cannot tell us which. If we got a batch of results like this in any kind of medical screening test we would begin by suspecting a malfunction in the assay or machine, or some administrative error. Failing that, we would consider the possibility that this batch of subjects was atypical in some way, and investigate whether or not this invalidated the results. (In the atheists example, the explanation could indeed be that professed atheists are several times more likely to lie than the general population - that would be quite consistent with the figures.)

My point is that it would be quite invalid to conclude that an anomalous high result well above the assumed base rate means that the subjects must be truthful and the polygraph incorrect.


If you knew that 80-90% of the people you were going to test were actually guilty, how hard do you think it would be to identify the guilty just through good interrogation alone?
As you point out, the accuracy of any test (false-positive and detection rates) is completely independent of the frequency of the condition (guilt and deception) in the test population. If you know that 80-90% of the subjects are guilty then yes, you could use interrogation techniques such that 80-90% confess, or a polygraph threshhold such that 80-90% test positive, but that does not 'identify the guilty'. If the test has no discriminatory power then you will still have equal detection and false positive rates - i.e. the test performs no better than chance. (This may be clear to you but it probably wasn't clear to most people reading this thread.)


But that's OK. "Polygraphs perform better than chance", so you have nothing to worry about. Nothing!!

It could be interesting to hear from those who are so impressed with the evidence just what they think the polygraphs can be used for.
The polygraph doesn't show that 80% of the subjects were lying. It shows that 80% of the subjects had a deviation from the baseline. To repeat and rephrase my earlier question which hasn't been answered yet (not directed particularly at you but for the proponents in general):

How can you tell the difference between a lie and a nervous reaction to a question when you don't know what the truthfull answer is?

CFLarsen, I see you have shifted your ground, and are no longer claiming that the basic concept is 'pseudoscience'. Good.

I don't think there are any 'proponents' of polygraphy here. My view is that there will probably never be any justification for routine use of the polygraph, and there is probably no justification for any (non-research) use at all at the moment. My motivation (and, I guess, skeptigirl's and drkitten's) was simply to correct a misconception - rather carelessly propagated by the anti-polygraph movement, and the 'skeptical' movement in general - that polygraph technology is bogus like Rife cancer treatment machines (http://en.wikipedia.org/wiki/Royal_Rife#Modern_revival_of_Rife.27s_work) and electronic exercise belts (http://www.ftc.gov/opa/2002/05/projectabsurd.shtm) and diagnostic dowsing machines (http://homeoinfo.com/08_non-classical_topics/dowsing/diagnostic_dowsing_machines.php) etc. The claim is often made by the 'anti' campaigners that the results are entirely due to a kind of reverse-placebo effect, and the actual discrimination is performed by the trained tester using information completely separate from the machine readings. The point is that this is the wrong argument, and can easily be demolished by polygraph enthusiasts (for example, by reference to the NA report, especially the results of laboratory studies (http://www.nap.edu/openbook.php?record_id=10420&page=123)). The right argument is that the accuracy of the unaided technology is far below what is claimed (and commonly believed), and that it is therefore neither safe nor effective for any current or proposed use.

How could we use the polygraph in a real-world setting when we don't know the 'correct' result? .13., the important point to grasp is that the polygraph is no different in this respect from any imperfect test – a medical screening test, for example. We do the basic research, conduct studies to determine the scope of validity and obtain the 'calibration' data. When we have enough confidence in the test we introduce it in the field, including QA to monitor and improve the test's performance.

As to distinguishing a nervous response from a guilty one, we assume that there are in principle some detectable differences between the two types of response, and try to refine the test to amplify these differences. As digithead suggests, there are theoretical reasons to suggest that the problem will be reduced by using GKT-type questions rather than CQT – I don't know how well this has been tested.

[.13.]

I really don't understand why you keep asking this. If something gets 70% correct when you know what the answers are, what makes you think it would be any different when you don't. To go back to your psychic example, if someone can guess which way up a coin will land 70% of the time under certain conditions that means that in the future, under the same conditions, if they guess a coin toss they have a 70% chance of getting it right. That's what 70% accuracy means.

Is it a fair coin?


To apply it to this topic, why does it matter if you know they are lying or not? You ask someone some questions and they answer, and you decide whether they are lying or not without you knowing what the answer actually is. Afterwards, you go back and check the actual answers. It turns out that out of 10 answers you said were lies, 7 actually were, ie. you were 70% accurate. Now go and ask them another question. If your machine says they are lying, what is the chance it is right? 70%. Whether you know the real answer or not.

I bolded the important part. You can't check your answers in real world conditions. In such a case how can you determine wether this particular person is lying or just had a nervous reaction to the question?

[CFLarsen]

CFLarsen, I see you have shifted your ground, and are no longer claiming that the basic concept is 'pseudoscience'.

No, I haven't.

[Belz...]

Say, Claus. Do polygraphs perform better than chance ?

It's a simple yes or no question. I'm just looking for your opinion, nothing else.

[LTC8K6]

What if I believe that my answers are true?

The PPA poster that I am required to post:

http://www.dol.gov/osbp/eppac.pdf

Which seems like it protects employees against them, but if you read it, it really doesn't.

[LTC8K6]

Do investigators dig out criminals or liars more often with a polygraph machine or with plain old interviews?

[Lucky]

CFLarsen, I see you have shifted your ground, and are no longer claiming that the basic concept is 'pseudoscience'.
No, I haven't.
CFLarsen, what is your unorthodox definition of 'pseudoscience', that includes a technology that performs at well above chance levels in laboratory tests?

Your assertion that the results have nothing to do with the technology, but are a fraud based on additional cues, trickery and intimidation, has been thoroughly debunked in this thread. For example:
People,

You are missing the point.

You can, in fact, use polygraphs to tell if people lie or not. What you can't use polygraphs for is to tell if people lie or not by reading the output.

Readings from polygraphs do not show if people lie or not. Using polygraphs puts people in a state of stress so they get confused, so they mess up their confession. That's what interrogations also do: You ask the same questions, again and again, in order to find out if the suspect has his story straight. Using a polygraph, you merely introduce a technobabble factor, to impress the lesser informed.

A polygraph is a an intimidation tool.
No, CFLarsen, you are missing the point that the National Academies report (http://books.nap.edu/openbook.php?record_id=10420&page=R1) examines all the data (from a literature search by the authors) that has been produced under laboratory conditions, and this data clearly shows that polygraph tests in these studies performed well above chance levels.

This research data is from studies in which:
the questions were of no real-world significance
and
the results had no consequences for the subjects
and
the testers did not know the answers.

Therefore the results can have nothing whatever to do with technobabble, intimidation, fear or gullibility.

Note that I am making no claims about the utility of polygraphy as a forensic technique. My post here (http://forums.randi.org/showthread.php?postid=3302366#post3302366) explains all of this in detail.


Here are some questions you have missed (any chance of an answer?):
So:
Cervical smear tests are pseudoscience because they don't reliably detect all cancers and ignore the healthy condition.
Academic examinations are pseudoscience because they don't reliably grade candidates into discrete categories of knowledge and ability.
Post mortems are pseudoscience because they don't reliably distinguish between death by natural and unnatural causes.

Do you begin to see the logical fallacy?

Do you seriously believe that any test (forensic, medical, whatever) that doesn't have false positive and false negative rates close to zero is 'pseudoscience'?
...

http://forums.randi.org/imagehosting/thum_2550477e34c28029c.jpg (http://forums.randi.org/vbimghost.php?do=displayimg&imgid=10080)

Do you understand what the ROC figure is telling us?
...
Almost the entire body of published data suggests that, in laboratory conditions, polygraph tests give results that are better than chance. How on earth can you (or anyone) suggest that this is in any way comparable to studies of homeopathy?
I'm not convinced that the polygraphs work.
You have been asked this question a number of times in this thread, and have not given a satisfactory answer:

What do you mean by 'work'?

If you are still claiming that the basic method (using blinded interpretation of the physiological measurements only) cannot be shown to produce significantly better than chance results in laboratory conditions, then that is plain wrong. There really is no debate here: the laboratory data and analysis in the NA report is conclusive on this point. Or are you saying that the accuracy rate is low enough that polygraphy is not (and perhaps never can be) safe and effective in real-world applications? If so, do you see that this is a completely different claim?
...
Polygraphy is one of many policy matters that require the public and its representatives to have a good grounding in the basic scientific issues, and the ability to disentangle them from the social/political ones. You (and many of us here) wish to educate the public in this kind of thinking, but do you acknowledge that we can only be effective by taking every opportunity to discuss and explain the relevant science?

[CFLarsen]

Lucky,

You really, really need to read the report.

[Lucky]

CFLarsen: Who on earth do you think you're impressing with this foolery?

Lucky,

You really, really need to read the report.


I have read the main scientific and analytical sections, (1) the Executive Summary, (3) The Scientific Basis for Polygraph Testing, (4) Evidence from Polygraph Research: Qualitative Assessment, (5) Evidence from Polygraph Research: Quantitative Assessment, (8) Conclusions and Recommendations, in detail, and skimmed the rest. Plainly, you have not (you seem not to have read my posts, either). It is utterly impossible to make a serious study of the report (in particular (3) The Scientific Basis for Polygraph Testing) and honestly maintain that polygraphy is a 'pseudoscience'.

So, point me to the important features of the report that you think I've missed, and I will be happy to discuss them with you.

Perhaps I overwhelmed you with my list of questions, so I'll condense it:

1) Do you agree or disagree with the NA report's statement that:
"features of polygraph charts and the judgments made from them are correlated with deception in a variety of controlled situations involving naïve examinees untrained in countermeasures: for such examinees and test contexts, the polygraph has an accuracy greater than chance. Random variation and biases in study design are highly implausible explanations for these results"?

2) Do you acknowledge that, if this statement is correct, polygraphy cannot be a 'pseudoscience'?

3) Do you agree or disagree with my view that, although polygraphy is based on plausible principles, and is supported by laboratory studies, its level of accuracy in real-world situations is likely to be so low that it will never be safe and effective for routine use (and that its utility has been grossly exaggerated by its proponents)?

[digithead]

I have read the main scientific and analytical sections, (1) the Executive Summary, (3) The Scientific Basis for Polygraph Testing, (4) Evidence from Polygraph Research: Qualitative Assessment, (5) Evidence from Polygraph Research: Quantitative Assessment, (8) Conclusions and Recommendations, in detail, and skimmed the rest. Plainly, you have not (you seem not to have read my posts, either). It is utterly impossible to make a serious study of the report (in particular (3) The Scientific Basis for Polygraph Testing) and honestly maintain that polygraphy is a 'pseudoscience'.

So, point me to the important features of the report that you think I've missed, and I will be happy to discuss them with you.

Perhaps I overwhelmed you with my list of questions, so I'll condense it:

1) Do you agree or disagree with the NA report's statement that:
"features of polygraph charts and the judgments made from them are correlated with deception in a variety of controlled situations involving naïve examinees untrained in countermeasures: for such examinees and test contexts, the polygraph has an accuracy greater than chance. Random variation and biases in study design are highly implausible explanations for these results"?

2) Do you acknowledge that, if this statement is correct, polygraphy cannot be a 'pseudoscience'?

3) Do you agree or disagree with my view that, although polygraphy is based on plausible principles, and is supported by laboratory studies, its level of accuracy in real-world situations is likely to be so low that it will never be safe and effective for routine use (and that its utility has been grossly exaggerated by its proponents)?

Lucky,

Their conclusion about the scientific evidence for the polygraph was that one cannot generalize from any results obtained from subjects in lab or field studies because there is no good scientific theory of why the CQT polygraph works the way it does in the lab. Additionally, there are so many potential confounders (i.e. conditions that have a similar physiological response as deception) for the CQT polygraph that haven't been properly studied that it is unknown how those confounders affect whatever lab accuracy CQT polygraph may have shown.

Is lying correlated with nervousness and emotional response? Yes, but it is not a 1 to 1 correlation. CQT polygraphy in my opinion makes the fallacy of composition. It is pseudoscientific to assume that all liars are nervous and can be detected from physiological response because some liars are nervous and can be detected by physiological response.

Additionally, when I worked in environmental science, we had immunoassay soil test kits that could detect certain chemicals down to the ppm level (pentachlorophenol I think but it's been 15 years). One of the caveats was that the test did not work in soils with high levels of certain heavy metals (iron I think) as it rendered the test useless. Would it be scientific to use that test anyway in an environment with high concentrations of the confounding element and report the results despite the fact we knew they were useless? No, it would be pseudoscientific to use it that way and that's why CQT polygraph is considered pseudoscientific. It's how and when it's used, not that it might be able to detect lies above chance levels in the lab.

Now I've gotta get back to my other stuff. School started yesterday and I've got lectures to prepare. I've also neglected my dissertation for a week and I got the evil eye from my chair this afternoon...

But I have been given food for thought on my paper on the polygraph and how to better argue against its use in sex offender treatment...

That's why I love being on the JREF forum. When the dialogue and arguments are respectful, one can learn a lot from some very smart people, even if there's disagreement...

Regards...

[CFLarsen]

Lucky,

You say polygraphs are not pseudoscience. I disagree. People can make up their own minds.

Since you think polygraphs are not pseudoscience, how should we use them?

[Cuddles]

I bolded the important part. You can't check your answers in real world conditions. In such a case how can you determine wether this particular person is lying or just had a nervous reaction to the question?

That's the whole point. You work out the accuracy under known conditions. Once you know it will correctly identify 70% of liars, you can say that anyone flagged as a liar has a 70% chance of actually being one. I really don't see how this can be hard to understand.

[.13.]

I don't think there are any 'proponents' of polygraphy here.

Then I had misunderstood drkitten and skeptigirl. I doubt it so I'll wait untill either of them corrects me on their position.


My view is that there will probably never be any justification for routine use of the polygraph, and there is probably no justification for any (non-research) use at all at the moment.

I mostly agree with you.


The claim is often made by the 'anti' campaigners that the results are entirely due to a kind of reverse-placebo effect, and the actual discrimination is performed by the trained tester using information completely separate from the machine readings. The point is that this is the wrong argument, and can easily be demolished by polygraph enthusiasts (for example, by reference to the NA report, especially the results of laboratory studies (http://www.nap.edu/openbook.php?record_id=10420&page=123)).

That's not what I have claimed.


The right argument is that the accuracy of the unaided technology is far below what is claimed (and commonly believed), and that it is therefore neither safe nor effective for any current or proposed use.

I haven't studied the statistics but if this is accurate then it sure is a good argument against the use of polygraph. But this isn't my argument either.


How could we use the polygraph in a real-world setting when we don't know the 'correct' result? .13., the important point to grasp is that the polygraph is no different in this respect from any imperfect test – a medical screening test, for example. We do the basic research, conduct studies to determine the scope of validity and obtain the 'calibration' data. When we have enough confidence in the test we introduce it in the field, including QA to monitor and improve the test's performance.

But there is a difference. You can't verify the polygraph results. If you could you wouldn't need the polygraph in the first place.

Let's consider an example medical test: You test a patient for some viral disease. You get a negative result and send the patient home. Next morning he comes back showing symptoms. Now you know that your test was wrong.

Now consider this somewhat facetious example: You perform a polygraph test on an employee. He passes it. Next morning he comes back looking guilty: "I lied in my polygraph yesterday." And now you know your test was wrong.

:)


As to distinguishing a nervous response from a guilty one, we assume that there are in principle some detectable differences between the two types of response, and try to refine the test to amplify these differences. As digithead suggests, there are theoretical reasons to suggest that the problem will be reduced by using GKT-type questions rather than CQT – I don't know how well this has been tested.

Is that a valid assumption? Which measurment could potentially show this difference?

But in anycase regardless of if it could be done in the future or not: Surely the machine can't do that with current technology?

[.13.]

That's the whole point. You work out the accuracy under known conditions. Once you know it will correctly identify 70% of liars, you can say that anyone flagged as a liar has a 70% chance of actually being one. I really don't see how this can be hard to understand.

It was claimed that the machine can tell the difference between a nervous reaction to a question and a lie. Argue that if you agree with the claim.

my earlier question repeated:
I really don't understand why you keep asking this. If something gets 70% correct when you know what the answers are, what makes you think it would be any different when you don't. To go back to your psychic example, if someone can guess which way up a coin will land 70% of the time under certain conditions that means that in the future, under the same conditions, if they guess a coin toss they have a 70% chance of getting it right. That's what 70% accuracy means.

Is it a fair coin?

[CFLarsen]

That's the whole point. You work out the accuracy under known conditions. Once you know it will correctly identify 70% of liars, you can say that anyone flagged as a liar has a 70% chance of actually being one. I really don't see how this can be hard to understand.

How do you know the conditions are known outside the lab?

[Belz...]

Aw, come on, Claus!

[CFLarsen]

Aw, come on, Claus!

That is the sore spot, isn't it?

[Thanz]

Lucky,

You say polygraphs are not pseudoscience. I disagree. People can make up their own minds.

Since you think polygraphs are not pseudoscience, how should we use them?
Here's a question then: If a polygraph operator were to apply for the million saying that he can, under controlled laboratory conditions, identify falsehoods at a rate of 75-80%, would that qualify for the challenge in your opinion? I recognize that Randi's opinion may differ, and he has the final say, but I am interested in your opinion.

[CFLarsen]

Here's a question then: If a polygraph operator were to apply for the million saying that he can, under controlled laboratory conditions, identify falsehoods at a rate of 75-80%, would that qualify for the challenge in your opinion? I recognize that Randi's opinion may differ, and he has the final say, but I am interested in your opinion.

You'll have to ask Randi about that.

[Professor Yaffle]

Why would he have to ask Randi about your opinion?

[Thanz]

You'll have to ask Randi about that.
I am asking for your opinion. If you were running the prize, would it qualify? IF Skeptic Report had its own challenge, on the same rules as the JREF prize (but with a smaller prize, I am assuming) would this qualify?

[Cuddles]

It was claimed that the machine can tell the difference between a nervous reaction to a question and a lie. Argue that if you agree with the claim.

my earlier question repeated:


Is it a fair coin?

How do you know the conditions are known outside the lab?

Why are you both babbling about irrelevant nonsense? It was stated that if something is shown to be 70% accurate, then when you run it once, there is a 70% chance that it is correct. .13. questioned this. I pointed out that actually, yes, that is entirely true, and is in fact what 70% accurate means. It doesn't matter if you're talking about polygraphs, tossing coins, predicting the weather or anything else. If you are right 70% of the time, then any single outcome has a 70% chance of being right. End of story.

You can argue whether any particular thing, like polygraphs, really are 70% accurate, and I haven't said anything about that. The fact remains that if they are 70% accurate, then they are 70% accurate. The more you argue with this, the more it looks like either you can't understand infant level maths or are just trolling because you can't make a real argument.

[Belz...]

That is the sore spot, isn't it?

What sore spot ? I asked you a straightforward question about 6 times, now, and you refuse to answer. I wonder why.

Could it be because if you admit that they perform better than chance (which I admit, I have no idea), you will be in the impossible predicament of having to subsequently admit that they are not woo ?

[CFLarsen]

Why would he have to ask Randi about your opinion?

I can't say whether it would qualify or not. That is entirely up to Randi.

If you were running the prize

I'm not.

IF Skeptic Report had its own challenge

It doesn't.

Why are you both babbling about irrelevant nonsense? It was stated that if something is shown to be 70% accurate, then when you run it once, there is a 70% chance that it is correct. .13. questioned this. I pointed out that actually, yes, that is entirely true, and is in fact what 70% accurate means. It doesn't matter if you're talking about polygraphs, tossing coins, predicting the weather or anything else. If you are right 70% of the time, then any single outcome has a 70% chance of being right. End of story.

You can argue whether any particular thing, like polygraphs, really are 70% accurate, and I haven't said anything about that. The fact remains that if they are 70% accurate, then they are 70% accurate. The more you argue with this, the more it looks like either you can't understand infant level maths or are just trolling because you can't make a real argument.

You can create an artificial situation in the lab and test just about anything. But you can't automatically extrapolate that to the real world, because the test isn't applicable if the situation is different.

That's why it is highly misleading to throw out the generalized "polygraphs perform better than chance" quip. It is certainly not what the report says.

Do you think I can create an artificial situation in the lab where I show that astrologers "perform better than chance"?

[CFLarsen]

What sore spot ? I asked you a straightforward question about 6 times, now, and you refuse to answer. I wonder why.

Could it be because if you admit that they perform better than chance (which I admit, I have no idea), you will be in the impossible predicament of having to subsequently admit that they are not woo ?

See my post above.

[Cuddles]

You can create an artificial situation in the lab and test just about anything. But you can't automatically extrapolate that to the real world, because the test isn't applicable if the situation is different.

That's why it is highly misleading to throw out the generalized "polygraphs perform better than chance" quip. It is certainly not what the report says.

Do you think I can create an artificial situation in the lab where I show that astrologers "perform better than chance"?

And I really don't care. I haven't said a single thin about whether polygraphs are accurate or not. All I did was try to clarify something for .13.. He asked if something being 70% accurate really means that any single output has a 70% chance of being correct. The answer is yes. Your bizzare rants are utterly irrelevant to anything I have said. That said, .13.'s recent posts give the impression that he is also being deliberately obtuse rather than actually being interested in any answer, so there doesn't seem much point in being here at all.

[CFLarsen]

And I really don't care. I haven't said a single thin about whether polygraphs are accurate or not. All I did was try to clarify something for .13.. He asked if something being 70% accurate really means that any single output has a 70% chance of being correct. The answer is yes. Your bizzare rants are utterly irrelevant to anything I have said. That said, .13.'s recent posts give the impression that he is also being deliberately obtuse rather than actually being interested in any answer, so there doesn't seem much point in being here at all.

We are not talking about any experiment. We are talking about an experiment where people's ability to lie is tested.

You have to make a lot of assumptions when you perform a polygraph tests. E.g., one is that the subjects don't have the skills to cheat the polygraph.

How do you know this? Ask them, and expect them to tell you the truth? In a lie detector test?