SteveJ
13th July 2010, 12:18 PM
Hi everyone. Been listening to Skeptoid podcasts for over a week straight, trying to catch up. I hear Brian Dunning's voice whenever I close my eyes. It's scary.
In Canada, a news story has been published by the CBC in which Paul Gendreau, a professor and expert in criminal justice issues, declared a recent pilot project on electronic anklets for parolees as a "unmitigated disaster".
You can google for the story...I'm new to the forums and can't post links yet.
I posted an article on my blog taking issue with Gendreau. The points I made were that:
The report on the pilot made it clear the shortcomings (small sample size, technical problems) that make the report good as a jumping point for further work, but not as the basis for a policy decision.
As a "pilot", the goal is proof-of-concept, not a working version, like a "beta" test.
Paul Gendreau comes with some red flags.
The red flags include a gushing description of Gendreau at the University of New Brunswick website ("The success of Dr. Gendreau's research is shown by the esteem in which he is held around the world. Other indicators of his phenomenal success include..." and so on -- it goes on for some length), and that Gendreau's specialty is "meta-analysis".
The (weird) rock star status of Gendreau makes me wonder if he has learned that harsh words like "unmitigated disaster" gets him mentioned in the press, which feeds into the Gendreau-is-great meme. Remember, be skeptical of an opinion from someone who is selling the product. Maybe Gendreau's opinion is meant to sell Gendreau.
As for the meta-analysis, just how big a red flag is an opinion coming from someone who does meta-analysis, versus a researcher who actually runs blinded research projects? Meta-analysis is, by its nature, not blinded, and terribly prone to selection bias by the analyst who decides what studies to include into his analysis. And Gendreau's violent reaction to this study ("unmitigated disaster...they would have been better off just to keep people locked up in jail..." and the like) immediately makes me skeptical that this is merely an academic criticism of the study protocol.
Any opinions? Am I being too skeptical of meta-analysis in general? Am I alone in perceiving the red flags?
[Remember, google "paul gendreau electronic anklets" to see the original article, which includes a link to the report that got Gendreau so worked up.]
In Canada, a news story has been published by the CBC in which Paul Gendreau, a professor and expert in criminal justice issues, declared a recent pilot project on electronic anklets for parolees as a "unmitigated disaster".
You can google for the story...I'm new to the forums and can't post links yet.
I posted an article on my blog taking issue with Gendreau. The points I made were that:
The report on the pilot made it clear the shortcomings (small sample size, technical problems) that make the report good as a jumping point for further work, but not as the basis for a policy decision.
As a "pilot", the goal is proof-of-concept, not a working version, like a "beta" test.
Paul Gendreau comes with some red flags.
The red flags include a gushing description of Gendreau at the University of New Brunswick website ("The success of Dr. Gendreau's research is shown by the esteem in which he is held around the world. Other indicators of his phenomenal success include..." and so on -- it goes on for some length), and that Gendreau's specialty is "meta-analysis".
The (weird) rock star status of Gendreau makes me wonder if he has learned that harsh words like "unmitigated disaster" gets him mentioned in the press, which feeds into the Gendreau-is-great meme. Remember, be skeptical of an opinion from someone who is selling the product. Maybe Gendreau's opinion is meant to sell Gendreau.
As for the meta-analysis, just how big a red flag is an opinion coming from someone who does meta-analysis, versus a researcher who actually runs blinded research projects? Meta-analysis is, by its nature, not blinded, and terribly prone to selection bias by the analyst who decides what studies to include into his analysis. And Gendreau's violent reaction to this study ("unmitigated disaster...they would have been better off just to keep people locked up in jail..." and the like) immediately makes me skeptical that this is merely an academic criticism of the study protocol.
Any opinions? Am I being too skeptical of meta-analysis in general? Am I alone in perceiving the red flags?
[Remember, google "paul gendreau electronic anklets" to see the original article, which includes a link to the report that got Gendreau so worked up.]