PDA

View Full Version : Beyes' theorem and justice


Mojo
3rd October 2011, 05:29 AM
Interesting article in today's Guardian:

http://www.guardian.co.uk/law/2011/oct/02/formula-justice-bayes-theorem-miscarriage

ETA: not sure if it should be here or in Social Issues, but absolutely certain that I've misspelled the title.

Ivor the Engineer
3rd October 2011, 07:13 AM
Not sure Bayesian reasoning is being banned from court as the article suggests, but rather only when shaky estimates for base-rates are used. I suppose it depends on how firm estimates are expected to be before Bayesian reasoning is admissible as to how sensible this verdict is.

Aepervius
3rd October 2011, 08:19 AM
If the expert is not able to cite a statistic with error bars, and the source of it, then I would call that a faulty evidence, as there is no way to cross check it.

IMHO the judge was right to squash it out.

TubbaBlubba
4th October 2011, 02:30 AM
In Swedish, the words for "theorem" and "batch of ejaculation" are one and the same (sats).

Ivor the Engineer
4th October 2011, 02:34 AM
In Swedish, the words for "theorem" and "batch of ejaculation" are one and the same (sats).

Any particular reason?

HansMustermann
4th October 2011, 03:11 AM
TBH, I see nothing wrong with that ruling. Garbage in, garbage out. A formula can be mathematically water-proof, but if you run numbers you just pulled out the ass through it, the result is meaningless.

To quote Charles Babbage, "On two occasions I have been asked,"Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" In one case a member of the Upper, and in the other a member of the Lower, House put this question. I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."

But it's a widespread delusion, isn't it? We're even living through a recession caused, basically, if we're to believe the interviews, by some ass-hats putting a number they guessed wildly wrong into a computer and then trusting the result. Because, hey, it came out of a computer, and computers are never wrong ;) It would be almost funny, if not for the frightening realization that it's worse than we thought. Self-interest, greed, even evil, or whatever, are predictable, but here we have the world's finances depending on some crap-for-brains idiots who don't even understand what they're doing.

And in this case, some ass-hats basically were using a formula to mask the fact that they don't actually have data, but make it seem all scientific enough to send someone to jail anyway.

That's all the judge complained about.

The rest of the BS in the article about judges not understanding statisticians, or the prosecutor's fallacy, are at best red herrings and at worst strawmen. The prosecutor's fallacy didn't even come up in that ruling. All that happened was that a judge had enough wits to understand that you can't run a number you took wild guesses at through a formula, and expect get the correct result at the other end.

Kudos for the judge, as far as I'm concerned.

TubbaBlubba
4th October 2011, 03:23 AM
Any particular reason?

Well, it's a general word for "batch", really - mathematical batch, batch of bread, batch of semen.

edd
4th October 2011, 03:26 AM
in the area of footwear evidence, no attempt could realistically be made in the generality of cases to use a formula to calculate the probabilities. That practice had no sound basis and it was clear from settled law that outside the field of DNA, Bayes theorem and likelihood ratios should not be used.
via http://www.criminallawandjustice.co.uk/index.php?/CLJ-Reporter/r-v-t-2010-all-er-d-240-oct-2010-ewca-crim-2439.html

I would say this is patent nonsense.

That's not to say that the rest of the points made aren't valid, and that the whole footwear business should probably have been left out altogether or much much more carefully qualified, but it's certainly complete twaddle to propose banning Bayes' theorem.

TubbaBlubba
4th October 2011, 03:35 AM
via http://www.criminallawandjustice.co.uk/index.php?/CLJ-Reporter/r-v-t-2010-all-er-d-240-oct-2010-ewca-crim-2439.html

I would say this is patent nonsense.

That's not to say that the rest of the points made aren't valid, and that the whole footwear business should probably have been left out altogether or much much more carefully qualified, but it's certainly complete twaddle to propose banning Bayes' theorem.

Yeah, it seems like an excellent case to use Bayes' theorem - what are the odds for someone being at a particular scene, given a matching footprint?

Of course, with uncertainties, and possibly a number of divisions, the resultant answer may very well come out useless. But any statistician should know to take that into account.

Acleron
4th October 2011, 03:39 AM
I'm not sure the Guardian article is accurately reporting what happened.

From the article itself, the judge is complaining, as others above have pointed out, about the data used. To draw the conclusion that Bayes theorem be excluded appears misleading.

But on the other hand, I can't see a professor of mathematics arguing that dodgy data should be accepted by the court.

Like so many cases reported by journalists, only a transcript of the court case will reveal the exact claims of each party.

edd
4th October 2011, 04:38 AM
http://www.lexisnexis.com/uk/legal/results/listview/listview.do?risb=21_T12882726185&startDocNo=1&sort=null&format=CATCHWORDS&BCT=G0 - not sure whether that's behind a paywall for some of you.

HansMustermann
4th October 2011, 04:55 AM
via http://www.criminallawandjustice.co.uk/index.php?/CLJ-Reporter/r-v-t-2010-all-er-d-240-oct-2010-ewca-crim-2439.html

I would say this is patent nonsense.

That's not to say that the rest of the points made aren't valid, and that the whole footwear business should probably have been left out altogether or much much more carefully qualified, but it's certainly complete twaddle to propose banning Bayes' theorem.

From the article, it doesn't sound like they excluded Bayes per se. It just says you can't use it without firm data to run through it. Which, frankly, guesswork as to how many soles of that type might have been sold in the USA was not.

And from your link, it sounds like reasonable objections all the way to me. Essentially the name of the theorem was used just to make something sound like a scientific fact, when it was (A) just based on unsound data, (B) in a non-transparent way, so basically the jury had to just take someone's word that TEH SCIENCE damns the defendant, and (C) it actually turns that the calculation was more favourable to the defendant being not guilty than the jury were lead to believe.

And especially the latter makes me even wonder WTH DID they show the jury anyway? Did someone just do a smoke and mirror show and mention theorems and statistics just to impress the laymen? How DO you use maths in a valid way, if the relevant results are actually not told to the jury?

edd
4th October 2011, 05:03 AM
From the article, it doesn't sound like they excluded Bayes per se.

It was submitted to the court the approach adopted was a Bayesian analysis which this Court had robustly rejected for non DNA evidence in a number of cases: R v Dennis Adams [1996] 2 Cr App R 467, R v Adams (No 2) [1998] 1 Cr App R 377; R v Doheny [1997] 1 Cr App R 369.

It is quite clear therefore that outside the field of DNA (and possibly other areas where there is a firm statistical base), this court has made it clear that Bayes theorem and likelihood ratios should not be used.

(I think personally that latter bracketing alleviates many of my concerns about too many restrictions on the uses for it in court)

HansMustermann
4th October 2011, 05:29 AM
Well, that bracketing is really what I see as important. If you have some firm data, they're not forbidding the use of Bayes.