Statistics Gurus - Help needed

MortFurd · 21st February 2007, 12:45 AM

Could someone help me make sense of the jargon in these sentences:
From here:
RESULTS: Only 13% of the studies met the inclusion criteria. Meta-analyses revealed that a significant reduction in depression symptom severity was associated with bright light treatment (eight studies, having an effect size of 0.84 and 95% confidence interval [CI] of 0.60 to 1.08) and dawn simulation in seasonal affective disorder (five studies; effect size=0.73, 95% CI=0.37 to 1.08) and with bright light treatment in nonseasonal depression (three studies; effect size=0.53, 95% CI=0.18 to 0.89). Bright light as an adjunct to antidepressant pharmacotherapy for nonseasonal depression was not effective (five studies; effect size=–0.01, 95% CI=–0.36 to 0.34).

From the rest of the text, I gather that getting zapped with a bright light helps depression - a good bit for people with SAD and not quite as much for people with other kinds of depression. I don't understand the expressions used in the statistics, so I'm not sure how positive the authors are of the claims nor am I clear on how large a percentage of the patients were helped by the bright light therapy.

I also see that "dawn simulation" isn't rated as effective as bright light therapy, but the "effect size" doesn't really mean much to me other than that higher seems to be better.

Stimpson J. Cat · 21st February 2007, 01:03 AM

Without knowing the inclusion criteria, it is impossible to judge. That said, one should be aware that even under the best of circumstances, all that can be concluded from such a meta-analysis is that there may be an effect present which was missed in the original studies due to the sample sizes being too small or to poor controls and standards in the experiments. In such a case, the proper course of action would be to perform a new study with sufficient sample size and more rigorous controls.

That said, this seems to be precisely what they state in their conclusions. They state that the results suggest that light therapy may be efficacious, but that new experiments with more rigorous protocals must be performed to determine whether it is.

Dr. Stupid

MortFurd · 21st February 2007, 02:16 AM

So, what is "effect size," and how is it determined?

sphenisc · 21st February 2007, 03:15 AM

Originally Posted by MortFurd

So, what is "effect size," and how is it determined?

From here:
http://en.wikipedia.org/wiki/Effect_size

There seem to be three main effect size measures; Pearson's r, Cohen's d and Cohen's f².

From the full article ( you did click on 'Full Text' right?), they refer to " the standardized mean difference effect size ", which suggests to me that they're using Cohen's d. (You may be able to confirm this in Lipsey and Wilson).

Hope this helps.

fls · 21st February 2007, 03:42 AM

Originally Posted by MortFurd

Could someone help me make sense of the jargon in these sentences:
From here:
RESULTS: Only 13% of the studies met the inclusion criteria. Meta-analyses revealed that a significant reduction in depression symptom severity was associated with bright light treatment (eight studies, having an effect size of 0.84 and 95% confidence interval [CI] of 0.60 to 1.08) and dawn simulation in seasonal affective disorder (five studies; effect size=0.73, 95% CI=0.37 to 1.08) and with bright light treatment in nonseasonal depression (three studies; effect size=0.53, 95% CI=0.18 to 0.89). Bright light as an adjunct to antidepressant pharmacotherapy for nonseasonal depression was not effective (five studies; effect size=–0.01, 95% CI=–0.36 to 0.34).

From the rest of the text, I gather that getting zapped with a bright light helps depression - a good bit for people with SAD and not quite as much for people with other kinds of depression. I don't understand the expressions used in the statistics, so I'm not sure how positive the authors are of the claims nor am I clear on how large a percentage of the patients were helped by the bright light therapy.

I also see that "dawn simulation" isn't rated as effective as bright light therapy, but the "effect size" doesn't really mean much to me other than that higher seems to be better.

They used a "standardized mean difference" for their effect size (from the full text). The difference (no matter how it is measured) between the two groups (treated and untreated) is divided by the standard deviation, which has the effect of converting all the measurements to a universal scale so they can be directly combined and compared.

Generally, an effect size of 0.20 is small, 0.50 is medium and 0.80 is large. Zero represents no effect and negative numbers would represent harm (in this analysis). The numbers are not limited and can be greater than 1.

Confidence intervals give an idea of how precise the measurement of the effect size is (and whether it is statistically significant). The 95 percent confidence interval tells you that (based on the mean and variance of this experiment) if you repeated the experiment, you are confident that the results would fall somewhere within that range. Or you are 95 percent confident the "true" treatment effect (given that you are working with a sample which likely differs somewhat from the whole) falls within that range. A narrow range is more precise than a wide range. And your interpretation of the clinical usefulness depends upon where the low and high end of the range fall.

The effect size for the use of light in SAD was 0.84 with a CI of 0.60 to 1.08, which means that the effect was large, and the range of effect covered "high medium" to "very large". You can be confident that it will have a large effect in this condition. On the other hand, the use of light in non-SAD had an effect of 0.53 with a CI of 0.18 to 0.89. The effect was "medium" but the range went from "small" to "large". You are less confident about the magnitude of effect you may expect, although you can be confident of at least some effect.

Linda

MortFurd · 21st February 2007, 06:51 AM

Ah.

Thank you, Linda. That's an explanation that puts some meat on the definitions of the terms. The concept makes perfect sense. Sort of like normalising audio files so that the volume is consistent and comparable between all songs on an album.

And thanks for the interpretation of the numbers.

Ivor the Engineer · 21st February 2007, 07:53 AM

Originally Posted by MortFurd

Ah.

Thank you, Linda. That's an explanation that puts some meat on the definitions of the terms. The concept makes perfect sense. Sort of like normalising audio files so that the volume is consistent and comparable between all songs on an album.

And thanks for the interpretation of the numbers.

Signal processing is statistics with different terminology and point of view. I find it very confusing at times.

MortFurd · 21st February 2007, 08:30 AM

Originally Posted by Ivor the Engineer

Signal processing is statistics with different terminology and point of view. I find it very confusing at times.

Reminds me of my favorite puzzling question:
How do folks account for phase changes when averaging monthly (or daily or whatever) values into yearly values?

The signal processing approach would be to use a symetrical FIR filter so you have an absolute delay, then just pick up the proper output data. On a 100 tap filter, the first 100 data points out are bogus. Drop 'em, then the data from that point on is properly adjusted for the filter delay and the signals (interesting variations) within the data maintain their proper phase relationships.

They've gotta do SOMETHING, but I don't know what and it never seems to get mentioned.

21st February 2007, 12:45 AM	#1
MortFurd Graduate Poster Join Date: Aug 2006 Location: An American in Germany Posts: 1,975	Statistics Gurus - Help needed Could someone help me make sense of the jargon in these sentences: From here: RESULTS: Only 13% of the studies met the inclusion criteria. Meta-analyses revealed that a significant reduction in depression symptom severity was associated with bright light treatment (eight studies, having an effect size of 0.84 and 95% confidence interval [CI] of 0.60 to 1.08) and dawn simulation in seasonal affective disorder (five studies; effect size=0.73, 95% CI=0.37 to 1.08) and with bright light treatment in nonseasonal depression (three studies; effect size=0.53, 95% CI=0.18 to 0.89). Bright light as an adjunct to antidepressant pharmacotherapy for nonseasonal depression was not effective (five studies; effect size=–0.01, 95% CI=–0.36 to 0.34). From the rest of the text, I gather that getting zapped with a bright light helps depression - a good bit for people with SAD and not quite as much for people with other kinds of depression. I don't understand the expressions used in the statistics, so I'm not sure how positive the authors are of the claims nor am I clear on how large a percentage of the patients were helped by the bright light therapy. I also see that "dawn simulation" isn't rated as effective as bright light therapy, but the "effect size" doesn't really mean much to me other than that higher seems to be better.

21st February 2007, 01:03 AM	#2
Stimpson J. Cat Graduate Poster Join Date: Sep 2001 Location: Oxford, UK Posts: 1,833	Without knowing the inclusion criteria, it is impossible to judge. That said, one should be aware that even under the best of circumstances, all that can be concluded from such a meta-analysis is that there may be an effect present which was missed in the original studies due to the sample sizes being too small or to poor controls and standards in the experiments. In such a case, the proper course of action would be to perform a new study with sufficient sample size and more rigorous controls. That said, this seems to be precisely what they state in their conclusions. They state that the results suggest that light therapy may be efficacious, but that new experiments with more rigorous protocals must be performed to determine whether it is. Dr. Stupid
	__________________ A poke in the eye makes Baby Jesus cry.

21st February 2007, 03:15 AM	#4
sphenisc Illuminator Join Date: Jul 2004 Posts: 3,749	Originally Posted by MortFurd So, what is "effect size," and how is it determined? From here: http://en.wikipedia.org/wiki/Effect_size There seem to be three main effect size measures; Pearson's r, Cohen's d and Cohen's f². From the full article ( you did click on 'Full Text' right?), they refer to " the standardized mean difference effect size ", which suggests to me that they're using Cohen's d. (You may be able to confirm this in Lipsey and Wilson). Hope this helps.
sphenisc Illuminator Join Date: Jul 2004 Posts: 3,749

21st February 2007, 03:42 AM	#5
fls Penultimate Amazing Join Date: Jan 2005 Posts: 10,236	Originally Posted by MortFurd Could someone help me make sense of the jargon in these sentences: From here: RESULTS: Only 13% of the studies met the inclusion criteria. Meta-analyses revealed that a significant reduction in depression symptom severity was associated with bright light treatment (eight studies, having an effect size of 0.84 and 95% confidence interval [CI] of 0.60 to 1.08) and dawn simulation in seasonal affective disorder (five studies; effect size=0.73, 95% CI=0.37 to 1.08) and with bright light treatment in nonseasonal depression (three studies; effect size=0.53, 95% CI=0.18 to 0.89). Bright light as an adjunct to antidepressant pharmacotherapy for nonseasonal depression was not effective (five studies; effect size=–0.01, 95% CI=–0.36 to 0.34). From the rest of the text, I gather that getting zapped with a bright light helps depression - a good bit for people with SAD and not quite as much for people with other kinds of depression. I don't understand the expressions used in the statistics, so I'm not sure how positive the authors are of the claims nor am I clear on how large a percentage of the patients were helped by the bright light therapy. I also see that "dawn simulation" isn't rated as effective as bright light therapy, but the "effect size" doesn't really mean much to me other than that higher seems to be better. They used a "standardized mean difference" for their effect size (from the full text). The difference (no matter how it is measured) between the two groups (treated and untreated) is divided by the standard deviation, which has the effect of converting all the measurements to a universal scale so they can be directly combined and compared. Generally, an effect size of 0.20 is small, 0.50 is medium and 0.80 is large. Zero represents no effect and negative numbers would represent harm (in this analysis). The numbers are not limited and can be greater than 1. Confidence intervals give an idea of how precise the measurement of the effect size is (and whether it is statistically significant). The 95 percent confidence interval tells you that (based on the mean and variance of this experiment) if you repeated the experiment, you are confident that the results would fall somewhere within that range. Or you are 95 percent confident the "true" treatment effect (given that you are working with a sample which likely differs somewhat from the whole) falls within that range. A narrow range is more precise than a wide range. And your interpretation of the clinical usefulness depends upon where the low and high end of the range fall. The effect size for the use of light in SAD was 0.84 with a CI of 0.60 to 1.08, which means that the effect was large, and the range of effect covered "high medium" to "very large". You can be confident that it will have a large effect in this condition. On the other hand, the use of light in non-SAD had an effect of 0.53 with a CI of 0.18 to 0.89. The effect was "medium" but the range went from "small" to "large". You are less confident about the magnitude of effect you may expect, although you can be confident of at least some effect. Linda
	__________________ God:a capricious creative or controlling force said to be the subject of a religion. Evidence is anything that tends to make a proposition more or less true.-Loss Leader SCAM will now be referred to as DIM (Demonstrably Ineffective Medicine) Look how nicely I'm not reminding you you're dumb.-Happy Bunny When I give an example, do not assume I am excluding every other possible example. Thank you.

21st February 2007, 02:16 AM	#3
MortFurd Graduate Poster Join Date: Aug 2006 Location: An American in Germany Posts: 1,975	So, what is "effect size," and how is it determined?

21st February 2007, 06:51 AM	#6
MortFurd Graduate Poster Join Date: Aug 2006 Location: An American in Germany Posts: 1,975	Ah. Thank you, Linda. That's an explanation that puts some meat on the definitions of the terms. The concept makes perfect sense. Sort of like normalising audio files so that the volume is consistent and comparable between all songs on an album. And thanks for the interpretation of the numbers.

21st February 2007, 07:53 AM	#7
Ivor the Engineer Philosopher Join Date: Feb 2006 Location: South Britain, near the middle Posts: 9,553	Originally Posted by MortFurd Ah. Thank you, Linda. That's an explanation that puts some meat on the definitions of the terms. The concept makes perfect sense. Sort of like normalising audio files so that the volume is consistent and comparable between all songs on an album. And thanks for the interpretation of the numbers. Signal processing is statistics with different terminology and point of view. I find it very confusing at times.

21st February 2007, 08:30 AM	#8
MortFurd Graduate Poster Join Date: Aug 2006 Location: An American in Germany Posts: 1,975	Originally Posted by Ivor the Engineer Signal processing is statistics with different terminology and point of view. I find it very confusing at times. Reminds me of my favorite puzzling question: How do folks account for phase changes when averaging monthly (or daily or whatever) values into yearly values? The signal processing approach would be to use a symetrical FIR filter so you have an absolute delay, then just pick up the proper output data. On a 100 tap filter, the first 100 data points out are bogus. Drop 'em, then the data from that point on is properly adjusted for the filter delay and the signals (interesting variations) within the data maintain their proper phase relationships. They've gotta do SOMETHING, but I don't know what and it never seems to get mentioned.