Is peer-reviewed data "THE DATA"?

Alric · 11th February 2008, 06:25 PM

And discussions in peer-reviewed articles trump "some guy's blog".

Science does change and data is disputed with more data. However, the latest scientific consensus is what we should use to make decisions.

Can we agree to this and move on?

Scott1972 · 11th February 2008, 07:19 PM

It's not clear to me what you are trying to say.

Being able to reproduce the data is what is important, not a person's or group's data.

For theoretical predictions, it's the prediction conforming to empirical data, and those measurements being reproducible.

Pipirr · 12th February 2008, 11:39 AM

Blog commentary can be very useful in understanding peer reviewed research. Where things get silly is when some guy with a blog is trying to rewrite science as we know it, and their unpublished, non-peer reviewed blog posts become elevated to a position of privilege.

Take D'Aleo, with his long paper* about problems in the US temperature record: those are grand claims. But are they valid claims, or has he just got it all wrong?

If there really is a problem with the temperature record, then publish*. The people that need to know would notice it. If it is unpublished, I think it is reasonable to wonder why not. If research has merit, get it through peer review, especially if the claims that are made and the conclusions that are reached are grand. Otherwise, what is there to distinguish it from any other blog post by yet another internet crazy?

*The ability to convert a file into PDF format does not a published paper make.

Alric · 12th February 2008, 11:43 AM

Right. That is why I started this thread. I see it happen with evolution and climate change where people will start linking to blogs in the face of publication as valid counterpoints.

Ivor the Engineer · 12th February 2008, 12:32 PM

While I generally agree (and hope) peer-reviewed journal articles have higher validity on average than unpublished work and points of view, I'll just mention Alan Sokal.

Plus let's not forget other factors such as researcher bias, publication bias and, dare I say it, elitism.

Whether a piece of research is published or unpublished, all that really matters are the method and the results.

Alric · 12th February 2008, 12:37 PM

Yeah but what can you expect when the publication is on a post-modernist journal. I don't think you could pull the same stunt in Science magazine, at least not for long. And that was precisely the point.

Ivor the Engineer · 12th February 2008, 12:44 PM

That's what I thought, but I'm sure I've seen positive results for homeopathy in "proper" medical journals, which members of this forum with suitable qualifications have subsequently torn to shreds. (The contents of the articles, not the journals.

).

It makes me wonder what the peer review process is supposed to be weeding out from publication.

Pipirr · 12th February 2008, 01:00 PM

Originally Posted by Ivor the Engineer

{snip}It makes me wonder what the peer review process is supposed to be weeding out from publication.

It isn't just weeding out. The process of peer review itself can benefit the author and the readers by improving the quality of the paper.

Although sometimes, junk does get through.

Pipirr · 12th February 2008, 01:13 PM

Originally Posted by Alric

Right. That is why I started this thread. I see it happen with evolution and climate change where people will start linking to blogs in the face of publication as valid counterpoints.

And with climate science, there is so much published research, why ignore it and focus on the unpublished?

Methinks shenanigans.

mhaze · 12th February 2008, 01:17 PM

Originally Posted by Alric

And discussions in peer-reviewed articles trump "some guy's blog".

Science does change and data is disputed with more data. However, the latest scientific consensus is what we should use to make decisions.

Can we agree to this and move on?

Data is not the same as scientific method, procedure, conclusion, hypotheses. Data in most cases should or is separate.

the latest scientific consensus is what we should use to make decisions.
Can we agree to this and move on?

No.

Dr. Imago · 12th February 2008, 04:23 PM

And, on a sidenote, a nice little article here...

http://news.bbc.co.uk/1/hi/sci/tech/290121.stm

If you can't trust the science was conducted with integrity, subsequent peer review means nothing. It is inherently expected within the process that the manuscript being submitted is accurate and the data (and conclusions) are based on events that actually took place. Sometimes this, sadly, just isn't the case.

I'm not suggesting (nor could I possible know) what, if any, science being speculated about here has been fraudulently produced and submitted. But, it (again) underscores the importance of independent validation of all landmark findings.

-Dr. Imago

11th February 2008, 06:25 PM	#1
Alric Muse Join Date: Apr 2007 Posts: 554	Is peer-reviewed data "THE DATA"? And discussions in peer-reviewed articles trump "some guy's blog". Science does change and data is disputed with more data. However, the latest scientific consensus is what we should use to make decisions. Can we agree to this and move on?
Alric Muse Join Date: Apr 2007 Posts: 554	Last edited by Alric; 11th February 2008 at 06:27 PM.

11th February 2008, 07:19 PM	#2
Scott1972 Student Join Date: Aug 2007 Posts: 48	It's not clear to me what you are trying to say. Being able to reproduce the data is what is important, not a person's or group's data. For theoretical predictions, it's the prediction conforming to empirical data, and those measurements being reproducible.
Scott1972 Student Join Date: Aug 2007 Posts: 48

12th February 2008, 11:43 AM	#4
Alric Muse Join Date: Apr 2007 Posts: 554	Right. That is why I started this thread. I see it happen with evolution and climate change where people will start linking to blogs in the face of publication as valid counterpoints.
Alric Muse Join Date: Apr 2007 Posts: 554

12th February 2008, 12:32 PM	#5
Ivor the Engineer Philosopher Join Date: Feb 2006 Location: South Britain, near the middle Posts: 9,553	While I generally agree (and hope) peer-reviewed journal articles have higher validity on average than unpublished work and points of view, I'll just mention Alan Sokal. Plus let's not forget other factors such as researcher bias, publication bias and, dare I say it, elitism. Whether a piece of research is published or unpublished, all that really matters are the method and the results.
	__________________ My Blog.

12th February 2008, 12:37 PM	#6
Alric Muse Join Date: Apr 2007 Posts: 554	Yeah but what can you expect when the publication is on a post-modernist journal. I don't think you could pull the same stunt in Science magazine, at least not for long. And that was precisely the point.
Alric Muse Join Date: Apr 2007 Posts: 554

12th February 2008, 11:39 AM	#3
Pipirr Graduate Poster Join Date: Mar 2006 Location: North Posts: 1,457	Blog commentary can be very useful in understanding peer reviewed research. Where things get silly is when some guy with a blog is trying to rewrite science as we know it, and their unpublished, non-peer reviewed blog posts become elevated to a position of privilege. Take D'Aleo, with his long paper* about problems in the US temperature record: those are grand claims. But are they valid claims, or has he just got it all wrong? If there really is a problem with the temperature record, then publish. The people that need to know would notice it. If it is unpublished, I think it is reasonable to wonder why not. If research has merit, get it through peer review, especially if the claims that are made and the conclusions that are reached are grand. Otherwise, what is there to distinguish it from any other blog post by yet another internet crazy? The ability to convert a file into PDF format does not a published paper make.

12th February 2008, 12:44 PM	#7
Ivor the Engineer Philosopher Join Date: Feb 2006 Location: South Britain, near the middle Posts: 9,553	That's what I thought, but I'm sure I've seen positive results for homeopathy in "proper" medical journals, which members of this forum with suitable qualifications have subsequently torn to shreds. (The contents of the articles, not the journals.). It makes me wonder what the peer review process is supposed to be weeding out from publication.
	__________________ My Blog. Last edited by Ivor the Engineer; 12th February 2008 at 12:45 PM.

12th February 2008, 01:00 PM	#8
Pipirr Graduate Poster Join Date: Mar 2006 Location: North Posts: 1,457	Originally Posted by Ivor the Engineer {snip}It makes me wonder what the peer review process is supposed to be weeding out from publication. It isn't just weeding out. The process of peer review itself can benefit the author and the readers by improving the quality of the paper. Although sometimes, junk does get through.

12th February 2008, 01:13 PM	#9
Pipirr Graduate Poster Join Date: Mar 2006 Location: North Posts: 1,457	Originally Posted by Alric Right. That is why I started this thread. I see it happen with evolution and climate change where people will start linking to blogs in the face of publication as valid counterpoints. And with climate science, there is so much published research, why ignore it and focus on the unpublished? Methinks shenanigans.

12th February 2008, 01:17 PM	#10
mhaze Penultimate Amazing Join Date: Jan 2007 Location: Woo*(+-1.10)^20=AGWwoo Posts: 15,385	Originally Posted by Alric And discussions in peer-reviewed articles trump "some guy's blog". Science does change and data is disputed with more data. However, the latest scientific consensus is what we should use to make decisions. Can we agree to this and move on? Data is not the same as scientific method, procedure, conclusion, hypotheses. Data in most cases should or is separate. the latest scientific consensus is what we should use to make decisions. Can we agree to this and move on? No.
	__________________ Get your climate widget Climategate - History's Biggest Scam Poptech's List Climate Acronyms

12th February 2008, 04:23 PM	#11
Dr. Imago Master Poster Join Date: Aug 2001 Posts: 2,051	And, on a sidenote, a nice little article here... http://news.bbc.co.uk/1/hi/sci/tech/290121.stm If you can't trust the science was conducted with integrity, subsequent peer review means nothing. It is inherently expected within the process that the manuscript being submitted is accurate and the data (and conclusions) are based on events that actually took place. Sometimes this, sadly, just isn't the case. I'm not suggesting (nor could I possible know) what, if any, science being speculated about here has been fraudulently produced and submitted. But, it (again) underscores the importance of independent validation of all landmark findings. -Dr. Imago
Dr. Imago Master Poster Join Date: Aug 2001 Posts: 2,051	__________________ DISCLAIMER: The above post is for informational and/or educational purposes only. It is not a substitute for the professional judgment of, in direct consultation with, a health care professional in diagnosing, treating, and/or preventing any disease or disorder. It is not to be construed as individualized medical advice, diagnosis, or a treatment recommendation. Your reliance upon the information obtained or used by you at, through, or as a result of this post is solely at your own risk. Last edited by Dr. Imago; 12th February 2008 at 04:24 PM.