Science Buffs - Is There A Method Guaranteed By Theory Which Has Not Been Calibrated?

[Dr Adequate]

OK, let me explain. By 'method' I mean method of finding things out. A telescope will do as a concrete example.

Now when Galileo made his telescope, the laws of optics told him that it was justified in theory. But he could also calibrate it: he could use it to look at a distant object, and then go and look at it close up, and check that the telescope really was magnifying: he could check it against the 'standard yardstick' of the naked eye. Then he can turn his attention to the night sky.

But suppose that for some reason he couldn't make such tests. Then his observations would be guranteed only by the laws of optics, giving them a slightly lower epistemological status.

Now, can anyone think of any method in current or past science that has that position? They should be few and far between. If theory tells you you've built a fish detector, you can go get some fish and calibrate it. But such methods could exist. I just can't think of one.

I thought I had one with the age of stars, but a bit of thought showed me that the methods for finding this can in fact be calibrated against an accurate stopwatch.

Can anyone think of an example, present, or failing that, past? Alternatively, if you'd like to post saying "I work in field X and I can't think of such a thing", that would also be enlightening.

Thank you.

[drkitten]

Originally posted by Dr Adequate


Now, can anyone think of any method in current or past science that has that position? They should be few and far between. If theory tells you you've built a fish detector, you can go get some fish and calibrate it. But such methods could exist. I just can't think of one.


Cryptography and cryptanalysis.

Specifically, trying to determine if one cryptographic system is "stronger" than another. For example, the strength of the RSA cryptosystem is well-known to be derived from the difficulty of the factoring problem; if I can factor the product pq of two large primes into those primes p and q, I can easily break the system. I could name a number of other, less well-known systems that hinge on other problems, such as the difficulty of taking a discrete log in a finite field, or a known NP-complete problem such as filling knapsacks or Boolean satisfiabiliity.

I cannot, however, cross-calibrate these problems. So far, no one knows a fast method of factoring or of taking discrete logs. But there are a lot of very bright people working on both. Which problem will fall first? So I can't really state that "this is a better system than that." I can say that "under currently known methods, this system is better than that,"... but our knowledge will almost certainly change.

Possibly by this time tomorrow.

[Dr Adequate]

This doesn't really fit the description.

Where is the method that works in theory but can't be calibrated?

If you had some theoretically sound method of cryptanalysis, then you could try it out on some samples of cypher with known decryptions to see if it decrypts properly, which would be calibration. Having calibrated it, you could then use it with some confidence on samples of cypher for which you have no decryption. This would not be a problem.

Anyone else?

Perhaps one of the geological clocks? ... but then, it could be checked against some of those which can be calibrated, which would constitute calibration.

I'm stumped.

[geni]

High resolution IR when used to determine the length of bonds in linear molecules.

[Rob Lister]

Originally posted by new drkitten
Cryptography and cryptanalysis...

...Which problem will fall first?...

Possibly by this time tomorrow.

Which is an interesting thread in itself. Suppose I was a really smart guy and figured out a way to quickly factor integers.

The illumina would certainly have me killed, not to mention the banks, cia, nsa, my wife (for different reasons)... :)

[drkitten]

Originally posted by Dr Adequate
This doesn't really fit the description.

Where is the method that works in theory but can't be calibrated?

If you had some theoretically sound method of cryptanalysis....

Wrong direction. We have some theoretically sound methods of cryptography that cannot be calibrated.

Specifically, we can confirm that messages can be successfully encoded and the encoded messages can be successfully decoded. In this sense, we can "calibrate" the cryptographic protocols to make sure that they work to transmit messages.

We have some theoretical results about how hard it would be for someone to cryptanalyze them. And we can confirm that for the cryptanalytic methods we know about, the observed results match the theoretical ones. We cannot, however, confirm (or calibrate) our theoretical measures of the method's strength against unknown cryptanalytic techniques.

[JamesM]

In theoretical chemistry, to the extent that quantum mechanics works, you can attempt to correlate calculated properties (e.g. electrostatic potential at an atom) with experimental properties (say, hydrogen bonding strength) - the very definition of calibration, to be sure.

But you could then extrapolate the results to calculate hydrogen-bonding values for a molecule for which that property is not defined, e.g. due to it being too reactive under the conditions the other molecules were measured. For that specific molecule, you would have a meaningful result only by virtue of quantum mechanics, empirical confirmation would be impossible.

That's the best I can think of right now.

[Dr Adequate]

Dr k --- I'm not sure your example works. We do not know in theory whether NP is larger than P.

But a similar example might go like this: "We know in theory that there are only so many sporadic finite simple groups. We can never verify this in practice because we'd have to list all possible finite groups and check how many are sporadic and simple, and there's an infinite number of them."

This example is not what I'm looking for for two reasons. Firstly, the method should be based on a physical theory, not a mathematical theorem. Secondly, in maths, the theorem is the "standard yardstick" for finding out whether something is true or false --- trying an infinite number of possibilities is not.

I'm looking for something like a physical fact --- the age of a certain rock, say, which can be given by a method which should work in theory but which, for some reason, cannot be not tested against an object the age of which we know. Something like that.

JamesM --- that is, in fact, what I mean by calibration. If we find a method works when we can test it agaainst some existing yardstick, then we're justified in using it when we don't have that yardstick. (Otherwise, what's the use of the method?)

So Galileo could check that a certain telescope made things look thirty times closer --- by testing it against terrestrial objects. He could then use it with reasonable confidence on celestial bodies, although he couldn't get thirty times closer to them to check his results. The instrument was calibrated.

In the same way, if we check that the quantum theoretical method of finding such-and-such a value works whenever we can test it, then we can regard it as calibrated and use it when we can't test it. Otherwise, what would be the point of having quantum mechanics?

This relies, of course, on the deepest principle of science: that nature is not jerking us around.

Or as Einstein put it: "God is subtle, but he is not malicious".

[JamesM]

Originally posted by Dr Adequate
If we find a method works when we can test it agaainst some existing yardstick, then we're justified in using it when we don't have that yardstick. (Otherwise, what's the use of the method?)[/b]
This is a side point, but: the use would be when making the real measurement would be too time-consuming or costly, but still possible. Estimates of the total number of potential organic molecules are between 10¹⁸ - 10²⁰⁰ - there simply aren't enough atoms in the universe to make even one molecule of each, assuming anyone had the time or money to attempt such an undertaking. A theoretical enumeration of the properties of such molecules is becoming possible, however.

In the same way, if we check that the quantum theoretical method of finding such-and-such a value works whenever we can test it, then we can regard it as calibrated and use it when we can't test it. Otherwise, what would be the point of having quantum mechanics?
No calibration can be universally reliable. Some applications are a larger extrapolation than others. The larger the extrapolation, the less your confidence in the results.

Not really answering the question you posed, I know.

[Dr Adequate]

Originally posted by JamesM
This is a side point, but: the use would be when making the real measurement would be too time-consuming or costly, but still possible. You're absolutely right, it would have practical use. But yes, you know what I mean.No calibration can be universally reliable. Some applications are a larger extrapolation than others. The larger the extrapolation, the less your confidence in the results. For example, a relation which may look linear over the range you can test it might be non-linear, if that range is small. This sort of discussion does interest me, fire away.

There are cases the other way round, which have been justified in practice but not in theory. Spectrography is an example --- am I right? --- the dark bands could be correlated in practice with elements before anyone knew the underlying quantum theory. I think I'm also right in saying that the first "standard candles" in astronomy were found to be standard before anyone knew why. Is there any current example of such a thing, I wonder, or is our theory completely up with our practice? Can anyone think of other examples from the present, or, failing that, the past?

[JamesM]

Originally posted by Dr Adequate
You're absolutely right, it would have practical use. But yes, you know what I mean.
Yes, although I wasn't trying to be facetious, those practical considerations are of enormous significance in the field of drug discovery.

For example, a relation which may look linear over the range you can test it might be non-linear, if that range is small. This sort of discussion does interest me, fire away.
This is the sort of the problem that anyone involved in computational drug (and more generally, molecular) design must grapple with daily. It's not that we don't have theoretical approach that ought to work, it's just it's of no practical use due to computational and speed constraints, so we must fall back on approximations.

We know these approximations have only a limited validity, but reliably establishing the limits of the validity remains an unsolved problem. I can use fancy machine learning techniques to learn a relationship between molecular properties and drug activity, but if you present me with a new chemical and ask how good a drug it is, I am unlikely to be able to quantify the degree of confidence in the prediction. This is not a problem restricted to chemistry, of course, it cuts across machine learning, statistics and data mining.

But are we still doing science? I would say so.

Can anyone think of other examples from the present, or, failing that, the past?
Other quantum examples are Planck's explanation of blackbody radiation, Balmer's discovery of the line spectra of hydrogen, and Bohr's model of the hydrogen atom. These all seemed to work, but could not be reconciled to classical electromagnetic theory.

A slightly more esoteric example: people were happily using genetic algorithms for a while, without proof that they converged to the correct solution within a certain time. I don't know what the status is for other popular optimisation heuristics like Particle Swarm, Artificial Ant and tabu search. I suppose that's why they're called heuristics.

[Dr Adequate]

Originally posted by JamesM
Other quantum examples are Planck's explanation of blackbody radiation, Balmer's discovery of the line spectra of hydrogen, and Bohr's model of the hydrogen atom. These all seemed to work, but could not be reconciled to classical electromagnetic theory. These weren't 'methods' in my sense (ways of finding out facts) lacking a deeper theory, but rather theories lacking a deeper theory.

(And of course, the deepest theories we know have precisely that status. Balmer (IIRC) said restrospectively of his formula that it was "mere numerology", but it might have turned out to have been The Law. This is by-the-by.)

[rppa]

It might be that answers to the important question of "what is the neutrino mass" might fit this bill, but perhaps not. In asking about this on physics forums, I've learned that all we seem to know from experimental record is: (a) the mass isn't zero and (b) the different types of neutrinos have different masses. There doesn't seem to be an experiment which can actually say "the mass of the electron neutrino is at least 3 eV" (due to the small numbers involved, particle masses are usually quoted in terms of their E = mc^2 equivalent, on which scale the electron is about half a million eV).

[69dodge]

Originally posted by Dr Adequate
I'm looking for something like a physical fact --- the age of a certain rock, say, which can be given by a method which should work in theory but which, for some reason, cannot be not tested against an object the age of which we know. Something like that.I don't think you'll find what you're looking for. What does "should work in theory" mean? It means, "should work according to some theory we use because it has some sort of experimental support." We don't just make up our scientific theories out of thin air, after all. There's always some connection, more or less tenuous, to something that has been calibrated.

[Soapy Sam]

Dr.A.
I question your example in the OP.

When Galileo checked things with the naked eye, he was relying on the laws of optics just as much as when he looked through the telescope with the naked eye- ie not at all.

Particularly if you mean Newtonian Optical Laws, which the (as yet unborn) Newton had not written at the time.

(Thanks to a typo, I just invented the "klaws of optics", which would be Infra red in tooth and klaw).

[Dr Adequate]

Originally posted by 69dodge
I don't think you'll find what you're looking for. I'd bet against it. But I'm obliged to look. What does "should work in theory" mean? It means, "should work according to some theory we use because it has some sort of experimental support." We don't just make up our scientific theories out of thin air, after all. There's always some connection, more or less tenuous, to something that has been calibrated. I said that examples should be very scarce. But they are possible in theory.

For example, suppose that the only way we could find out about ages of stars was to run a theoretical model of the laws of physics and so find out, in theory, what a star of such and such an age should, in theory, look like. This would be a method justified by theory, but which could not (so far as I can see) be calibrated, if that was the only method we had, against the "standard yardstick" of subtracting the time of a thing's origin from the time now. So in that case, the only justification for saying "the star is so-and-so million years old" would be that this follows from the known laws of physics.

(This example fails because there is at least one more way of getting information about the age of a star, which can be calibrated. Then this method can be calibrated against that.)

[Dr Adequate]

Originally posted by Soapy Sam
Dr.A.
I question your example in the OP.

When Galileo checked things with the naked eye, he was relying on the laws of optics just as much as when he looked through the telescope with the naked eye- ie not at all.

Particularly if you mean Newtonian Optical Laws, which the (as yet unborn) Newton had not written at the time. Galileo knew enough about optics to construct a telescope from a description of what it did. Obviously, he then looked at distant objects to see if it really did magnify them. If he hadn't been able to make such a check before looking at the night sky, he'd have been relying on theory alone. (And, if the theory was too poor to explain chromatic aberation, he might then have taken it as a genuine celestial phenomenon, revealed by the telescope.)

[Bruce]

There's simple light absorption technique called Nephalometry (sp?) that measures the turbidity of a solution. I had one of these at my old work place. I don't quite remember the theory off the top of my head, but I remember the manufacturer saying that it didn't need to be calibrated.

They had such a difficult time explaining this to those who didn't understand theory that they sold standards anyway. The standards were outrageously expensive and only had a 1 month shelf life. They were useless anyway because there was no way to adjust the instrument. It literally used a detector to measure the difference in intesity from a regular light bulb, with and without the sample.

Naturally, my boss bought the standards every two months and had the techs run them anyway. I'm so glad I got out of there. :)

[Dr Adequate]

Originally posted by Bruce
There's simple light absorption technique called Nephalometry (sp?) that measures the turbidity of a solution. I had one of these at my old work place. I don't quite remember the theory off the top of my head, but I remember the manufacturer saying that it didn't need to be calibrated. This is an interesting story, but it doesn't give an example of what I'm looking for. Theory led them to build the device. Then, before they advertised it, took orders, and shipped it, they themselves (I bet dollars to donuts) tested their device against given standards. They calibrated it. They made sure their method gave the right results according to some pre-existing "standard yardstick". Then they went to market.

I do see the point of the rest of your anecdote, but really you should send it to Scott Adams. The question of how much time and money you should spend calibrating an individual measuring device guaranteed by the manufacturer is a practical and economic matter.

And, as it turned out, you could calibrate these devices agaist standards if you wanted to. You just didn't need to, as it turned out. I'm looking for examples where calibration is impossible, not where it's unnecessary.

[davefoc]

Suppose that the carbon 14 dating test was not accurate enough to measure the age of organic substances that are of recent enough origin to have a known date associated with them.

Then the use of Carbon 14 dating would be based entirely on a theory that the decay rate of carbon 14 was constant and carbon 14 would be an example of a method of determining the age of something that could not be calibrated.

In fact, that was more or less the situation when carbon 14 dating was invented. Subsequently carbon 14 dating has been calibrated with wood from trees of known age based on their estimated age derived from tree ring analysis. Modifications were made to carbon 14 dating methodologies based on the tree ring calibration.

So carbon 14 dating, initially, was something of an uncalibrated methodology.

I am less familiar (not that I'm that familiar with carbon 14 dating) with the other radiometric dating techniques but I suspect that one could quibble with the various calibration techniques that have been used to test their accuracy. No matter what one doesn't just walk into the dessert and pick up a few rocks marked with their ages and use them to calibrate these techniques. The accuracy of these techiques depends on theories that various decay rates have stayed constant for whichI don't think there is any reliable method of calibration to insure that they are constant over great amounts of time. The accuracy also depends on the notion that we know what the test sample was made of in the first place a notion which is clearly not subject to validation by calibration.

[Bruce]

Originally posted by Dr Adequate

And, as it turned out, you could calibrate these devices agaist standards if you wanted to. You just didn't need to, as it turned out. I'm looking for examples where calibration is impossible, not where it's unnecessary.

True ab initio, eh? I knew many computational chemists that claimed to have programs that could predict such things as bond distances, angles, and energies of compounds that had not yet been made. They were so certain their programs were accurate that they would fault the experimentalists if the results didn't match when the compound was actually synthesized. Yes, the experimentalists must have done something wrong, those oafs.

This is nothing new in computational chemistry universe. Whenever reality doesn't match theory, and the scientific community is convinced that the experimental results are accurate, the computational chemists returned to their computers and come back with the new and improved program that takes care of all those previous problems. When I graduated, Density Functional Theory (DFT) was the big thing. I'm not sure what it is now.

The computational chemists that I knew found the "real world" to be a hard slap in the face. I don't know a single one that ever found a career in computational chemistry outside of academics.

Must be that whole "gaurantee" thing that customers are so picky about. :rolleyes:

[Carn]

Originally posted by Dr Adequate

For example, suppose that the only way we could find out about ages of stars was to run a theoretical model of the laws of physics and so find out, in theory, what a star of such and such an age should, in theory, look like. This would be a method justified by theory, but which could not (so far as I can see) be calibrated, if that was the only method we had, against the "standard yardstick" of subtracting the time of a thing's origin from the time now. So in that case, the only justification for saying "the star is so-and-so million years old" would be that this follows from the known laws of physics.

(This example fails because there is at least one more way of getting information about the age of a star, which can be calibrated. Then this method can be calibrated against that.)

Measuring the age of universe fits that description.
No control universe and some of the physic laws and constants could have been slightly different in the past than today.
And no solid theory of the intial stages.

Carn

[drkitten]

Originally posted by Dr Adequate

For example, suppose that the only way we could find out about ages of stars was to run a theoretical model of the laws of physics and so find out, in theory, what a star of such and such an age should, in theory, look like.



But where does that theoretical model come from?

The very evidence that is used to construct the theory could also be used to calibrate it.

[Drooper]

What about a mass spectrometer?

Based on the theory that would describe the movement of ionised particles it measures the relative concentration of molecules or elements.

However, we can't take a sample and count them to check.

[davefoc]

What about a mass spectrometer?

It is possible, I think, to very accurately way chemically pure substances and make an estimate of the isotope ratios. This could serve as a calibrating method I think.

But I could also be totally wrong about this. I await more informed opinions.

[davefoc]

davefoc wrote:...to very accurately way chemically pure...

I'm pretty sure that I meant "to very accurately weigh" in that sentence.

[Dr Adequate]

Originally posted by davefoc
Suppose that the carbon 14 dating test was not accurate enough to measure the age of organic substances that are of recent enough origin to have a known date associated with them.

Then the use of Carbon 14 dating would be based entirely on a theory that the decay rate of carbon 14 was constant and carbon 14 would be an example of a method of determining the age of something that could not be calibrated.

In fact, that was more or less the situation when carbon 14 dating was invented. Subsequently carbon 14 dating has been calibrated with wood from trees of known age based on their estimated age derived from tree ring analysis. Modifications were made to carbon 14 dating methodologies based on the tree ring calibration.

So carbon 14 dating, initially, was something of an uncalibrated methodology. That would seem fit the bill.

You write "have a known date associated with them". Does "associated" include all the other methods of dating which were then known, if any?

Thanks for the information.