Posted by Yon Bard on July 03, 1999 at 11:00:07:
I have just sent the following letter to HALI:
Your report on the Liestal conference (HALI 104 pp. 82-85) was most interesting, but left some open questions on how to interpret the results. Taking, for example, the Salor chuval fragment from the bottom of page 82, we are told that with 95% confidence, the piece has a 74.6% probability of dating from 1485-1608, and an 18% probability of dating from 1742-1808. The unwary reader may get the impression that the rug probably dates from the 16th century, but, as the following thought experiment demonstrates, nothing could be further from the truth: Suppose we analyzed a large number of Salors made, say, around 1800 from the same materials as the one shown. They will all have the same C14 content, hence (subject to some experimental error) will all be described as in the quoted example. Yet, 100% rather than 18% of them will have come from the second period.
So what do these interval probabilities mean? They apply to a situation where we pick rugs at random from a uniformly distributed population of rugs, i.e., having equal numbers from each year in history (or at least from 1485 to1808), and subject them to C14 analysis. We continue until we have obtained a large number of rugs whose C14 level matches that of the Salor. Then, within this set of rugs, 71% (=74.6x0.95) will have been made between 1485 and 1608 and 17% (=18x0.95) between 1742 and 1808. Since the assumption of uniform so-called prior distribution does not hold in practice, the interval probabilities stated in the article are inapplicable.
Let us see how the results of the analysis need to be modified to conform with what we know, or think we know, about a rug. Suppose, for example, that we knew with certainty that our Salor was woven in 1802. The C14 analysis neither confirms nor refutes this knowledge – the analysis is irrelevant, i.e., it produces no new information. The same would be true if we knew the piece to date from 1999. If we knew for sure that the piece dated from 1800 to 1820, then after the C14 analysis we would be able to assert that with 95% probability it actually dated from 1800-1808. On the other hand, if in view of our general knowledge we felt only 80% sure that it was made between 1800 and 1820, then after the C14 analysis we could place it in the 1800-1808 period with something like 80x0.95, or 76% probability.
The real point of this essay is to point out that if C14 analysis produces ranges that are in conflict with what we know, or even think we know, about the piece, then we can recognize that these ranges do not constitute new information, and they can be safely rejected out of hand as mere artifacts of the methodology.
Technical notes:
The statistical analysis that combines prior knowledge about an item with information derived from new observations is an example of Bayesian inference.
Our probability calculations were somewhat simplified for the purpose of exposition, but the exact results would not differ substantially from the ones presented.
Regards,
Yonathan Bard