Range of Application of Program
The program has been verified to converge as long as
- The sample standard deviation s is in the range [0.1, 10.0]
(Note that this range is for the log-transformed values, if one
is dealing with the lognormal distribution)
- The degrees of freedom is in the range [2,1000] (in most cases, this
is equivalent to the number of data points being in the range [3,1001])
How Can You Be Sure that the Calculated Confidence Limits are Correct?
One simple method of verification involves the following:
- Assume values for the mean m, s2,
and number of samples n
- For any possible sample mean y and sample variance
s2, it is known that
- y = yp = m
+ (s n-1/2) F(p)
for some p in (0,1), where F(p)
is the pth percentile of the standard normal distribution
- s2 = s2q = s2
c2(q) / (n-1) for some q
(0,1), where c2(q) is the
qth percentile of the chi-squared distribution with n-1
degrees of freedom
- Perform Monte Carlo sampling for p and q
from (0,1), thus obtaining samples of the sample mean and variance
- For each Monte Carlo sample, calculate the confidence
limits for (m +
s2/2) of level a
- Verify that the fraction of samples for which the calculated
confidence limits fail is approximately equal to 1-a
(we can tell whether they fail or not because we know what the actual
mean and variance are)
A problem with this method is that it is vulnerable to sampling variation
in the Monte Carlo sampling itself. It is possible, however, to perform
a more precise test. Basically, with this method one calculates the area
of the region in (p,q) space for which the calculated confidence
limits fail (see report for details). Due to nature of the particular
sampling distributions involved, the area of this region can be calculated
without having to deal with impacts of sampling variation. A program that
calculates this area was implemented and used to verify the accuracy of
the confidence limits. This program is available on the download page.
It was applied over the range of values shown in the table below. You
can use this program to convince yourself that the program is actually
working over the range of s, a,
and/or n that you expect to encounter in your
application (note that the program is not sensitive to the value of the
sample mean y or l, since a normalization
is possible that allows the most general case to be determined from the
special case y=0 and l=1/2).
Parameter |
Values for Which Verification Program
Explicitly Applied |
Sample standard deviation s |
0.1, 1.0, 10.0 |
Confidence Level a |
0.001, 0.005, 0.01, 0.1, 0.9, 0.95, 0.99,
0.995, 0.999 |
Degrees of freedom n
|
2,3,5,10,100,1000 |
|