Uncertainty in Estimates

Level of uncertainty

Considerable care was taken to identify a wide range of auxiliary variables that could be used in the modeling, and extensive efforts were made to select the sets of auxiliary variables that best predicted the percentages of adults lacking Basic Prose Literacy Skills (BPLS) for the 2003 and 1992 models. However, while the sets did contribute significantly in the models, their predictive ability is limited. As a result, the precision of the indirect county estimates is often fairly low. The state estimates are more precise, and gains in precision are achieved in the estimates for states participating in SAAL and SALS as a result of the increased sample sizes in these states.

It is important to take the prediction error in model-dependent indirect estimates into account in their interpretation. Users need to pay careful attention to the 95 percent credible interval bounds that are provided along with the indirect estimates to assess the range of uncertainty in the estimates. The credible intervals for the county indirect estimates are often very wide.

Credible intervals

Credible intervals have been computed to indicate the levels of uncertainty in the indirect estimates of the percentages of adults lacking Basic Prose Literacy Skills (BPLS). A credible interval is a posterior probability interval, used in Bayesian statistics for purposes similar to those of a confidence interval in frequentist statistics. A 95 percent credible interval for an estimate of the percentage of adults in a county lacking BPLS gives the range for which there is a probability of 0.95 that the interval contains the true percent lacking BPLS. For example, suppose a county's estimate for the percent lacking BPLS is 12 percent with a 95 percent credible interval of 5 to 25 percent (as in this example, the intervals are generally asymmetric around the estimate). This indicates that there is a probability of 0.95 that the actual value was between 5 and 25 percent.

Comparisons

Exact 95 percent credible intervals have been produced for the differences between the estimated percentages lacking BPLS in 2003 for all pairs of counties within the same state. However, exact credible intervals have not been computed for all other comparisons. For these other comparisons, the analysis has been restricted to determining whether or not the 95 percent credible interval contains 0, using the following approach:

• If the credible interval for the estimate for county (or state) i does not overlap with the credible interval for the estimate for county (or state) j, then the credible interval for the difference does not contain 0.
• If the credible interval for the estimate for county (or state) i is fully nested within the credible interval for the estimate for county (or state) j, then the credible interval for the difference contains 0.
• If the credible intervals for the estimates for the two counties or two states partially overlap, then the determination is based on the calculation of an approximate credible interval.

For more detail on the comparison of estimates, see Estimation Approach.

Users should keep in mind that the credible intervals for the differences in county indirect estimates are often very wide (see General Cautions).

Top