Purpose: The purpose of this standard is to ensure that statistically sound practices are used to generate estimates from models for information products.
Scope: The Census Bureau’s statistical quality standards apply to all information products released by the Census Bureau and the activities that generate those products, including products released to the public, sponsors, joint partners, or other customers. All Census Bureau employees and Special Sworn Status individuals must comply with these standards; this includes contractors and other individuals who receive Census Bureau funding to develop and release Census Bureau information products.
In particular, this standard applies to the production of estimates from models for Census Bureau information products. This standard applies to models (e.g., regression, economic, and log-linear) used to produce estimates, such as:
Exclusions:
In addition to the global exclusions listed in the Preface, this standard does not apply to:
Key Terms: Autocorrelation function, autoregressive integrated moving average (ARIMA), cross-validation, goodness-of-fit, heteroscedastic, homoscedastic, model, model validation, Monte Carlo simulation, multicollinearity, projection, regression, revisions history, residual, sanitized data, seasonal adjustment, sensitivity analysis, sliding spans, small area estimation, and spectral graphs.
Requirement D2-1: Throughout all processes associated with estimation, unauthorized release of protected information or administratively restricted information must be prevented by following federal laws (e.g., Title 13, Title 15, and Title 26), Census Bureau policies (e.g., Data Stewardship Policies), and additional provisions governing the use of the data (e.g., as may be specified in a memorandum of understanding or data-use agreement). (See Statistical Quality Standard S1, Protecting Confidentiality.)
Requirement D2-2: A plan must be developed that addresses:
Note: Statistical Quality Standard A1, Planning a Data Program, addresses overall planning requirements, including estimates of schedule and costs.
Requirement D2-3: Models must be developed and implemented using statistically sound practices.
Examples of statistically sound model development practices include:
Examples of statistically sound practices for demographic estimates and projections include:
Examples of statistically sound practices for seasonal adjustments include:
Sub-Requirement D2-3.1: Model results must be evaluated and validated, and the results of the evaluation and validation must be documented.
Examples of evaluation and validation activities include:
Note: Evaluation and validation is required when the model is developed. Models used in a continuing production setting must be re-evaluated periodically as appropriate.
Sub-Requirement D2-3.2: Specifications for the modeling and estimation systems must be developed and implemented.
Examples of issues that specifications might address include:
Sub-Requirement D2-3.3:; Estimation systems must be verified and tested to ensure that all components function as intended.
Examples of verification and testing activities include:
Sub-Requirement D2-3.4: Methods and systems must be developed and implemented to verify the modeled estimates and evaluate their quality.
Examples of verification and evaluation activities include:
Note: Statistical Quality Standard D3, Producing Measures and Indicators of Nonsampling Error, provides requirements for measuring and evaluating nonsampling error.
Sub-Requirement D2-3.4.1: The seasonal adjustment process and results must be reviewed annually by the program manager (or the appropriate mathematical statistician) to identify needed changes in the X-12-ARIMA specification files. Using the required secure data transmission protocols, the program manager (or the appropriate mathematical statistician) must provide the following to the Time Series Methods Staff (TSMS) of the Office of Statistical Methods and Research for Economic Programs (OSMREP):
Sub-Requirement D2-3.4.2: For indicator releases, any routine revisions to the annual review process, such as benchmarking and updating of seasonality factors, must be consolidated and released simultaneously. See Statistical Policy Directive No. 3. Deviations from this requirement must be approved as specified in the directive.
Requirement D2-4: Documentation needed to replicate and evaluate the modeling activities must be produced. The documentation must be retained, consistent with applicable policies and data use agreements, and must be made available to Census Bureau employees who need it to carry out their work. (See Statistical Quality Standard S2, Managing Data and Documents.)
Examples of documentation include:
Notes: