Uncertainty Quantification/Verification & Validation Seminar Series (Internal at Sandia)

This talk will describe a pragmatic interval-based approach to model validation where significant aleatory and epistemic sources of uncertainty exist in the experiments and simulations. The validation comparison of experimental and simulation results, and corresponding criteria and procedures for model affirmation or refutation, take place in “real space” as opposed to “difference space” where subtractive differences between experiments and simulations are assessed. The versatile model validation framework handles difficulties associated with representing and aggregating aleatory and epistemic uncertainties from multiple correlated and uncorrelated source types, including:

• experimental variability from multiple repeat experiments
• uncertainty of experimental inputs
• experimental output measurement uncertainties
• uncertainties that arise in data processing and inference from raw simulation and experiment outputs
• parameter and model-form uncertainties intrinsic to the model
• numerical solution uncertainty from model discretization effects

Significantly, the framework and uncertainty processing machinery of the new model validation methodology can serve dual use for model calibration under uncertainty. It will be explained how the framework provides connectivity of sub-scale model validation & calibration activities into hierarchical modeling efforts, such as QMU analysis. Recent applications in the QASPR and fire-modeling programs will be presented, along with several other application examples.

Our basic goal in model validation is to perform an assessment of the usefulness of a model or simulation for its intended purpose. But we
know from the outset that all modeling is approximation, and thus that all simulations exhibit error. Further complicating our efforts is that, for a variety of reasons, we cannot rely on obtaining an exact description for a given physical scenario through experiment either, nor can we perform a sufficiently exhaustive experiment suite to achieve a full characterization for a given class of physical events. Finally, it is difficult, or impossible, to segregate various sources of discrepancy between model-based simulations and experiments. All of the above contribute to the existence of essential uncertainty in the validation process, both inherent and epistemic.

In this discussion, I will present some ideas on building a mathematical framework for developing validation methods and algorithms that are capable of accommodating error and various forms of uncertainty. I also suggest the framework as a basis on which we might build to allow deeper exploration via the assessment process. For example, one within which we can address such issues as: (1) The relevance of acquired data; (2) The performance of cost-benefit trade-offs for ascertaining where best to apportion resources; and, finally, (3) The development of better up-front tools for specifying accuracy requirements.

NOTE: Many people expressed interest in obtaining the slides from the Model Validation/UQ tutorial that was given on January 21st. The slides and videostream are available on the web site.