Operational geospace model validation

Groundbased magnetic perturbations dB/dt and Regional-K studies

Calculations and study results in the time plotting tool
Community-wide validation of geospace model ground magnetic field perturbation predictions to support model transition to operations, Antti Pulkkinen et al, 2013
Phase I Report on dB/dt Study
- Supplementary materials for phase I Report (above)
Phase II Report on Regional-K Study
Calculation of K index values from model magnetic perturbations, Alex Glocer et al, presented at GEM 2013 Summer Workshop
Presentations from GEM 2013 Summer Workshop
Presentations from GEM 2012 Summer Workshop

About the study
To support the transition of space weather research models to operations, the CCMC is leading the effort to assess the ability of geospace models to forecast geomagnetic disturbances. The final Phase I Report on dB/dt Study have been submitted to NOAA/SWPC during the 2013 Space Weather Workshop. The results of the Phase II study on Regional-K index was presented and discussed at the GEM 2013 Summer Workshop.

This geospace model validation project is built upon the ground magnetic perturbations (delta-B) metrics study which is a part of the GEM Modeling Challenge (Pulkkinen et al., 2010, 2011; Rastaetter et al., 2011). The community-wide GEM Modeling Challenge was initiated in the summer of 2008 by the GGCM Metrics and Validation Focus Group. The CCMC is supporting the Challenge.

NOAA SWPC is planning to utilize the results of this geospace model validation project in their selection of operational geospace model. The time derivative of the horizontal magnetic field vector dB/dt and regional K index were identified as physical parameters of primary interest to NOAA SWPC customers.

Three global MHD magnetosphere models (SWMF v20110131, OpenGGCM v4.0, and CMIT-LFM-MIX vLTR-2_1_1) and two empirical delta-B models (Weimer and Weigel) participate in this activity.

Global MHD models were delivered to CCMC in January 2011. There was extensive communication with the model developers to ensure the correct installation and appropriate settings of the models. Model installation, robustness tests, necessary revisions performed by the code developers in response to CCMC feedback, and the final selection of model settings were accomplished by mid August 2011. Test simulations for all models were performed on 64 processors (or less) of the CCMC Beowulf Cluster. Simulation settings for all models were selected so that the simulations take no longer than twice the duration of the simulated event.

In support of the project the CCMC developed a tool that extracts the ground magnetic field perturbations from the global MHD model outputs (through integration over magnetospheric and ionospheric current systems).

The CCMC performed a series of sensitivity studies: sensitivity to methods of dB/dt calculations (e.g.,the CCMC post-processing tool vs. run-time calculations implemented in SWMF); sensitivity to ground station locations sensitivity to model output frequency. The results of the studies were presented to code developers and discussed at special sessions of the GEM and mini-GEM Workshops.

One of the lessons learned from the GEM Modeling Challenge is that the results of metrics studies strongly depend on the selected metrics type (how the skill score characterizing the model performance is calculated). The CCMC has performed a series of test metrics studies on the outputs of the GEM Challenge simulations. Different approaches to skill score calculations were demonstrated and discussed. Per discussion with the model developers and NOAA SWPC it was agreed to implement the threshold-based model-data comparisons and calculate the skills such as probability of detection (POD), probability of false detection (POFD), and Heidke skill score [Lopez et al, 2007, Pulkkinen et al, 2010].

Six events that include the original four GEM Challenge events and two "surprise" events were selected for the final evaluation. All simulations were conducted at the CCMC. Full 3D results are available for on-line visualization. Calculated ground magnetic perturbations dB/dt and Regional-K for all models participating in the study together with observational data for each event and ground stations can be viewed using the online time-series plotting tool.

The final Phase I Report on dB/dt Study have been submitted to NOAA/SWPC during the 2013 Space Weather Workshop. The results of the Phase II study on Regional-K index was presented and discussed at the "GGCM Modeling Challenge" session at the GEM 2013 Summer WorkshopThe final Phase I Report on dB/dt Study have been submitted to NOAA/SWPC during the 2013 Space Weather Workshop. The results of the Phase II study on Regional-K index was presented and discussed at the "GGCM Modeling Challenge" session at the GEM 2013 Summer Workshop

Publications and presentations:

Glocer, A., L. Rastätter, M. Kuznetsova, A. Pulkkinen, H. J. Singer, D. Weimer, D. Welling, M. Wiltberger, J. Raeder, R.S. Weigel, J. McCollough, S. Wing (2016), Community-wide validation of geospace model local K-index predictions to support model transition to operations, Space Weather, 14, 469–480, doi:10.1002/2016SW001387
Rastaetter, L., G. Toth, M. M. Kuznetsova, and A. A. Pulkkinen (2014), CalcDeltaB: An efficient postprocessing tool to calculate ground-level magnetic perturbations from global magnetosphere simulations, Space Weather, 11, doi:10.1002/2014SW001083.
A. Pulkkinen, L. Rastätter, M. Kuznetsova, H. Singer, C. Balch, D. Weimer, G. Toth, A. Ridley, T. Gombosi, M. Wiltberger, J. Raeder, and R. Weigel, Community-wide validation of geospace model ground magnetic field perturbation predictions to support model transition to operations, Space Weather Journal, vol.11, 369-385, doi:10.1002.swe.20056, 2013.
A. Pulkkinen, M.Kuznetsova, A.Ridley, J.Raeder, A.Vapirev, D.Weimer, R.S.Weigel, M.Wiltberger, G.Milward, L.Rastaetter, M.Hesse, H.J.Singer, A.Chulaki, Geospace Environment Modeling 2008-2009 Challenge: Ground magnetic field perturbations, Space Weather Journal, vol.9, SO2004, doi:10.1029/2010SW000600, 2011.
A.Pulkkinen, L.Rastaetter, M.Kuznetsova, M.Hesse, A.Ridley, J.Raeder, H.J.Singer, A.Chulaki, Systematic evaluation of ground and geostationary magnetic field predictions generated by global magnetohydrodynamic models, JGR, vol.115, A03206, doi:10.1029/2009JA014537, 2010.
R.Lopez, S.Hernandez, M.Wiltberger, C.-L.Huang., E.L.Kepko, H.Spence, C.C.Goodrich, J.G.Lyon, Predicting magnetopause crossings at geosynchronous orbit during the Halloween storms, Space Weather, vol.5, S01005, doi:10.1029/2006SW00222, 2007
Phase I Report on Ground Magnetic Field Perturbations dB/dt, CCMC report submitted to NOAA/SWPC during the 2013 Space Weather Workshop
Supplementary material for Phase I Report
Calculation of K index values from model magnetic perturbations, Alex Glocer, GEM Summer Workshop, 2013
Assessing Geospace Models for Use in Space Weather Operations, Howard Singer, Space Weather Workshop, 2012
Sensitivity analyses for evaluation of ground magnetic field predictions, A.Pulkkinen, M.Kuznetsova, L.Rastaetter, 2011
From GEM metrics studies to operational geospace model selection, A.Pulkkinen, M.Kuznetsova, M.Hesse, L.Rastaetter, A.Chulaki, J.S.Shim, GEM-CEDAR Workshop, 2011
Geospace Model Project Goals, Howard Singer, Geospace Model Workshop, 2011
Model Delivery to CCMC, Masha Kuznetsova, Geospace Model Workshop, 2011
GEM 2008-2009 Challenge: ground magnetic field perturbations, A.Pulkkinen, M.Kuznetsova, A.Ridley, J. Raeder, D.Weimer, R.Weige, M.Wiltberger, G.Millward, L.Rastaetter, M.Hesse, H.J.Singer, A.Chulaki, AGU GEM mini-workshop, 2009

Feedback Collection