注：包括SYM-H Index和Storm Intensity index

A transformer-based framework for predicting geomagnetic indices with uncertainty quantification

Journal of Intelligent Information Systems 18 November 2023

A transformer-based framework for predicting geomagnetic indices with uncertainty quantification | Journal of Intelligent Information Systems (springer.com)

Abstract

Geomagnetic activities have a crucial impact on Earth, which can affect spacecraft and electrical power grids. Geospace scientists use a geomagnetic index, called the Kp index, to describe the overall level of geomagnetic activity. This index is an important indicator of disturbances in the Earth’s magnetic field and is used by the U.S. Space Weather Prediction Center as an alert and warning service for users who may be affected by the disturbances. Another commonly used index, called the ap index, is converted from the Kp index. Early and accurate prediction of the Kp and ap indices is essential for preparedness and disaster risk management. In this paper, we present a deep learning framework, named GNet, to perform short-term forecasting of the Kp and ap indices. Specifically, GNet takes as input time series of solar wind parameters’ values, provided by NASA’s Space Science Data Coordinated Archive, and predicts as output the Kp and ap indices respectively at time point t+w hours for a given time point t where w ranges from 1 to 9. GNet combines transformer encoder blocks with Bayesian inference, which is capable of quantifying both aleatoric uncertainty (data uncertainty) and epistemic uncertainty (model uncertainty) in making predictions. Experimental results show that GNet outperforms closely related machine learning methods in terms of the root mean square error and R-squared score. Furthermore, GNet can provide both data and model uncertainty quantification results, which the existing methods cannot offer. To our knowledge, this is the first time that Bayesian transformers have been used for geomagnetic activity prediction.

A Transformer-Based Framework for Geomagnetic Activity Prediction

Foundations of Intelligent Systems (ISMIS 2022)

A Transformer-Based Framework for Geomagnetic Activity Prediction | SpringerLink

Abstract

Geomagnetic activities have a crucial impact on Earth, which can affect spacecraft and electrical power grids. Geospace scientists use a geomagnetic index, called the Kp index, to describe the overall level of geomagnetic activity. This index is an important indicator of disturbances in the Earth’s magnetic field and is used by the U.S. Space Weather Prediction Center as an alert and warning service for users who may be affected by the disturbances. Early and accurate prediction of the Kp index is essential for preparedness and disaster risk management. In this paper, we present a novel deep learning method, named KpNet, to perform short-term, 1–9 hour ahead, forecasting of the Kp index based on the solar wind parameters taken from the NASA Space Science Data Coordinated Archive. KpNet combines transformer encoder blocks with Bayesian inference, which is capable of quantifying both aleatoric uncertainty (data uncertainty) and epistemic uncertainty (model uncertainty) when making Kp predictions. Experimental results show that KpNet outperforms closely related machine learning methods in terms of the root mean square error and R-squared score. Furthermore, KpNet can provide both data and model uncertainty quantification results, which the existing methods cannot offer. To our knowledge, this is the first time that Bayesian transformers have been used for Kp prediction.

Use of Classification Algorithms to Predict the Grade of Geomagnetic Disturbance

Advances in Neural Computation, Machine Learning, and Cognitive Research VI(NEUROINFORMATICS 2022)

Use of Classification Algorithms to Predict the Grade of Geomagnetic Disturbance | SpringerLink

Abstract

This paper presents different approaches for predicting the grade of geomagnetic Kp index using machine learning algorithms. The Kp index is considered to be an indicator of the energy input from the solar wind into the Earth’s magnetosphere. In this study, a wide range of machine learning algorithms were tested for the purpose of classifying Kp index grade, such as gradient boosting models, linear models, and neural networks. The main challenge of this classification task is a strong class imbalance, due to the fact that extreme values of Kp index are rarely observed. To overcome the issue, the SMOTE technique for minority classes oversampling was utilized. It is shown that SMOTE improves quality of the classification at far horizons. We also test time-series cross-validation for hyperparameters optimization and show that it does not improve the quality. All the models are scored against an out-of-sample test set to assess their quality and compare the results. Finally, we highlight the directions of further research based on the results obtained in this study.

Forecasting Geomagnetic Storm Disturbances and Their Uncertainties Using Deep Learning

Space Weather, 13 November 2023

Forecasting Geomagnetic Storm Disturbances and Their Uncertainties Using Deep Learning - Conde - 2023 - Space Weather - Wiley Online Library

Abstract

Severe space weather produced by disturbed conditions on the Sun results in harmful effects both for humans in space and in high-latitude flights, and for technological systems such as spacecraft or communications. Also, geomagnetically induced currents (GICs) flowing on long ground-based conductors, such as power networks, potentially threaten critical infrastructures on Earth. The first step in developing an alarm system against GICs is to forecast them. This is a challenging task given the highly non-linear dependencies of the response of the magnetosphere to these perturbations. In the last few years, modern machine-learning models have shown to be very good at predicting magnetic activity indices. However, such complex models are on the one hand difficult to tune, and on the other hand they are known to bring along potentially large prediction uncertainties which are generally difficult to estimate. In this work we aim at predicting the SYM-H index characterizing geomagnetic storms multiple-hour ahead, using public interplanetary magnetic field (IMF) data from the Sun-Earth L1 Lagrange point and SYM-H data. We implement a type of machine-learning model called long short-term memory (LSTM) network. Our scope is to estimate the prediction uncertainties coming from a deep-learning model in the context of forecasting the SYM-H index. These uncertainties will be essential to set reliable alarm thresholds. The resulting uncertainties turn out to be sizable at the critical stages of the geomagnetic storms. Our methodology includes as well an efficient optimization of important hyper-parameters of the LSTM network and robustness tests.

Prediction of the SYM-H Index Using a Bayesian Deep Learning Method With Uncertainty Quantification

Space Weather, 14 February 2024

Prediction of the SYM‐H Index Using a Bayesian Deep Learning Method With Uncertainty Quantification - Abduallah - 2024 - Space Weather - Wiley Online Library

Abstract

We propose a novel deep learning framework, named SYMHnet, which employs a graph neural network and a bidirectional long short-term memory network to cooperatively learn patterns from solar wind and interplanetary magnetic field parameters for short-term forecasts of the SYM-H index based on 1- and 5-min resolution data. SYMHnet takes, as input, the time series of the parameters' values provided by NASA's Space Science Data Coordinated Archive and predicts, as output, the SYM-H index value at time point t + w hours for a given time point t where w is 1 or 2. By incorporating Bayesian inference into the learning framework, SYMHnet can quantify both aleatoric (data) uncertainty and epistemic (model) uncertainty when predicting future SYM-H indices. Experimental results show that SYMHnet works well at quiet time and storm time, for both 1- and 5-min resolution data. The results also show that SYMHnet generally performs better than related machine learning methods. For example, SYMHnet achieves a forecast skill score (FSS) of 0.343 compared to the FSS of 0.074 of a recent gradient boosting machine (GBM) method when predicting SYM-H indices (1 hr in advance) in a large storm (SYM-H = −393 nT) using 5-min resolution data. When predicting the SYM-H indices (2 hr in advance) in the large storm, SYMHnet achieves an FSS of 0.553 compared to the FSS of 0.087 of the GBM method. In addition, SYMHnet can provide results for both data and model uncertainty quantification, whereas the related methods cannot.

Figure 2

The SYMHnet framework: (a) the overall architecture of SYMHnet, (b) the architecture of its GNN component, and (c) the architecture of its BiLSTM component. The input parameter graph is for illustration; the actual graph in the implementation is a fully connected graph (FCG). B = IMF magnitude (B), By = By component, Bz = Bz component, EF = Electric field, N_p = Proton density, P_dyn = Flow pressure, and V = Flow speed.

A Time-efficient, Data-driven Modeling Approach for Predicting the Geomagnetic Impact of Coronal Mass Ejections

The Astrophysical Journal Letters, 950:L11 (11pp), 2023 June 20

A Time-efficient, Data-driven Modeling Approach for Predicting the Geomagnetic Impact of Coronal Mass Ejections (iop.org)

Abstract

To understand the global-scale physical processes behind coronal mass ejection (CME)–driven geomagnetic storms and predict their intensity as a space weather forecasting measure, we develop an interplanetary CME flux rope–magnetosphere interaction module using 3D magnetohydrodynamics. The simulations adequately describe CME-forced dynamics of the magnetosphere including the imposed magnetotail torsion. These interactions also result in induced currents, which are used to calculate the geomagnetic perturbation. Through a suitable calibration, we estimate a proxy of geoeffectiveness—the Storm Intensity index (STORMI)—that compares well with the Dst/ SYM-H index. Simulated impacts of two contrasting CMEs quantified by the STORMI index exhibit a high linear correlation with the corresponding Dst and SYM-H indices. Our approach is relatively simple, has fewer parameters to be fine-tuned, and is time efficient compared to complex fluid-kinetic methods. Furthermore, we demonstrate that flux rope erosion does not significantly affect our results. Thus our method has the potential to significantly extend the time window for predictability—an outstanding challenge in geospace environment forecasting—if early predictions of near-Earth CME flux rope structures based on near-Sun observations are available as inputs. This study paves the way for early warnings based on operational predictions of CME-driven geomagnetic storms.

Figure 2. Simulated 3D view of the planetary magnetosphere from a viewpoint just above the ecliptic plane. The magnetospheric fields are depicted using colored lines to distinguish among the Earth’s polar open field lines (orange), the closed inner magnetospheric lines (cyan), and IMF (green). The strong event (event 1) that occurred on 2003 November 20 is shown in panel (a), and the moderate event (event 2) of 2006 April 14 is shown in panel (b). The white arrows in both images denote the rotation axis of Earth. The magnitude of the current density (J) is plotted on the equatorial planes to demonstrate the current formation around the Earth right after the passage of the leading halves of the flux ropes for event 1 (top) and event 2 (bottom). The yellow arrows designate the Sun-side (along the x-axis).

-------------------------------

Model Evaluation Guidelines for Geomagnetic Index Predictions

Space Weather, December 2018 

Model Evaluation Guidelines for Geomagnetic Index Predictions - Liemohn - 2018 - Space Weather - Wiley Online Library

Abstract

Geomagnetic indices are convenient quantities that distill the complicated physics of some region or aspect of near-Earth space into a single parameter. Most of the best-known indices are calculated from ground-based magnetometer data sets, such as Dst, SYM-H, Kp, AE, AL, and PC. Many models have been created that predict the values of these indices, often using solar wind measurements upstream from Earth as the input variables to the calculation. This document reviews the current state of models that predict geomagnetic indices and the methods used to assess their ability to reproduce the target index time series. These existing methods are synthesized into a baseline collection of metrics for benchmarking a new or updated geomagnetic index prediction model. These methods fall into two categories: (1) fit performance metrics such as root-mean-square error and mean absolute error that are applied to a time series comparison of model output and observations and (2) event detection performance metrics such as Heidke Skill Score and probability of detection that are derived from a contingency table that compares model and observation values exceeding (or not) a threshold value. A few examples of codes being used with this set of metrics are presented, and other aspects of metrics assessment best practices, limitations, and uncertainties are discussed, including several caveats to consider when using geomagnetic indices.