Contents
1 Introduction 1
1.1 Overview of Data Science in Air Quality Monitoring 2
1.1.1 Importance of Air Quality Monitoring 2
1.1.2 The Role of Data Science in Environmental Monitoring 6
1.1.3 Characteristics and Challenges of Air Quality Data 10
1.1.4 Current Application of Data Science and Technology in Air Quality Monitoring 13
1.2 Key Problems Data Science in Air Quality Monitoring 16
1.2.1 Data Processing 16
1.2.2 Data Decomposition 21
1.2.3 Data Identification 25
1.2.4 Data Clustering 27
1.2.5 Data Forecasting 33
1.2.6 Data Interpolation 36
1.3 Scope of the Book 42
References 44
2 Data Preprocessing in Air Quality Monitoring 49
2.1 Introduction 49
2.2 Data Acquisition 51
2.3 Characteristic Analysis of Air Quality Data 52
2.3.1 Temporal Characteristics 52
2.3.2 Spatial Characteristics 55
2.4 Missing Data Imputation of Air Quality Data 56
2.4.1 Missing Data Imputation Performance Evaluation 58
2.4.2 Univariate Missing Data Imputation Based on K-Nearest Neighbors 60
2.4.3 Multivariate Missing Data Imputation
Based on Self-Organizing Map 61
2.5 Outlier Detection of Air Quality Data 65
2.5.1 Outlier Detection Performance Evaluation 66
2.5.2 Outlier Detection Based on Unsupervised Isolation Forest 67
2.5.3 Outlier Detection Based on Hampel Filter 70
2.5.4 Outlier Detection Based on Deep Learning Forecasting 75
2.6 Preprocessing Performance Comparison 78
2.6.1 Performance Comparison of Missing Data Imputation 78
2.6.2 Performance Comparison of Outlier Detection 81
2.7 Conclusions 82
References 83
3 Data Decomposition in Air Quality Monitoring 85
3.1 Introduction 85
3.1.1 Application of Wavelet Decomposition in Air Quality Data Analysis 86
3.1.2 Application of Modal Decomposition in Air Quality Data Analysis 86
3.1.3 Deficiencies and Challenges of Existing Research 87
3.1.4 Temporal Resolution 87
3.1.5 Frequency Resolution 87
3.1.6 Boundary Effect 88
3.1.7 Noise Reduction Effect 88
3.2 Wavelet Decomposition of Air Quality Data 90
3.2.1 Time-Frequency Localization Characteristics 90
3.2.2 Multi-resolution Analysis 90
3.2.3 Strong Sparse Representation Capability 91
3.2.4 Discrete Wavelet Transform 92
3.3 Top Layer: Approximation Coefficients 97
3.4 Detail Coefficients 97
3.4.1 Reconstruction Error 98
3.4.2 Signal-to-Noise Ratio (SNR) 98
3.4.3 Correlation Coefficient 99
3.4.4 Various Wavelet Basis Functions 100
3.4.5 Continuous Wavelet Transform 104
3.5 Mode Decomposition of Air Quality Data 106
3.5.1 Empirical Mode Decomposition 106
3.5.2 Variations and Improvements of the Traditional EMD Method 110
3.6 Decomposition Performance Comparison 113
3.6.1 Decomposition Accuracy 113
3.6.2 Computational Complexity 114
3.6.3 Boundary Effect 115
3.7 Conclusions 116
References 116
4 Data Identification in Air Quality Monitoring 119
4.1 Introduction 119
4.1.1 The Importance of Data Identification in Air Quality Monitoring 120
4.1.2 Methods for Data Identification in Air
Quality Monitoring 121
4.2 Data Acquisition 122
4.3 Feature Selection of Air Quality Data 123
4.3.1 Feature Selection Performance Evaluation 123
4.3.2 Filter Methods 125
4.3.3 Wrapper Methods 126
4.4 Forward Selection 128
4.5 Backward Elimination 128
4.6 Recursive Feature Elimination (RFE) 129
4.6.1 Modeling Step 129
4.6.2 Embedded Methods 131
4.7 Feature Extraction of Air Quality Data 131
4.7.1 Feature Extraction Performance Evaluation 131
4.7.2 Statistical Feature Extraction 132
4.7.3 Time-Frequency Analysis 134
4.8 Identification Performance Comparison 137
4.8.1 Performance Comparison of Feature Selection 137
4.8.2 Performance Comparison of Feature Extraction 140
4.9 Conclusions 143
References 144
5 Data Preprocessing in Air Quality Monitoring 147
5.1 Introduction 147
5.2 Data Acquisition 148
5.3 Temporal Clustering of Air Quality Data 151
5.3.1 Definition and Role of Temporal Clustering 151
5.3.2 DBSCAN Temporal Clustering 152
5.3.3 AE-DBSCAN Temporal Clustering 154
5.3.4 CAE-DBSCAN Temporal Clustering 157
5.4 Spatial Clustering of Air Quality Data 159
5.4.1 K-Means Clustering 159
5.4.2 GMM 160
5.4.3 GAE -Kmeans 162
5.4.4 Modeling Step 164
5.5 Clustering Performance Comparison 165
5.5.1 Evaluation with Silhouette Score 165
5.5.2 Evaluation with Base Model 166
5.5.3 Comparison of Spatial Clustering 168
5.6 Conclusions 171
References 171
6 Data Forecasting in Air Quality Monitoring 173
6.1 Introduction 173
6.2 Data Acquisition 176
6.3 Deterministic Forecasting of Air Quality Data 178
6.3.1 Extreme Learning Machine 178
6.3.2 Gated Recurrent Unit 180
6.3.3 Bidirectional Long Short-term Memory 182
6.3.4 Deep Extreme Learning Machine 184
6.3.5 Transformer 185
6.4 Probabilistic Forecasting of Air Quality Data 187
6.4.1 Bayesian Neural Networks 187
6.4.2 Quantile Recurrent Neural Networks 189
6.5 Forecasting Performance Comparison 190
6.5.1 Evaluation Indicator 190
6.5.2 Deterministic Forecasting Performance 192
6.5.3 Probabilistic Forecasting Performance 199
6.6 Conclusions 207
References 208
7 Data Interpolation in Air Quality Monitoring 211
7.1 Introduction 211
7.2 Data Acquisition 212
7.3 Temporal Interpolation of Air Quality Data 214
7.3.1 Linear Interpolation 215
7.3.2 Polynomial Interpolation 216
7.3.3 Spline Interpolation 218
7.3.4 Interpolation Based on Statistical Model 219
7.4 Spatial Interpolation of Air Quality Data 220
7.4.1 Nearest Neighbor Interpolation 221
7.4.2 Inverse Distance Weighted Interpolation 223
7.4.3 Kriging Interpolation 224
7.4.4 Radial Basis Function Interpolation 225
7.5 Interpolation Performance Comparison 228
7.5.1 Comparison Between Temporal Interpolations 228
7.5.2 Comparison Between Spatial Interpolations 231
7.5.3 Comparison Between Temporal Interpolation and Spatial Interpolation 236
7.6 Conclusions 237
References 238
List of Figures
Table 1.1 Overview of data processing methods 20
Table 1.2 Overview of data decomposition methods 24
Table 1.3 Comparison of different clustering methods 32
Table 1.4 Overview of data forecasting methods 37
Table 1.5 Overview of data interpolation methods 41
Table 2.1 The maximum cliques and their correlation indexes 57
Table 2.2 The percentages of the missing gaps in series #1 59
Table 2.3 The missing data imputation performance of the KNN and SOM for 25% missing ratio 80
Table 2.4 Performance of the outlier detection methods and without outlier detection 82
Table 3.1 The performance of each level of different wavelet classification 104
Table 3.2 Performance of different data decomposition methods 114
Table 4.1 General guidelines for the interpretation of Pearson
correlation coefficient 125
Table 4.2 The feature selection performance of the filter method and wrapper method 138
Table 4.3 The feature extraction performance of the statistical analysis and DWT 141
Table 5.1 Station classification center 160
Table 5.2 Silhouette score for different models 166
Table 5.3 Table of evaluation indicators 167
Table 5.4 Comparison of evaluation indicators 170
Table 6.1 A summary of the reviewed deterministic forecasting literature 175
Table 6.2 A summary of the reviewed probabilistic forecasting literature 176
Table 6.3 The statistical descriptions of four PM2.5 data sets 177
Table 6.4 The calculation formulas of deterministic forecasting evaluation
indicators 191
Table 6.5 The calculation formulas of probabilistic forecasting evaluation indicators 192
Table 6.6 The error evaluation of the forecasting results in Changsha 197
Table 6.7 The error evaluation of the forecasting results in Seoul 198
Table 6.8 Error evaluation of forecasting results in Changsha 203
Table 6.9 Error evaluation of forecasting results in Seoul 206
Table 6.10 Prediction interval coverage and width metrics for Changsha 206
Table 6.11 Prediction interval coverage and width metrics for Seoul 207
Table 7.1 The feature selection performance of different temporal interpolations 230
Table 7.2 The feature selection performance of different spatial interpolations 235
Table 7.3 The comparison between temporal interpolation and spatial interpolation 237