Abstract
The existence of outliers can seriously influence the analysis of variational data assimilation. Quality control allows us to effectively eliminate or absorb these outliers to produce better analysis fields. In particular, variational quality control (VarQC) can process gray zone outliers and is thus broadly used in variational data assimilation systems. In this study, governing equations are derived for two VarQC algorithms that utilize different contaminated Gaussian distributions (CGDs): Gaussian plus flat distribution and Huber norm distribution. As such, these VarQC algorithms can handle outliers that have non-Gaussian innovations. Then, these VarQC algorithms are implemented in the Global/Regional Assimilation and PrEdiction System (GRAPES) model-level three-dimensional variational data assimilation (m3DVAR) system. Tests using artificial observations indicate that the VarQC method using the Huber distribution has stronger robustness for including outliers to improve posterior analysis than the VarQC method using the Gaussian plus flat distribution. Furthermore, real observation experiments show that the distribution of observation analysis weights conform well with theory, indicating that the application of VarQC is effective in the GRAPES m3DVAR system. Subsequent case study and long-period data assimilation experiments show that the spatial distribution and amplitude of the observation analysis weights are related to the analysis increments of the mass field (geopotential height and temperature). Compared to the control experiment, VarQC experiments have noticeably better posterior mass fields. Finally, the VarQC method using the Huber distribution is superior to the VarQC method using the Gaussian plus flat distribution, especially at the middle and lower levels.
摘 要
离群值的存在会严重影响变分资料同化分析的准确性. 质量控制能够使同化系统有效地剔除或吸收离群值观测, 从而获得更佳的分析场, 尤其是变分质量控制 (VarQC) 可以有效处理灰色区域的离群值. 因此, VarQC在变分资料同化系统中得到了广泛的应用. 本文利用 “高斯分布+均匀分布” 和Huber norm分布的污染**态分布模型分别推导了两种VarQC算法. 这两种VarQC算法可以质量控制具有非高斯新息向量分布的离群值观测. 随后, 两种VarQC算法被应用于 GRAPES模式层的三维变分 (m3DVAR) 资料同化系统. 理想观测同化试验的检验表明, Huber分布的VarQC方法比“高斯分布+均匀分布”的VarQC方法具有更**的稳健性. 实际观测同化试验结果表明, 两种VarQC试验的观测分析权重分布与各自理论的权重曲线变化一致, 证明了基于GRAPES m3DVAR系统建立的VarQC方案应用的**确性. 随后的个例和长期同化试验分析表明, 观测分析权重大小的变化幅度和分布与质量场 (位势高度和温度) 的分析增量变化具有一致的响应关系. 与控制试验相比, VarQC试验具有更准确的后验分析质量场, 证明了Huber分布的VarQC算法优于“高斯分布+均匀分布”的VarQC算法, 尤其在对流层中下层.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
An, X. Q., S. X. Zhai, M. **, S. L. Gong, and Y. Wang, 2016: Development of an adjoint model of GRAPES-CUACE and its application in tracking influential haze source areas in north China. Geosci. Model Dev., 9, 2153–2165, https://doi.org/10.5194/gmd-9-2153-2016.
Anderson, E., and H. Järvinen, 1999: Variational quality control. Quart. J. Roy. Meteor. Soc., 125, 697–722, https://doi.org/10.1002/qj.49712555416.
Auligné, T., 2014: Multivariate minimum residual method for cloud retrieval. Part II: Real observations experiments. Mon. Wea. Rev., 142, 4399–4415, https://doi.org/10.1175/MWR-D-13-00173.1.
Bao, X. H., and F. Q. Zhang, 2013: Evaluation of NCEP-CFSR, NCEP-NCAR, ERA-Interim, and ERA-40 reanalysis datasets against independent sounding observations over the Tibetan Plateau. J. Climate, 26, 206–214, https://doi.org/10.1175/JCLI-D-12-00056.1.
Bauer, P., and Coauthors, 2011: Satellite cloud and precipitation assimilation at operational NWP centres. Quart. J. Roy. Meteor. Soc., 137, 1934–1951, https://doi.org/10.1002/qj.905.
Cardinali, C., L. Isaksen, and E. Andersson, 2003: Use and impact of automated aircraft data in a global 4DVAR data assimilation system. Mon. Wea. Rev., 131, 1865–1877, https://doi.org/10.1175//2569.1.
Chan, M. Y., J. L. Anderson, and X. C. Chen, 2020: An efficient bi-Gaussian ensemble Kalman filter for satellite infrared radiance data assimilation. Mon. Wea. Rev., 148, 5087–5104, https://doi.org/10.1175/MWR-D-20-0142.1.
Chen, D. H., and Coauthors, 2008: New generation of multi-scale NWP system (GRAPES): General scientific design. Chin. Sci. Bull., 53, 3433–3445, https://doi.org/10.1007/s11434-008-0494-z.
Collins, W. G., and L. S. Gandin, 1990: Comprehensive hydrostatic quality control at the National Meteorological Center. Mon. Wea. Rev., 118, 2752–2767, https://doi.org/10.1175/1520-0493(1990)118<2752:CHQCAT>2.0.CO;2.
Courtier, P., and Coauthors, 1998: The ECMWF implementation of three-dimensional variational assimilation (3D-Var). I: Formulation. Quart. J. Roy. Meteor. Soc., 124, 1783–1807, https://doi.org/10.1002/qj.49712455002.
Dee, D. P., and S. Uppala, 2009: Variational bias correction of satellite radiance data in the ERA-Interim reanalysis. Quart. J. Roy. Meteor. Soc., 135, 1830–1841, https://doi.org/10.1002/qj.493.
Duan, B. H., W. M. Zhang, X. F. Yang, H. J. Dai, and Y. Yu, 2017: Assimilation of typhoon wind field retrieved from scatterometer and SAR based on the Huber norm quality control. Remote Sens., 9, 987, https://doi.org/10.3390/rs9100987.
Fiebrich, C. A., C. R. Morgan, A. G. McCombs, P. K. Hall, and R. A. McPherson, 2010: Quality assurance procedures for mesoscale meteorological data. J. Atmos. Oceanic Technol., 27, 1565–1582, https://doi.org/10.1175/2010jtecha1433.1.
Fowler, A., and P. J. Van Leeuwen, 2013: Observation impact in data assimilation: The effect of non-Gaussian observation error. Tellus A, 65, 20035, https://doi.org/10.3402/tellusa.v65i0.20035.
Gandin, L. S., 1988: Complex quality control of meteorological observations. Mon. Wea. Rev., 116, 1137–1156, https://doi.org/10.1175/1520-0493(1988)116<1137:CQCOMO>2.0.CO;2.
Gandin, L. S., L. L. Morone, and W. G. Collins, 1993: Two years of operational comprehensive hydrostatic quality control at the National Meteorological Center. Wea. Forecasting, 8, 57–72, https://doi.org/10.1175/1520-0434(1993)008<0057:TYOOCH>2.0.CO;2.
Geer, A. J., and P. Bauer, 2011: Observation errors in all-sky data assimilation. Quart. J. Roy. Meteor. Soc., 137, 2024–2037, https://doi.org/10.1002/qj.830.
Guitton, A., and W. W. Symes, 2003: Robust inversion of seismic data using the Huber norm. Geophysics, 68, 1310–1319, https://doi.org/10.1190/1.1598124.
Hampel, F. R., 1977: Rejection rules and robust estimates of location: An analysis of some Monte Carlo results. Preprints, Transactions of the Seventh Prague Conference on Information Theory, Statistical Decision Functions, and Random Processes, Dordrecht, Hingham, MA, E. M. o. Statisticians, 187–194.
Hampel, F. R., 2001: Robust statistics: A brief introduction and overview. Preprints, Research report/Seminar für Statistik, Eidgenössische Technische Hochschule, Zurich, Switzerland, E. T. Hochschule, 1–5, https://doi.org/10.3929/ethz-a-004158597.
Harnisch, F., M. Weissmann, and Á. Periáñez, 2016: Error model for the assimilation of cloud-affected infrared satellite observations in an ensemble data assimilation system. Quart. J. Roy. Meteor. Soc., 142, 1797–1808, https://doi.org/10.1002/qj.2776.
He, D. X., Z. M. Zhou, Z. P. Kang, and L. Liu, 2019a: Numerical studies on forecast error correction of GRAPES model with variational approach. Adv. Meteor., 2019, 2856289, https://doi.org/10.1155/2019/2856289.
He, J., and Coauthors, 2019b: Development and evaluation of an ensemble-based data assimilation system for regional reanalysis over the Tibetan Plateau and surrounding regions. J. Adv. Model. Earth Sy., 11, 2503–2522, https://doi.org/10.1029/2019MS001665.
Honda, T., and Coauthors, 2018: Assimilating all-sky Himawari-8 satellite infrared radiances: A case of Typhoon Soudelor (2015). Mon. Wea. Rev., 146, 213–229, https://doi.org/10.1175/MWR-D-16-0357.1.
Houtekamer, P. L., and H. L. Mitchell, 1998: Data assimilation using an ensemble Kalman filter technique. Mon. Wea. Rev., 126, 796–811, https://doi.org/10.1175/1520-0493(1998)126<0796:DAUAEK>2.0.CO;2.
Huber, P. J., 1972: The 1972 wald lecture robust statistics: A review. Ann. Math. Statist., 43, 1041–1067, https://doi.org/10.1214/aoms/1177692459.
Huber, P. J., 2011: Robust statistics. International Encyclopedia of Statistical Science, M. Lovric, Ed., Springer-Verlag, 1248–1251, https://doi.org/10.1007/978-3-642-04898-2_594.
Ingleby, N. B., and A. C. Lorenc, 1993: Bayesian quality control using multivariate normal distributions. Quart. J. Roy. Meteor. Soc., 119, 1195–1225, https://doi.org/10.1002/qj.49711951316.
Järvinen, H., and P. Undén, 1997: Observation Screening and background Quality Control in the ECMWF 3D-Var Data Assimilation System. ECMWF Technical Memoranda No. 236, 33 pp, https://doi.org/10.21957/lyd3q81.
Kalnay, E., 2003: Atmospheric Modeling, Data Assimilation and Predictability. Cambridge University Press, 368 pp.
Lahoz, W. A., and P. Schneider, 2014: Data assimilation: Making sense of earth observation. Front. Environ. Sci., 2, 16, https://doi.org/10.3389/fenvs.2014.00016.
Legrand, R., Y. Michel, and T. Montmerle, 2016: Diagnosing non-Gaussianity of forecast and analysis errors in a convective-scale model. Nonlin. Processes Geophys., 2, 1–12, https://doi.org/10.5194/npg-23-1-2016.
Lorenc, A. C., 1984: Analysis methods for the quality control of observations. Preprints, Workshop on the Use and Quality Control of Meteorological Observations, ECMWF, 397–428.
Lorenc, A. C., 1986: Analysis methods for numerical weather prediction. Quart. J. Roy. Meteor. Soc., 112, 1177–1194, https://doi.org/10.1002/qj.49711247414.
Ma, X. L., Z. R. Zhuang, J. S. Xue, and W. S. Lu, 2009: Development of 3-D variational data assimilation system for the nonhydrostatic numerical weather prediction model-GRAPES. Acta Meteor. Sinica, 67, 50–60, https://doi.org/10.3322/j.issn:0577-6619.2009.01.006. (in Chinese with English abstract)
Ma, X. L., J. He, B. Y. Zhou, L. L. Li, Y. X. Ji, and H. Guo, 2017: Effect of variational quality control of Non-Gaussian distribution observation error on heavy rainfall prediction. Trans. Atmos. Sci., 40, 170–180, https://doi.org/10.13878/j.cnki.dqkxxb.20150911001. (in Chinese with English abstract)
Minamide, M., and F. Q. Zhang, 2017: Adaptive observation error inflation for assimilating all-sky satellite radiance. Mon. Wea. Rev., 145, 1063–1081, https://doi.org/10.1175/MWR-D-16-0257.1.
Pires, C. A., O. Talagrand, and M. Bocquet, 2010: Diagnosis and impacts of non-Gaussianity of innovations in data assimilation. Physica D, 239, 1701–1717, https://doi.org/10.1016/j.physd.2010.05.006.
Purser, R. J., 1984: A new approach to optimal assimilation of meteorological data by iterative Bayesian analysis. Preprints, Conf. on Weather Forecasting and Analysis, Clearwater Beach, FL, 102–105.
Rabier, F., H. Järvinen, E. Klinker, J. F. Mahfouf, and A. Simmons, 2000: The ECMWF operational implementation of four-dimensional variational assimilation. I: Experimental results with simplified physics. Quart. J. Roy. Meteor. Soc., 126, 1143–1170, https://doi.org/10.1002/qj.49712656415.
Rawlins, F., S. P. Ballard, K. J. Bovis, A. M. Clayton, D. Li, G. W. Inverarity, A. C. Lorenc, and T. J. Payne, 2007: The Met Office global four-dimensional variational data assimilation scheme. Quart. J. Roy. Meteor. Soc., 133, 347–362, https://doi.org/10.1002/qj.32.
Storto, A., 2016: Variational quality control of hydrographic profile data with non-Gaussian errors for global ocean variational data assimilation systems. Ocean Model., 104, 226–241, https://doi.org/10.1016/j.ocemod.2016.06.011.
Tavolato, C., and L. Isaksen, 2015: On the use of a Huber norm for observation quality control in the ECMWF 4D-Var. Quart. J. Roy. Meteor. Soc., 141, 1514–1527, https://doi.org/10.1002/qj.2440.
Tukey, J. W., 1960: A survey of sampling from contaminated distributions. Contributions to Probability and Statistics: Essays in Honor of Harold Hotelling, I. Olkin, Ed., Stanford University Press, 448–485.
Vickers, D., and L. Mahrt, 1997: Quality control and flux sampling problems for tower and aircraft data. J. Atmos. Oceanic Technol., 14, 512–526, https://doi.org/10.1175/1520-0426(1997)014<0512:QCAFSP>2.0.CO;2.
Wang, A. H., and X. B. Zeng, 2012: Evaluation of multireanalysis products with in situ observations over the Tibetan Plateau. J. Geophys. Res.-Atmos., 117, D05102, https://doi.org/10.1029/2011jd016553.
Wang, L. L., D. H. Chen, and H. J. Bao, 2016: The improved Noah land surface model based on storage capacity curve and Muskingum method and application in GRAPES model. Atmos. Sci. Lett., 17, 190–198, https://doi.org/10.1022/asl.642.
Xu, J. W., W. C. Zhang, Z. Y. Zheng, M. Y. Jiao, and J. Chen, 2012: Early flood warning for Linyi watershed by the GRAPES/XXT model using TIGGE data. Acta Meteor. Sinica, 26, 103–111, https://doi.org/10.1007/s13351-012-0110-7.
Yang, Y. X., 1991: Robust Bayesian estimation. Bulletin Géodésique, 65, 145–150, https://doi.org/10.1007/BF00806343.
Zhang, R. H., and X. S. Shen, 2008: On the development of the GRAPES—A new generation of the national operational NWP system in China. Chin. Sci. Bull., 53, 3429–3432, https://doi.org/10.1007/s11434-008-0462-7.
Zhu, J. J., 1996: Robustness and the robust estimate. J. Geodesy, 70, 586–590, https://doi.org/10.1007/BF00867867.
Zhu, J. J., and Z. Q. Zeng, 1999: The theory of surveying adjustment under contaminated error model. Acta Geodaet. Cartograph. Sinica, 28, 215–220, https://doi.org/10.3321/j.issn:1001-1595.1999.03.006. (in Chinese with English abstract)
Acknowledgments
The authors thank Yinghui LU for his helpful advice and grammar correction. Jie HE is supported by the China Scholarship Council. This work is primarily sponsored by the National Key R&D Program of China (Grant No. 2018YFC1506702 and Grant No. 2017YFC1502000). We acknowledge the High Performance Computing Center of Nan**g University of Information Science & Technology (NUIST) for their support of this work. The datasets in this paper are archived and accessible on the supercomputer of NUIST.
Author information
Authors and Affiliations
Corresponding author
Additional information
Article Highlights
• The VarQC method using the Huber distribution (Huber-VarQC) is implemented in the GRAPES m3DVAR system.
• Huber-VarQC reveals strong robustness against outliers.
• Huber-VarQC is superior to the VarQC method using the Gaussian plus flat distribution (Flat-VarQC) in the improvement of the mass field in the GRAPES m3DVAR system.
Rights and permissions
About this article
Cite this article
He, J., Ma, X., Ge, X. et al. Variational Quality Control of Non-Gaussian Innovations in the GRAPES m3DVAR System: Mass Field Evaluation of Assimilation Experiments. Adv. Atmos. Sci. 38, 1510–1524 (2021). https://doi.org/10.1007/s00376-021-0336-3
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00376-021-0336-3