Abstract
Recently large quantities of data from many different field experiments have become available to facilitate the examination of various proposed models of atmospheric dispersion. However, these data sets are invariably corrupted by some form of random noise and, also, significant baseline drift is often recorded. Consequently, these problems require careful consideration and treatment before the data can be used in model testing. In many papers, the noise is simply treated by ‘thresholding’ but this is unacceptable since many valid readings are discarded. This paper examines the performance of two different noise removal methods that are more soundly based, both physically and mathematically. The first is a Wiener filter with certain modifications, and the second is a maximum entropy inversion technique. A comparison of the results of applying these methods is presented, with the emphasis on the practical treatment of the numerical and computational problems that arise. The problem of baseline drift is treated initially by applying a number of subjective fits. Subsequently the noise removal methods are applied. In general, it is found that the maximum entropy method performs better than the Wiener filter for the data sets examined here.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bryan, R. K., Bansal, M., Folkhand, W., Nave, C., and Marvin, D. A.: 1983, ‘Maximum-Entropy Calculations of the Electron Density at 4 Å Resolution ofPf1 Filamentous Bacteriophage’,Proc. Nat. Acad. Sci., USA 80, 4728–4731.
Bryan, R. K. and Skilling, J.: 1980, ‘Deconvolution by Maximum Entropy as Illustrated by Applications to the Jet of M87’,Mon. Not. R. Astr. Soc. 191, 69–79.
Chatwin, P. C. and Hajian, N. T.: 1990, ‘Concentration Fluctuations in Atmospheric Dispersion’,Final Technical Report on Agreement No. 2066/62, CBDE Porton Down, Wiltshire.
Chatwin, P. C. and Sullivan, P. J.: 1989, ‘The Intermittency Factor of Scalars in Turbulence’,Phys. Fluids A1(4), 761–763.
Davison, A. C. and Smith, R. L.: 1990, ‘Models for Exceedances over High Thresholds’,J. R. Statist. Soc. B 52(3), 393–442.
Dowling, D. R. and Dimotakis, P. E.: 1988,On Mixing and Structure of the Concentration Field of Turbulent Jets, AIAA/ASME/SIAM/APS. 1st National Fluid Dynamics Congress, Cincinnati, Part 2, pp. 982–988.
Flannery, B. P., Press, W. H., Teukolsky, S. A., and Vetterling, W. T.: 1990,Numerical Recipes, Cambridge University Press.
Frieden, B. R.: 1972, ‘Restoring with Maximum Likelihood and Maximum Entropy’,J. Opt. Soc. Am. 62, 511–518.
Frieden, B. R. and Swindell, W.: 1976, ‘Restored Pictures of Ganymede, Moon of Jupiter’,Science 191, 1237–1241.
Frieden, B. R. and Wells, D. C.: 1978, ‘Restoring with Maximum Entropy III. Poisson Sources and Backgrounds’,J. Opt. Soc. Amer. 68, 93–108.
Fox, D. G.: 1984, ‘Uncertainty in Air Quality Modelling’,Bull. Amer. Meteorol. Soc. 65, 27–36.
Gull, S. E. and Daniell, G. J.: 1978, ‘Image Reconstruction from Incomplete and Noisy Data,Nature 272, 686–690.
Gull, S. F. and Skilling, J.: 1983,IAU/URSI Symposium on Indirect Imaging, Sydney, Australia, Cambridge University Press, pp. 267–279.
Jaynes, E. T.: 1957, ‘Information Theory and Statistical Mechanics I and II’,Phys. Rev. 106, 620–630 and108, 171–190.
Kemp, M. C.: 1980,International Symposium on Radionuclide Imaging, IAEA-SM-247, Heidelberg, p. 128.
Mole, N.: 1989, ‘Estimating Statistics of Concentration Fluctuations from Measurements,Seventh Symposium on Turbulent Shear Flows, Stanford University, pp. 29.5.1–29.5.6.
Mole, N.: 1990, ‘Turbulent Dispersion and Statistics’,Environmetrics 1, 179–194.
Mylne, K. R. and Mason, P. J.: 1991, ‘Concentration Fluctuation Measurements in a Dispersing Plume at a Range of up to 1000 m’,Quart. J. R. Meteorol. Soc. 117, 177–206.
Narayan, R. and Nityananda, R.: 1986, ‘Maximum Entropy Image Reconstruction in Astronomy,Ann. Rev. Astron. Astrophys. 24, 127–170.
O'Sullivan, F.: 1986, ‘A Statistical Perspective on Ill-Posed Inverse Problems’,Statistical Science 1(4), 502–527.
Papoulis, A.: 1987,Signal Analysis, McGraw Hill, Inc.
Shannon, C. E.: 1948, ‘A Mathematical Theory of Communication I and II’,Bell System Tech. J. 27, 379–423 and 623–656.
Skilling, J. and Bryan, R. K.: 1984, ‘Maximum Entropy Image Reconstruction: General Algorithm’,Mon. Not. R. Astr. Soc. 211, 111–124.
Titterington, D. M.: 1985, ‘Common Structure of Smoothing Techniques in Statistics’,International Statistic Rev. 53(2), 141–170.
Tretter, S. A.: 1976,Introduction to Discrete-Time Signal Processing, John Wiley and Sons, Inc.
Venkatram, A.: 1982, ‘A Framework for Evaluating Air Quality Models’,Boundary-Layer Meteorol. 24, 371–385.
Weil, J. C., Sykes, R. I., and Venkatram, A.: 1992, ‘Evaluating Air-Quality Models: Review and Outlook’,J. Appl. Meteorol. 31, 1121–1145.
Wyngaard, J. C.: 1973, ‘On Surface Layer Turbulence’, in Duane A. Haugen (ed.),Workshop on Micrometeorology, AMS.
Yee, E., Kosteniuk, P. R., Chandler, G. M., Biltoft, C. A., and Bowers, J. F.: 1993, ‘Statistical Characteristics of Concentration Fluctuations in Dispersing Plumes in the Atmospheric Surface Layer’,Boundary-Layer Meteorol. 65, 69–109.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Lewis, D.M., Chatwin, P.C. The treatment of atmospheric dispersion data in the presence of noise and baseline drift. Boundary-Layer Meteorol 72, 53–85 (1995). https://doi.org/10.1007/BF00712390
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF00712390