Abstract
One of the most powerful algorithms for obtaining maximum likelihood estimates for many incomplete-data problems is the EM algorithm. However, when the parameters satisfy a set of nonlinear restrictions, It is difficult to apply the EM algorithm directly. In this paper, we propose an asymptotic maximum likelihood estimation procedure under a set of nonlinear inequalities restrictions on the parameters, in which the EM algorithm can be used. Essentially this kind of estimation problem is a stochastic optimization problem in the M-step. We make use of methods in stochastic optimization to overcome the difficulty caused by nonlinearity in the given constraints.
Similar content being viewed by others
References
A P Dempster, N Laird, D B Rubin. Maximum likelihood estimation from incomplete data via the EM algorithm (with discussion), Journal of the American Statistical Association B, 1977, 39(1), 1–38.
D A D’Esopo. A convex programming procedure, Naval Research Logistics Quarterly, 1959, 6(1): 33–42.
F Eicker. Asymptotic normality and consistency of the least squares estimators for families of linear regressions, Annals of Mathematical Statistics, 1963, 34(2): 447–456.
C G Hildreth. A quadratic programming procedure, Naval Research Logistics Quarterly, 1957, 4(1): 79–85.
D K Kim, J M G Taylor. The restricted EM Algorithm for maximum likelihood estimation under linear restrictions on the parameters, Journal of the American Statistical Association, 1995, 90(430): 708–716.
C K Liew. Inequality constrained least-squares estimation, Journal of the American Statistical Association, 1976, 71(355): 746–751.
R J A. Little, D Rubin. Statistical Analysis with Missing Data, Wiley, New York, 1987.
R J A Little, D Rubin. The ECME algorithm: a simple extension of EM and ECM with faster monotone convergence, Biometrika, 1994, 81(4): 633–648.
X S Liu, J D Wang. Testing for increasing convex order in several populations, Annals of Institute Statistical Mathematics, 2003, 55(1): 121–136.
X L Meng, D B Rubin. Maximum likelihood estimation via the ECM algorithm: a general framework, Biometrika, 1993, 80(2): 267–278.
N Nagaraj, W Fuller. Estimation of the parameters of linear time series models subject to nonlinear regressions, Annals of Statistics, 1991, 19(3): 1143–1154.
H Nyquist. Restricted estimation of generalized linear models, Applied statistics, 1991, 40(1): 133–141.
R Prakasa. Asymptotic theory of statistical inference. Wiley, New york, 1987.
R Ranjana, B Huanga, A Fatehiab. Robust Gaussian process modeling using EM algorithm, Journal of Process Control, 2016, 42(6): 125–136.
N Z Shi, S R Zheng, J H Guo. The restricted EM algorithm under inequality restrictions on the parameters, Journal of Multivariate Analysis, 2005, 92(1): 53–76.
J D Wang. The asymptotics of least-squares estimators for constrained nonlinear regression, Annals of Statistics, 1996, 24(3): 1316–1326.
D Wei, X K S Peter. EM algorithm in Gaussian copula with missing data, Computational Statistics and Data Analysis, 2016, 101(12): 1–11.
C F J Wu. on the convergence properties of the EM algorithm, Annals of Statistics, 1969, 11(1): 95–103.
Author information
Authors and Affiliations
Corresponding author
Additional information
Supported by Teaching reform project of Zhengzhou University of Science and Technology (KFCZ201909), National Foundation for Cultivating Scientific Research Projects of Zhengzhou Institute of Technology (GJJKTPY2018K4), Henan Big Data Double Base of Zhengzhou Institute of Technology (20174101546503022265) and the Key Scientific Research Foundation of Education Bureau of Henan Province (20B110020).
Rights and permissions
About this article
Cite this article
Shen, Qx., Miao, P. & Liang, Ys. The EM algorithm for ML Estimators under nonlinear inequalities restrictions on the parameters. Appl. Math. J. Chin. Univ. 34, 393–402 (2019). https://doi.org/10.1007/s11766-019-3484-9
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11766-019-3484-9