SpringerLink
Log in

Fine-Tuned Parallel Piecewise Sequential Confidence Interval and Point Estimation Strategies for the Mean of a Normal Population: Big Data Context

  • Chapter
  • First Online:
Artificial Intelligence, Big Data and Data Science in Statistics

  • 673 Accesses

Abstract

In this paper, we provide some new perspectives on sequential experimental designs for statistical inference in the context of big data. A fine-tuned parallel piecewise sequential procedure is developed for estimating the mean of a normal population having an unknown variance. With the help of such fine-tuning, asymptotic unbiasedness of the terminal sample size can be achieved along with the added operational efficiency as a result of utilizing the parallel processing or distributed computing. Theory and methodology will go hand-in-hand followed by illustrations from large-scale data analyses based on simulated data as well as real data from a health study.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
EUR 29.95
Price includes VAT (Thailand)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
EUR 149.79
Price includes VAT (Thailand)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
EUR 179.99
Price excludes VAT (Thailand)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info
Hardcover Book
EUR 179.99
Price excludes VAT (Thailand)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Aerts, M., Geertsema, J.C.: Bounded length confidence intervals in nonparametric regression. Seq. Anal. 9, 171–192 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  2. Anscombe, F.J.: Large-sample theory of sequential estimation. Proc. Camb. Philol. Soc. 48, 600–607 (1952)

    Article  MathSciNet  MATH  Google Scholar 

  3. Anscombe, F.J.: Sequential estimation. J. R. Stat. Soc. Ser. B 15, 1–29 (1953)

    MathSciNet  MATH  Google Scholar 

  4. Aoshima, M., Mukhopadhyay, N., Kobayashi, Y.: Two-stage procedures for estimating the difference of means when the sampling cost is different. Seq. Anal. 30, 160–171 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  5. Chow, Y.S., Robbins, H.: On the asymptotic theory of fixed width confidence intervals for the mean. Ann. Math. Stat. 36, 457–462 (1965)

    Article  MathSciNet  MATH  Google Scholar 

  6. Dantzig, G.B.: On the non-existence of tests of student’s hypothesis having power functions independent of σ. Ann. Math. Stat. 11, 186–192 (1940)

    Google Scholar 

  7. Geertsema, J.C.: Sequential confidence intervals based on rank tests. Ann. Math. Stat. 41, 1016–1026 (1970)

    Article  MathSciNet  MATH  Google Scholar 

  8. Ghosh, M., Mukhopadhyay, N.: On two fundamental problems of sequential estimation. Sankhyā Ser. B 38, 203–218 (1976)

    MathSciNet  MATH  Google Scholar 

  9. Ghosh, M., Mukhopadhyay, N.: Consistency and asymptotic efficiency of two-stage and sequential procedures. Sankhyā Ser. A 43, 220–227 (1981)

    MathSciNet  MATH  Google Scholar 

  10. Ghosh, M., Mukhopadhyay, N., Sen, P.K.: Sequential Estimation. Wiley, New York (1997)

    Book  MATH  Google Scholar 

  11. Ghosh, B.K., Sen, P.K.: Handbook of Sequential Analysis, edited volume. Dekker, New York (1991)

    Google Scholar 

  12. Henke, N., Bughin, J., Chui, M., Manyika, J., Saleh, T., Wiseman, B., Sethupathy, G.: The age of analytics: competing in a data-driven world. McKinsey Global Institute report (2016). https://www.mckinsey.com/business-functions/mckinsey-analytics/our-insights/the-age-of-analytics-competing-in-a-data-driven-world

  13. Jurečkovā, J., Sen, P.K.: Robust Statistical Procedures. Wiley, New York (1996)

    MATH  Google Scholar 

  14. Lai, T.L., Siegmund, D.: A nonlinear renewal theory with applications to sequential analysis I. Ann. Stat. 5, 946–954 (1977)

    Article  MathSciNet  MATH  Google Scholar 

  15. Lai, T.L., Siegmund, D.: A nonlinear renewal theory with applications to sequential analysis II. Ann. Stat. 7, 60–76 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  16. Lombard, F., Swanepoel, J.W.H.: On finite and infinite confidence sequences. South African Stat. J. 12, 1–24 (1978)

    MathSciNet  MATH  Google Scholar 

  17. Manyika, J., Chui, M., Brown, B., Bughin, J., Dobbs, R., Roxburgh, C., Byers, A.H.: Big data: the next frontier for innovation, competition, and productivity. McKinsey Global Institute report (2011). https://www.mckinsey.com/business-functions/digital-mckinsey/our-insights/big-data-the-next-frontier-for-innovation

  18. Mukhopadhyay, N.: Sequential estimation of location parameters in exponential distributions. Calcutta Statist. Assoc. Bull. 23, 85–93 (1974)

    Article  MathSciNet  MATH  Google Scholar 

  19. Mukhopadhyay, N.: A consistent and asymptotically efficient two-stage procedure to construct fixed-width confidence intervals for the mean. Metrika 27, 281–284 (1980)

    Article  MathSciNet  MATH  Google Scholar 

  20. Mukhopadhyay, N.: A study of the asymptotic regret while estimating the location of an exponential distribution. Calcutta Statist. Assoc. Bull. 31, 207–213 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  21. Mukhopadhyay, N.: Sequential estimation problems for negative exponential populations. Commun. Stat. Theory Methods Ser. A 17, 2471–2506 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  22. Mukhopadhyay, N.: Two-stage and multi-stage estimation. In: Balakrishnan, N., Basu, A.P. (eds.) The Exponential Distribution: Theory, Methods and Application, Chapter 26, pp. 429–452. Gordon and Breach, Amsterdam (1995)

    Google Scholar 

  23. Mukhopadhyay, N.: Probability and Statistical Inference. Dekker, New York (2000)

    MATH  Google Scholar 

  24. Mukhopadhyay, N., Datta, S.: On fine-tuning a purely sequential procedure and the associated second-order properties. Sankhyā Ser. A 57, 100–117 (1995)

    MathSciNet  MATH  Google Scholar 

  25. Mukhopadhyay, N., de Silva, B.M.: Sequential Methods and Their Applications. Chapman & Hall/CRC, Boca Raton (2009)

    Google Scholar 

  26. Mukhopadhyay, N., Sen, P.K.: Replicated piecewise stop** numbers and sequential analysis. Seq. Anal. 12, 179–197 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  27. Mukhopadhyay, N., Solanky, T.K.S.: Multistage Selection and Ranking Procedures. Dekker, New York (1994)

    MATH  Google Scholar 

  28. Mukhopadhyay, N., Vik, G.: Asymptotic results for stop** times based on U-statistics. Seq. Anal. 4, 83–110 (1985)

    Article  MathSciNet  MATH  Google Scholar 

  29. Mukhopadhyay, N., Zhang, C.: EDA on the asymptotic normality of the standardized sequential stop** times, part-I: parametric models. Seq. Anal. 37, 342–374 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  30. Mukhopadhyay, N., Zhang, C.: EDA on the asymptotic normality of the standardized sequential stop** times, part-II: distribution-free models. Seq. Anal. 39, 367–398 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  31. Ray, W.D.: Sequential confidence intervals for the mean of a normal population with unknown variance. J. R. Stat. Soc. Ser. B 19, 133–143 (1957)

    MathSciNet  MATH  Google Scholar 

  32. Robbins, H.: Sequential estimation of the mean of a normal population. In: Cramér volume, H., Grenander, U. (eds.) Probability and Statistics, pp. 235–245. Almquist and Wiksell, Uppsala (1959)

    Google Scholar 

  33. Sen, P.K.: Sequential Nonparametrics. Wiley, New York (1981)

    MATH  Google Scholar 

  34. Siegmund, D.: Sequential Analysis: Tests and Confidence Intervals. Springer, New York (1985)

    Book  MATH  Google Scholar 

  35. Starr, N.: The performance of a sequential procedure for fixed-width interval estimate. Ann. Math. Stat. 36, 36–50 (1966)

    Article  MathSciNet  MATH  Google Scholar 

  36. Starr, N.: On the asymptotic efficiency of a sequential procedure for estimating the mean. Ann. Math. Stat. 37, 1173–1185 (1966)

    Article  MathSciNet  MATH  Google Scholar 

  37. Starr, N., Woodroofe, M.: Remarks on sequential point estimation. Proc. Natl. Acad. Sci. USA 63, 285–288 (1969)

    Article  MathSciNet  MATH  Google Scholar 

  38. Stein, C.: A Two sample test for a linear hypothesis whose power is independent of the variance. Ann. Math. Stat. 16, 243–258 (1945)

    Article  MathSciNet  MATH  Google Scholar 

  39. Stein, C.: Some problems in sequential estimation (abstract). Econometrica 17, 77–78 (1949)

    Google Scholar 

  40. Steland, A., Chang, Y.-T.: High-confidence nonparametric fixed-width uncertainty intervals and applications to projected high-dimensional data and common mean estimation. Seq. Anal. 40, 97–124 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  41. Swanepoel, J.W.H., van Wyk, J.W.J.: Fixed width confidence intervals for the location parameter of an exponential distribution. Commun. Stat. Theory Methods 11, 1279–1289 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  42. Woodroofe, M.: Second order approximations for sequential point and interval estimation. Ann. Stat. 5, 984–995 (1977)

    Article  MathSciNet  MATH  Google Scholar 

  43. Woodroofe, M.: Nonlinear Renewal Theory in Sequential Analysis, CBMS #39. SIAM, Philadelphia (1982)

    Book  MATH  Google Scholar 

Download references

Acknowledgements

We remain indebted to Professor Ansgar Steland and the referees for critically evaluating this invited contribution. Their feedback has improved an original version of our work. We take this opportunity to thank them.

Author information

Authors and Affiliations

  1. Department of Statistics, University of Connecticut, Storrs, CT, USA

    Nitis Mukhopadhyay & Chen Zhang

Authors
  1. Nitis Mukhopadhyay
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chen Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nitis Mukhopadhyay .

Editor information

Editors and Affiliations

  1. Institute of Statistics and AI Center, RWTH Aachen University, Aachen, Germany

    Ansgar Steland

  2. Grado Department of Industrial and Systems Engineering, Virginia Polytechnic Institute and State University, Blacksburg, VA, USA

    Kwok-Leung Tsui

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Mukhopadhyay, N., Zhang, C. (2022). Fine-Tuned Parallel Piecewise Sequential Confidence Interval and Point Estimation Strategies for the Mean of a Normal Population: Big Data Context. In: Steland, A., Tsui, KL. (eds) Artificial Intelligence, Big Data and Data Science in Statistics. Springer, Cham. https://doi.org/10.1007/978-3-031-07155-3_3

Download citation

Publish with us

Policies and ethics

Search

Navigation