A Brief History of Permutation Methods

  • Chapter
  • First Online:
A Primer of Permutation Statistical Methods

Abstract

This chapter provides a brief history and overview of the early beginnings and subsequent development of permutation statistical methods, organized by decades from the 1920s to the present.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (Canada)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (Canada)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (Canada)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info
Hardcover Book
USD 119.99
Price excludes VAT (Canada)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Authors’ note: Statistics on the Table and The Seven Pillars of Statistical Wisdom by Stephen Stigler are comprehensible and lucid texts written for readers with limited statistical training.

  2. 2.

    Authors’ note: The Rise of Statistical Thinking by Theodore Porter and The Lady Tasting Tea by David Salsburg are well-written and appropriate for readers with limited statistical training.

  3. 3.

    Jerzy Spława-Neyman later shortened his name to Jerzy Neyman, emigrated to the USA, and assumed a position at the University of California, Berkeley, in 1938. Neyman founded the Department of Statistics at UC Berkeley in 1955.

  4. 4.

    The original symbol for the variance-ratio test statistic used by Fisher was z. In 1934 George Snedecor published tabled values in a small monograph for Fisher’s z statistic and rechristened the test statistic F [89].

  5. 5.

    The exact probability test for 2×2 contingency tables was independently developed by Frank Yates in 1934 [104] and Joseph Irwin in 1935 [52].

  6. 6.

    Also see an article on this topic by E.J. Burr in 1960 [19].

  7. 7.

    Relatively speaking, there were no “high-speed” computers in 1960. Since Robertson worked at the Sandia National Laboratory in Albuquerque, New Mexico, he had access to a Royal McBee LGP-30. The Royal McBee Librascope General Purpose (LGP) computer was considered a desktop computer, even though it weighed 740 pounds. The LGP-30 contained a 4096-word magnetic drum, and had a clock rate of only 120 kHz.

  8. 8.

    Authors’ note: After 40-plus years, this 1973 article by Feinstein remains as perhaps the clearest non-mathematical introduction to permutation methods ever written and should be consulted by all researchers new to the field of permutation methods.

  9. 9.

    The journal Applied Statistics is also known as Journal of the Royal Statistical Society, Series C.

  10. 10.

    Eugene Edgington, a dominating force in the promotion of permutation statistical methods for 50 years, passed away on September 2, 2013, at the age of 89.

  11. 11.

    StatXact is a statistical software package for analyzing data using exact statistics. It is marketed by Cytel Inc. [4].

  12. 12.

    Technically, R was first developed in 1995, but only came into wide use in the period 2000–2009.

References

  1. Agresti, A.: Exact inference for categorical data: recent advances and continuing controversies. Stat. Med. 20, 2709–2722 (2001)

    Article  Google Scholar 

  2. Agresti, A., Wackerly, D.: Some exact conditional tests of independence for R × C cross-classification tables. Psychometrika 42, 111–125 (1977)

    Article  Google Scholar 

  3. Amonenk, I.S., Robinson, J.: A new permutation test statistic for complete block designs. Ann. Stat. 43, 90–101 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  4. Author: StatXact for Windows. Cytel Software, Cambridge (2000)

    Google Scholar 

  5. Berry, K.J., Mielke, P.W.: Goodman and Kruskal’s tau-b statistic: a nonasymptotic test of significance. Sociol. Method. Res. 13, 543–550 (1985)

    Article  Google Scholar 

  6. Berry, K.J., Mielke, P.W.: A generalization of Cohen’s kappa agreement measure to interval measurement and multiple raters. Educ. Psychol. Meas. 48, 921–933 (1988)

    Article  Google Scholar 

  7. Berry, K.J., Mielke, P.W.: A Monte Carlo investigation of the Fisher Z transformation for normal and nonnormal distributions. Psychol. Rep. 87, 1101–1114 (2000)

    Article  Google Scholar 

  8. Berry, K.J., Kvamme, K.L., Mielke, P.W.: A permutation technique for the spatial analysis of artifacts into classes. Am. Antiquity 45, 55–59 (1980)

    Article  Google Scholar 

  9. Berry, K.J., Johnston, J.E., Mielke, P.W.: Permutation methods. Comput. Stat. 3, 527–542 (2011)

    Article  Google Scholar 

  10. Berry, K.J., Johnston, J.E., Mielke, P.W.: A Chronicle of Permutation Statistical Methods: 1920–2000 and Beyond. Springer, Cham (2014)

    Book  MATH  Google Scholar 

  11. Berry, K.J., Mielke, P.W., Johnston, J.E.: Permutation Statistical Methods: An Integrated Approach. Springer, Cham (2016)

    Book  MATH  Google Scholar 

  12. Berry, K.J., Johnston, J.E., Mielke, P.W.: The Measurement of Association: A Permutation Statistical Approach. Springer, Cham (2018)

    Book  MATH  Google Scholar 

  13. Berry, K.J., Johnston, J.E., Mielke, P.W., Johnston, L.A.: Permutation methods. Part II. Comput. Stat. 10, 1–31 (2018)

    MathSciNet  MATH  Google Scholar 

  14. Biondini, M.E., Mielke, P.W., Berry, K.J.: Data-dependent permutation techniques for the analysis of ecological data. Vegetatio 75, 161–168 (1988). [The name of the journal was changed to Plant Ecology in 1997]

    Google Scholar 

  15. Box, G.E.P., Andersen, S.L.: Permutation theory in the derivation of robust criteria and the study of departures from assumption (with discussion). J. R. Stat. Soc. B Methodol. 17, 1–34 (1955)

    MATH  Google Scholar 

  16. Boyett, J.M.: Algorithm 144: R × C tables with given row and column totals. J. R. Stat. Soc. C Appl. 28, 329–332 (1979)

    MATH  Google Scholar 

  17. Bradley, J.V.: Distribution-free Statistical Tests. Prentice–Hall, Englewood Cliffs (1968)

    Google Scholar 

  18. Brooks, E.B.: Frank Wilcoxon, 2 Sept 1892 – 18 Nov 1965. Tales of Statisticians. http://www.umass.edu/wsp/statistics/tales/wilcoxon.html. Accessed 1 Apr 2012

  19. Burr, E.J.: The distribution of Kendall’s score S for a pair of tied rankings. Biometrika 47, 151–171 (1960)

    Article  MathSciNet  MATH  Google Scholar 

  20. Cade, B.S., Richards, J.D.: A permutation test for quantile regression. J. Agric. Biol. Environ. Stat. 11, 106–126 (2006)

    Article  Google Scholar 

  21. Curran-Everett, D.: Explorations in statistics: permutation methods. Adv. Physiol. Educ. 36, 181–187 (2012)

    Article  Google Scholar 

  22. Dabrowska, D.M., Speed, T.P.: On the application of probability theory to agricultural experiments. Essay on principles. Section 9. Stat. Sci. 5, 465–480 (1990). [This is a translation by Dabrowska and Speed of an article originally published in Polish in Annals of Agricultural Sciences by Jerzy Spława–Neyman in 1923]

    Google Scholar 

  23. David, F.N.: Studies in the history of probability and statistics: I. Dicing and gaming. Biometrika 42, 1–15 (1955)

    MATH  Google Scholar 

  24. David, F.N.: Games, Gods, and Gambling: The Origin and History of Probability and Statistical Ideas from the Earliest Times to the Newtonian Era. Hafner, New York (1962)

    MATH  Google Scholar 

  25. Dwass, M.: Modified randomization tests for nonparametric hypotheses. Ann. Math. Stat. 28, 181–187 (1957)

    Article  MathSciNet  MATH  Google Scholar 

  26. Eden, T., Yates, F.: On the validity of Fisher’s z test when applied to an actual example of non-normal data. J. Agric. Sci. 23, 6–17 (1933)

    Article  Google Scholar 

  27. Edgington, E.S.: Randomization tests. J. Psychol. 57, 445–449 (1964)

    Article  Google Scholar 

  28. Edgington, E.S.: Statistical inference and nonrandom samples. Psychol. Bull. 66, 485–487 (1966)

    Article  Google Scholar 

  29. Edgington, E.S.: Statistical inference from N = 1 experiments. J. Psychol. 65, 195–199 (1967)

    Article  Google Scholar 

  30. Edgington, E.S.: Approximate randomization tests. J. Psychol. 72, 143–149 (1969)

    Article  Google Scholar 

  31. Edgington, E.S.: Statistical Inference: The Distribution-free Approach. McGraw–Hill, New York (1969)

    Google Scholar 

  32. Edgington, E.S.: Randomization Tests. Marcel Dekker, New York (1980)

    MATH  Google Scholar 

  33. Feinstein, A.R.: Clinical Biostatistics XXIII: The role of randomization in sampling, testing, allocation, and credulous idolatry (Part 2). Clin. Pharmacol. Ther. 14, 898–915 (1973)

    Article  Google Scholar 

  34. Festinger, L.: The significance of differences between means without reference to the frequency distribution function. Psychometrika 11, 97–105 (1946)

    Article  MathSciNet  MATH  Google Scholar 

  35. Fisher, R.A.: Statistical Methods for Research Workers. Oliver and Boyd, Edinburgh (1925)

    MATH  Google Scholar 

  36. Fisher, R.A.: The Design of Experiments. Oliver and Boyd, Edinburgh (1935)

    Google Scholar 

  37. Fisher, R.A.: The logic of inductive inference (with discussion). J. R. Stat. Soc. 98, 39–82 (1935)

    Article  MATH  Google Scholar 

  38. Freeman, G.H., Halton, J.H.: Note on an exact treatment of contingency, goodness of fit and other problems of significance. Biometrika 38, 141–149 (1951)

    Article  MathSciNet  MATH  Google Scholar 

  39. Frick, R.W.: Interpreting statistical testing: process and propensity, not population and random sampling. Behav. Res. Methods Instrum. Comput. 30, 527–535 (1998)

    Article  Google Scholar 

  40. Gani, J. (ed.): The Making of Statisticians. Springer, New York (1982)

    MATH  Google Scholar 

  41. Geary, R.C.: Some properties of correlation and regression in a limited universe. Metron 7, 83–119 (1927)

    MATH  Google Scholar 

  42. Gebhard, J., Schmitz, N.: Permutation tests — a revival?! I. Optimum properties. Stat. Pap. 39, 75–85 (1998)

    MathSciNet  MATH  Google Scholar 

  43. Gebhard, J., Schmitz, N.: Permutation tests — a revival?! II. An efficient algorithm for computing the critical region. Stat. Pap. 39, 87–96 (1998)

    MathSciNet  MATH  Google Scholar 

  44. Ghent, A.W.: A method for exact testing of 2 × 2, 2 × 3, 3 × 3, and other contingency tables, employing binomial coefficients. Am. Midl. Nat. 88, 15–27 (1972)

    Article  Google Scholar 

  45. Gianchristofaro, R.A., Pesarin, F., Salmaso, L.: Permutation Anderson–Darling type and moment-based test statistics for univariate ordered categorical data. Commun. Stat. Simul. Comput. 36, 139–150 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  46. Green, B.F.: A practical interactive program for randomization tests of location. Am. Stat. 31, 37–39 (1977)

    Google Scholar 

  47. Hald, A.: History of Probability and Statistics and Their Applications Before 1750. Wiley, New York (1990)

    Book  MATH  Google Scholar 

  48. Hald, A.: A History of Mathematical Statistics from 1750 to 1930. Wiley, New York (1998)

    MATH  Google Scholar 

  49. Hill, I.D., Pike, M.C.: Algorithm 4: TWOBYTWO. Comput. Bull. 9, 56–63 (1965). [Reprinted in Comput J 22, 87–88 (1979). Addenda in Comput. J. 9, 212 (1966) and 9, 416 (1967)]

    Google Scholar 

  50. Hollander, M.: A conversation with Ralph A. Bradley. Stat. Sci. 16, 75–100 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  51. Hotelling, H., Pabst, M.R.: Rank correlation and tests of significance involving no assumption of normality. Ann. Math. Stat. 7, 29–43 (1936)

    Article  MATH  Google Scholar 

  52. Irwin, J.O.: Tests of significance for differences between percentages based on small numbers. Metron 12, 83–94 (1935)

    MATH  Google Scholar 

  53. Janssen, A., Pauls, T.: How do bootstrap and permutation tests work? Ann. Stat. 31, 768–806 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  54. Kempthorne, O., Doerfler, T.E.: The behaviour of some significance tests under experimental randomization. Biometrika 56, 231–248 (1969)

    Article  MathSciNet  MATH  Google Scholar 

  55. Kendall, M.G.: A new measure of rank correlation. Biometrika 30, 81–93 (1938)

    Article  MATH  Google Scholar 

  56. Kendall, M.G.: Rank Correlation Methods. Griffin, London (1948)

    MATH  Google Scholar 

  57. Kendall, M.G., Babington Smith, B.: The problem of m rankings. Ann. Math. Stat. 10, 275–287 (1939)

    Article  MathSciNet  MATH  Google Scholar 

  58. Kendall, M.G., Plackett, R.L. (eds.): Studies in the History of Statistics and Probability, vol. II. Griffin, London (1977)

    MATH  Google Scholar 

  59. Kennedy, P.E., Cade, B.S.: Randomization tests for multiple regression. Commun. Stat. Simul. Comput. 25, 923–936 (1996)

    Article  MATH  Google Scholar 

  60. Kruskal, W.H., Wallis, W.A.: Use of ranks in one-criterion variance analysis. J. Am. Stat. Assoc. 47, 583–621 (1952). [Erratum: J. Am. Stat. Assoc. 48, 907–911 (1953)]

    Article  MATH  Google Scholar 

  61. Lehmann, E.L., Stein, C.M.: On the theory of some non-parametric hypotheses. Ann. Math. Stat. 20, 28–45 (1949)

    Article  MATH  Google Scholar 

  62. Leslie, P.H.: A simple method of calculating the exact probability in 2 × 2 contingency tables with small marginal totals. Biometrika 42, 522–523 (1955)

    MathSciNet  MATH  Google Scholar 

  63. Ludbrook, J., Dudley, H.A.F.: Why permutation tests are superior to t and F tests in biomedical research. Am. Stat. 52, 127–132 (1998)

    Google Scholar 

  64. MacKenzie, D.: Statistics in Britain, 1865–1930: The Social Construction of Scientific Knowledge. Edinburgh University Press, Edinburgh (1981)

    Google Scholar 

  65. Mann, H.B., Whitney, D.R.: On a test of whether one of two random variables is stochastically larger than the other. Ann. Math. Stat. 18, 50–60 (1947)

    Article  MathSciNet  MATH  Google Scholar 

  66. May, R.B., Hunter, M.A.: Some advantages of permutation tests. Can. Psychol. 34, 401–407 (1993)

    Article  Google Scholar 

  67. Mehta, C.R., Patel, N.R.: A network algorithm for performing Fisher’s exact test in r × c contingency tables. J. Am. Stat. Assoc. 78, 427–434 (1983)

    MathSciNet  MATH  Google Scholar 

  68. Mielke, P.W.: Meteorological applications of permutation techniques based on distance functions. In: Krishnaiah, P.R., Sen, P.K. (eds.) Handbook of Statistics, vol. IV, pp. 813–830. North-Holland, Amsterdam (1984)

    Google Scholar 

  69. Mielke, P.W., Berry, K.J.: Cumulant methods for analyzing independence of r-way contingency tables and goodness-of-fit frequency data. Biometrika 75, 790–793 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  70. Mielke, P.W., Berry, K.J.: Permutation tests for common locations among samples with unequal variances. J. Educ. Behav. Stat. 19, 217–236 (1994)

    Article  Google Scholar 

  71. Mielke, P.W., Berry, K.J.: Permutation Methods: A Distance Function Approach. Springer, New York (2001)

    Book  MATH  Google Scholar 

  72. Mielke, P.W., Berry, K.J.: Multivariate multiple regression analyses: a permutation method for linear models. Psychol. Rep. 91, 3–9 (2002)

    Article  Google Scholar 

  73. Mielke, P.W., Berry, K.J., Johnson, E.S.: Multi-response permutation procedures for a priori classifications. Commun. Stat. Theor. Methods 5, 1409–1424 (1976)

    Article  MATH  Google Scholar 

  74. Mielke, P.W., Berry, K.J., Neidt, C.O.: A permutation test for multivariate matched-pairs analyses: comparisons with Hotelling’s multivariate matched-pairs T 2 test. Psychol. Rep. 78, 1003–1008 (1996)

    Article  Google Scholar 

  75. Mielke, P.W., Berry, K.J., Johnston, J.E.: Resampling programs for multiway contingency tables with fixed marginal frequency totals. Psychol. Rep. 101, 18–24 (2007)

    Article  Google Scholar 

  76. Mielke, P.W., Berry, K.J., Johnston, J.E.: Resampling probability values for weighted kappa with multiple raters. Psychol. Rep. 102, 606–613 (2008)

    Article  Google Scholar 

  77. Mielke, P.W., Long, M.A., Berry, K.J., Johnston, J.E.: g-treatment ridit analysis: resampling permutation methods. Stat. Methodol. 6, 223–229 (2009)

    Article  MathSciNet  Google Scholar 

  78. Patefield, W.M.: Algorithm 159: an efficient method of generating random r × c tables with given row and column totals. J. R. Stat. Soc. C Appl. 30, 91–97 (1981)

    MATH  Google Scholar 

  79. Pearson, E.S., Kendall, M.G. (eds.): Studies in the History of Statistics and Probability, vol. I. Griffin, London (1970)

    Google Scholar 

  80. Pesarin, F., Salmaso, L.: Exact permutation tests for unreplicated factorials. Appl. Stoch. Model. Bus. 18, 287–299 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  81. Pitman, E.J.G.: Significance tests which may be applied to samples from any populations. Suppl. J. R. Stat. Soc. 4, 119–130 (1937)

    Article  MATH  Google Scholar 

  82. Pitman, E.J.G.: Significance tests which may be applied to samples from any populations: II. The correlation coefficient test. Suppl. J. R. Stat. Soc. 4, 225–232 (1937)

    Article  MATH  Google Scholar 

  83. Pitman, E.J.G.: Significance tests which may be applied to samples from any populations: III. The analysis of variance test. Biometrika 29, 322–335 (1938)

    MATH  Google Scholar 

  84. Porter, T.M.: The Rise of Statistical Thinking, 1820–1900. Princeton University Press, Princeton (1986)

    Google Scholar 

  85. Robertson, W.H.: Programming Fisher’s exact method of comparing two percentages. Technometrics 2, 103–107 (1960)

    Article  MathSciNet  MATH  Google Scholar 

  86. Sakoda, J.M., Cohen, B.H.: Exact probabilities for contingency tables using binomial coefficients. Psychometrika 22, 83–86 (1957)

    Article  MathSciNet  MATH  Google Scholar 

  87. Salsburg, D.S.: The Lady Tasting Tea: How Statistics Revolutionized Science in the Twentieth Century. Holt, New York (2001)

    MATH  Google Scholar 

  88. Scheffé, H.: Statistical inference in the non-parametric case. Ann. Math. Stat. 14, 305–332 (1943)

    Article  MathSciNet  MATH  Google Scholar 

  89. Snedecor, G.W.: Calculation and Interpretation of Analysis of Variance and Covariance. Collegiate Press, Ames (1934)

    Book  Google Scholar 

  90. Spława–Neyman, J.: Próba uzasadnienia zastosowań rachunku prawdopodobieństwa do doświadczeń polowych (On the application of probability theory to agricultural experiments. Essay on principles. Section 9). Rocz Nauk Rolnicz (Ann. Agric. Sci.) 10, 1–51 (1923). [Translated from the original Polish by D. M. Dabrowska and T. P. Speed and published in Statist. Sci. 5, 465–472 (1990)]

    Google Scholar 

  91. Stelmach, J.: On permutation tests with several test statistics to compare two populations. In: Vojackova, H. (ed.) Mathematical Methods in Economics 2013, pp. 850–855. Coll Polytechnics Jihlava, Jihlava (2013)

    Google Scholar 

  92. Stigler, S.M.: American Contributions to Mathematical Statistics in the Nineteenth Century, vol. I. Arno Press, New York (1980)

    MATH  Google Scholar 

  93. Stigler, S.M.: American Contributions to Mathematical Statistics in the Nineteenth Century, vol. II. Arno Press, New York (1980)

    MATH  Google Scholar 

  94. Stigler, S.M.: The History of Statistics: The Measurement of Uncertainty Before 1900. Harvard University Press, Cambridge (1986)

    MATH  Google Scholar 

  95. Stigler, S.M.: Statistics on the Table: The History of Statistical Concepts and Methods. Harvard University Press, Cambridge (1999)

    MATH  Google Scholar 

  96. Stigler, S.M.: The Seven Pillars of Statistical Wisdom. Harvard University Press, Cambridge (2016)

    Book  MATH  Google Scholar 

  97. Tedin, O.: The influence of systematic plot arrangements upon the estimate of error in field experiments. J. Agric. Sci. 21, 191–208 (1931)

    Article  Google Scholar 

  98. Thakur, A.K., Berry, K.J., Mielke, P.W.: A FORTRAN program for testing trend and homogeneity in proportions. Comput. Prog. Biomed. 19, 229–233 (1985)

    Article  Google Scholar 

  99. Tukey, J.W.: Randomization and re-randomization: the wave of the past in the future. In: Statistics in the Pharmaceutical Industry: Past, Present and Future. Philadelphia Chapter of the American Statistical Association (1988). [Presented at a Symposium in Honor of Joseph L. Ciminera held in June 1988 at Philadelphia, Pennsylvania]

    Google Scholar 

  100. Umlauft, M., Konietschke, F., Pauly, M.: Rank-based permutation approaches for non-parametric factorial designs. Brit. J. Math. Stat. Psychol. 70, 368–390 (2017)

    Article  MATH  Google Scholar 

  101. White, C.: The use of ranks in a test of significance for comparing two treatments. Biometrics 8, 33–41 (1952)

    Article  MATH  Google Scholar 

  102. Wilcoxon, F.: Individual comparisons by ranking methods. Biometrics Bull. 1, 80–83 (1945)

    Article  Google Scholar 

  103. Wilks, S.S.: Order statistics. Bull. Am. Math. Soc. 54, 6–50 (1948)

    Article  MathSciNet  MATH  Google Scholar 

  104. Yates, F.: Contingency tables involving small numbers and the χ 2 test. Suppl. J. R. Stat. Soc. 1, 217–235 (1934)

    Article  MATH  Google Scholar 

  105. Yeo, I.K.: An algorithm for computing the exact distribution of the Wilcoxon signed-rank statistic. J. Korean Stat. Soc. 46, 328–338 (2017)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Berry, K.J., Johnston, J.E., Mielke, P.W. (2019). A Brief History of Permutation Methods. In: A Primer of Permutation Statistical Methods. Springer, Cham. https://doi.org/10.1007/978-3-030-20933-9_2

Download citation

Publish with us

Policies and ethics

Navigation