Abstract
This chapter provides a brief history and overview of the early beginnings and subsequent development of permutation statistical methods, organized by decades from the 1920s to the present.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Authors’ note: Statistics on the Table and The Seven Pillars of Statistical Wisdom by Stephen Stigler are comprehensible and lucid texts written for readers with limited statistical training.
- 2.
Authors’ note: The Rise of Statistical Thinking by Theodore Porter and The Lady Tasting Tea by David Salsburg are well-written and appropriate for readers with limited statistical training.
- 3.
Jerzy Spława-Neyman later shortened his name to Jerzy Neyman, emigrated to the USA, and assumed a position at the University of California, Berkeley, in 1938. Neyman founded the Department of Statistics at UC Berkeley in 1955.
- 4.
The original symbol for the variance-ratio test statistic used by Fisher was z. In 1934 George Snedecor published tabled values in a small monograph for Fisher’s z statistic and rechristened the test statistic F [89].
- 5.
- 6.
Also see an article on this topic by E.J. Burr in 1960 [19].
- 7.
Relatively speaking, there were no “high-speed” computers in 1960. Since Robertson worked at the Sandia National Laboratory in Albuquerque, New Mexico, he had access to a Royal McBee LGP-30. The Royal McBee Librascope General Purpose (LGP) computer was considered a desktop computer, even though it weighed 740 pounds. The LGP-30 contained a 4096-word magnetic drum, and had a clock rate of only 120 kHz.
- 8.
Authors’ note: After 40-plus years, this 1973 article by Feinstein remains as perhaps the clearest non-mathematical introduction to permutation methods ever written and should be consulted by all researchers new to the field of permutation methods.
- 9.
The journal Applied Statistics is also known as Journal of the Royal Statistical Society, Series C.
- 10.
Eugene Edgington, a dominating force in the promotion of permutation statistical methods for 50 years, passed away on September 2, 2013, at the age of 89.
- 11.
StatXact is a statistical software package for analyzing data using exact statistics. It is marketed by Cytel Inc. [4].
- 12.
Technically, R was first developed in 1995, but only came into wide use in the period 2000–2009.
References
Agresti, A.: Exact inference for categorical data: recent advances and continuing controversies. Stat. Med. 20, 2709–2722 (2001)
Agresti, A., Wackerly, D.: Some exact conditional tests of independence for R × C cross-classification tables. Psychometrika 42, 111–125 (1977)
Amonenk, I.S., Robinson, J.: A new permutation test statistic for complete block designs. Ann. Stat. 43, 90–101 (2015)
Author: StatXact for Windows. Cytel Software, Cambridge (2000)
Berry, K.J., Mielke, P.W.: Goodman and Kruskal’s tau-b statistic: a nonasymptotic test of significance. Sociol. Method. Res. 13, 543–550 (1985)
Berry, K.J., Mielke, P.W.: A generalization of Cohen’s kappa agreement measure to interval measurement and multiple raters. Educ. Psychol. Meas. 48, 921–933 (1988)
Berry, K.J., Mielke, P.W.: A Monte Carlo investigation of the Fisher Z transformation for normal and nonnormal distributions. Psychol. Rep. 87, 1101–1114 (2000)
Berry, K.J., Kvamme, K.L., Mielke, P.W.: A permutation technique for the spatial analysis of artifacts into classes. Am. Antiquity 45, 55–59 (1980)
Berry, K.J., Johnston, J.E., Mielke, P.W.: Permutation methods. Comput. Stat. 3, 527–542 (2011)
Berry, K.J., Johnston, J.E., Mielke, P.W.: A Chronicle of Permutation Statistical Methods: 1920–2000 and Beyond. Springer, Cham (2014)
Berry, K.J., Mielke, P.W., Johnston, J.E.: Permutation Statistical Methods: An Integrated Approach. Springer, Cham (2016)
Berry, K.J., Johnston, J.E., Mielke, P.W.: The Measurement of Association: A Permutation Statistical Approach. Springer, Cham (2018)
Berry, K.J., Johnston, J.E., Mielke, P.W., Johnston, L.A.: Permutation methods. Part II. Comput. Stat. 10, 1–31 (2018)
Biondini, M.E., Mielke, P.W., Berry, K.J.: Data-dependent permutation techniques for the analysis of ecological data. Vegetatio 75, 161–168 (1988). [The name of the journal was changed to Plant Ecology in 1997]
Box, G.E.P., Andersen, S.L.: Permutation theory in the derivation of robust criteria and the study of departures from assumption (with discussion). J. R. Stat. Soc. B Methodol. 17, 1–34 (1955)
Boyett, J.M.: Algorithm 144: R × C tables with given row and column totals. J. R. Stat. Soc. C Appl. 28, 329–332 (1979)
Bradley, J.V.: Distribution-free Statistical Tests. Prentice–Hall, Englewood Cliffs (1968)
Brooks, E.B.: Frank Wilcoxon, 2 Sept 1892 – 18 Nov 1965. Tales of Statisticians. http://www.umass.edu/wsp/statistics/tales/wilcoxon.html. Accessed 1 Apr 2012
Burr, E.J.: The distribution of Kendall’s score S for a pair of tied rankings. Biometrika 47, 151–171 (1960)
Cade, B.S., Richards, J.D.: A permutation test for quantile regression. J. Agric. Biol. Environ. Stat. 11, 106–126 (2006)
Curran-Everett, D.: Explorations in statistics: permutation methods. Adv. Physiol. Educ. 36, 181–187 (2012)
Dabrowska, D.M., Speed, T.P.: On the application of probability theory to agricultural experiments. Essay on principles. Section 9. Stat. Sci. 5, 465–480 (1990). [This is a translation by Dabrowska and Speed of an article originally published in Polish in Annals of Agricultural Sciences by Jerzy Spława–Neyman in 1923]
David, F.N.: Studies in the history of probability and statistics: I. Dicing and gaming. Biometrika 42, 1–15 (1955)
David, F.N.: Games, Gods, and Gambling: The Origin and History of Probability and Statistical Ideas from the Earliest Times to the Newtonian Era. Hafner, New York (1962)
Dwass, M.: Modified randomization tests for nonparametric hypotheses. Ann. Math. Stat. 28, 181–187 (1957)
Eden, T., Yates, F.: On the validity of Fisher’s z test when applied to an actual example of non-normal data. J. Agric. Sci. 23, 6–17 (1933)
Edgington, E.S.: Randomization tests. J. Psychol. 57, 445–449 (1964)
Edgington, E.S.: Statistical inference and nonrandom samples. Psychol. Bull. 66, 485–487 (1966)
Edgington, E.S.: Statistical inference from N = 1 experiments. J. Psychol. 65, 195–199 (1967)
Edgington, E.S.: Approximate randomization tests. J. Psychol. 72, 143–149 (1969)
Edgington, E.S.: Statistical Inference: The Distribution-free Approach. McGraw–Hill, New York (1969)
Edgington, E.S.: Randomization Tests. Marcel Dekker, New York (1980)
Feinstein, A.R.: Clinical Biostatistics XXIII: The role of randomization in sampling, testing, allocation, and credulous idolatry (Part 2). Clin. Pharmacol. Ther. 14, 898–915 (1973)
Festinger, L.: The significance of differences between means without reference to the frequency distribution function. Psychometrika 11, 97–105 (1946)
Fisher, R.A.: Statistical Methods for Research Workers. Oliver and Boyd, Edinburgh (1925)
Fisher, R.A.: The Design of Experiments. Oliver and Boyd, Edinburgh (1935)
Fisher, R.A.: The logic of inductive inference (with discussion). J. R. Stat. Soc. 98, 39–82 (1935)
Freeman, G.H., Halton, J.H.: Note on an exact treatment of contingency, goodness of fit and other problems of significance. Biometrika 38, 141–149 (1951)
Frick, R.W.: Interpreting statistical testing: process and propensity, not population and random sampling. Behav. Res. Methods Instrum. Comput. 30, 527–535 (1998)
Gani, J. (ed.): The Making of Statisticians. Springer, New York (1982)
Geary, R.C.: Some properties of correlation and regression in a limited universe. Metron 7, 83–119 (1927)
Gebhard, J., Schmitz, N.: Permutation tests — a revival?! I. Optimum properties. Stat. Pap. 39, 75–85 (1998)
Gebhard, J., Schmitz, N.: Permutation tests — a revival?! II. An efficient algorithm for computing the critical region. Stat. Pap. 39, 87–96 (1998)
Ghent, A.W.: A method for exact testing of 2 × 2, 2 × 3, 3 × 3, and other contingency tables, employing binomial coefficients. Am. Midl. Nat. 88, 15–27 (1972)
Gianchristofaro, R.A., Pesarin, F., Salmaso, L.: Permutation Anderson–Darling type and moment-based test statistics for univariate ordered categorical data. Commun. Stat. Simul. Comput. 36, 139–150 (2007)
Green, B.F.: A practical interactive program for randomization tests of location. Am. Stat. 31, 37–39 (1977)
Hald, A.: History of Probability and Statistics and Their Applications Before 1750. Wiley, New York (1990)
Hald, A.: A History of Mathematical Statistics from 1750 to 1930. Wiley, New York (1998)
Hill, I.D., Pike, M.C.: Algorithm 4: TWOBYTWO. Comput. Bull. 9, 56–63 (1965). [Reprinted in Comput J 22, 87–88 (1979). Addenda in Comput. J. 9, 212 (1966) and 9, 416 (1967)]
Hollander, M.: A conversation with Ralph A. Bradley. Stat. Sci. 16, 75–100 (2000)
Hotelling, H., Pabst, M.R.: Rank correlation and tests of significance involving no assumption of normality. Ann. Math. Stat. 7, 29–43 (1936)
Irwin, J.O.: Tests of significance for differences between percentages based on small numbers. Metron 12, 83–94 (1935)
Janssen, A., Pauls, T.: How do bootstrap and permutation tests work? Ann. Stat. 31, 768–806 (2003)
Kempthorne, O., Doerfler, T.E.: The behaviour of some significance tests under experimental randomization. Biometrika 56, 231–248 (1969)
Kendall, M.G.: A new measure of rank correlation. Biometrika 30, 81–93 (1938)
Kendall, M.G.: Rank Correlation Methods. Griffin, London (1948)
Kendall, M.G., Babington Smith, B.: The problem of m rankings. Ann. Math. Stat. 10, 275–287 (1939)
Kendall, M.G., Plackett, R.L. (eds.): Studies in the History of Statistics and Probability, vol. II. Griffin, London (1977)
Kennedy, P.E., Cade, B.S.: Randomization tests for multiple regression. Commun. Stat. Simul. Comput. 25, 923–936 (1996)
Kruskal, W.H., Wallis, W.A.: Use of ranks in one-criterion variance analysis. J. Am. Stat. Assoc. 47, 583–621 (1952). [Erratum: J. Am. Stat. Assoc. 48, 907–911 (1953)]
Lehmann, E.L., Stein, C.M.: On the theory of some non-parametric hypotheses. Ann. Math. Stat. 20, 28–45 (1949)
Leslie, P.H.: A simple method of calculating the exact probability in 2 × 2 contingency tables with small marginal totals. Biometrika 42, 522–523 (1955)
Ludbrook, J., Dudley, H.A.F.: Why permutation tests are superior to t and F tests in biomedical research. Am. Stat. 52, 127–132 (1998)
MacKenzie, D.: Statistics in Britain, 1865–1930: The Social Construction of Scientific Knowledge. Edinburgh University Press, Edinburgh (1981)
Mann, H.B., Whitney, D.R.: On a test of whether one of two random variables is stochastically larger than the other. Ann. Math. Stat. 18, 50–60 (1947)
May, R.B., Hunter, M.A.: Some advantages of permutation tests. Can. Psychol. 34, 401–407 (1993)
Mehta, C.R., Patel, N.R.: A network algorithm for performing Fisher’s exact test in r × c contingency tables. J. Am. Stat. Assoc. 78, 427–434 (1983)
Mielke, P.W.: Meteorological applications of permutation techniques based on distance functions. In: Krishnaiah, P.R., Sen, P.K. (eds.) Handbook of Statistics, vol. IV, pp. 813–830. North-Holland, Amsterdam (1984)
Mielke, P.W., Berry, K.J.: Cumulant methods for analyzing independence of r-way contingency tables and goodness-of-fit frequency data. Biometrika 75, 790–793 (1988)
Mielke, P.W., Berry, K.J.: Permutation tests for common locations among samples with unequal variances. J. Educ. Behav. Stat. 19, 217–236 (1994)
Mielke, P.W., Berry, K.J.: Permutation Methods: A Distance Function Approach. Springer, New York (2001)
Mielke, P.W., Berry, K.J.: Multivariate multiple regression analyses: a permutation method for linear models. Psychol. Rep. 91, 3–9 (2002)
Mielke, P.W., Berry, K.J., Johnson, E.S.: Multi-response permutation procedures for a priori classifications. Commun. Stat. Theor. Methods 5, 1409–1424 (1976)
Mielke, P.W., Berry, K.J., Neidt, C.O.: A permutation test for multivariate matched-pairs analyses: comparisons with Hotelling’s multivariate matched-pairs T 2 test. Psychol. Rep. 78, 1003–1008 (1996)
Mielke, P.W., Berry, K.J., Johnston, J.E.: Resampling programs for multiway contingency tables with fixed marginal frequency totals. Psychol. Rep. 101, 18–24 (2007)
Mielke, P.W., Berry, K.J., Johnston, J.E.: Resampling probability values for weighted kappa with multiple raters. Psychol. Rep. 102, 606–613 (2008)
Mielke, P.W., Long, M.A., Berry, K.J., Johnston, J.E.: g-treatment ridit analysis: resampling permutation methods. Stat. Methodol. 6, 223–229 (2009)
Patefield, W.M.: Algorithm 159: an efficient method of generating random r × c tables with given row and column totals. J. R. Stat. Soc. C Appl. 30, 91–97 (1981)
Pearson, E.S., Kendall, M.G. (eds.): Studies in the History of Statistics and Probability, vol. I. Griffin, London (1970)
Pesarin, F., Salmaso, L.: Exact permutation tests for unreplicated factorials. Appl. Stoch. Model. Bus. 18, 287–299 (2002)
Pitman, E.J.G.: Significance tests which may be applied to samples from any populations. Suppl. J. R. Stat. Soc. 4, 119–130 (1937)
Pitman, E.J.G.: Significance tests which may be applied to samples from any populations: II. The correlation coefficient test. Suppl. J. R. Stat. Soc. 4, 225–232 (1937)
Pitman, E.J.G.: Significance tests which may be applied to samples from any populations: III. The analysis of variance test. Biometrika 29, 322–335 (1938)
Porter, T.M.: The Rise of Statistical Thinking, 1820–1900. Princeton University Press, Princeton (1986)
Robertson, W.H.: Programming Fisher’s exact method of comparing two percentages. Technometrics 2, 103–107 (1960)
Sakoda, J.M., Cohen, B.H.: Exact probabilities for contingency tables using binomial coefficients. Psychometrika 22, 83–86 (1957)
Salsburg, D.S.: The Lady Tasting Tea: How Statistics Revolutionized Science in the Twentieth Century. Holt, New York (2001)
Scheffé, H.: Statistical inference in the non-parametric case. Ann. Math. Stat. 14, 305–332 (1943)
Snedecor, G.W.: Calculation and Interpretation of Analysis of Variance and Covariance. Collegiate Press, Ames (1934)
Spława–Neyman, J.: Próba uzasadnienia zastosowań rachunku prawdopodobieństwa do doświadczeń polowych (On the application of probability theory to agricultural experiments. Essay on principles. Section 9). Rocz Nauk Rolnicz (Ann. Agric. Sci.) 10, 1–51 (1923). [Translated from the original Polish by D. M. Dabrowska and T. P. Speed and published in Statist. Sci. 5, 465–472 (1990)]
Stelmach, J.: On permutation tests with several test statistics to compare two populations. In: Vojackova, H. (ed.) Mathematical Methods in Economics 2013, pp. 850–855. Coll Polytechnics Jihlava, Jihlava (2013)
Stigler, S.M.: American Contributions to Mathematical Statistics in the Nineteenth Century, vol. I. Arno Press, New York (1980)
Stigler, S.M.: American Contributions to Mathematical Statistics in the Nineteenth Century, vol. II. Arno Press, New York (1980)
Stigler, S.M.: The History of Statistics: The Measurement of Uncertainty Before 1900. Harvard University Press, Cambridge (1986)
Stigler, S.M.: Statistics on the Table: The History of Statistical Concepts and Methods. Harvard University Press, Cambridge (1999)
Stigler, S.M.: The Seven Pillars of Statistical Wisdom. Harvard University Press, Cambridge (2016)
Tedin, O.: The influence of systematic plot arrangements upon the estimate of error in field experiments. J. Agric. Sci. 21, 191–208 (1931)
Thakur, A.K., Berry, K.J., Mielke, P.W.: A FORTRAN program for testing trend and homogeneity in proportions. Comput. Prog. Biomed. 19, 229–233 (1985)
Tukey, J.W.: Randomization and re-randomization: the wave of the past in the future. In: Statistics in the Pharmaceutical Industry: Past, Present and Future. Philadelphia Chapter of the American Statistical Association (1988). [Presented at a Symposium in Honor of Joseph L. Ciminera held in June 1988 at Philadelphia, Pennsylvania]
Umlauft, M., Konietschke, F., Pauly, M.: Rank-based permutation approaches for non-parametric factorial designs. Brit. J. Math. Stat. Psychol. 70, 368–390 (2017)
White, C.: The use of ranks in a test of significance for comparing two treatments. Biometrics 8, 33–41 (1952)
Wilcoxon, F.: Individual comparisons by ranking methods. Biometrics Bull. 1, 80–83 (1945)
Wilks, S.S.: Order statistics. Bull. Am. Math. Soc. 54, 6–50 (1948)
Yates, F.: Contingency tables involving small numbers and the χ 2 test. Suppl. J. R. Stat. Soc. 1, 217–235 (1934)
Yeo, I.K.: An algorithm for computing the exact distribution of the Wilcoxon signed-rank statistic. J. Korean Stat. Soc. 46, 328–338 (2017)
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Berry, K.J., Johnston, J.E., Mielke, P.W. (2019). A Brief History of Permutation Methods. In: A Primer of Permutation Statistical Methods. Springer, Cham. https://doi.org/10.1007/978-3-030-20933-9_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-20933-9_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-20932-2
Online ISBN: 978-3-030-20933-9
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)