SpringerLink
Log in

Do Students Know What They Know and What They Don’t Know? Using a Four-Tier Diagnostic Test to Assess the Nature of Students’ Alternative Conceptions

  • Published:
Research in Science Education Aims and scope Submit manuscript

Abstract

This study reports on the development and application of a four-tier multiple-choice (4TMC) diagnostic instrument, which has not been reported in the literature. It is an enhanced version of the two-tier multiple-choice (2TMC) test. As in 2TMC tests, its answer and reason tiers measure students’ content knowledge and explanatory knowledge, respectively. The two additional tiers measure the level of confidence of students in the correctness of their chosen options for the answer and reason tiers respectively. The 4TMC diagnostic test focused on the properties and propagation of mechanical waves. It was administered to 598 upper secondary students after they were formally instructed on the foregoing topics. The vast majority of the respondents were found to have an inadequate grasp of the topics tested. Mean scores and mean confidence associated with the answer tier was higher than those associated with the reason tier. The students tended to be poorly discriminating between what they know and what they do not know. Familiarity with the topic tested was associated with greater percentage of students giving correct answers, higher confidence, and better discrimination quotient. Nine genuine alternative conceptions (which were expressed with moderate levels of confidence by students) were identified.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. Mainstream schools are non-autonomous and non-independent schools. They follow the curriculum and implement school programmes which are set by the Ministry of Education. They do not offer niche programmes. They admit students solely on the basis of the students’ scores in the Primary School Leaving Examination (PSLE)—a national examination given at the end of Grade Six. (Ministry of Education 2008).

  2. Based on the students’ aggregate scores in the PSLE, students are directed into different streams of academic ability. The stream, in which the present sample belongs, requires aggregate scores of about 50% and above of the maximum score, for eligibility.

References

  • Al-Rubayea, A. A. M. (1996). An analysis of Saudi Arabia high school students’ misconceptions about physics concepts. Dissertation Abstracts International, 57(04), 1462, (UMI No. 9629018).

    Google Scholar 

  • Altman, D. G. (1991). Practical statistics for medical research. London: Chapman & Hall.

    Google Scholar 

  • Barman, C. R., & Barman, S. N. (1996). Two teaching methods and students’ understanding of sound. School Science and Mathematics, 96(2), 63–68.

    Article  Google Scholar 

  • Clement, J., Brown, D. E., & Zietsman, A. (1989). Not all preconceptions are misconceptions: finding ‘anchoring conceptions’ for grounding instruction on students’ intuition. International Journal of Science Education, 11(5), 554–565. doi:10.1080/0950069890110507.

    Article  Google Scholar 

  • diSessa, A. A. (1993). Toward an epistemology of physics. Cognition and Instruction, 10(2 & 3), 105–225.

    Google Scholar 

  • Echternacht, G. J. (1972). The use confidence testing in objective tests. Review of Educational Research, 42–2, 217–236.

    Google Scholar 

  • Eshach, H., & Schwarz, J. L. (2006). Sound stuff: naïve materialism in middle-school students’ conceptions of sound. International Journal of Science Education, 7(1), 733–764.

    Article  Google Scholar 

  • Franklin, B. J. (1992). The development and application of a two-tier diagnostic instrument to detect misconceptions in the area of force, heat light and electricity. Dissertation Abstracts International, 53(12), 4186, (UMI No. 9301049).

    Google Scholar 

  • Glenberg, A. M., & Epstein, W. (1985). Calibration of comprehension. Journal of Experimental Psychology: Learning Memory and Cognition, 11, 702–708.

    Article  Google Scholar 

  • Glenberg, A. M., Sanocki, T., Epstein, W., & Morris, C. (1987). Enhancing calibration of comprehension. Journal of Experimental Psychology: General, 116, 119–136.

    Article  Google Scholar 

  • Griffard, P. B., & Wandersee, J. H. (2001). The two-tier instrument on photosynthesis: what does it diagnose? International Journal of Science Education, 23(10), 1039–1052.

    Article  Google Scholar 

  • Hasan, S., Bagayoko, D., & Kelley, E. L. (1999). Misconceptions and the Certainty of Response Index (CRI). Physics Education, 34(5), 294–299.

    Article  Google Scholar 

  • Hill, G. D. (1997). Conceptual change through the use of student-generated analogies of photosynthesis and respiration by college non-science majors. Dissertation Abstracts International, 58(06), 2142, (UMI No. 9735480).

    Google Scholar 

  • Lang, H. G. (1982). Criterion-referenced tests in science: an investigation of reliability, validity and standards-setting. Journal of Research in Science Teaching, 19(8), 665–674.

    Article  Google Scholar 

  • Linder, C. J. (1993). University physics students’ conceptualizations of factors affecting the speed of sound propagation. International Journal of Science Education, 15(6), 655–662.

    Article  Google Scholar 

  • Lundeberg, M. A., Fox, P. W., & Punćhochaŕ, J. (1994). Highly confident but wrong: gender differences and similarities in confidence judgments. Journal of Educational Psychology, 86(1), 114–121.

    Article  Google Scholar 

  • Lundeberg, M. A., Fox, P. W., Brown, A. C., & Elbedour, S. (2000). Cultural influences on confidence: country and gender. Journal of Educational Psychology, 92(1), 152–159.

    Article  Google Scholar 

  • Maurines, L. (1992). Spontaneous reasoning on the propagation of visible mechanical signals. International Journal of Science Education, 14(3), 279–293.

    Article  Google Scholar 

  • McKelvie, S. J. (1978). Effects of some variations in rating scale on the means and reliabilities of ratings. British Journal of Psychology, 69, 185–202.

    Google Scholar 

  • McKelvie, S. (1992). Does memory contaminate test-retest reliability? The Journal of General Psychology, 119(1), 59–72.

    Google Scholar 

  • Menchen, K. V. P., & Thomson, J. R. (2004). Pre-service teachers’ understanding of propagation and resonance in sound phenomena. In J. Max, S. Franklin, & J. Cummings (Eds.), 2003 Physics education Research Conference (pp. 65–68). New York: American Institute of Physics.

    Google Scholar 

  • Ministry of Education [MOE] (2008). Independent schools, special assistance plan (SAP) schools, autonomous schools and niche programme schools. http://www.moe.gov.sg/education/admissions/secondary-one-posting/files/secondary-one-posting-english.pdf.

  • Morris, C. (1990). Retrieval process underlying confidence in comprehension judgments. Journal of Experimental Psychology: Learning Memory and Cognition, 16, 223–232.

    Article  Google Scholar 

  • Palacios, F. J. P., Cazorla, F. N., & Cervantes, A. (1989). Misconceptions on geometric optics and their association with relevant educational variables. International Journal of Science Education, 11(3), 273–286.

    Article  Google Scholar 

  • Popham, W. J., & Husek, T. R. (1969). Implications of criterion-referenced measurement. Journal of Educational Measurement, 6(1), 1–9.

    Article  Google Scholar 

  • Reiner, M., Slotta, J. D., Chi, M. T. H., & Resnick, L. B. (2000). Naïve physics reasoning: a commitment to substance-based conceptions. Cognition and Instruction, 18(1), 1–34.

    Article  Google Scholar 

  • Reynolds, C. R., Livingston, R. B., & Willson, V. (2006). Measurement and assessment in education. Boston: Pearson.

    Google Scholar 

  • Stankov, L., & Crawford, J. D. (1997). Self-confidence and performance on test of cognitive abilities. Intelligence, 25(2), 93–109.

    Article  Google Scholar 

  • Stankov, L., & Dolph, B. (2000). Metacognitive aspects of test-taking and intelligence. Psychologische Beiträge, 42(2), 213–227.

    Google Scholar 

  • Tobin, K. G., & Capie, W. (1981). The development and validation of a group test of logical thinking. Educational and Psychological Measurement, 11, 413–423.

    Article  Google Scholar 

  • Treagust, D. F. (1986). Evaluating students’ misconceptions by means of diagnostic multiple-choice items. Research in Science Education, 16, 199–207.

    Article  Google Scholar 

  • Treagust, D. F. (1988). Development and use of diagnostic tests to evaluate students’ misconceptions in science. International Journal of Science Education, 10, 159–170.

    Article  Google Scholar 

  • Tsai, C. C., & Chou, C. (2002). Diagnosing students’ alternative conceptions in science. Journal of Computer Assisted Learning, 18, 157–165.

    Article  Google Scholar 

  • Voska, K. W., & Heikkinen, H. W. (2000). Identification and analysis of student conceptions used to solve chemical equilibrium problems. Journal of Research in Science Teaching, 37(2), 160–176.

    Article  Google Scholar 

  • Witmann, M. C. (2002). The object coordination class applied to wave pulses: analysing student reasoning in wave physics. International Journal of Science Education, 24(1), 97–118.

    Article  Google Scholar 

  • Wittmann, M. C., Steinberg, R. N., & Redish, E. F. (1999). Making sense of how students make sense of mechanical waves. The Physics Teacher, 37, 15–21.

    Article  Google Scholar 

  • Wittmann, M. C., Steinberg, R. N., & Redish, E. F. (2003). Understanding and affecting student reasoning about sound waves. International Journal of Science Education, 25(8), 991–1013.

    Article  Google Scholar 

  • Zakay, D., & Glicksohn, J. (1992). Overconfidence in a multiple-choice test and its relationship to achievement. Psychological Record, 42(4), 519–525.

    Google Scholar 

Download references

Acknowledgment

We thank the Nanyang Technological University for the award of a research scholarship to the first author and a research grant (RI 9/06 RS) to the second author.

Author information

Authors and Affiliations

  1. National Institute of Education, Nanyang Technological University, 1 Nanyang Walk, Singapore, 637616, Singapore

    Imelda S. Caleon & R. Subramaniam

Authors
  1. Imelda S. Caleon
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. R. Subramaniam
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to R. Subramaniam.

Appendix*

Appendix*

Rights and permissions

Reprints and permissions

About this article

Cite this article

Caleon, I.S., Subramaniam, R. Do Students Know What They Know and What They Don’t Know? Using a Four-Tier Diagnostic Test to Assess the Nature of Students’ Alternative Conceptions. Res Sci Educ 40, 313–337 (2010). https://doi.org/10.1007/s11165-009-9122-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11165-009-9122-4

Keywords

Search

Navigation