Abstract
Students’ understanding of models in science has been subject to a number of investigations. The instruments the researchers used are suitable for educational research but, due to their complexity, cannot be employed directly by teachers. This article presents forced choice (FC) tasks, which, assembled as a diagnostic instrument, are supposed to measure students’ understanding of the nature of models efficiently, while being sensitive enough to detect differences between individuals. In order to evaluate if the diagnostic instrument is suitable for its intended use, we propose an approach that complies with the demand to integrate students’ responses to the tasks into the validation process. Evidence for validity was gathered based on relations to other variables and on students’ response processes. Students’ understanding of the nature of models was assessed using three methods: FC tasks, open-ended tasks and interviews (N = 448). Furthermore, concurrent think-aloud protocols (N = 30) were performed. The results suggest that the method and the age of the students have an effect on their understanding of the nature of models. A good understanding of the FC tasks as well as a convergence in the findings across the three methods was documented for grades eleven and twelve. This indicates that teachers can use the diagnostic instrument for an efficient and, at the same time, valid diagnosis for this group. Finally, the findings of this article may provide a possible explanation for alternative findings from previous studies as a result of specific methods that were used.
Similar content being viewed by others
References
Abd-El-Khalick, F. (2012). Nature of science in science education: toward a coherent framework for synergistic research and development. In B. Fraser, K. Tobin, & C. McRobbie (Eds.), Second international handbook of science education (pp. 1041–1060). Dordrecht: Springer.
Adams, R., & Wu, M. (2002). PISA 2000 technical report. Paris: OECD.
Al-Balushi, S. (2011). Students’ evaluation of the credibility of scientific models that represent natural entities and phenomena. International Journal of Science and Mathematics Education, 9, 571–601.
American Educational Research Association, American Psychological Association & National Council on Measurement in Education [AERA, APA, & NCME]. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
Anastasi, A. (1976). Psychological testing. New York: Macmillan.
Bailer-Jones, D. (2009). Scientific models in philosophy of science. Pittsburgh, PA: University of Pittsburgh.
Bennett, R. (1993). On the meaning of constructed response. In R. Bennett & W. Ward (Eds.), Construction versus choice in cognitive measurement: issues in constructed response, performance testing, and portfolio assessment (pp. 1–28). Hillsdale, NJ: Erlbaum.
Böckenholt, U. (2004). Comparative judgments as an alternative to ratings: identifying the scale origin. Psychological Methods, 9, 453–465.
Bond, T., & Fox, C. (2001). Applying the Rasch model. Mahwah, NJ: Erlbaum.
Brennan, R., & Prediger, D. (1981). Coefficient kappa: some uses, misuses, and alternatives. Educational and Psychological Measurement, 41, 687–699.
Campbell, D., & Fiske, D. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81–105.
Chittleborough, G., Treagust, D., Mamiala, T., & Mocerino, M. (2005). Students’ perceptions of the role of models in the process of science and in the process of learning. Research in Science and Technological Education, 23, 195–212.
Crawford, B., & Cullin, M. (2005). Dynamic assessments of preservice teachers’ knowledge of models and modelling. In K. Boersma, M. Goedhart, O. de Jong, & H. Eijkelhof (Eds.), Research and the quality of science education (pp. 309–323). Dordrecht: Springer.
Danusso, L., Testa, I., & Vicentini, M. (2010). Improving prospective teachers’ knowledge about scientific models and modelling: design and evaluation of a teacher education intervention. International Journal of Science Education, 32, 871–905.
Eggert, S., & Bögeholz, S. (2010). Students’ use of decision-making strategies with regard to socioscientific issues—an application of the Rasch partial credit model. Science Education, 94, 230–258.
Eid, M., & Diener, E. (2006). Handbook of multimethod measurement in psychology. Washington, DC: American Psychological Association.
Eid, M., & Schmidt, K. (2014). Testtheorie und Testkonstruktion [Test theory—test construction]. Göttingen: Hogrefe & Huber.
Embretson, S., & Gorin, J. (2001). Improving construct validity with cognitive psychology principles. Journal of Educational Measurement, 38, 343–368.
Embretson, S., & Reise, S. (Eds.). (2000). Item response theory for psychologists. Mahwah, NJ: Erlbaum.
Ericsson, K., & Simon, H. (1998). How to study thinking in everyday life: contrasting think-aloud protocols with descriptions and explanations of thinking. Mind, Culture, and Activity, 5, 178–186.
Gibbs, A., & Lawson, A. (1992). The nature of scientific thinking as reflected by the work of biologists & by biology textbooks. The American Biology Teacher, 54, 137–152.
Giere, R. (1988). Explaining science. A cognitive approach. Chicago, IL: University of Chicago.
Gilbert, J. (2004). Models and modelling: routes to more authentic science education. International Journal of Science and Mathematics Education, 2, 115–130.
Gobert, J., O’Dwyer, L., Horwitz, P., Buckley, B., Levy, S., & Wilensky, U. (2011). Examining the relationship between students’ understanding of the nature of models and conceptual learning in biology, physics, and chemistry. International Journal of Science Education, 33, 653–684.
Gogolin, S., & Krüger, D. (2015). Nature of models — Entwicklung von Diagnoseaufgaben [Nature of models — development of diagnostic tasks]. In M. Hammann, J. Mayer, & N. Wellnitz (Eds.), Lehr- und Lernforschung in der Biologiedidaktik [Research on Teaching and Learning in Biology Education] 6 (pp. 27–41). Innsbruck: Studienverlag.
Gonzalez, E., & Rutkowski, L. (2010). Principles of multiple matrix booklet designs and parameter recovery in large-scale assessments. IERI Monograph Series: Issues and Methodologies in Large-Scale Assessments, 3, 125–156.
Grosslight, L., Unger, C., Jay, E., & Smith, C. (1991). Understanding models and their use in science: conceptions of middle and high school students and experts. Journal of Research in Science Teaching, 28, 799–822.
Grünkorn, J., Upmeier zu Belzen, A., & Krüger, D. (2014). Assessing students' understandings of biological models and their use in science to evaluate a theoretical framework. International Journal of Science Education, 36, 1651–1684.
Harrison, A., & Treagust, D. (2000). A typology of school science models. International Journal of Science Education, 22, 1011–1026.
Hartig, J., Klieme, E., & Leutner, D. (Eds.). (2008). Assessment of competencies in educational contexts: state of the art and future prospects. Göttingen: Hogrefe & Huber.
Hodson, D. (2014). Learning science, learning about science, doing science. International Journal of Science Education, 36, 2534–2553.
Hoyt, W. (2000). Rater bias in psychological research: when is it a problem and what can we do about it? Psychological Methods, 5, 64–86.
Justi, R., & Gilbert, J. (2003). Teacher’s views on the nature of models. International Journal of Science Education, 25, 1369–1386.
Justi, R., & Van Driel, J. (2005). The development of science teachers’ knowledge on models and modelling: promoting, characterizing, and understanding the process. International Journal of Science Education, 27, 549–573.
Katz, I., Bennett, E., & Rerger, A. (2000). Effects of response format on difficulty of SAT-Mathematics items: it’s not the strategy. Journal of Edueational Measurement, 37, 39–57.
Kauertz, A., & Fischer, H. (2006). Assessing students’ level of knowledge and analysing the reasons for learning difficulties in physics by Rasch analysis. In X. Liu & W. Boone (Eds.), Applications of Rasch Measurement in Science Education (pp. 212–245). Maple Grove, MN: JAM.
Kauertz, A., Neumann, K., & Haertig, H. (2012). Competence in science education. In B. Fraser, K. Tobin, & C. McRobbie (Eds.), Second international handbook of science education (pp. 711–721). Dordrecht: Springer.
Klieme, E., Hartig, J., & Rauch, D. (2008). The concept of competence in educational contexts. In J. Hartig, E. Klieme, & D. Leutner (Eds.), Assessment of competencies in educational contexts (pp. 3–22). Hogrefe & Huber: Göttingen.
Konferenz der Kultusminister der Länder in der Bundesrepublik Deutschland (KMK). (2005). Bildungsstandards im Fach Biologie für den Mittleren Schulabschluss [Biology education standards for the Mittlere Schulabschluss]. München: Wolters Kluwer.
Krell, M. (2012). Using polytomous IRT models to evaluate theoretical levels of understanding models and modeling in biology education. Science Education Review Letters, Theoretical Letters, 2012, 1–5.
Krell, M. (2013). Wie Schülerinnen und Schüler biologische Modelle verstehen [How students understand biological models]. Berlin: Logos.
Krell, M., Upmeier zu Belzen, A., & Krüger, D. (2012). Students’ understanding of the purpose of models in different biological contexts. International Journal of Biology Education, 2, 1–34.
Krell, M., Reinisch, B., & Krüger, D. (2015). Analyzing students’ understanding of models and modeling referring to the disciplines biology, chemistry, and physics. Research in Science Education, 45, 367–393.
Krell, M., Upmeier zu Belzen, A., & Krüger, D. (2014). Context-specificities in students’ understanding of models and modelling: an issue of critical importance for both assessment and teaching. In C. Constantinou, N. Papadouris, & A. Hadjigeorgiou (Eds.), E-Book proceedings of the ESERA 2013 conference. Retrieved from http://www.esera.org/media/esera2013/Moritz_Krell_07Feb2014.pdf. Accessed 12 January 2016.
Krell, M., Upmeier zu Belzen, A., & Krüger, D. (2014b). Students’ levels of understanding models and modelling in biology: global or aspect-dependent? Research in Science Education, 44, 109–132.
Kuckartz, U. (2012). Qualitative Inhaltsanalyse: Methoden, Praxis, Computerunterstützung [Qualitative content analysis: methods, practice, technological support]. Beltz Juventa: Weinheim.
Leach, J., Millar, R., Ryder, J., & Séré, M.-G. (2000). Epistemological understanding in science learning: the consistency of representations across contexts. Learning and Instruction, 10, 497–527.
Lee, S., Chang, H., & Wu, H. (2015). Students’ views of scientific models and modeling: do representational characteristics of models and students’ educational levels matter? Research in Science Education. doi:10.1007/s11165-015-9502-x.
Leighton, J. (2004). Avoiding misconception, misuse, and missed opportunities: the collection of verbal reports in educational achievement testing. Educational Measurement: Issues and Practice, 23, 6–15.
Mahr, B. (2011). On the epistemology of models. Rethinking Epistemology, 1, 301–352.
Martinez, M. (1999). Cognition and the question of test item format. Educational Psychologist, 34, 207–218.
Masters, G. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149–174.
Mayring, P. (2010). Qualitative Inhaltsanalyse: Grundlagen und Techniken [Qualitative content analysis: Basics and techniques]. Weinheim: Beltz.
McCloy, R., Heggestad, E., & Reeve, C. (2005). A silk purse from the sow’s ear: retrieving normative information from multidimensional forced-choice items. Organizational Research Methods, 8, 222–248.
McComas, W. (Ed.). (2013). The language of science education: an expanded glossary of key terms and concepts in science teaching and learning. Rotterdam: Sense.
Meisert, A. (2007). Über den Umgang mit Hypothesen [How to deal with hypotheses]. Der mathematische und naturwissenschaftliche Unterricht [Mathematical and science teaching], 60, 431–437.
Messick, S. (1995). Validity of psychological assessment: validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741–749.
Lead States, N. G. S. S. (Ed.). (2013). Next generation science standards: for states, by states. Washington, DC: National Academies.
Nisbett, R., & Wilson, T. (1977). Telling more than we can know: verbal reports on mental processes. Psychological Review, 84, 231–259.
Odom, A., & Barrow, L. (1995). Development and application of a two‐tier diagnostic test measuring college biology students’ understanding of diffusion and osmosis after a course of instruction. Journal of Research in Science Teaching, 32, 45–61.
Oh, P., & Oh, S. (2011). What teachers of science need to know about models: an overview. International Journal of Science Education, 33, 1109–1130.
Pant, H., Stanat, P., Schroeders, U., Roppelt, A., & Siegle, T. (Eds.). (2013). IQB-Ländervergleich 2012 [IQB – Cross-country comparison 2012]. Münster: Waxmann.
Passmore, C., Gouvea, J., & Giere, R. (2014). Models in science and in learning science. In M. Matthews (Ed.), International handbook of research in history, philosophy and science teaching (pp. 1171–1202). Dordrecht: Springer.
Patzke, C., Krüger, D., & Upmeier zu Belzen, A. (2015). Entwicklung von Modellkompetenz im Längsschnitt [Development of model competence in a longitudinal study]. In M. Hammann, J. Mayer, & N. Wellnitz (Eds.), Lehr- und Lernforschung in der Biologiedidaktik [Research on teaching and learning in biology education] 6 (pp. 43–58). Innsbruck: Studienverlag.
Rector Federer, M., Nehm, R., Opfer, J., & Pearl, D. (2014). Using a constructed-response instrument to explore the effects of item position and item features on the assessment of students’ written scientific explanations. Research in Science Education, 4, 527–553.
Rost, J. (2004). Lehrbuch Testtheorie – Testkonstruktion [Textbook test theory—test construction]. Bern: Hans Gruber.
Rost, J., & Walter, O. (2006). Multimethod item response theory. In M. Eid & E. Diener (Eds.), Handbook of multimethod measurement in psychology (pp. 249–268). Washington, DC: American Psychological Association.
Schwarz, C. (2002). The role of epistemic forms and games: perspectives on the role of metamodeling knowledge in learning with models. In P. Bell, R. Stevens, & T. Satwicz (Eds.), Kee** learning complex: the proceedings of the Fifth International Conference of the Learning Sciences (ICLS) (pp. 414–420). Mahwah, NJ: Erlbaum.
Schwarz, C., & White, B. (2005). Metamodelling knowledge: develo** students’ understanding of scientific modelling. Cognition and Instruction, 23, 165–205.
Schwarz, C., Reiser, B., Davis, E., Kenyon, L., Acher, A., Fortus, D., et al. (2009). Develo** a learning progression for scientific modeling: making scientific modeling accessible and meaningful for learners. Journal of Research in Science Teaching, 46, 632–654.
Schwarz, C., Reiser, B., Fortus, D., Krajcik, J., Roseman, J., Willard, T., & Acher, A. (2008). Designing and testing the MoDeLS learning progression. Paper presented at the annual conference of the National Association for Research in Teaching (NARST), Baltimore, MD. Retrieved from http://www.models.northwestern.edu/docs/NARST_MoDeLS_Intro_Paper_FINAL.pdf. Accessed 12 January 2016.
Sins, P., Savelsbergh, E., Van Joolingen, W., & Van Hout–Wolters, B. (2009). The relation between students’ epistemological understanding of computer models and their cognitive processing on a modelling task. International Journal of Science Education, 31, 1205–1229.
Terzer, E. (2013). Modellkompetenz im Kontext Biologieunterricht [Model competence in the context of biology education] (Dissertation). Humboldt Universität zu Berlin. Retrieved from http://edoc.hu-berlin.de/dissertationen/terzereva-2012-12-19/PDF/terzer.pdf. Accessed 12 January 2016.
Traub, R., & MacRury, K. (1990). Multiple-choice vs. free-response in the testing of scholastic achievement. In K. Ingenkamp & R. Jäger (Eds.), Tests und Trends [Tests and Trends]. 8. Jahrbuch der Pädagogischen Diagnostik (pp. 128–159). Beltz: Weinheim.
Treagust, D., Chittleborough, G., & Mamiala, T. (2002). Student’s understanding of the role of scientific models in learning science. International Journal of Science Education, 24, 357–368.
Treagust, D., Chittleborough, G., & Mamiala, T. (2004). Students’ understanding of the descriptive and predictive nature of teaching models in organic chemistry. Research in Science Education, 34, 1–20.
Trier, U., & Upmeier zu Belzen, A. (2009). ‘Die Wissenschaftler nutzen Modelle, um etwas Neues zu entdecken, und in der Schule lernt man einfach nur, dass es so ist’.: Schülervorstellungen zu Modellen [‘Scientists use models to discover something new and in school, you only learn that this is the case’: students’ conceptions of models]. Erkenntnisweg Biologiedidaktik [Path of knowledge in biology education], 8, 23–37.
Trier, U., Krüger, D., & Upmeier zu Belzen, A. (2014). Students’ versus scientists’ conceptions of models and modelling. In D. Krüger, & M. Ekborg (Eds.), Research in biological education (pp. 103–115). Retrieved from http://www.bcp.fu-berlin.de/biologie/arbeitsgruppen/didaktik/eridob_2012/eridob_proceeding/7-Students_Versus.pdf?1389177503. Accessed 12 January 2016.
Ubben, I., Nitz, S., Rousseau, M., & Upmeier zu Belzen, A. (2015). Modelle von und für Evolution in Schulbüchern [Models of and for evolution in textbooks]. [Abstract]. In U. Gebhard, M. Hammann, & B. Knälmann (Eds.), Bildung durch Biologieunterricht. 20. Internationale Tagung der Fachsektion Didaktik der Biologie (FDdB) im VBiO [Education through biology teaching. 20th international conference of the FDdB] (pp. 75–76). Hamburg. Retrieved from http://docplayer.org/15781310-Bildung-durch-biologieunterricht-14-17-september-2015.html. Accessed 05 September 2016.
Upmeier zu Belzen, A., & Krüger, D. (2010). Modellkompetenz im Biologieunterricht [Model competence in biology education]. Zeitschrift für Didaktik der Naturwissenschaften [Journal for Education in Science], 16, 41–57.
Van Der Valk, T., Van Driel, J., & De Vos, W. (2007). Common characteristics of models in present-day scientific practice. Research in Science Education, 37, 469–488.
Van Driel, J., & Verloop, N. (1999). Teachers’ knowledge of models and modelling in science. International Journal of Science Education, 21, 1141–1153.
Van Driel, J., & Verloop, N. (2002). Experienced teachers’ knowledge of teaching and learning of models and modelling in science education. International Journal of Science Education, 24, 1255–1277.
Van Someren, M., Barnard, Y., & Sandberg, J. (1994). The think-aloud method: a practical guide to modelling cognitive processes. San Diego, CA: Academic.
Vygotsky, L. (1962). Thought and language. Cambridge, MA: MIT.
Warm, T. (1989). Weighted likelihood estimation of ability in item response theory. Psychometrika, 54, 427–450.
Welzel, M., Haller, K., Bandiera, M., Hammelev, D., Koumaras, P., Niedderer, H., Paulsen, A., Robinault, K., & Von Aufschnaiter, S. (1998). Ziele, die Lehrende mit dem Experimentieren in der naturwissenschaftlichen Ausbildung verbinden [Aims, teachers pursue with experimenting as part of science education]. Zeitschrift für Didaktik der Naturwissenschaften [Journal for education in science], 4, 29–44.
Willard, T., & Roseman, J. (2010). Probing students’ ideas about models using standards-based assessment items. Paper presented at the annual conference of the National Association for Research in Teaching (NARST), Philadelphia, PA. Retrieved from http://www.models.northwestern.edu/docs/Models%20Willard%20NARST%203-19-10.pdf. Accessed 12 January 2016.
Wilson, M., De Boeck, P., & Carstensen, C. (2008). Explanatory item response models: a brief introduction. In J. Hartig, E. Klieme, & D. Leutner (Eds.), Assessment of competencies in educational contexts (pp. 91–120). Göttingen: Hogrefe & Huber.
Windschitl, M., Thompson, J., & Braaten, M. (2008). Beyond the scientific method: model-based inquiry as a new paradigm of preference for school science investigations. Science Education, 92, 941–967.
Wirtz, M., & Caspar, F. (2002). Beurteilerübereinstimmung und Beurteilerreliabilität [Interrater-accordance and interrater-reliability]. Göttingen: Hogrefe & Huber.
Wu, M., Adams, R., Wilson, M., & Haldane, S. (2007). ACER ConQuest. Camberwell, VIC: ACER.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Gogolin, S., Krüger, D. Diagnosing Students’ Understanding of the Nature of Models. Res Sci Educ 47, 1127–1149 (2017). https://doi.org/10.1007/s11165-016-9551-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11165-016-9551-9