SpringerLink
Log in

Modeling and Measuring High School Students’ Computational Thinking Practices in Science

  • Published:
Journal of Science Education and Technology Aims and scope Submit manuscript

Abstract

Despite STEM education communities recognizing the importance of integrating computational thinking (CT) into high school curricula, computation still remains a separate area of study in K-12 contexts. In addition, much of the research on CT has focused on creating generally agreed-upon definitions and curricula, but few studies have empirically tested assessments or used contemporary learning sciences methods to do so. In this paper, we outline the implementation of an assessment approach for a 10-day high school biology unit with computational thinking activities that examines student pre-post responses as well as responses to embedded assessments throughout the unit. Using pre-post scores, we identified students with both positive and negative gains and examined how each group’s CT practices developed as they engaged with the curricular unit. Our results show that (1) students exhibited science and computational learning gains after engaging with a science unit with computational models and (2) that the use of embedded assessments and discourse analytics tools reveals how students think differently with computational tools throughout the unit.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

Notes

  1. Braun & Clark (2006) note that when researchers use a bottom-up approach, they do not completely analyze their data in an “epistemological vacuum” because they “can not free themselves [completely] of their theoretical and epistemological commitments.” Even if researchers do not explicitly take a theoretical or epistemological stance, their implicit biases and points of view shape the analysis of the data.

References

  • Abrahamson, D., & Wilensky, U. (2007). Learning axes and bridging tools in a technology-based design for statistics. International Journal of Computers for Mathematical Learning, 12(1), 23–55.

    Article  Google Scholar 

  • Adams, W. K., & Wieman, C. E. (2011). Development and validation of instruments to measure learning of expert-like thinking. International Journal of Science Education, 33(9), 1289–1312. https://doi.org/10.1080/09500693.2010.512369.

    Article  Google Scholar 

  • Arastoopour, G., & Shaffer, D. W. (2013). Measuring social identity development in epistemic games. In Computer-Supported Collaborative Learning Conference, CSCL (Vol. 1).

    Google Scholar 

  • Arastoopour, G., Chesler, N. C., & Shaffer, D. W. (2014). Epistemic persistence: A simulation-based approach to increasing participation of women in engineering. Journal of Women and Minorities in Science and Engineering, 20(3). https://doi.org/10.1615/JWomenMinorScienEng.2014007317.

  • Arastoopour, G., Shaffer, D. W., Swiecki, Z., Ruis, A. R. R., & Chesler, N. C. N. C. (2016). Teaching and assessing engineering design thinking with virtual internships and epistemic network analysis. International Journal of Engineering Education, 32(3), 1–10.

    Google Scholar 

  • Atmatzidou, S., & Demetriadis, S. (2016). Advancing students’ computational thinking skills through educational robotics: A study on age and gender relevant differences. Robotics and Autonomous Systems, 75, 661–670. https://doi.org/10.1016/j.robot.2015.10.008.

    Article  Google Scholar 

  • Bagley, E. A., & Shaffer, D. W. (2009). When people get in the way: Promoting civic thinking through epistemic game play. International Journal of Gaming and Computer-Mediated Simulations, 1(1), 36–52.

    Article  Google Scholar 

  • Basu, S., Kinnebrew, J. S., & Biswas, G. (2014). Assessing student performance in a computational-thinking based science learning environment. Springer International Publishing, 476–481.

  • Berland, M., & Wilensky, U. (2015). Comparing virtual and physical robotics environments for supporting complex systems and computational thinking. Journal of Science Education and Technology, 24(5), 628–647. https://doi.org/10.1007/s10956-015-9552-x.

    Article  Google Scholar 

  • Bers, M. U., Flannery, L., Kazakoff, E. R., & Sullivan, A. (2014). Computational thinking and tinkering: Exploration of an early childhood robotics curriculum. Computers & Education, 72, 145–157. https://doi.org/10.1016/j.compedu.2013.10.020.

    Article  Google Scholar 

  • Bienkowski, M., Snow, E., Rutstein, D., & Grover, S. (2015). Assessment design patterns for computational thinking practices in secondary computer science, (December), 1–46.

  • Blikstein, P., & Wilensky, U. (2009). An atom is known by the company it keeps: A constructionist learning environment for materials science using agent-based modeling. International Journal of Computers for Mathematical Learning, 14(2), 81–119. https://doi.org/10.1007/s10758-009-9148-8.

    Article  Google Scholar 

  • Brady, C., Holbert, N., Soylu, F., Novak, M., & Wilensky, U. (2015). Sandboxes for model-based inquiry. Journal of Science Education and Technology, 24(2–3), 265–286. https://doi.org/10.1007/s10956-014-9506-8.

    Article  Google Scholar 

  • Brasiel, S., Close, K., Jeong, S., Lawanto, K., Janisiewicz, P., & Martin, T. (2017). Emerging research, practice, and policy on computational thinking. In P. J. Rich & C. B. Hodges (Eds.), Emerging research, practice, and policy on computational thinking (pp. 327–347). https://doi.org/10.1007/978-3-319-52691-1.

    Chapter  Google Scholar 

  • Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101.

    Article  Google Scholar 

  • Brennan, K., & Resnick, M. (2012). New frameworks for studying and assessing the development of computational thinking. In Annual Meeting of the American Educational Research Association. Vancouver, B.C. https://doi.org/10.1.1.296.6602.

  • Collier, W., Ruis, A. R., & Shaffer, D. W. (2016). Local versus global connection making in discourse. In International Conference of the Learning Sciences. Singapore.

  • Dabholkar, S., Hall, K., Woods, P., Bain, C., & Wilensky, U. (2017). From ecosystems to speciation. Evanston, IL: Center for Connected Learning and Computer-Based Modeling, Northwestern University.

  • Denner, J., Werner, L., Campe, S., & Ortiz, E. (2014). Pair programming: Under what conditions is it advantageous for middle school students? Journal of Research on Technology in Education, 46(3), 277–296. https://doi.org/10.1080/15391523.2014.888272.

    Article  Google Scholar 

  • di Sessa, A. A. (2001). Changing minds: Computers, learning, and literacy. Cambridge, MA: Mit Press.

    Google Scholar 

  • Dijkstra, E. W. (1974). Programming as a discipline of mathematical nature. The American Mathematical Monthly, 81(6), 608–612.

    Article  Google Scholar 

  • diSessa, A. A. (1993). Towards an epistemology of physics. Cognition and Instruction, 10(2), 105–225.

    Article  Google Scholar 

  • Eagan, B. R., Rogers, B., Serlin, R., Ruis, A. R., Arastoopour Irgens, G., & Shaffer, D. W. (2017). Can we rely on IRR? Testing the assumptions of inter-rater reliability. In Computer Supported Collaborative Learning. Philadelphia, PA.

  • Felsen, M., & Wilensky, U. (2007). NetLogo urban suite—pollution model. Evanston, IL: Center for Connected Learning and Computer-Based Modeling, Northwestern University.

  • Foster, I. (2006). A two-way street to science’s future. Nature, 440(March 23), 419. https://doi.org/10.1038/440419a.

    Article  Google Scholar 

  • Grover, S. (2017). Assessing algorithmic and computational thinking in K-12: Lessons from a middle school classroom. In P. J. Rich, & C. B. Hodges (Eds.), Emerging research, practice, and policy on computational thinking (1st ed.). pp. 269-288. Cham, Switzerland: Springer International Publishing

  • Grover, S., & Pea, R. (2013). Computational thinking in K-12: A review of the state of the field. Educational Research, 42(1), 38–43. https://doi.org/10.3102/0013189X12463051.

    Article  Google Scholar 

  • Grover, S., Pea, R., & Cooper, S. (2015). Designing for deeper learning in a blended computer science course for middle school students. Computer Science Education, 25(2), 199–237. https://doi.org/10.1080/08993408.2015.1033142.

    Article  Google Scholar 

  • Hall, K., & Wilensky, U. (2017). Ecosystem stability. Evanston, IL: Center for Connected Learning and Computer-Based Modeling, Northwestern University.

  • Hammer, D., & Elby, A. (2004). On the form of a personal epistemology, 169–190. https://doi.org/10.4324/9780203424964.

  • Hatfield, D. L. (2015). The right kind of telling: An analysis of feedback and learning in a journalism epistemic game. International Journal of Gaming and Computer-Mediated Simulations, 7(2), 1–23.

    Article  Google Scholar 

  • Israel, M., Pearson, J. N., Tapia, T., Wherfel, Q. M., & Reese, G. (2015). Supporting all learners in school-wide computational thinking: A cross-case qualitative analysis. Computers & Education, 82, 263–279. https://doi.org/10.1016/j.compedu.2014.11.022.

    Article  Google Scholar 

  • Kafai, Y. (1995). Minds in play: Video game design as a context for children’s learning. Hillsdale, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Knight, S., Arastoopour, G., Shaffer, D. W., Buckingham Shum, S., & Littleton, K. (2014). Epistemic networks for epistemic commitments. In Proceedings of the International Conference of the Learning Sciences. Boulder, CO.

  • Knuth, D. E. (1985). Algorithmic thinking and mathematical thinking. The American Mathematical Monthly, 92(3), 170–181.

    Article  Google Scholar 

  • Koh, K. H., Basawapatna, A., Nickerson, H., & Repenning, A. (2014a). Real time assessment of computational thinking. Proceedings of IEEE Symposium on Visual Languages and Human-Centric Computing, VL/HCC, 49–52. https://doi.org/10.1109/VLHCC.2014.6883021.

  • Koh, K. H., Nickerson, H., & Basawapatna, A. (2014b). Early validation of computational thinking pattern analysis. In Proceedings of the 2014 ITICSE. Uppsala, Sweden. https://doi.org/10.1145/2591708.2591724.

  • Lee, I., Martin, F., Denner, J., Coulter, B., Allan, W., Erickson, J., et al. (2011). Computational thinking for youth in practice. ACM Inroads, 2(1), 32. https://doi.org/10.1145/1929887.1929902.

    Article  Google Scholar 

  • Lund, K., & Burgess, C. (1996). Producing high-dimensional semantic spaces from lexical co-occurrence. Behavior Research Methods, Instruments, & Computers, 28(2), 203–208.

    Article  Google Scholar 

  • Moreno-León, J., Robles, G., & Román-González, M. (2015). Dr. Scratch: Automatic analysis of scratch projects to assess and foster computational thinking. RED. Revista de Educación a Distancia, 15(46), 1–23. https://doi.org/10.6018/red/46/10.

    Article  Google Scholar 

  • Moreno-León, J., Harteveld, C., Román-González, M., & Robles, G. (2017). On the automatic assessment of computational thinking skills: A comparison with human experts. Conference on Human Factors in Computing Systems (CHI), 2788–2795. https://doi.org/10.1145/3027063.3053216.

  • Nash, P., & Shaffer, D. W. (2013). Epistemic trajectories: Mentoring in a game design practicum. Instructional Science, 41(4), 745–771.

    Article  Google Scholar 

  • National Academy of Sciences, National Academy of Engineering, & Institute of Medicine. (2007). Rising above the gathering storm: Energizing and employing America for a brighter economic future. Washington, DC: The National Academies Press. https://doi.org/10.17226/11463.

    Book  Google Scholar 

  • National Research Council. (2010). Report of a workshop on the scope and nature of computational thinking. Washington, DC: The National Academies Press. https://doi.org/10.17226/12840.

    Book  Google Scholar 

  • National Research Council. (2012). A framework for K-12 science education: Practices, crosscutting concepts, and core ideas. Washington, DC: The National Academies Press. https://doi.org/10.17226/13165.

    Book  Google Scholar 

  • NGSS Lead States. (2013). Next generation science standards: for states, by states. Washington, DC: The National Academies Press.

    Google Scholar 

  • Papert, S. (1980). Mindstorms: Children, computers, and powerful ideas. New York, NY: Basic Books.

    Google Scholar 

  • Papert, S. (1996). An exploration in the space of mathematics educations. International Journal of Computers for Mathematical Learning, 1(1), 95–123.

    Google Scholar 

  • Papert, S., & Harel, I. (1991). Situating constructionism. In Constructionism (pp. 1–11). Ablex Publishing Corporation. https://doi.org/10.1111/1467-9752.00269.

  • Portelance, D. J., & Bers, M. U. (2015). Code and tell: Assessing young children’s learning of computational thinking using peer video interviews with ScratchJr. In Proceedings of the 14th international conference on interaction design and children (pp. 271–274). https://doi.org/10.1145/2771839.2771894.

    Chapter  Google Scholar 

  • Schanzer, E., Fisler, K., & Krishnamurthi, S. (2018). Assessing bootstrap: Algebra students on scaffolded and unscaffolded word problems. In Proceedings of the 49th ACM technical symposium on computer science education - SIGCSE’18 (pp. 8–13). Baltimore, Maryland: ACM Press. https://doi.org/10.1145/3159450.3159498.

  • Seiter, L., & Foreman, B. (2013). Modeling the learning progressions of computational thinking of primary grade students. In Proceedings of the Ninth Annual International ACM Conference on International Computing Education Research (pp. 59–66). https://doi.org/10.1145/2493394.2493403.

    Chapter  Google Scholar 

  • Sengupta, P., & Wilensky, U. (2009). Learning electricity with NIELS: Thinking with electrons and thinking in levels. International Journal of Computers for Mathematical Learning, 14(1), 21–50.

    Article  Google Scholar 

  • Sengupta, P., Kinnebrew, J. S., Basu, S., Biswas, G., & Clark, D. (2013). Integrating computational thinking with K-12 science education using agent-based computation: A theoretical framework. Education and Information Technologies, 18(2), 351–380. https://doi.org/10.1007/s10639-012-9240-x.

    Article  Google Scholar 

  • Shaffer, D. W. (2017). Quantitative ethnography. Madison, WI: Cathcart Press.

    Google Scholar 

  • Shaffer, D. W., & Resnick, M. (1999). Thick authenticity: New media and authentic learning. Journal of Interactive Learning Research, 10(2), 195–215.

    Google Scholar 

  • Shaffer, D. W., & Ruis, A. R. (2017). Epistemic network analysis: A worked example of theory-based learning analytics. In Handbook of learning analytics and educational data mining (p. in press).

    Google Scholar 

  • Shaffer, D. W., Hatfield, D., Svarovsky, G., Nash, P., Nulty, A., Bagley, E. A., et al. (2009). Epistemic network analysis: A prototype for 21st century assessment of learning. The International Journal of Learning and Media, 1(1), 1–21.

    Article  Google Scholar 

  • Shaffer, D. W., Borden, F., Srinivasan, A., Saucerman, J., Arastoopour, G., Collier, W., … Frank, K. A. (2015). The nCoder: a technique for improving the utility of inter-rater reliability statistics (Epistemic Games Group Working Paper No. 2015–01).

  • Shaffer, D. W., Collier, W., & Ruis, A. R. (2016). A tutorial on epistemic network analysis: Analyzing the structure of connections in cognitive, social, and interaction data. Journal of Learning Analytics, 3(3), 9–45.

    Article  Google Scholar 

  • Sherin, B. L. (2001). A comparison of programming languages and algebraic notation as expressive languages for physics. International Journal of Computers for Mathematical Learning, 6(1), 1–61. https://doi.org/10.1023/A:1011434026437.

    Article  Google Scholar 

  • Sherin, B. L. (2006). Common sense clarified: The role of intuitive knowledge in physics problem solving. Journal of Research in Science Teaching, 43(6), 535–555. https://doi.org/10.1002/tea.20136.

    Article  Google Scholar 

  • Shute, V. J., Sun, C., & Asbell-Clarke, J. (2017). Demystifying computational thinking. Educational Research Review, 22, 142–158. https://doi.org/10.1016/j.edurev.2017.09.003.

    Article  Google Scholar 

  • Siebert-Evenstone, A. L., Arastoopour Irgens, G., Collier, W., Swiecki, Z., Ruis, A. R., & Shaffer, D. W. (2017). In search of conversational grain size: Modeling semantic structure using moving stanza windows. Journal of Learning Analytics, 4(3), 123–139.

    Article  Google Scholar 

  • Swanson, H., Arastoopour Irgens, G., Bain, C., Hall, K., Woods, P., Rogge, C., et al. (2018). Characterizing computational thinking in high school science. In International Conference of the Learning Sciences. London, UK.

    Google Scholar 

  • Swanson, H., Anton, G., Bain, C., Horn, M., & Wilensky, U. (2019). Introducing and Assessing Computational Thinking in the Secondary Science Classroom. In Computational Thinking Education (pp. 99-117). Springer, Singapore.

  • Wagh, A., Cook-Whitt, K., & Wilensky, U. (2017). Bridging inquiry-based science and constructionism: Exploring the alignment between students tinkering with code of computational models and goals of inquiry. Journal of Research in Science Teaching, 54(5), 615–641. https://doi.org/10.1002/tea.21379.

    Article  Google Scholar 

  • Webb, D. C. (2010). Troubleshooting assessment: An authentic problem solving activity for it education. Procedia-Social and Behavioral Sciences, 9, 903–907.

    Article  Google Scholar 

  • Weintrop, D., Beheshti, E., Horn, M. S., Orton, K., Trouille, L., Jona, K., & Wilensky, U. (2014). Interactive assessment tools for computational thinking in high school STEM classrooms. In Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST, 136 LNICST, 22–25. https://doi.org/10.1007/978-3-319-08189-2_3.

    Chapter  Google Scholar 

  • Weintrop, D., Beheshti, E., Horn, M., Orton, K., Jona, K., Trouille, L., & Wilensky, U. (2016). Defining computational thinking for mathematics and science classrooms. Journal of Science Education and Technology, 25(1), 127–147. https://doi.org/10.1007/s10956-015-9581-5.

    Article  Google Scholar 

  • Werner, L., Denner, J., & Campe, S. (2012). The fairy performance assessment: Measuring computational thinking in middle school. In Proceedings of the 43rd ACM Technical Symposium on Computer Science Education - SIGCSE ‘12, 215–220. https://doi.org/10.1145/2157136.2157200.

    Chapter  Google Scholar 

  • Wilensky, U. (1991). Abstract meditation on the concrete and concrete implications for mathematics education. In Constructionism (pp. 193–204).

    Google Scholar 

  • Wilensky, U. (1998). NetLogo Virus model. Evanston, IL: Center for Connected Learning and Computer-Based Modeling, Northwestern University.

    Google Scholar 

  • Wilensky, U. (1999). NetLogo. Evanston, IL: Center for Connected Learning and Computer-Based Modeling, Northwestern University.

    Google Scholar 

  • Wilensky, U. (2003). Statistical mechanics for secondary school: The GasLab multi-agent modeling toolkit. International Journal of Computers for Mathematical Learning, 8(1), 1–41.

    Article  Google Scholar 

  • Wilensky, U., & Papert, S. (2010). Restructurations: Reformulating knowledge disciplines through new representational forms. In Constructionism (pp. 1–14) Paris, France.

    Google Scholar 

  • Wilensky, U., & Stroup, W. (1999). Learning through participatory simulations: Network-based design for systems learning in classrooms. Proceedings of the 1999 Conference on Computer Support for Collaborative Learning, (1), 80.

    Google Scholar 

  • Wilensky, U., & Stroup, W. (2002). Participatory simulations: Envisioning the networked classroom as a way to support systems learning for all. In Presented at the Annual meeting of the American Research Education Association, New Orleans, LA.

  • Wilensky, U., Novak, M., & Wagh, A. (2012). MSIM evolution unit. Evanston, IL: Center for Connected Learning and Computer-Based Modeling, Northwestern University.

    Google Scholar 

  • Wing, J. M. (2006). Computational thinking. Communications of the ACM, 49(3), 33. https://doi.org/10.1145/1118178.1118215.

    Article  Google Scholar 

  • Wing, J. M. (2017). Computational thinking’s influence on research and education for all. Italian Journal of Educational Technology, 25(2), 7–14. https://doi.org/10.17471/2499-4324/922.

    Article  Google Scholar 

  • Zhong, B., Wang, Q., Chen, J., & Li, Y. (2016). An exploration of three-dimensional integrated assessment for computational thinking. Journal of Educational Computing Research, 53(4), 562–590. https://doi.org/10.1177/0735633115608444.

    Article  Google Scholar 

Download references

Acknowledgments

This study was funded in part by the National Science Foundation (grants CNS-1138461, CNS-1441041, and DRL-1020101), the Spencer Foundation (Award #201600069), the National Science Foundation (DRL-1661036, DRL-1713110), the Wisconsin Alumni Research Foundation, and the Office of the Vice Chancellor for Research and Graduate Education at the University of Wisconsin-Madison. The opinions, findings, and conclusions do not reflect the views of the funding agencies, cooperating institutions, or other individuals. Thank you to Daisy Rutstein from SRI for providing feedback on the development of the pre-post assessments.

Author information

Authors and Affiliations

  1. Clemson University, Clemson, SC, USA

    Golnaz Arastoopour Irgens

  2. Northwestern University, Evanston, IL, USA

    Sugat Dabholkar, Connor Bain, Philip Woods, Kevin Hall, Hillary Swanson, Michael Horn & Uri Wilensky

Authors
  1. Golnaz Arastoopour Irgens
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sugat Dabholkar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Connor Bain
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Philip Woods
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kevin Hall
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hillary Swanson
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Michael Horn
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Uri Wilensky
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Golnaz Arastoopour Irgens.

Ethics declarations

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and national research committee. Informed consent was obtained from all individual participants included in the study.

Conflict of Interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

Table 4 Rubrics used to score student pre and post assessments. Students received one point for each item on the rubric.
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Arastoopour Irgens, G., Dabholkar, S., Bain, C. et al. Modeling and Measuring High School Students’ Computational Thinking Practices in Science. J Sci Educ Technol 29, 137–161 (2020). https://doi.org/10.1007/s10956-020-09811-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10956-020-09811-1

Keywords

Search

Navigation