Abstract
Although formative peer assessment (FPA) has become a prevailing learning activity in different educational settings, there are not enough suggestions on how to structure it with instructional supports in online environments to optimize its benefits. Therefore, this study aims to propose design guidelines for the development of an effective online FPA environment with instructional scaffolds in the context of writing activities at high schools. To this end, an online FPA environment was designed on the basis of an assessment model for regulated learning and teachers’ and students’ experiences. It was evaluated and refined three times. The formative evaluation findings suggested designing an online FPA environment with preparatory activities, information resources, goal setting and planning, anonymity, criteria form, sustainable and supportive dialog, motivational elements, and an easy-to-use interface. As a result, 11 design guidelines were produced. Overall, this research provides a better understanding of the essential design elements of online FPA environments.
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11423-022-10115-x/MediaObjects/11423_2022_10115_Fig1_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11423-022-10115-x/MediaObjects/11423_2022_10115_Fig2_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11423-022-10115-x/MediaObjects/11423_2022_10115_Fig3_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11423-022-10115-x/MediaObjects/11423_2022_10115_Fig4_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11423-022-10115-x/MediaObjects/11423_2022_10115_Fig5_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11423-022-10115-x/MediaObjects/11423_2022_10115_Fig6_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11423-022-10115-x/MediaObjects/11423_2022_10115_Fig7_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11423-022-10115-x/MediaObjects/11423_2022_10115_Fig8_HTML.png)
Similar content being viewed by others
References
Adachi, C., Tai, J., & Dawson, P. (2018). A framework for designing, implementing, communicating and researching peer assessment. Higher Education Research & Development, 37(3), 453–467. https://doi.org/10.1080/07294360.2017.1405913
Allal, L. (2010). Assessment and the regulation of learning. In P. Peterson, E. Baker, & B. McGaw (Eds.), International encyclopedia of education (pp. 348–352). Oxford: Elsevier
Allal, L. (2020). Assessment and the co-regulation of learning in the classroom. Assessment in Education: Principles, Policy & Practice, 27(4), 332–349. https://doi.org/10.1080/0969594X.2019.1609411
Alqassab, M., Strijbos, J. W., & Ufer, S. (2018). Training peer-feedback skills on geometric construction tasks: Role of domain knowledge and peer-feedback levels. European Journal of Psychology of Education, 33(1), 11–30. https://doi.org/10.1007/s10212-017-0342-0
Anderson, T., & Shattuck, J. (2012). Design-based research: A decade of progress in education research? Educational Researcher, 41(1), 16–25. https://doi.org/10.3102/0013189X11428813
Andrade, H. L. (2013). Classroom assessment in the context of learning theory and research. In J. H. McMillan (Ed.), Sage handbook of research on classroom learning (pp. 17–34). California: Sage Publications
Andrade, H., & Valtcheva, A. (2009). Promoting learning and achievement through self-assessment. Theory into Practice, 48(1), 12–19. https://doi.org/10.1080/00405840802577544
Ashenafi, M. M. (2017). Peer-assessment in higher education–twenty-first century practices, challenges and the way forward. Assessment & Evaluation in Higher Education, 42(2), 226–251. https://doi.org/10.1080/02602938.2015.1100711
Azevedo, R., Johnson, A., Chauncey, A., & Graesser, A. (2011). Use of hypermedia to assess and convey self-regulated learning. In B. J. Zimmerman,Schunk D. H. (Eds.) (Ed.), Handbook of self-regulation of learning and performance (pp. 102–121). New York, NY: Routledge
Baker, K. M. (2016). Peer review as a strategy for improving students’ writing process. Active Learning in Higher Education, 17(3), 179–192. https://doi.org/10.1177/1469787416654794
Bennett, S., Dawson, P., Bearman, M., Molloy, E., & Boud, D. (2017). How technology shapes assessment design: Findings from a study of university teachers. British Journal of Educational Technology, 48(2), 672–682. https://doi.org/10.1111/bjet.12439
Black, P., Harrison, C., Lee, C., Marshall, B., & Wiliam, D. (2003). Assessment for learning: Putting it into practice. England: McGraw-Hill Education
Black, P., & Wiliam, D. (2009). Develo** the theory of formative assessment. Educational Assessment, Evaluation and Accountability, 21(1), 5–31. https://doi.org/10.1007/s11092-008-9068-5
Bloxham, S., & Campbell, L. (2010). Generating dialogue in assessment feedback: Exploring the use of interactive cover sheets. Assessment & Evaluation in Higher Education, 35(3), 291–300. https://doi.org/10.1080/02602931003650045
Bolzer, M., Strijbos, J. W., & Fischer, F. (2015). Inferring mindful cognitive-processing of peer‐feedback via eye‐tracking: Role of feedback‐characteristics, fixation‐durations and transitions. Journal of Computer Assisted Learning, 31(5), 422–434. https://doi.org/10.1111/jcal.12091
Bourgeois, L. (2016). Supporting students’ learning: From teacher regulation to co-regulation. In L. Allal, & D. Laveault (Eds.), Assessment for learning: Meeting the challenge of implementation (pp. 345–363). Heidelberg: Springer
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa
Brown, G. T., Peterson, E. R., & Yao, E. S. (2016). Student conceptions of feedback: Impact on self-regulation, self‐efficacy, and academic achievement. British Journal of Educational Psychology, 86(4), 606–629. https://doi.org/10.1111/bjep.12126
Butler, D. L., & Winne, P. H. (1995). Feedback and self-regulated learning: A theoretical synthesis. Review of Educational Research, 65(3), 245–281. https://doi.org/10.3102/00346543065003245
Chen, T. (2016). Technology-supported peer feedback in ESL/EFL writing classes: A research synthesis. Computer Assisted Language Learning, 29(2), 365–397. https://doi.org/10.1080/09588221.2014.960942
Dawson, P., & Henderson, M. (2017). How does technology enable scaling up assessment for learning?. In D. Carless, S. Bridges, C. Ka Yuk Chan, & R. Glofcheski (Eds.), Scaling up assessment for learning in higher education (pp. 209–222). Singapore: Springer
Deeley, S. J. (2018). Using technology to facilitate effective assessment for learning and feedback in higher education. Assessment & Evaluation in Higher Education, 43(3), 439–448. https://doi.org/10.1080/02602938.2017.1356906
Dix, A., Finlay, J., Abowd, G. D., & Beale, R. (2004). Human-computer interaction. Essex: Pearson Education
Dochy, F. J. R. C., Segers, M., & Sluijsmans, D. (1999). The use of self-, peer and co-assessment in higher education: A review. Studies in Higher Education, 24(3), 331–350. https://doi.org/10.1080/03075079912331379935
Dumas, J. S., & Redish, J. C. (1999). A practical guide to usability testing. Portland, OR: Intellect Books
Erdinç, O., & Lewis, J. R. (2013). Psychometric evaluation of the T-CSUQ: The Turkish version of the computer system usability questionnaire. International Journal of Human-Computer Interaction, 29(5), 319–326. https://doi.org/10.1080/10447318.2012.711702
Hadwin, A. F., Järvelä, S., & Miller, M. (2011). Self-regulated, co-regulated, and socially shared regulation of learning. In B. J. Zimmerman,Schunk D. H. (Eds.) (Ed.), Handbook of self-regulation of learning and performance (pp. 65–84). New York, NY: Routledge
Howard, C. D., Barrett, A. F., & Frick, T. W. (2010). Anonymity to promote peer feedback: Pre-service teachers’ comments in asynchronous computer-mediated communication. Journal of Educational Computing Research, 43(1), 89–112. https://doi.org/10.2190/EC.43.1.f
Huisman, B., Saab, N., van den Broek, P., & van Driel, J. (2019). The impact of formative peer feedback on higher education students’ academic writing: A Meta-Analysis. Assessment & Evaluation in Higher Education, 44(6), 863–880. https://doi.org/10.1080/02602938.2018.1545896
Gielen, M., & De Wever, B. (2015). Structuring the peer assessment process: A multilevel approach for the impact on product improvement and peer feedback quality. Journal of Computer Assisted Learning, 31(5), 435–449. https://doi.org/10.1111/jcal.12096
Gielen, S., Peeters, E., Dochy, F., Onghena, P., & Struyven, K. (2010). Improving the effectiveness of peer feedback for learning. Learning and Instruction, 20(4), 304–315. https://doi.org/10.1016/j.learninstruc.2009.08.007
Gikandi, J. W., Morrow, D., & Davis, N. E. (2011). Online formative assessment in higher education: A review of the literature. Computers & Education, 57(4), 2333–2351. https://doi.org/10.1016/j.compedu.2011.06.004
Gikandi, J. W., & Morrow, D. (2016). Designing and implementing peer formative feedback within online learning environments. Technology, Pedagogy and Education, 25(2), 153–170. https://doi.org/10.1080/1475939X.2015.1058853
Graham, S., & Harris, K. R. (2000). The role of self-regulation and transcription skills in writing and writing development. Educational Psychologist, 35(1), 3–12. https://doi.org/10.1207/S15326985EP3501_2
Graham, S., Gillespie, A., & McKeown, D. (2013). Writing: Importance, development, and instruction. Reading and Writing, 26(1), 1–15. https://doi.org/10.1007/s11145-012-9395-2
Kauffman, H. (2015). A review of predictive factors of student success in and satisfaction with online learning. Research in Learning Technology, 23, 1–13. https://doi.org/10.3402/rlt.v23.26507
Kaufman, J. H., & Schunn, C. D. (2011). Students’ perceptions about peer assessment for writing: Their origin and impact on revision work. Instructional Science, 39(3), 387–406. https://doi.org/10.1007/s11251-010-9133-6
Kim, M. (2009). The impact of an elaborated assessee’s role in peer assessment. Assessment & Evaluation in Higher Education, 34(1), 105–114. https://doi.org/10.1080/02602930801955960
Kim, M., & Ryu, J. (2013). The development and implementation of a web-based formative peer assessment system for enhancing students’ metacognitive awareness and performance in ill-structured tasks. Educational Technology Research and Development, 61(4), 549–561. https://doi.org/10.1007/s11423-012-9266-1
Kollar, I., & Fischer, F. (2010). Peer assessment as collaborative learning: A cognitive perspective. Learning and Instruction, 20(4), 344–348. https://doi.org/10.1016/j.learninstruc.2009.08.005
Krings, R., Jacobshagen, N., Elfering, A., & Semmer, N. K. (2015). Subtly offending feedback. Journal of Applied Social Psychology, 45(4), 191–202. https://doi.org/10.1111/jasp.12287
Lewis, J. R. (1995). IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction, 7(1), 57–78. https://doi.org/10.1080/10447319509526110
Li, L. (2017). The role of anonymity in peer assessment. Assessment & Evaluation in Higher Education, 42(4), 645–656. https://doi.org/10.1080/02602938.2016.1174766
Martono, F., & Salam, U. (2017). Students’ learning in asynchronous discussion forums: A meta-analysis. International Journal of Information and Communication Technology Education (IJICTE), 13(1), 48–60
Narciss, S. (2008). Feedback strategies for interactive learning tasks. In J. M. Spector, M. D. Merrill, J. Van Merrienboer, & M. P. Driscoll (Eds.), Handbook of research on educational communications and technology (3rd ed., pp. 125–144). New York, NY: Lawrence Erlbaum Associates
Nicol, D. (2012). Resituating feedback from the reactive to the proactive. In D. Boud, & E. Molloy (Eds.), Feedback in higher and professional education: Understanding it and doing it well (pp. 34–49). London: Routledge
Nicol, D. (2014). Guiding principles for peer review: unlocking learners’ evaluative skills. In C. Kreber, C. Amdersen, N. Entwistle, & J. McArthur (Eds.), Advances and innovations in university assessment and feedback (pp. 197–224). Edinburgh: Edinburgh University Press
Nicol, D., Thomson, A., & Breslin, C. (2014). Rethinking feedback practices in higher education: A peer review perspective. Assessment & Evaluation in Higher Education, 39(1), 102–122. https://doi.org/10.1080/02602938.2013.795518
O-Donnell, A. M., & Top**, K. (1998). Peers assessing peers: Possibilities and problems. In K. Top**, & S. Ehly (Eds.), Peer-assisted learning (pp. 255–278). Mahwah, NJ: Lawrence Erlbaum
Pachler, N., Daly, C., Mor, Y., & Mellar, H. (2010). Formative e-assessment: Practitioner cases. Computers & Education, 54(3), 715–721. https://doi.org/10.1016/j.compedu.2009.09.032
Panadero, E. (2016). Is it safe? Social, interpersonal, and human effects of peer assessment: A review and future directions. In G. T. L. Brown, & L. R. Harris (Eds.), Handbook of social and human conditions in assessment (pp. 247–266). New York: Routledge
Panadero, E., Jonsson, A., & Alqassab, M. (2018). Providing formative peer feedback. In A. Lipnevich, & J. K. Smith (Eds.), The Cambridge handbook of instructional feedback (pp. 409–431). Cambridge: Cambridge University Press
Panadero, E., Jonsson, A., & Botella, J. (2017). Effects of self-assessment on self-regulated learning and self-efficacy: Four meta-analyses. Educational Research Review, 22, 74–98. https://doi.org/10.1016/j.edurev.2017.08.004
Panadero, E., Jonsson, A., & Strijbos, J. W. (2016). Scaffolding self-regulated learning through self-assessment and peer assessment: Guidelines for classroom implementation. In D. Laveault, & L. Allal (Eds.), Assessment for learning: Meeting the challenge of implementation (pp. 311–326). Dordrecht: Springer International Publishing
Parr, J. M. (2013). Classroom assessment in writing. In J. H. McMillan (Ed.), Sage handbook of research on classroom learning (pp. 489–501). California: Sage Publications
Patchan, M. M., & Schunn, C. D. (2015). Understanding the benefits of providing peer feedback: How students respond to peers’ texts of varying quality. Instructional Science, 43(5), 591–614. https://doi.org/10.1007/s11251-015-9353-x
Peters, O., Körndle, H., & Narciss, S. (2018). Effects of a formative assessment script on how vocational students generate formative feedback to a peer’s or their own performance. European Journal of Psychology of Education, 33(1), 117–143. https://doi.org/10.1007/s10212-017-0344-y
Pintrich, P. R. (1995). Understanding self-regulated learning. New Directions for Teaching and Learning, 63, 3–12. https://doi.org/10.1002/tl.37219956304
Pintrich, P. R. (2000). The role of goal orientation in self-regulated learning. In M. Boekaerts, P. R. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 451–502). San Diego, CA: Academic Press
Reigeluth, C. M., & Frick, T. W. (1999). Formative research: A methodology for creating and improving design theories. In C. M. Reigeluth (Ed.), Instructional-design theories and models (Volume II): A new paradigm of instructional theory (pp. 633–652). Mahwah, NJ: Lawrence Erlbaum Associates
Reigeluth, C. M., & An, Y. (2009). Theory building. In C. M. Reigeluth, & A. A. Carr-Chellman (Eds.), Instructional design theories and models (Volume III): Building a common knowledge base (pp. 365–386). New York, NY: Routledge
Reinholz, D. (2016). The assessment cycle: A model for learning through peer assessment. Assessment & Evaluation in Higher Education, 41(2), 301–315. https://doi.org/10.1080/02602938.2015.1008982
Richey, R. C., Klein, J. D., & Nelson, W. A. (2004). Developmental research: Studies of instructional design and development. In D. H. Jonassen (Ed.), Handbook of research for educational communications and technology (2nd ed., pp. 1099–1130). Mahwah, NJ: Lawrence Erlbaum Associates Publishers
Sadler, D. R. (1998). Formative assessment: Revisiting the territory. Assessment in Education: Principles, Policy & Practice, 5(1), 77–84. https://doi.org/10.1080/0969595980050104
Schunk, D. H. (2012). Learning theories (6. ed.). Boston, MA: Pearson Education Inc
Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189. https://doi.org/10.3102/0034654307313795
Strijbos, J. W., Ochoa, T. A., Sluijsmans, D. M., Segers, M. S., & Tillema, H. H. (2009). Fostering interactivity through formative peer assessment in (web-based) collaborative learning environments. In C. Mourlas, N. Tsianos, & P. Germanakod (Eds.), Cognitive and emotional processes in web-based education: Integrating human factors and personalization (pp. 375–395). Hershey, PA: Information Science Reference
Strobl, C., Ailhaud, E., Benetos, K., Devitt, A., Kruse, O., Proske, A., & Rapp, C. (2019). Digital support for academic writing: A review of technologies and pedagogies. Computers & Education, 131, 33–48. https://doi.org/10.1016/j.compedu.2018.12.005
Tenório, T., Bittencourt, I. I., Isotani, S., & Silva, A. P. (2016). Does peer assessment in online learning environments work? A systematic review of the literature. Computers in Human Behavior, 64, 94–107. https://doi.org/10.1016/j.chb.2016.06.020
Timmis, S., Broadfoot, P., Sutherland, R., & Oldfield, A. (2016). Rethinking assessment in a digital age: Opportunities, challenges and risks. British Educational Research Journal, 42(3), 454–476. https://doi.org/10.1002/berj.3215
Top**, K. (2009). Peer assessment. Theory into Practice, 48(1), 20–27. https://doi.org/10.1080/00405840802577569
van den Akker, J. (1999). Principles and methods of development research. In van den J. Akker, N. Nieveen, R. M. Branch, K. L. Gustafson, & T. Plomp (Eds.), Design methodology and developmental research in education and training (pp. 1–14). The Netherlands: Kluwer Academic Publishers
van Popta, E., Kral, M., Camp, G., Martens, R. L., & Simons, P. R. J. (2017). Exploring the value of peer feedback in online learning for the provider. Educational Research Review, 20, 24–34. https://doi.org/10.1016/j.edurev.2016.10.003
van Zundert, M. J., Könings, K. D., Sluijsmans, D. M. A., & van Merriënboer, J. J. G. (2012). Teaching domain-specific skills before peer assessment skills is superior to teaching them simultaneously. Educational Studies, 38(5), 541–557. https://doi.org/10.1080/03055698.2012.654920
Wang, X. M., Hwang, G. J., Liang, Z. Y., & Wang, H. Y. (2017). Enhancing students’ computer programming performances, critical thinking awareness and attitudes towards programming: An online peer-assessment attempt. Educational Technology & Society, 20(4), 58–68
Wanner, T., & Palmer, E. (2018). Formative self-and peer assessment for improved student learning: the crucial factors of design, teacher participation and feedback. Assessment & Evaluation in Higher Education, 43(7), 1032–1047. https://doi.org/10.1080/02602938.2018.1427698
Webb, N. M., & Mastergeorge, A. (2003). Promoting effective hel** behavior in peer-directed groups. International Journal of Educational Research, 39(1–2), 73–97. https://doi.org/10.1016/S0883-0355(03)00074-0
Wichmann, A., Funk, A., & Rummel, N. (2018). Leveraging the potential of peer feedback in an academic writing activity through sense-making support. European Journal of Psychology of Education, 33(1), 165–184. https://doi.org/10.1007/s10212-017-0348-7
Winstone, N. E., Nash, R. A., Parker, M., & Rowntree, J. (2017). Supporting learners’ agentic engagement with feedback: A systematic review and a taxonomy of recipience processes. Educational Psychologist, 52(1), 17–37. https://doi.org/10.1080/00461520.2016.1207538
Zheng, L., Zhang, X., & Cui, P. (2020). The role of technology-facilitated peer assessment and supporting strategies: A meta-analysis. Assessment & Evaluation in Higher Education, 45(3), 372–386. https://doi.org/10.1080/02602938.2019.1644603
Zhu, Q., & Carless, D. (2018). Dialogue within peer feedback processes: clarification and negotiation of meaning. Higher Education Research & Development, 37(4), 883–897. https://doi.org/10.1080/07294360.2018.1446417
Zimmerman, B. J., & Moylan, A. R. (2009). Self-regulation: Where metacognition and motivation intersect. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Handbook of metacognition in education (pp. 299–315). New York, NY: Routledge
Zimmerman, B. J., & Risemberg, R. (1997). Becoming a self-regulated writer: A social cognitive perspective. Contemporary Educational Psychology, 22(1), 73–101. https://doi.org/10.1006/ceps.1997.0919
Funding
Partial financial support was received under the scope of the Council of Higher Education-Scientific Human Resources Development program in Turkey.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest/competing interests
The authors have no relevant financial or non-financial interests to disclose.
Ethics approval
This study was approved by the Human Subject Ethics committee of Middle East Technical University (Approval number: 2018-EGT-050) and the Ministry of National Education in Turkey.
Informed consent
Informed consent was obtained from all individual participants included in the study.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Alemdag, E., Yildirim, Z. Design and development of an online formative peer assessment environment with instructional scaffolds. Education Tech Research Dev 70, 1359–1389 (2022). https://doi.org/10.1007/s11423-022-10115-x
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11423-022-10115-x