Log in

Design and development of an online formative peer assessment environment with instructional scaffolds

  • Development Article
  • Published:
Educational technology research and development Aims and scope Submit manuscript

Abstract

Although formative peer assessment (FPA) has become a prevailing learning activity in different educational settings, there are not enough suggestions on how to structure it with instructional supports in online environments to optimize its benefits. Therefore, this study aims to propose design guidelines for the development of an effective online FPA environment with instructional scaffolds in the context of writing activities at high schools. To this end, an online FPA environment was designed on the basis of an assessment model for regulated learning and teachers’ and students’ experiences. It was evaluated and refined three times. The formative evaluation findings suggested designing an online FPA environment with preparatory activities, information resources, goal setting and planning, anonymity, criteria form, sustainable and supportive dialog, motivational elements, and an easy-to-use interface. As a result, 11 design guidelines were produced. Overall, this research provides a better understanding of the essential design elements of online FPA environments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Germany)

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Adachi, C., Tai, J., & Dawson, P. (2018). A framework for designing, implementing, communicating and researching peer assessment. Higher Education Research & Development, 37(3), 453–467. https://doi.org/10.1080/07294360.2017.1405913

    Article  Google Scholar 

  • Allal, L. (2010). Assessment and the regulation of learning. In P. Peterson, E. Baker, & B. McGaw (Eds.), International encyclopedia of education (pp. 348–352). Oxford: Elsevier

    Chapter  Google Scholar 

  • Allal, L. (2020). Assessment and the co-regulation of learning in the classroom. Assessment in Education: Principles, Policy & Practice, 27(4), 332–349. https://doi.org/10.1080/0969594X.2019.1609411

    Article  Google Scholar 

  • Alqassab, M., Strijbos, J. W., & Ufer, S. (2018). Training peer-feedback skills on geometric construction tasks: Role of domain knowledge and peer-feedback levels. European Journal of Psychology of Education, 33(1), 11–30. https://doi.org/10.1007/s10212-017-0342-0

    Article  Google Scholar 

  • Anderson, T., & Shattuck, J. (2012). Design-based research: A decade of progress in education research? Educational Researcher, 41(1), 16–25. https://doi.org/10.3102/0013189X11428813

    Article  Google Scholar 

  • Andrade, H. L. (2013). Classroom assessment in the context of learning theory and research. In J. H. McMillan (Ed.), Sage handbook of research on classroom learning (pp. 17–34). California: Sage Publications

    Google Scholar 

  • Andrade, H., & Valtcheva, A. (2009). Promoting learning and achievement through self-assessment. Theory into Practice, 48(1), 12–19. https://doi.org/10.1080/00405840802577544

    Article  Google Scholar 

  • Ashenafi, M. M. (2017). Peer-assessment in higher education–twenty-first century practices, challenges and the way forward. Assessment & Evaluation in Higher Education, 42(2), 226–251. https://doi.org/10.1080/02602938.2015.1100711

    Article  Google Scholar 

  • Azevedo, R., Johnson, A., Chauncey, A., & Graesser, A. (2011). Use of hypermedia to assess and convey self-regulated learning. In B. J. Zimmerman,Schunk D. H. (Eds.) (Ed.), Handbook of self-regulation of learning and performance (pp. 102–121). New York, NY: Routledge

    Google Scholar 

  • Baker, K. M. (2016). Peer review as a strategy for improving students’ writing process. Active Learning in Higher Education, 17(3), 179–192. https://doi.org/10.1177/1469787416654794

    Article  Google Scholar 

  • Bennett, S., Dawson, P., Bearman, M., Molloy, E., & Boud, D. (2017). How technology shapes assessment design: Findings from a study of university teachers. British Journal of Educational Technology, 48(2), 672–682. https://doi.org/10.1111/bjet.12439

    Article  Google Scholar 

  • Black, P., Harrison, C., Lee, C., Marshall, B., & Wiliam, D. (2003). Assessment for learning: Putting it into practice. England: McGraw-Hill Education

    Google Scholar 

  • Black, P., & Wiliam, D. (2009). Develo** the theory of formative assessment. Educational Assessment, Evaluation and Accountability, 21(1), 5–31. https://doi.org/10.1007/s11092-008-9068-5

  • Bloxham, S., & Campbell, L. (2010). Generating dialogue in assessment feedback: Exploring the use of interactive cover sheets. Assessment & Evaluation in Higher Education, 35(3), 291–300. https://doi.org/10.1080/02602931003650045

    Article  Google Scholar 

  • Bolzer, M., Strijbos, J. W., & Fischer, F. (2015). Inferring mindful cognitive-processing of peer‐feedback via eye‐tracking: Role of feedback‐characteristics, fixation‐durations and transitions. Journal of Computer Assisted Learning, 31(5), 422–434. https://doi.org/10.1111/jcal.12091

    Article  Google Scholar 

  • Bourgeois, L. (2016). Supporting students’ learning: From teacher regulation to co-regulation. In L. Allal, & D. Laveault (Eds.), Assessment for learning: Meeting the challenge of implementation (pp. 345–363). Heidelberg: Springer

    Chapter  Google Scholar 

  • Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa

    Article  Google Scholar 

  • Brown, G. T., Peterson, E. R., & Yao, E. S. (2016). Student conceptions of feedback: Impact on self-regulation, self‐efficacy, and academic achievement. British Journal of Educational Psychology, 86(4), 606–629. https://doi.org/10.1111/bjep.12126

    Article  Google Scholar 

  • Butler, D. L., & Winne, P. H. (1995). Feedback and self-regulated learning: A theoretical synthesis. Review of Educational Research, 65(3), 245–281. https://doi.org/10.3102/00346543065003245

    Article  Google Scholar 

  • Chen, T. (2016). Technology-supported peer feedback in ESL/EFL writing classes: A research synthesis. Computer Assisted Language Learning, 29(2), 365–397. https://doi.org/10.1080/09588221.2014.960942

    Article  Google Scholar 

  • Dawson, P., & Henderson, M. (2017). How does technology enable scaling up assessment for learning?. In D. Carless, S. Bridges, C. Ka Yuk Chan, & R. Glofcheski (Eds.), Scaling up assessment for learning in higher education (pp. 209–222). Singapore: Springer

    Chapter  Google Scholar 

  • Deeley, S. J. (2018). Using technology to facilitate effective assessment for learning and feedback in higher education. Assessment & Evaluation in Higher Education, 43(3), 439–448. https://doi.org/10.1080/02602938.2017.1356906

    Article  Google Scholar 

  • Dix, A., Finlay, J., Abowd, G. D., & Beale, R. (2004). Human-computer interaction. Essex: Pearson Education

    Google Scholar 

  • Dochy, F. J. R. C., Segers, M., & Sluijsmans, D. (1999). The use of self-, peer and co-assessment in higher education: A review. Studies in Higher Education, 24(3), 331–350. https://doi.org/10.1080/03075079912331379935

    Article  Google Scholar 

  • Dumas, J. S., & Redish, J. C. (1999). A practical guide to usability testing. Portland, OR: Intellect Books

    Google Scholar 

  • Erdinç, O., & Lewis, J. R. (2013). Psychometric evaluation of the T-CSUQ: The Turkish version of the computer system usability questionnaire. International Journal of Human-Computer Interaction, 29(5), 319–326. https://doi.org/10.1080/10447318.2012.711702

    Article  Google Scholar 

  • Hadwin, A. F., Järvelä, S., & Miller, M. (2011). Self-regulated, co-regulated, and socially shared regulation of learning. In B. J. Zimmerman,Schunk D. H. (Eds.) (Ed.), Handbook of self-regulation of learning and performance (pp. 65–84). New York, NY: Routledge

    Google Scholar 

  • Howard, C. D., Barrett, A. F., & Frick, T. W. (2010). Anonymity to promote peer feedback: Pre-service teachers’ comments in asynchronous computer-mediated communication. Journal of Educational Computing Research, 43(1), 89–112. https://doi.org/10.2190/EC.43.1.f

    Article  Google Scholar 

  • Huisman, B., Saab, N., van den Broek, P., & van Driel, J. (2019). The impact of formative peer feedback on higher education students’ academic writing: A Meta-Analysis. Assessment & Evaluation in Higher Education, 44(6), 863–880. https://doi.org/10.1080/02602938.2018.1545896

    Article  Google Scholar 

  • Gielen, M., & De Wever, B. (2015). Structuring the peer assessment process: A multilevel approach for the impact on product improvement and peer feedback quality. Journal of Computer Assisted Learning, 31(5), 435–449. https://doi.org/10.1111/jcal.12096

    Article  Google Scholar 

  • Gielen, S., Peeters, E., Dochy, F., Onghena, P., & Struyven, K. (2010). Improving the effectiveness of peer feedback for learning. Learning and Instruction, 20(4), 304–315. https://doi.org/10.1016/j.learninstruc.2009.08.007

    Article  Google Scholar 

  • Gikandi, J. W., Morrow, D., & Davis, N. E. (2011). Online formative assessment in higher education: A review of the literature. Computers & Education, 57(4), 2333–2351. https://doi.org/10.1016/j.compedu.2011.06.004

    Article  Google Scholar 

  • Gikandi, J. W., & Morrow, D. (2016). Designing and implementing peer formative feedback within online learning environments. Technology, Pedagogy and Education, 25(2), 153–170. https://doi.org/10.1080/1475939X.2015.1058853

    Article  Google Scholar 

  • Graham, S., & Harris, K. R. (2000). The role of self-regulation and transcription skills in writing and writing development. Educational Psychologist, 35(1), 3–12. https://doi.org/10.1207/S15326985EP3501_2

    Article  Google Scholar 

  • Graham, S., Gillespie, A., & McKeown, D. (2013). Writing: Importance, development, and instruction. Reading and Writing, 26(1), 1–15. https://doi.org/10.1007/s11145-012-9395-2

    Article  Google Scholar 

  • Kauffman, H. (2015). A review of predictive factors of student success in and satisfaction with online learning. Research in Learning Technology, 23, 1–13. https://doi.org/10.3402/rlt.v23.26507

    Article  Google Scholar 

  • Kaufman, J. H., & Schunn, C. D. (2011). Students’ perceptions about peer assessment for writing: Their origin and impact on revision work. Instructional Science, 39(3), 387–406. https://doi.org/10.1007/s11251-010-9133-6

    Article  Google Scholar 

  • Kim, M. (2009). The impact of an elaborated assessee’s role in peer assessment. Assessment & Evaluation in Higher Education, 34(1), 105–114. https://doi.org/10.1080/02602930801955960

    Article  Google Scholar 

  • Kim, M., & Ryu, J. (2013). The development and implementation of a web-based formative peer assessment system for enhancing students’ metacognitive awareness and performance in ill-structured tasks. Educational Technology Research and Development, 61(4), 549–561. https://doi.org/10.1007/s11423-012-9266-1

    Article  Google Scholar 

  • Kollar, I., & Fischer, F. (2010). Peer assessment as collaborative learning: A cognitive perspective. Learning and Instruction, 20(4), 344–348. https://doi.org/10.1016/j.learninstruc.2009.08.005

    Article  Google Scholar 

  • Krings, R., Jacobshagen, N., Elfering, A., & Semmer, N. K. (2015). Subtly offending feedback. Journal of Applied Social Psychology, 45(4), 191–202. https://doi.org/10.1111/jasp.12287

    Article  Google Scholar 

  • Lewis, J. R. (1995). IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction, 7(1), 57–78. https://doi.org/10.1080/10447319509526110

    Article  Google Scholar 

  • Li, L. (2017). The role of anonymity in peer assessment. Assessment & Evaluation in Higher Education, 42(4), 645–656. https://doi.org/10.1080/02602938.2016.1174766

    Article  Google Scholar 

  • Martono, F., & Salam, U. (2017). Students’ learning in asynchronous discussion forums: A meta-analysis. International Journal of Information and Communication Technology Education (IJICTE), 13(1), 48–60

    Article  Google Scholar 

  • Narciss, S. (2008). Feedback strategies for interactive learning tasks. In J. M. Spector, M. D. Merrill, J. Van Merrienboer, & M. P. Driscoll (Eds.), Handbook of research on educational communications and technology (3rd ed., pp. 125–144). New York, NY: Lawrence Erlbaum Associates

    Google Scholar 

  • Nicol, D. (2012). Resituating feedback from the reactive to the proactive. In D. Boud, & E. Molloy (Eds.), Feedback in higher and professional education: Understanding it and doing it well (pp. 34–49). London: Routledge

    Google Scholar 

  • Nicol, D. (2014). Guiding principles for peer review: unlocking learners’ evaluative skills. In C. Kreber, C. Amdersen, N. Entwistle, & J. McArthur (Eds.), Advances and innovations in university assessment and feedback (pp. 197–224). Edinburgh: Edinburgh University Press

    Google Scholar 

  • Nicol, D., Thomson, A., & Breslin, C. (2014). Rethinking feedback practices in higher education: A peer review perspective. Assessment & Evaluation in Higher Education, 39(1), 102–122. https://doi.org/10.1080/02602938.2013.795518

    Article  Google Scholar 

  • O-Donnell, A. M., & Top**, K. (1998). Peers assessing peers: Possibilities and problems. In K. Top**, & S. Ehly (Eds.), Peer-assisted learning (pp. 255–278). Mahwah, NJ: Lawrence Erlbaum

    Google Scholar 

  • Pachler, N., Daly, C., Mor, Y., & Mellar, H. (2010). Formative e-assessment: Practitioner cases. Computers & Education, 54(3), 715–721. https://doi.org/10.1016/j.compedu.2009.09.032

    Article  Google Scholar 

  • Panadero, E. (2016). Is it safe? Social, interpersonal, and human effects of peer assessment: A review and future directions. In G. T. L. Brown, & L. R. Harris (Eds.), Handbook of social and human conditions in assessment (pp. 247–266). New York: Routledge

    Google Scholar 

  • Panadero, E., Jonsson, A., & Alqassab, M. (2018). Providing formative peer feedback. In A. Lipnevich, & J. K. Smith (Eds.), The Cambridge handbook of instructional feedback (pp. 409–431). Cambridge: Cambridge University Press

    Chapter  Google Scholar 

  • Panadero, E., Jonsson, A., & Botella, J. (2017). Effects of self-assessment on self-regulated learning and self-efficacy: Four meta-analyses. Educational Research Review, 22, 74–98. https://doi.org/10.1016/j.edurev.2017.08.004

    Article  Google Scholar 

  • Panadero, E., Jonsson, A., & Strijbos, J. W. (2016). Scaffolding self-regulated learning through self-assessment and peer assessment: Guidelines for classroom implementation. In D. Laveault, & L. Allal (Eds.), Assessment for learning: Meeting the challenge of implementation (pp. 311–326). Dordrecht: Springer International Publishing

    Chapter  Google Scholar 

  • Parr, J. M. (2013). Classroom assessment in writing. In J. H. McMillan (Ed.), Sage handbook of research on classroom learning (pp. 489–501). California: Sage Publications

    Google Scholar 

  • Patchan, M. M., & Schunn, C. D. (2015). Understanding the benefits of providing peer feedback: How students respond to peers’ texts of varying quality. Instructional Science, 43(5), 591–614. https://doi.org/10.1007/s11251-015-9353-x

    Article  Google Scholar 

  • Peters, O., Körndle, H., & Narciss, S. (2018). Effects of a formative assessment script on how vocational students generate formative feedback to a peer’s or their own performance. European Journal of Psychology of Education, 33(1), 117–143. https://doi.org/10.1007/s10212-017-0344-y

    Article  Google Scholar 

  • Pintrich, P. R. (1995). Understanding self-regulated learning. New Directions for Teaching and Learning, 63, 3–12. https://doi.org/10.1002/tl.37219956304

    Article  Google Scholar 

  • Pintrich, P. R. (2000). The role of goal orientation in self-regulated learning. In M. Boekaerts, P. R. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 451–502). San Diego, CA: Academic Press

    Chapter  Google Scholar 

  • Reigeluth, C. M., & Frick, T. W. (1999). Formative research: A methodology for creating and improving design theories. In C. M. Reigeluth (Ed.), Instructional-design theories and models (Volume II): A new paradigm of instructional theory (pp. 633–652). Mahwah, NJ: Lawrence Erlbaum Associates

    Google Scholar 

  • Reigeluth, C. M., & An, Y. (2009). Theory building. In C. M. Reigeluth, & A. A. Carr-Chellman (Eds.), Instructional design theories and models (Volume III): Building a common knowledge base (pp. 365–386). New York, NY: Routledge

    Chapter  Google Scholar 

  • Reinholz, D. (2016). The assessment cycle: A model for learning through peer assessment. Assessment & Evaluation in Higher Education, 41(2), 301–315. https://doi.org/10.1080/02602938.2015.1008982

    Article  Google Scholar 

  • Richey, R. C., Klein, J. D., & Nelson, W. A. (2004). Developmental research: Studies of instructional design and development. In D. H. Jonassen (Ed.), Handbook of research for educational communications and technology (2nd ed., pp. 1099–1130). Mahwah, NJ: Lawrence Erlbaum Associates Publishers

    Google Scholar 

  • Sadler, D. R. (1998). Formative assessment: Revisiting the territory. Assessment in Education: Principles, Policy & Practice, 5(1), 77–84. https://doi.org/10.1080/0969595980050104

    Article  Google Scholar 

  • Schunk, D. H. (2012). Learning theories (6. ed.). Boston, MA: Pearson Education Inc

    Google Scholar 

  • Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189. https://doi.org/10.3102/0034654307313795

    Article  Google Scholar 

  • Strijbos, J. W., Ochoa, T. A., Sluijsmans, D. M., Segers, M. S., & Tillema, H. H. (2009). Fostering interactivity through formative peer assessment in (web-based) collaborative learning environments. In C. Mourlas, N. Tsianos, & P. Germanakod (Eds.), Cognitive and emotional processes in web-based education: Integrating human factors and personalization (pp. 375–395). Hershey, PA: Information Science Reference

    Chapter  Google Scholar 

  • Strobl, C., Ailhaud, E., Benetos, K., Devitt, A., Kruse, O., Proske, A., & Rapp, C. (2019). Digital support for academic writing: A review of technologies and pedagogies. Computers & Education, 131, 33–48. https://doi.org/10.1016/j.compedu.2018.12.005

    Article  Google Scholar 

  • Tenório, T., Bittencourt, I. I., Isotani, S., & Silva, A. P. (2016). Does peer assessment in online learning environments work? A systematic review of the literature. Computers in Human Behavior, 64, 94–107. https://doi.org/10.1016/j.chb.2016.06.020

    Article  Google Scholar 

  • Timmis, S., Broadfoot, P., Sutherland, R., & Oldfield, A. (2016). Rethinking assessment in a digital age: Opportunities, challenges and risks. British Educational Research Journal, 42(3), 454–476. https://doi.org/10.1002/berj.3215

    Article  Google Scholar 

  • Top**, K. (2009). Peer assessment. Theory into Practice, 48(1), 20–27. https://doi.org/10.1080/00405840802577569

    Article  Google Scholar 

  • van den Akker, J. (1999). Principles and methods of development research. In van den J. Akker, N. Nieveen, R. M. Branch, K. L. Gustafson, & T. Plomp (Eds.), Design methodology and developmental research in education and training (pp. 1–14). The Netherlands: Kluwer Academic Publishers

    Google Scholar 

  • van Popta, E., Kral, M., Camp, G., Martens, R. L., & Simons, P. R. J. (2017). Exploring the value of peer feedback in online learning for the provider. Educational Research Review, 20, 24–34. https://doi.org/10.1016/j.edurev.2016.10.003

    Article  Google Scholar 

  • van Zundert, M. J., Könings, K. D., Sluijsmans, D. M. A., & van Merriënboer, J. J. G. (2012). Teaching domain-specific skills before peer assessment skills is superior to teaching them simultaneously. Educational Studies, 38(5), 541–557. https://doi.org/10.1080/03055698.2012.654920

    Article  Google Scholar 

  • Wang, X. M., Hwang, G. J., Liang, Z. Y., & Wang, H. Y. (2017). Enhancing students’ computer programming performances, critical thinking awareness and attitudes towards programming: An online peer-assessment attempt. Educational Technology & Society, 20(4), 58–68

    Google Scholar 

  • Wanner, T., & Palmer, E. (2018). Formative self-and peer assessment for improved student learning: the crucial factors of design, teacher participation and feedback. Assessment & Evaluation in Higher Education, 43(7), 1032–1047. https://doi.org/10.1080/02602938.2018.1427698

    Article  Google Scholar 

  • Webb, N. M., & Mastergeorge, A. (2003). Promoting effective hel** behavior in peer-directed groups. International Journal of Educational Research, 39(1–2), 73–97. https://doi.org/10.1016/S0883-0355(03)00074-0

    Article  Google Scholar 

  • Wichmann, A., Funk, A., & Rummel, N. (2018). Leveraging the potential of peer feedback in an academic writing activity through sense-making support. European Journal of Psychology of Education, 33(1), 165–184. https://doi.org/10.1007/s10212-017-0348-7

    Article  Google Scholar 

  • Winstone, N. E., Nash, R. A., Parker, M., & Rowntree, J. (2017). Supporting learners’ agentic engagement with feedback: A systematic review and a taxonomy of recipience processes. Educational Psychologist, 52(1), 17–37. https://doi.org/10.1080/00461520.2016.1207538

    Article  Google Scholar 

  • Zheng, L., Zhang, X., & Cui, P. (2020). The role of technology-facilitated peer assessment and supporting strategies: A meta-analysis. Assessment & Evaluation in Higher Education, 45(3), 372–386. https://doi.org/10.1080/02602938.2019.1644603

    Article  Google Scholar 

  • Zhu, Q., & Carless, D. (2018). Dialogue within peer feedback processes: clarification and negotiation of meaning. Higher Education Research & Development, 37(4), 883–897. https://doi.org/10.1080/07294360.2018.1446417

    Article  Google Scholar 

  • Zimmerman, B. J., & Moylan, A. R. (2009). Self-regulation: Where metacognition and motivation intersect. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Handbook of metacognition in education (pp. 299–315). New York, NY: Routledge

    Google Scholar 

  • Zimmerman, B. J., & Risemberg, R. (1997). Becoming a self-regulated writer: A social cognitive perspective. Contemporary Educational Psychology, 22(1), 73–101. https://doi.org/10.1006/ceps.1997.0919

    Article  Google Scholar 

Download references

Funding

Partial financial support was received under the scope of the Council of Higher Education-Scientific Human Resources Development program in Turkey.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ecenaz Alemdag.

Ethics declarations

Conflict of interest/competing interests

The authors have no relevant financial or non-financial interests to disclose.

Ethics approval

This study was approved by the Human Subject Ethics committee of Middle East Technical University (Approval number: 2018-EGT-050) and the Ministry of National Education in Turkey.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alemdag, E., Yildirim, Z. Design and development of an online formative peer assessment environment with instructional scaffolds. Education Tech Research Dev 70, 1359–1389 (2022). https://doi.org/10.1007/s11423-022-10115-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11423-022-10115-x

Keywords

Navigation