Assessing and giving feedback to students’ written work: closing the gap between expert and novice raters
List of Authors
  • Jariah Mohd. Jan , Jerry, Clarence , Samuel, Moses

Keyword
  • Assessing writing, Expert raters, Novice raters, Conceptual mental model, Writing knowledge and skill, Training workshop

Abstract
  • The present study focuses on the feasibility of narrowing the gap in the differences between expert and novice raters in terms of expertise in assessing and evaluating writing in a training/workshop through the use of a tentative mental model which was developed in a prior research. The conceptual mental model of assessing writing deployed in this study helped the researchers to understand how expert raters could be differentiated from the novice raters in terms of their mental cognitive processes. This paper reports the understanding which was translated into a training module and trialled out in the workshop. Five participants out of twenty five course participants who came for the workshop were interviewed and the preliminary probe into the usefulness of the mental model in training yielded positive results. The findings of this study indicated that with proper intervention, beginning teachers can be trained to be “expert raters” themselves by closely following the tentative mental model recommended in the study.

Reference
  • Baba, K. (2009). Aspects of lexical proficiency in writing summaries in a foreign language. Journal of Second Language Writing 18(3), 191-208.
    Bukta, K. (2007). Assessment of written performance: Tracing raters’ decision making process. Portal Linguarum 8, 21-40.
    Bracewell, R. J. (1994). Withered wisdom: A reply to Dobrin. In P. Smagorinsky (ed.), Speaking about Writing (pp. 290-294). Thousand Oaks, CA: Sage.
    Cardelle, M. & Corno, L. (1981). Effects on second language learning of variations in written feedback on homework assignments. TESOL Quarterly 15(3), 251-261.
    Flower, L. S. (1979). Writer-based prose: A cognitive basis for problems in writing. College English 41(1), 19-37.
    Grabe, M. & Kaplan, R. B. (1998). Theory & practice of writing: An applied linguistic perspective. New York: Longman.
    Greene, S. & Higgins, L. (1994). “Once Upon a Time”: The use of retrospective accounts in building theory in composition. In P. Smagorinsky (ed.). Speaking about Writing (115-140). Thousand Oaks, CA: Sage.
    Hassebrock, F. & Prietula, M. J. (1992). A Protocol-based coding schemes for the analysis of medical reasoning. Int. J. Man-Machine Studies 37, 613-652.
    Hattie, J. & Timperley, H. (2007). The power of feedback. Review of Educational Research 77(1), 81-112.
    Hoffman, R. R., Neville, K. & Fowlkes, J. (2009). Using cognitive task analysis to explore issues in the procurement of intelligent decision support systems, Cognition, Technology and Work 11(1), 57-70.
    Inbar-Lourie, O. (2008). Constructing a language assessment knowledge base: A focus on language assessment courses. Language Testing 25(3), 385-402.
    Kuhar, M. (1998). Critical thinking – a framework for problem solving in the occupational setting, AAOHN Journal 46, 80-81.
    Merriam, S. B. (2001). Case study research in education: A qualitative approach. San Francisco: Jossey-Bass.
    Militellon, G. L. & Hoffman, R. (2008). Perspectives on cognitive task analysis. USA: Psychology Press.
    Nelson, M. M. & Schunn, D. S. (2009). The nature of feedback: how different types of peer feedback affect writing performance. Instructional Science 37(4), 375-401.
    Newell, A. & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice-Hall.
    Orellana, M. F. (1995). Literacy as a gendered social practice: Tasks, texts, talk, and take-up. Reading Research Quarterly 30(4), 674-708.
    Weigle, S. C. (1999). Investigating rater/prompt interactions in writing assessment: Quantitative and qualitative Approaches. Assessing Writing 6(2), 145-178.
    Wells, G. & Chang-Wells, G. L. (1992). Constructing knowledge together: Classrooms as centers of inquiry and literacy. Portsmouth, NH: Heinemann.
    Wollman-Bonilla, J. E. (2000). Teaching science writing to first graders: Genre learning and recontextualization. Research in the Teaching of English 35(1), 35-65.
    Yin, R. K. (1994). Case Study Research: Design and Methods (2nd Ed.). Thousand Oaks, CA: Sage.
    Zellermayer, M. (1989). The study of teachers’ written feedback to student writing: Changes in the theoretical considerations and the expansion of research contexts. Instructional Science 18, 145-165.