Peer Review and Scoring Guides

Printer friendly version: http://www.calstatela.edu/academic/english/comp/PeerReviewAndScoringGuides.pdf

Scoring guides can ensure consistency in grading practices and transparency in the grading process by serving as a shared understanding of both the requirements of an assignment and the expectations of the instructor. Made available to students early in the term the scoring guide helps students understand what is expected of them and how they will be evaluated. But a scoring guide is not merely a convenience for students and teachers. It represents the expectations of the discourse community towards which students are moving. True members don’t merely join, they define what it means to belong.

While we might want to believe that once we distribute the scoring guide for a course and perhaps discuss it in class, students will know what to expect and perform accordingly, the reality is very different. As experienced writing teachers know, inexperienced writers (and sometimes experienced writers too!) often struggle to see problems in their own writing, though, they are usually able to see the same problems in other people’s writing.

This inability to see the log in our own eye is the basis for peer review and forms the foundation for an effective class activity that uses small groups to peer review essays, an activity commonly known as read around groups, or RAGs. The innovation is that the peer review groups are first subjected to the “norming” process familiar to holistic scorers and then are asked to score or grade the anonymous essays that they read. Here’s an overview of the process:

1. On the first day, students are “normed” using the scoring guide and sample essays that represent the range of the scoring guide.
2. On the second day, students are placed in small groups (three or four students per group).
3. Each group collects the essay drafts written by the group members and passes them to another group.
4. Each group member then reads each essay and assigns it a score or grade based on the scoring guide. Each group member is also encouraged to take brief notes that justify the score or grade assigned.
5. When each member in the group has read each essay, the group discusses each essay and the score or grade it should be given.
6. After the score or grade has been assigned, the group passes its essays to the next group, receives a new set of essays, and repeats steps 3 and 4.
   

This combined peer review/scoring guide “norming” activity usually requires parts of two class periods and many variations are possible. Students might be asked to bring multiple copies of their draft to facilitate group reading. The groups might read the essays aloud. Group scores and comments might be recorded and returned to the writer. Instead of scoring or grading, the group might vote on the best essay in each set, justifying their choice by referring to specific language in the scoring guide.

While students might focus on the voting and scoring, as any participant in holistic scoring knows the real benefit is the discussion of how writing is valued. The scoring guide provides students with language to talk about what makes writing interesting and effective. As they apply this language to the writing of others, the silent work of incorporation moves them closer to the academic discourse community.


What is a RAG?
A Read-Around Group (RAG) is a small-group activity commonly used in composition classes. Students in small groups read and respond to drafts of one another’s essays. Besides gaining feedback on their own writing, students gain insight into the assignment and learn useful strategies from other writers.

Helpful Hints

• Have students identify their papers with their initials or a code instead of with their name.
  
• Have each group select a group member to serve as note-taker.
  
• Give the groups a specific task to perform; don’t overload them with multiple tasks.
  
• Set time limits for the reading and stick to them.
  
• Consider having the groups read the papers aloud. (You will need to plan for the added noise.)
  
• Consider how comfortable you are with having students score, grade, or select the best paper. If you are uncomfortable with these tasks, consider having students select papers that best exemplify specific criteria in the scoring guide.
  
• Be sure to allow enough time for a whole class discussion.
  

Assessment and Scoring Guides

Printer friendly version: http://www.calstatela.edu/academic/english/comp/AssessmentAndScoringGuides.pdf

Beloved of accrediting bodies, scoring guides (or rubrics as they are sometimes called) are all the rage. A well-designed scoring guide can streamline the grading process by making clear to both the instructor and students what constitutes different levels of work. Scoring guides can ensure consistency in grading practices and transparency in the grading process by serving as a shared understanding of both the requirements of a project and the expectations of the instructor.


What exactly is a scoring guide? Briefly, it is a set of scoring guidelines for evaluating student work. The scoring guide indicates both the criteria that will be used to judge the work and distinguishes between different levels of performance. By identifying the criteria to be used to judge the work, a good scoring guide clearly ariculates the instructor's expectations. And by distinguishing between different levels of performance a good scoring guide helps students determine how to meet those expectations. And when a scoring guide is shared with students early in the process, students become better judges of the quality of their own work and in peer review the work of others. The scoring guide also provides the instructor with language to give feedback on the quality of work.


All four composition courses at CSULA have scoring guides. The ENGL 095 and 096 scoring guides are used for evaluating end-of-the-quarter portfolios, while the ENGL 101 and ENGL 102 scoring guides are used for assigning grades to individual essays.

095 and 096 Scoring Guides

Since grades for ENGL 095 and ENGL 096 are determined by the student’s portfolio scores, the scoring guides for these courses represent not simply an instructor’s idiosyncratic do’s and don’ts, but program and university expectations. Most experienced 095 and 096 instructors distribute the course scoring guide to students, spend class time reading and interpreting it, organize “norming” sessions to help students apply it, and use language from the scoring guide in their comments on student writing.

This last point is perhaps the most unappreciated, but possibly the most important. After having spent some time in class discussing the scoring guide, an instructor might note on an 096 student’s draft that at present it does “not respond to the text at all (see ‘3’ score),” or that it “consists mostly of generalizations without support (see ‘3’ score).” The scoring guide gives both student and instructor a shared language of assessment, which the instructor can invoke in his or her comments.

101 and 102 Scoring Guides

Assigning grades to individual essays is not an objective activity and yet it is also not purely subjective and idiosyncratic. Experienced teachers of writing tend to apply similar criteria and arrive at similar assessments and grader consistency is improved when a common scoring guide is used. The scoring guides for ENGL 101 and ENGL 102 ensure that students in different classes with different instructors will receive comparable grades for comparable work.

As in ENGL 095 and 096, the scoring guide is most effective when it is made available to students early in the course, is the subject of class discussion, and is specifically referred to in instructor comments.

Using Scoring Guides

• Make the scoring guide available to students, preferably early in the term
  
• Devote class time to discussing the scoring guide
  
• Provide students with sample essays that exemplify the scoring guide
  
• Consider “norming” students by having them read and score sample student essays (or each other’s essays) as part of an extended peer review
  
• Refer to the specific language of the scoring guide when commenting on student essays