Menu

Eli Review Learning Resources

Coaching Review Tasks

Eli Review provides both course and individual level reports on feedback given and feedback received. This brief tutorial helps instructors use these four reports, presenting strategies and methods for coaching better commenting and revisions:

Theories behind coaching writing

Coaching is one way teachers can show students how to learn. When coaching, instructors tell students what they were looking for, what patterns they see, and what actions should follow from those insights.

Coaching engages students in metacognition*. While instructors debrief, students see and hear how habits of mind play out in real time.

Coaching is important “pre-game,” “in-game,” and “post-game.” As described in our detailed look at evidence-based teaching using formative feedback, pre-game and in-game coaching helps students make adjustments that improve their learning. Formative feedback is distinguished from post-game (summative feedback) in two important ways:

  1. Timing: Formative feedback is provided while there is still time to make changes. Summative feedback comes at the end of a process through grades or test scores.
  2. Quality: Formative feedback offers descriptive as well as suggestive advice. Summative feedback is normative—placing the writing/writer into a ranking system. Summative feedback may include suggestions, but since the time for improvement is past, students cannot act on them.

Eli Review focuses on formative feedback. It helps instructors with pre-game and in-game coaching so students practice skills before being graded on them. In Eli, writers get feedback that is timely, consistent, and actionable. This formative feedback for writers also provides formative feedback for instructors who are actively tracking learning while it’s happening.

Time is of the essence with formative feedback. Managing all these details in time for students to benefit from an instructor’s insights is challenging, even with analytics. The next two sections describe strategies for figuring out what to say as well as the best places to say it.

*Learn more about metacognition: for the tip of the iceberg, see Yancey’s Reflection in the Writing Classroom, 1998; Bransford et al, How People Learn, 2000; Bean’s Engaging Ideas, Part 3, 2011; for practical applications, see University of Michigan Sweetland Writing Center’s Metacognition-Cultivating Reflection for College instructors; University of Virginia’s The Coaching Role for K-12 instructors.

Strategies for what (and when) to coach

After students have completed a review, instructors need to know four things:

  1. How much effort did students put into the review?
  2. How well did they understand the criteria when evaluating others’ work?
  3. How helpful were their comments?
  4. What’s next?

Each review will not emphasize each issue equally. Pick and choose the issues that fit your learning goals. If you are thinking like a researcher in your own classroom, answering these questions with Eli’s analytics can help you design your coaching strategies.

How much effort did students put into the review?

Students’ engagement in the process of giving and getting feedback is an indication of learning.

To assess how much effort students put into a review, use these analytics:

These analytics indicate the level and nature of students’ efforts. These displays also exert positive social pressure. When instructors project them, students can see how their engagement in giving and getting feedback compares to the rest of the class. For an example of how to encourage longer comments, see the comment volume blog.

How well did they understand the criteria when evaluating others’ work?

Building students’ confidence about the value of peer review depends on developing a shared understanding of the criteria. Students need to trust themselves and their peers to correctly recognize when a draft meets criteria and when it doesn’t. Repeated opportunities to practice applying the criteria help students gain confidence that peer are giving them accurate feedback.

trait-id-debrief

Within a single review, instructors can use the following analytics to explore how accurately students align their feedback with criteria.

By paying attention to how well reviewers understand criteria, instructors can increase the quality of peer feedback. If writers get and follow criteria-driven comments, their writing will improve.

How helpful were students’ comments?

Comments are a critical part of the feedback students use in Eli to plan revision. We have student materials and instructor materials that provide strategies for teaching helpful comments.

In Eli, helpfulness ratings are a way for writers to tell reviewers the extent to which their comments helped them revise. For helpfulness ratings to be powerful, writers need to be “stingy with the stars.”

In the "Feedback and Helpfulness" table we can see how feedback given by students was rated over the duration of the course.

If instructors have prepped students to give helpful feedback as reviewers and to rate helpful feedback as writers, Eli’s helpfulness rating analytics provide a way to monitor the quality of students’ comments:

What’s the next step?

Good coaches know how to use analytics to inform what they do in class. (Want a basketball analogy? Read this blog post.) After instructors examine students’ efforts, their alignment with criteria, and their helpfulness, instructors have to decide what to do.

Methods for Delivering Coaching Advice

Coaching requires knowing about content, skills, motivation, and expected changes in performance over time. Coaching is also about finding the right time and place to talk with students about their performance.

1. Convey insights about the whole class to all students.

With Eli’s real-time analytics, instructors can decide to address students’ performance through

  1. Extemporaneous comments delivered while students are working through the review and before they leave to work on revision plans (see Ann Shiver-McNair’s Live Feed article)
  2. Prepared comments delivered after students have completed a review task but before they’ve submitted revision plans
    • Spoken in class or in synchronous online environment
    • Spoken via screencast
    • Written via class blog, e-mail, announcement, etc.

2. Convey insights to individual students.

With Eli’s aggregated view of all the comments writers received and reviewers gave, instructors can address students’ performance through

revision-plan

Digging Deeper

Formative feedback is messier than summative feedback, and pre-game coaching of students’ work as writers and reviewers is challenging. Contact us if you’d like assistance interpreting the trends in your courses. If you’re looking for other tutorials, consider:

Other resources for learning about teaching with Eli Review include:

[resource-table]

You can also find human support to help you learn how to use Eli Review effectively. You can contact Melissa Meeks, Eli’s director of professional development, and:

Have any additional questions about how to use Eli Review? Contact us at [email protected].

upward arrow button