Eli Review Learning Resources

Making the Process Count

Eli Review does not include a gradebook.

Eli is a workshop space where students improve their drafts alongside peers; it’s not showcase space where finished products are assessed. Using Eli’s analytics, instructors see evidence of learning and coach students accordingly.

The work students do in Eli Review as writers and reviewers, however, is valuable enough to grade.

Many instructors using Eli Review use contract grading. This document shares their policies, and you are welcome to add your own.

Consider making Eli Review tasks worth 10-15+% of the course grade.

If you are assigning rapid feedback iterations such that students are writing smaller bits, reviewing those bits for a few criteria, and revising some of those drafts using a revision plan and/or a resubmission, students will do a lot of work in Eli. For example, many courses end up with 40-50+ tasks, even in accelerated terms.

As described below, three approaches can be used for assessment:

Grading on Task Completion

Eli’s purpose is to help students get better at giving feedback and revising, and they need deliberate practice in order to improve. Grading a single instance of that practice on its quality penalizes students for engaging in the learning process. Instead, grade across time.

Eli’s Completion downloads can simplify grading. The Completion Report shows whether students submitted on-time, late, or not at all. The visual cues on screen or the .csv download makes grading completion across the term simple.

The completion report shows the status of students' submissions in each task.

 

Engagement

Assess engagement in giving feedback across multiple reviews.

This strategy works well for instructors who are emphasizing better commenting and revision. At the end of a project that has included multiple write-review-revise cycles or at the end of the term, instructors can use Eli’s Analytics to assess students’ engagement with the process. As a formative feedback tool, Eli measures “engagement” as completing assigned writing, review, and revision tasks as well as counting various aspects of that work (word count, quantity of comments, quantity of endorsed comments, etc). To simplify, instructors can get an overall sense from the “Engagement Data” analytics per student.

  1. Click “Roster” from anywhere in your course.
  2. Click the name of any student.
  3. Click the “Engagement Data” tab.
  4. Click Trend Graph > Word Count.

These analytics show how a student’s engagement compares to the top 30%, average, and bottom 30% of the class. For ease, instructors might decide that students who consistently fall in

NOTE: In a class full of similarly capable and engaged students, the difference between the top and bottom 30% may be negligible. So, the grading scale might go from B+ to A+.

In the image below, Matthew contributed as much feedback as the bottom 30% for the first four reviews, but then contributed as much as the top 30% for the last four. His grade for his contributions as a reviewer might be a B.

Student's personal word count trend across reviews compared to the top 30% and bottom 30%.

The other reports in the student-level analytics provide more ways to assess individual student engagement and quality.

Reflection

Students’ completion of review and revision tasks in Eli are acts of learning; asking them to reflect on that body of work is a way of seeing how explicit their knowledge is.

Ask students to reflect on their growth as writers.

Reflections about how writers have grown over the term are common in portfolios*. If students have been working on revision plans throughout the term, their course dashboards have a rich archive of the decisions they’ve made along the way. In an end-of-term reflection, instructors can offer a prompt along these lines:

Look back over the feedback you’ve received and your revision plans. What were the three most important comments you received from peers, and how did they affect the revisions you completed? Explain the comments, your reactions to them, why they were helpful, your revisions, and how these comments may impact your future work.

*See portfolio principles from NCTE and CCCC as well as resources from AAEEBL.

Ask students to reflect on their growth as reviewers.

Eli’s unique features in review tasks aggregate all the comments reviewers’ give peers in a single digest. The comment digest plus helpfulness ratings makes it easy for instructors to explicitly teach students to give helpful comments and to treat comments as a genre.  Through deliberate practice over time, reviewers should improve. If instructors have emphasized the qualities of helpful feedback, then students can look back over their work as reviewers and find evidence of their growth.

Because it is unusual to assign reflection specifically about review work, we offer several examples. Consider this directive prompt that asks reviewers to connect their performance with the criteria of helpful feedback:

Eli Review helps you become a better reviewer because the review task helps you read drafts according to criteria and then guides you in giving better feedback as you explain to writers what you see. Write a 300-word reflection that begins “My work in Eli Review helped me become a (slightly better/better/much better) reviewer because I improved my ability to (explain).” In your reflection, use evidence from the “Review Feedback” tab in the “review tasks” you’ve completed this term to show you improved in at least three of the following areas:

Asking students to argue for the grade their work as reviewers deserves can also facilitate meaningful reflection:

Eli Review’s analytics aggregate your helpfulness ratings over the whole term; on the “Review Feedback” tab in the “review tasks,” you can also access all the comments you’ve given to your peers this term. Write me a letter in which you argue for the grade you think your work as a reviewer deserves, and use your helpfulness rating and examples from the comments you gave in Eli to support your claim.

Another type of reflective prompt asks students to describe helpful feedback using only comments they’ve made this term:

We’ve spent a lot of time this term discussing the qualities of helpful feedback. Using only three comments you gave reviewers this term (locate them on the “Review Feedback” tab in the “review tasks”), write a set of guidelines for giving helpful feedback that would be appropriate for me to share with my students next term. Be sure to connect your advice to specific aspects of the comments and to explain why comments that follow your suggestions will be helpful for writers.


Digging Deeper

This tutorial covers ways to assess the work students complete in Eli. Related resources include: