Eli Review provides a digital scaffold for review and revision, and one of its innovations is the way comments are captured, used, and reported in the system. That innovation emerges from three core ideas:
- Giving helpful feedback helps writers improve.
- Comments are a genre instructors need to teach.
- Comments are a performance record of students’ learning.
Within the app, instructors and students can view and interact with comments in several ways to see trends in comment volume, frequency, and helpfulness. Using these displays, Eli makes it easy to get a high-level view of reviewers’ comments.
Our own work as teachers and researchers as well as our work with several research teams has inspired harder questions about reviewers’ comments:
- How well do comment counts capture the amount of feedback students receive?
- Since peer learning dynamics depend students’ perceptions of getting back what they put in, how close to reciprocity are reviewers in term of giving and getting comments?
- Are comments aligned with appropriate criteria?
- Do writers actually use feedback when they revise?
Answering these questions means being a researcher in your own classroom. To do that, you can download the comment digest that Eli Review provides from individual review activities or your entire course. Then, grab a copy of our Google Sheet Template and follow the instructions for merging the comment digest into the spreadsheet. You can also watch the video below for step-by-step instructions:
This hack can help you use Eli as a lever for heavier lifting in several teaching scenarios:
- Analyze the comment digest to identify trends to share with students when you debrief the review. For example, in a couple of courses we’ve researched, average word count was correlated with helpfulness. Longer comments (about 25 words or one sentence) tend to be more helpful because (perhaps) longer comments give writers enough detail to understand the reviewers’ suggestion. By connecting comment length with what writers need to revise, you can help reviewers set a concrete goal for giving more helpful (longer) comments.
- Analyze the substance of student comments to understand their focus. Although it takes a little bit of time, coding comments to identify their focus can be instructive. Use the comment focus report, you can assess the balance of peer editing versus higher-order comments.
- Involve students in self-assessment using these reports. Share the Google sheet with your students. To make the data anonymous, you can use “find and replace” to change names to ids. Ask them to reflect on their work as reviewers by self-assessing. This assignment is structured as a progress report to be completed at midterm and at the end of the semester.
Are you doing research on your class using Eli data? If you are and if you try this hack, we’d love to hear from you. (Be careful with non-anonymized student data.)
