Montcalm Community College Language Arts/Humanities department is emphasizing multiple drafts, peer feedback, and revision in English 100 and 101 this academic year (2016-2017). Three full-time faculty and 8-10 adjuncts are committed to improving student learning outcomes related to the writing process. The department selected Eli Review to help them create a better scaffold for feedback and revision, and their institutional purchase provides 680 students, including 75 fully online students, with access to the app. Although it’s just midterm, the program as a whole, and two courses in particular, are seeing great results.
Program-wide Use
We can already tell that more feedback rounds are happening. Currently, instructors in 12 sections have assigned:
- 45 writing tasks,
- 41 review tasks, and
- 10 revision plan tasks.
Montcalm’s usage pattern is very strong, particularly for a first semester. On average, each student has completed three online review tasks. Combined with the writing workshops instructors are doing outside the app, it’s clear that the department is on track to meet their goals for multiple drafts (Montcalm’s English 100 Goal 2C and English 101 Goal 1E).
Another goal relates to students providing peer feedback. It’s too soon to see improvement in the quality of feedback, but Eli’s analytics already show that students are engaging in giving feedback at different levels. In courses where 3-4 reviews have been assigned, three levels emerge:
- Highly engaged reviewers (top 30%) have offered just over 1,000 words in comments.
- Engaged reviewers (middle 40%) have offered around 300 words in comments.
- Weakly engaged reviewers (bottom 30%) have offered around 80 words in comments.
This pattern is expected but a little wider than with other courses we’ve analyzed. Usually, each level doubles the word count of the level below it. So far, highly engaged reviewers are writing comments with 3x the words as engaged reviewers and 10x the words as weakly engaged reviewers.
This snapshot of engagement helps instructors intervene with those who are struggling. Students putting forth the least effort in peer learning (writing the fewest words in comments) are missing out on opportunities to bring the criteria into focus; they aren’t getting better at talking about writing with their peers; and they aren’t likely to improve as writers. Closing the gap with direct instruction in how to give helpful feedback encourages all students to practice enough to improve.
F2F Students Are “Buying into Eli”
For students to invest in giving feedback, they have to see the value of the work, which includes using Eli. One of the face-to-face instructors is already noticing this change in her students.
Trisha Winn is a new adjunct at Montcalm who was hired after the on-campus training with Eli; she joined one of Eli’s online professional development workshops and has been a regular contributor to our email discussions about peer learning pedagogy. She shared this report about a recent peer workshop in her face-to-face class:
We did a two review/revision process this time. They were required to submit a sketch (very rough draft) and give feedback on big ideas and thesis support only (no worries about grammar or wording), then make a revision plan. I asked them to make sure to add notes to their revision plans this time, rather than just adding peer feedback comments. They revised into a complete rough draft, then had to do feedback and another revision plan. Students were given full participation points for each portion of this cycle if they did their work and their feedback on time. They received half points for completing the assignments late.
I’m not sure if the grading really helped. I’m positive it helped motivate students who were late—they didn’t just blow it off, they actually completed the work and didn’t complain about half points.
[S]tudents commented that the quality of feedback is indeed getting better! And they seem to be getting more discerning in their [helpfulness] star ratings.
But overall, I think they are just really buying into the whole Eli system. They like getting feedback, they like the prompts I give them for giving feedback, because they help them know what to look for. I am getting more specific with feedback requirements. I used trait identifications, star ratings, and asked them to highlight specific things and comment. I’m not sure what the take-away is, other than Eli is really working for my students, and their writing is improving. I think the more I learn about using Eli effectively, the more students will get out of it.
Online CC Students Say Eli is Easy and Inspiring
Professor and department chair Greta Skogseth reported her fully online students’ enthusiasm for Eli after their first review too. For online students, ease is super important. Greta posts links to Eli Review in Canvas (Montcalm’s learning management system) so that her students move seamlessly between the two apps. In Eli, students paste/upload their drafts and give comments. Then, they return to Canvas and complete their other assignments. After the first review in Eli, Greta assigned a discussion board where students reflected on their experience giving and receiving feedback in Eli. With permission, we share a selection of their comments below.
Trend 1: Eli Review is easy to use.
These first-semester, fully-online first-year students were able to get on board and start giving feedback quickly. Thumbs up!
Jordyn Kass-Yowtz: Eli Review is very helpful for these type of projects. It gave me directions on where to upload my things, and it was very easy to maneuver. I liked how when we had to review our peers papers they had the stars you could fill in, and with an explanation box underneath. I’ve never peer reviewed anything that way before, it was great. Also, the weren’t any malfunctions when loading my paper or doing the reviews, so that’s always a plus!
Samantha Hanes: Overall I think Eli Review is great tool for us online students. This is giving us an opportunity to get constructive criticism from more than one peer even though we are not in a classroom. I found Eli Review very easy to maneuver around and upload my paper.
Trend 2: My teacher’s review task in Eli helped me
In Eli, instructors create a review tasks by setting up questions that reviewers answer. When students say how much they benefitted from the review task and talk about features they liked such as star ratings with explanations, the app is part of what they like. Primarily, they like the way Greta’s questions helped them focus on the criteria.
Danni Welke: I found that I really liked Eli Review! I really liked how you could give the author a star rating and then explain why you awarded the author with that rating. I actually find this site to be more helpful rather than passing actual papers back and forth. It is so much harder in my opinion to pass papers, because usually there is no guide to what we are supposed to be looking for, other than the normal spelling, punctuation, and so on.
Emily Denny: I really like the way you can give feedback. I enjoy being able to give a star rating, but then being able to explain why I gave that rating. I also like that there are prompted things to critique them with. Almost like questions to ask myself about my peers writing to be able to give them proper feedback. Lastly, I like that I have 2 people reviewing my work to ensure I can receive the best feedback possible. Overall, I really like this as a review process.
Alec Hallwachs: I really liked Eli Review! It made the review process easy and it also made it easier to give better detail to the author. I liked that it allowed you to give a star rating and then the comments section so you could tell the author why you thought it was three or so stars and give examples of how you came to that conclusion. It helped break down each section that we were basing our review off of so we could give clear feedback.
Julia Hauenstein: With Eli Review, it gives you questions so you’re able to know exactly what you’re trying to look for in your peers paper to help them see what they need to fix, and the things they did right. Overall, this is a very helpful and user friendly reviewing process.
Trend 3: Eli makes me want to give better feedback next time.
Giving better feedback comes with practice. Part of that practice is getting used to using Eli as a reviewer, but the larger part is learning how to give helpful comments.
Samantha Hanes: This being only the second time for me reviewing a peer’s paper I hope I was able to do them justice by giving enough detailed ideas.
Michael Wojcik: Eli review was very helpful for me. It gave me great instructions on what to do when I created the account. The set-up of the site is great and very easy to navigate through. I’ve never had to do any peer review for people I don’t know especially people whose writing habits I don’t know. I could have probably given a little more detail in my final comment, which I wish I would have after submitting my review. All and all Eli review gets five stars from me!
Amanda Block: I was a little confused. I did not realize we were supposed grade with the stars so i do apologize to those i reviewed i will get it right next time. I did leave comments tho. I afterwards played around a little bit on the Eli website and feel like in the future it will be very useful now that I know how to used it and how to get around the website.
Two weeks after this initial feedback, Greta wrote to say that students were “were much more conservative with stars” for helpfulness ratings. She explained that comments rated 1 or 2 stars because they didn’t give writers what they needed to revise made reviewers “feel a bit ‘low’ this time around.”
In a group email chain, we talked about why a dip in helpfulness means students now have higher expectations for peer feedback. Students feel worse because they realize they need to give better comments! Awesome.
To turn their discouragement over giving unhelpful comments around, Greta has lots of options for teaching students how to give better feedback using Eli’s review task displays and downloads of comments. Because she can see their comments and the helpfulness ratings writers gave them (1 star to 5 stars), Greta can walk students through the parts of a strong/weak comments using a screencast or a handout. She can guide them to reflect on the comments they gave and help them set goals. She celebrate with them next week when their work as reviewers improve. Eureka! She gets to teach students to write comments because students are motivated to learn how to improve!
Our team loves hearing that the app’s ease-of-use lets students and teachers focus on what’s really important: better feedback and better revision. We’re excited to see how Montcalm CC’s patterns of use and engagement change over time. We’re confident they’ll continue getting great results and thrilled to be part of their success.
