If you’re interested in doing research in your own classrooms, revision plans in Eli Review offer a great site of inquiry. Revision plans capture the decisions writers are making between getting feedback and revising. A revision plans shows which comments writers selected, how helpful those comments were, and how the writer plans to follow the advice. Students generate a substantial amount of data while assembling revision plans, which instructors can access through the engagement data downloads in single reviews or in the analytics feature.
Texas Tech University PhD candidate Andrea Beaudin outlines an approachable method for examining students’ learning in revision plans, though her own work focuses on instructor commenting. She outlined her method in a poster that won the Chair’s Awards for Best Poster for Best Research Design at the College Conference on Composition and Communication in Tampa in April 2015.
The poster and its accompanying paper “Usability of Instructor Feedforward” describe Beaudin’s usability research, which compares how students comprehend instructors’ inline comments versus their veedback (video feedback). Her research investigates the technical affordances that spur students to use feedback as they revise. She describes her interests this way:
As an instructor, one of my concerns is whether my students “get” the feedback they receive, and if so, are they able to feed it forward into future composing practices … [but since much of the foundational research on feedback uptake was published] students are less likely to receive handwritten comments and more likely to receive embedded text commentary through a learning management system such as Blackboard or Canvas, or, in some situations, receive commentary as audio or video. I wanted to ascertain whether–and if so, how–factors such as media and learning styles as well as feedback content and design may affect what students “get” out of feedback.
In particular, she examines the inline vs video feedback from the perspective of the “The 5Es of usability,” introduced by Whitney Quesenbery:
- Effective: How completely and accurately the work or experience is completed or goals reached
- Efficient: How quickly this work can be completed
- Engaging: How well the interface draws the user into the interaction and how pleasant and satisfying it is to use
- Error Tolerant: How well the product prevents errors and can help the user recover from mistakes that do occur
- Easy to Learn: How well the product supports both the initial orientation and continued learning throughout the complete lifetime of use
Beaudin’s application of Quesenbery’s usability principles to the uptake of instructor commenting is interesting because so much of the existing research shows how easy it is for students to ignore instructor’s comments. Understanding how medium affects the way students use feedback in an important line of inquiry.
We’re also interested in how her method can be applied to the uptake of peer comments. As noted above, Eli’s revision plan is very intentional about how it asks students to select comments from review, add them to the plan, then prioritize selected comments, and finally add notes reflecting on how they will revise accordingly. If desired, instructors can add comments to writers’ plans too, so it’s possible to investigate both the uptake of peer and instructor feedback from revision plans.
Although users have given us lots of anecdotal evidence about the technical affordances, Beaudin’s methodology offers a manageable usability study using only 5 writers (5 participants is “usually sufficient to identify 55-80% of usability issues,” according to Beaudin). If you’d like to conduct usability research on revision plans following Beaudin’s methodology or a different one, contact us at [email protected] so that we can help you gather the data you need and connect you with other researchers in our community.
About Andrea Beaudin: Andrea is a doctoral candidate at Texas Tech University, completing a dissertation which is a UX (User Experience) analysis of feedback media. Her professional background as a graphic designer has influenced both her pedagogy and scholarship in considering how multimodality (the creation of, reception of, and interaction with) affects student learning. She can be reached via email, Twitter, or academia.edu.
