Tim Amidon (Twitter, Facebook, LinkedIn) is an assistant professor in English at Colorado State University. He used Eli Review first in Spring 2015 but returned to it in Spring 2016 with a new sense of how Eli could help him get students talking about the higher order concerns he was always writing in marginal comments. His goal was to use Eli to “be wicked specific and nuanced about how [he responded] to writers as individuals while targeting aggregate trends in large group meeting/discussions.”
Tim shares the strategies he learned for framing the value of peer learning and designing review tasks.
What motivated you to try Eli Review?
I knew that Eli was built around the Vygotsky’s concept of the zone of proximal development, which posits that learners can do more after/when partnering with a peer who is slightly better at a task. In Fall 2016, I talked to Bill Hart-Davidson and Melissa Meeks about my experiences with Eli during Spring 2015. I knew that Eli offered more tools than I had made use of, and I wanted to improve how I was using Eli in my writing course.
After our conversation, Melissa worked with me to build a sandbox of tasks based on assignments I had shared. I began to better see how Eli scaffolds high-impact moments within the composing process, from brainstorm and peer-feedback to revision and submission.
The first time using any new strategy or tool can be difficult. What were your challenges, and how did you work through them?
Looking back, I had made some less successful choices in Spring 2015 about how to integrate the tool. Specifically, I had initially envisioned Eli as a peer-review tool, so I considered much of the composing process as unrelated to Eli. Students were consistently moving content in/out of the interface rather than using the interface as the digital space where we could share work and engage in co-learning activities.
In spring 2016, I wanted Eli to be the place where we could perform the collaborative, social work of composing and learning together. What I’ve come to understand is that Eli is much more than a review interface, but rather a learning technology that is built around the best research we have regarding the types of feedback and support interventions that instructors can leverage to foster student growth.
How did Eli help you accomplish your goals?
In trying Eli for a second time, one of my primary aims was to focus on higher order concerns like global organization because I have found that is where my students need the most support and attention:
- Claims may be strong, but use of evidence is weak.
- Use of evidence might be strong, but there is no discussion that frames how the audiences might understand that evidence.
- Claims are generic and bland despite integration of rich and sophisticated evidence.
Initial drafts from almost all students tend to have some dimension of organization, development, evidence integration that can be enriched or better coordinated, so I really wanted to focus on building prompts to help students revise these things that I normally comment on.
Melissa’s sandbox used trait identification and Likerts in ways that really go beyond the claim, evidence, and discussion model that I’ve been using to scaffold paragraph level concerns in argumentative writing. This different way of designing review-prompts gives reviewers more ways of describing the draft’s contents. That description within the review survey then becomes a map of possible revisions/additions that writers might consider.
Trait Identification: Tick the box if the report writer discusses these appeals to ethos that the article author uses to establish credibility and authority with an audience:
NOTE: Not all of these appeals are necessary; this list is meant to inspire you to consider whether additional details are needed.
- ways the article author brings and builds personal credibility
- ways the article author builds authority by citing reputable sources
- ways the article author reveals their moral character and values
- ways the article author shows that they have the best interests of readers in mind
- ways the article author accounts for different points of view or counterarguments and resolves contradictions
- an overall sense of how aggressively or artfully the article author uses ethos and avoids ethical fallacies (ad hominem, guilt by association, and false authority)
This trait identification set helps student reviewers look for specific moves in a student report writer’s analysis of a professional magazine author’s ethos. Writers aren’t required to include all of them, but the list can inspire their revisions.
Likert Scale #1: Choose the issue that is the MOST well-developed in this introductory background section and then explain your choice.
NOTE: It’s possible that these issues do not need be fully-developed in this section. If so, explain why you think that.
- how the article author’s views align with the moderate stance of the publication
- why the article author’s writing would benefit or detract from this publication
- what ethical implications follow from hiring this article author a columnist
- None of the above
Likert Scale #2: Choose the issue that is the LEAST well-developed in this introductory background section and then explain your choice.
NOTE: It’s possible that these issues do not need be fully-developed in this section. If so, explain why you think that.
- how the article author’s views align with the moderate stance of the publication
- why the article author’s writing would benefit or detract from this publication
- what ethical implications follow from hiring this article author a columnist
- All of the above are equally well-developed.
These two Likert ratings ask reviewers to identify the most and least well-developed aspect of a draft, which helps writers know where to put their energy.
From this sandbox, I could see how to use review tasks to embed really clear paths for writers to discover opportunities to enrich content, develop arguments, or integrate required elements that were missing. I got a better sense of how to design reviews that targeted global organizational problems. I also feel like, as an instructor, I am learning to be more concrete and descriptive about the elements of writing that I’m looking for in different sections of a document. These criteria, then, become a type of heuristic that students can use to strategize the types of rhetorical appeals they might incorporate in order to enrich, rethink, revise, or expand on earlier ideas. That’s powerful.
What makes you confident that students are learning more using Eli?
I found that students fell into two groups: resistant and engaged.
Resistant Students: To be fair, not all of my students did learn more in Eli this term. Some students seemed resistant to buy in not because of Eli itself, but because they seemed to reject the notions of writing as a social, iterative process that undergird Eli. They did not value the type of peer accountability that gives rise to a community of practice. For this group, the underlying ideology seemed to be that learning occurs through individual enterprise. Some of these students dropped right way; others left midterm.
Again, this isn’t an Eli issue, because these are the principles that inform my pedagogy and the pedagogy that guide writing studies as a field.What this lesson taught me is that there is still much work for me to do as an educator. It’s about bringing students into conversations about pedagogy on day one, and I am finding that super exciting for class discussions because we start talking about why and how different forms of pedagogy that differentially attend to components of writing (e.g., process; audience; feedback; revision) are unique to composition courses.
I also think that the way peer learning and the writing-review-revise process of Eli makes who is doing what, when, and to what degree super visible, and I think some students need extra attention to develop comfort with that.
Engaged Students: The core of students who really engaged with Eli this semester, those students showed me just how transformative Eli can be. I saw improvement and growth on a scale that I haven’t seen in previous courses. So, generally, I think that Eli makes a lot of tacit things that happen in a writing classroom way more visible. It gives us opportunities to talk about those things and have dialogue about strategies for responding to them.
What I’m trying to do now is make more clear to students how Eli helps peers and myself learn how well students are attending to these different elements of their writing:
- Do they have a strong grasp on the aims of the assignment?
- Have they used strategies appropriate to the genre?
- Are they framing arguments in ways that their audience might find compelling?
Seeing how students are responding helps me see what dispositions students are bringing to the classroom regarding writing generally. They also have opportunities to get feedback that can help them course correct early and often.
Eli gives me opportunities to reach out early and often and say, “Hey John, I see you didn’t do X,” is there something about that task I can help you with.” “Yeah, I don’t get this part or I’m struggling with Y.” “Ok let’s talk about that.” “Let me help you with that.”
For those who were willing to engage, it truly created a community of practice, and I guess the part I’m working through now is how can I as an educator use the tool even more effectively to engage those who seem disinterested in the community.
Has working with Eli changed how you teach?
In creating prompts for reviews, Eli requires me to be more concrete and descriptive about what elements and qualities of writing I’m asking students to identify and evaluate. That makes the complex tasks they’re managing as writers less complex. Eli makes it easier to say, “Hey, see X variable, that’s still missing, and you had Y feedback from peers and me that said that X was missing.” Then I can frame that thing as an area to really target on helping that individual student work on. I mean, I have abilities to be wicked specific and nuanced about how I respond to writers as individuals while targeting aggregate trends in large group meeting/discussions. That’s super helpful.
Have you had any feedback from students or colleagues about your use of Eli?
I’ve had mixed feedback, perhaps, because I’m still improving on the ways I’m integrating the tool. From those that engaged in the learning we performed this semester, I had really, really great feedback:
Without Eli, I wouldn’t have improved the way I did. I got feedback from peers that was so powerful and insightful, and it helped me realize what I could do to change certain elements.
Perhaps the best feedback isn’t what students say directly, but the marked way that I see them acquire ways of talking about elements of writing that are more nuanced than “this paragraph” or “this sentence.” Students are talking about how they are strategizing different moves in a body paragraph, they’re talking about why they are using X evidence instead of Y evidence. They’re talking about audience with one another. Usually those marginal conversations are limited to me speaking directly to 24+ students when I comment. Eli allows students opportunities to start talking about elements of writing like aims, audience, organization, genre much earlier in the writing process; it allows students much more agency over how/when they’ll revise.
What tip would you give an instructor using Eli for the first time?
Be patient. Spend the time on the front end, and work with Melissa, Mike, Bill, and Jeff. The people behind the technology know it better than I do. There are folks who have been using this tool for a while now, and they have learned the hard lessons through trial and error. Talk with them, and trust what they say. Commit in the ways they suggest because they will offer you support so that Eli works in the powerful, transformative ways that only peer-learning can.
