Eli Review was built by writing professors frustrated by how hard it was to coordinate effective peer feedback fast enough to get multiple drafts revised in just a few weeks.
Every aspect of Eli has been informed by study into how people learn and especially how they learn to write. At the same time, Eli produces rich data about student engagement and revision – more rich than any other tool in existence, opening up many new opportunities for researchers.
Research Powering Eli Review
Two major theoretical approaches inform Eli’s creation and functionality:
- Revision is central to learning – it is through the process of re-evaluating ideas that we learn what works and what doesn’t.
- Feedback is one of the most powerful ways to facilitate revision – learning theory tells us that through coaching and modeling we are exposed to different ways to solve problems.
The challenge in supporting these approaches is that neither teachers nor students are necessarily good at providing quality feedback. To revise effectively, one needs quality feedback; to give good feedback, one must be taught and have practiced doing so.
Eli is meant to facilitate this improvement. It makes the review process transparent, making it easy for instructors to coach students into giving better feedback. It makes the revision process transparent by having students prepare plans for how they will improve their work based on the feedback they received.
Eli makes available a rich set of teacher development modules meant to help instructors grow their own research-driven review practices, as well as a diverse set of instructional materials reflecting these practices that can be put to use in classrooms immediately. Of particular note:
- Feedback and Revision: The Components of Powerful Writing Pedagogy
- Designing Effective Reviews: Helping Students Give Helpful Feedback
- Teaching Revision: Helping Students Rethink Their Writing
- Evidence-Based Teaching: Formative Feedback and Writing Instruction
Relevant Publications and Invention by Eli’s Creators
- William Hart-Davidson, Michael McLeod, Christopher Klerkx, and Michael Wojcik. 2010. A method for measuring helpfulness in online peer review. In Proceedings of the 28th ACM International Conference on Design of Communication (SIGDOC ’10). ACM, New York, NY, USA, 115-121. DOI=10.1145/1878450.1878470 http://doi.acm.org/10.1145/1878450.1878470
- Michael McLeod, William Hart-Davidson, Jeff Grabill. ” Theorizing & Building Online Writing Environments: User-Centered Design Beyond the Interface.” Designing Web-Based Applications for 21st Century Writing Classrooms. Ed. George Pullman and Baotong Gu. New York: Baywood, March 2013.
- US Patent Application 61313106, “Social Writing Application Platform,” William Hart-Davidson, Michael McLeod, Jeffrey Grabill. Filed March 11, 2010.
- US Patent Application 61313108, “Systems and Methods for Tracking and Evaluating Review Tasks,” William Hart-Davidson, Michael McLeod, Jeffrey Grabill. Filed March 11, 2010.
Research Powered by Eli Review
A side effect of building a system supporting research-driven pedagogy is an abundance of data about writers and their interactions with feedback. This data was first surfaced within Eli as a means of helping instructors coach their students through better revisions, but it has much bigger implications for teachers and teacher researchers.
- Qualitative data: Eli collects all comments exchanged between reviewers. It displays them in a real-time feed for instructors but also makes them available for download.
- Quantitative data: Eli also collects data about student engagement around review feedback. This data is likewise surfaced during individual reviews but can be utilized to observe reviewer trends over time.
This data is revealed in the following places within Eli:
- Individual reviews: reports of student progress from individual review activities.
- Course-level analytics: an ongoing report that tracks progress among all students over the duration of a course. The more reviews instructors assign, the more data.
- Student-level analytics: similar to course-level analytics, these reports show an individual student’s performance over time and positions them against class averages.
- Data downloads: each report (individual, course, and student) includes features that allow instructors to repurpose their student data in a variety of ways. Both qualitative and quantitative data can be downloaded into flat files that can then be used to create custom sorts and queries to support a wide variety of research purposes.
Interested in doing research with Eli Review?
We are eager to work with teachers and scholars interested in utilizing Eli’s data for research purposes. We can help brainstorm, design studies, clear institutional review boards, and more. Please contact us for more information.
Research by Scholars Utilizing Eli Review
-
Laflen, Angela. 2020. “Using Eli Review as a Strategy for Feedback in Online Courses.” Assessing Writing 46 (October): 100486. https://doi.org/10.1016/j.asw.2020.100486.Gogan, Brian, and Samantha Atkins. 2020. “Peer Review Practice, Student Identity, and Success in a First-Year Writing Pilot Curriculum: An Equity-Minded Analysis.” The Journal of Writing Analytics 4 (1): 55–103. https://doi.org/10.37514/JWA-J.2020.4.1.04.
-
Hart-Davidson, William, and Melissa Graham Meeks. 2020. “Feedback Analytics for Peer Learning: Indicators of Writing Improvement in Digital Environments.” In Improving Outcomes: Disciplinary Writing, Local Assessment, and the Aim of Fairness, edited by Diane Kelly-Riley and Norbert Elliot, 79–92. New York: Modern Language Association.
- Ching, Kory Lawson and Stacy Wittstock. “Teaching with Digital Peer Response: Four Cases of Technology Appropriation, Resistance, and Transformation.” Research in the Teaching of English; Urbana Vol. 54, Iss. 2, (Nov 2019): 161-182.
- Amidon, Tim. (Dis)owning Tech: Ensuring Value And Agency At The Moment Of Interface (2016, September 8). Digital Hybrid Pedagogy.
- McLeod, Michael, Ann Shivers-McNair, Jeff Grabill. (2016, May 21). Tools for Writing Researchers, by Writing Researchers. Computers & Writing, Rochester, NY.
- McLeod, Michael, Bill Hart-Davidson, Melissa Meeks. (2016, April 8). Better Learning Technologies: Taking Action to Reframe #edtech Development for Writers and Writing Teachers. College Conference on Composition and Communication, Houston, TX.
- Digitally-Mediated Peer Review in Multimodal Composition Courses: Implications for Writing Pedagogies. Erin Zimmerman, Kathy Rose, Eric York, Iowa State University, Susan Pagnac, Central College. Presentation at the Annual Convention of the Conference on College Composition and Communication, Indianapolis, IN, March 19-22, 2014.
