Today we’re announcing some significant changes to the Engagement Data report generated after each review activity. These changes include new data points as well as different ways to to help you spot who’s working hard and who isn’t.
The previous version of the Engagement Data report included data about comment counts and helpfulness ratings for comments given and received. The new version of that report expands on the existing data with these new data points:
- Word Count: how many words were written in all comments
- Comments with Word Count < 20: how many comments contain less than 20 words
- Endorsements: how many of the comments received an instructor endorsement
- Comments Added to Revision Plans: how many comments the writer added to a plan
These measurements are displayed both for the comments students receive from reviewers but also on feedback they gave. There is also a new Word Count Ratio measurement, making it easy to see the reciprocity in the number of words written to classmates versus received from them. All of this new data is also available in the Engagement Data download.
Adding eight new data points makes for an incredibly large table, so we’ve given the engagement data a horizontal scroll to help it fit on a single screen.
It’s also difficult to make meaning from so much data, so we’ve added “data stories” to the report. Clicking on a data story will change the columns displayed in the table, making it easier to focus on one aspect of what the data can reveal.
These data stories include:
- Intensity: how many words were exchanged in comments, and did comments contain enough words?
- Reciprocity: did the student contribute as much help to group mates as they received?
- Responsiveness: did the student act on the feedback they received (rating it, adding it to plans, etc)
We’ll add more stories to the engagement data table over time, but these three offer significant insight into student engagement in our review activities.
Thank you, as always, to your comments and suggestions about how to improve Eli Review. These improvements in particular are the direct result of our collaboration with teachers and researchers interested in engagement. If you have any feedback, please let us know.
