The Course Analytics Report is meant to provide a portrait of the aggregate activity of all students throughout an entire course. Activity from all the individual reviews completed to date is combined in this report. When a course is first created, the report contains no data because there’s no activity on which to report; as instructors lead students through rounds of the write-review-revise cycles, the report expands.
Course Analytics provides a course-level view of student performance, focused on averages to determine trends; its companion, the Student Analytics Report, provides similar information but looks at the performances of an individual over the duration of the course to compare against class averages.
Every Eli course has a unique analytics report compiled from the data generated by students during the review process. The course-level analytics report can be accessed by clicking the “Analytics” link in the primary navigation bar under the title for any given course.
Each report contains five sub-sections with unique displays of student engagement data. Each section can be accessed via the report navigation bar.
The report sections include the following:
When using any of the reports, keep in mind the following:
The first section of the Course Analytics report is the Engagement Highlights display. It is designed to provide a quick overview of some of the most important engagement data collected about students. Its two tables offer a glimpse:
The data in the Feedback and Helpfulness table can be sorted by clicking any of the column headers. It can also be downloaded as a CSV file for more advanced searching and sorting. The Engagement by Task Type report provides similar review data but is expanded upon significantly.
These reports consider individual engagement data metrics and graph their performance over time, revealing trends in student performance. The three engagement metrics currently available are:
Each graph is accompanied by a table containing all the raw data that went into plotting it as well as a toolbar enabling a number of different ways to download those materials.
Each point on a graph and each column in the data tables correspond to a single review completed by students in the course. If an instructor has assigned seven reviews, there will be seven points on each graph and seven columns in each table. Hovering the cursor over any data point or column header will reveal the name of that review and its relevant measure; clicking any data point on a chart will reveal the individual report for that review.
Switching between different graphs is quick and easy; above each graph is a dropdown labeled “Switch Trend Graphs” – just click this dropdown for easy access to each of the three reports.
This graph is meant to help instructors see, in general, how much feedback each reviewer has given in relation to how much they’ve received. The given/received ratio is calculated by looking at the number of contextual comments exchanged during reviews and, for the class average, the individual averages are aggregated into a single score.
The given/received ratio tells us the following:
Ideally, this graph would show a straight line – the closer the ratio is to 1, the more evenly distributed feedback is between writers and reviewers.
While this data is not scientific, it can give helpful insight into student behavior and suggest how that student might benefit from some direct instruction. Here are some ways to think about ratios:
This graph is meant to help an instructor see, in general, how much feedback is being exchanged during reviews.
Seeing comment volumes plotted in this chart can demonstrate a number of things:
The table below the volume chart lists the exact number of comments given by every reviewer who participated in a review as well as the class average for comparison.
Writers are given the option to rate each comment they receive on a scale of 1-5 stars. While it’s up to the individual instructor to coach students to rate their feedback and how to assess it (what a one-star comment looks like compared to a five-star comment), those ratings are compiled in this trend graph.
This chart can help provide answers to questions like:
Common trends: ratings graphs often slope downward over time. In early reviews, writers are eager to rate all of the feedback they receive very highly our of a desire to be friendly and collegial. Over time, as they get better at differentiating helpful from unhelpful feedback, ratings will often decline as they begin applying higher ratings more infrequently.
The table below the helpfulness chart lists the helpfulness averages for individual students on the comments they gave as part of a review.
This report is intended to help instructors understand what the performance of an average student looks like in their Eli reviews. When assessing overall performance, knowing what the average student did in a course as well as what kind of feedback they both gave and received can help illustrate where more coaching may be helpful.
Additionally, the data in these tables is intended to help instructors answer larger questions like “how much help did each student receive?” when compiling reports on student performance and engagement.
This report provides instructors with statistics about student performance on the different types of tasks they have assigned to their students.
The table will help an instructor understand, at a class-level, how students engaged with each type of tasks and will help them better understand how one might design new tasks for this class or for future classes.
These features give instructors the ability to download all of their raw student data. While most of this data is accessible through a browser, these tools allow instructors to take the uncooked student data and perform their own queries and sorts with it. This is particularly helpful for teacher researchers investigating their own practice or writing researchers studying student review behaviors.
For a complete breakdown of the data available in these downloads, and about formats and methods for using that data, see the Data Download FAQ. You can also view samples of the data available via download:
There are a few factors to keep in mind about what data is included in these reports:
Back to top