The Eli Review Blog

Scott Kushner, assistant professor in the Department of Communication Studies in the Harrington School of Communication and Media at University of Rhode Island, started his class this Spring with a one-word writing assignment. Yes, you read that correctly. Writers typed one word and submitted it as the first draft in Eli Review.

This one-word assignment is deceptively simple. The word is the keyword for students’ semester-long inquiry research project. Kushner calls it a “Mock Keyword Selection” (Figure 1) so that students understand that the assignment is exploratory. They’re not locked into the keyword beyond this one assignment.

Mock Keyword Selection

This one is simple.  In the box below, enter one single media keyword to which you might consider devoting your work this semester.  (Don’t worry: this is just to get started—we’ll make real selections next week.)

What makes a good media keyword?

• A word that describes a practice or an object that has currency in today’s media environment.
• A word that somehow reveals something about culture (global, sub-, high/low, whatever).
• A word through which you can see something about the inner workings of some part of contemporary society.

Here are ten examples, just to give you an idea of what kinds of words can work:

1. Album
2. File
3. Movie
4. Paper
5. Radio
6. Show
7. Site
8. Song
9. Tap
10. Television

Enter your one single word below.

Figure 1. Kushner’s Mock Keyword Selection Prompt.

The genius of this assignment is not the one-word writing prompt, though it is a nifty way to get students started early on a long project. Finding a keyword isn’t that difficult. In the end, any keyword will do, really. It’s the thinking about the keyword that matters, especially the criteria that Scott highlights as important for a good keyword: currency, culture, and (revealing of) inner workings of media in society. In inquiry projects, writers need to find a good angle with enough interesting evidence from which to build a compelling analysis.

Experienced researchers quickly consider many options and identify a productive vein. Their internal dialogue is complex: “This can lead here, which might lead here, which helps show that, but it might dead-end here.” These are the mental muscles that make nearly every keyword satisfactory. For students taking up media studies for the first time, this kind of mental forecasting takes a lot of practice, however.

What should students practice?

The genius of this assignment is asking students to practice that forecasting skill as reviewers on other writers’ keywords. This is where they come face to face with the three criteria. They must think carefully about these in relation to other writers’ choices. Kushner’s review task (Figure 2) guides students in rating the keyword for its currency, cultural influences, and the ways in which it reveals society’s inner working. Reviewers are invited to explain each rating, and wrap up with a comment about what else they’d like to know about the topic.

Figure 2. Student preview of Kushner’s Review of Mock Keyword Selection prompt (note fake names and text)

This quick activity is designed to help students gain familiarity with the criteria and how other students are approaching the project. Once internalized, these criteria can be a resource for writers to develop their own inquiry focus and the writing that will follow from it. It’s the right obstacle course at the right time.

Did students practice effectively?

Except most students breezed through it. If they internalized the criteria, we don’t see evidence of reviewers using these criteria in their comments. We can tell because of patterns in the quantity and theme of students’ comments to each other.

Too little to learn

Using the comment digest and criteria & scaled downloads in Eli Review, we analyzed the word counts in reviewers’ rating explanations and open-ended comments. The table below shows that all 53 students wrote something to the writers in their groups, but 15 offered no answer to “What do you want to know about this keyword?”

This table also shows that students wrote comments of very different lengths: the top 30% wrote twice as much as the middle who wrote twice as much as the bottom 30%. This is a pattern our team now expects because we’ve seen it across disciplines, courses, and activities. The intensity pattern itself isn’t necessarily worrisome. If the bottom group writes enough to learn, the pattern is productive for everyone. The red flag in this data is that the bottom 30% wrote six word comments.

Short comments aren’t necessarily unhelpful, but, by virtue of their brevity, they have less potential to be helpful. In a six word comment, each word has to pack a punch. And, that’s rare for an open-ended prompt designed as a practice round for brainstorming an inquiry research project.

Here’s a sample of what students said in their short comments:

• Related to everything we do in today’s society
• What were your thoughts when choosing this word?
• Great keyword – very relevant

None of this feedback is wrong or rude. But giving it or getting it accomplishes little in relation to the focus of this assignment, which is all about those three criteria. None of these comments help the writer find a good angle or an interesting body of evidence. Little of it focuses the writer’s attention on meeting the criteria. By offering this feedback, reviewers didn’t tune into the point of the review task or of the research project.

To re-cap, we see evidence that 15 students engaged in no exploratory thinking related to their partners’ keywords, and 11 said something, but what they said didn’t push the keyword any further. So, about half the class missed out on the most important practice in the activity.

Enough of the right indicators

What about the other half of the class? Did the students who engaged more in giving feedback practice the key skill? To assess that, we looked at how reviewers answer the open-ended “What do you want to know about this keyword?” We coded every comment twice. First, we labeled which criteria from the prompt the comment mentioned. Second, we identified the theme of the comment.

Table 1 shows that 43 of 64 comments (67%) did not mention any of the required criteria. Of the three criteria, students talked most about “inner working.”

Table 1. Number of Comments Reflecting Criteria from the Prompt

The themes in reviewers’ comments (Table 2 below) help explain why so many were unrelated to the primary criteria for the assignment.

• About one third of the comments offered praise.
• Another third referred to history or impact, which were lines of inquiry discussed in class.

The other themes had fewer than 5 comments each. These reflect those knee-jerk things students have learned to say about research projects: your keyword is too narrow (ha!), add an example, clarify your term, and add personal significance.

Table 2. Themes of Comments with Number and Length

These trends tell us that students don’t automatically know to zero in on the criteria and use these in their comments to one another. Confused, they assure each other that they’re doing it right. These sample comments from the middle have that vibe:

• The internet is very important to a lot of people, its value to people demonstrates its importance in society.
• Good keyword.  Obviously advertisements are in almost every form of media, especially in the modern world.  There is a lot you can discuss.
• Find a way to make Website less broad, but otherwise, you chose a good word

Highly engaged reviewers are better able to push a single keyword until there’s an angle or a body of evidence. But, these comments still read like conclusions, not opening questions.

• Sometimes I feel that a song will demonstrate the inner parts of contemporary society through the music itself. May it be through lyrics, the tone of the instruments used, And so on. The tones could be high or low, expressing the feel and meaning behind the song.
• Photography also has to do with the news and social media. People are constantly posting pictures all over the media which could be negative or positive. This definitely helps us see the inner workings of society.
• Virtually all Americans, and specifically young adults, use the Internet, and the Internet is accessed through websites. Websites can maintained by anyone, and reflect society through organizations, interests, political movements, etc.

Again, the prompt for these comments was: What else do you want to know about this keyword? For these students, on the first activity of the term, answering a question with a question couldn’t be the right answer.

What’s next?

This write-review sequence is powerful because it is so simple yet so revealing. In Eli, debriefing a review is when instructors read the trends, models, and comments from peer review in order to figure out how to coach students toward revision.  After reading through the spreadsheet analysis of students’ comments, Scott figured out which criteria students weren’t talking about and why. He re-taught a section from the readings. He modeled more examples of how to approach the research project. He contrasted “Good job!” from praise that helps writers know what they did well. By reading through students’ conversations with each other in peer learning, he was able to a make a just-in-time intervention.

The intervention during debriefing is the key in this assignment in particular. Prevention would fail. Think about this assignment from a student’s perspective:

• The professor just wants ONE word as a draft. This is a cakewalk!
• The professor wants a few ratings and a couple of short answers. And the cake comes with ice cream!
• The professor checked our thinking. This was practice, not dessert? Wait, what?

Now, ahead of that second bullet, imagine a 5-minute pep talk on being a good peer reviewer of a one-word draft. I mean, come on. How hard can it be to respond to one word? Few students could hear such framing without rolling their eyes. By putting the pep talk afterward in a debrief, Scott assures that everyone in the room feels like he is talking to them.

Because he is. Only a handful of comments out of 338 exchanged in this review had enough substance and direction to help writers take the next step in developing their keywords. Only those reviewers practiced working with the criteria in a way likely to benefit their own writing.

This kind of activity shifts students from rehearsal to practice, something Bill Hart-Davidson and I wrote about previously. These learning analytics have all the signs of rehearsal. Students did what they always have done. They did not recognize that Scott was asking them to engage in deliberate practice of inquiry thinking. A strong cue at the start would not have changed the overall result of this review because the writing felt too easy. Once he stops the rehearsal, Scott can lead students in practice. From each other and the class’s work together, they’ll learn in a deliberate way how to push a keyword toward an interesting angle with a compelling body of evidence to analyze.