Feedback Manager Implementation
Click the [+] to expand a table and [-] to collapse.
All representative cases will open in a new window.
Click here to download a printable document containing all Implemenation tables.
Important Information For Success
What We've Learned[+]
| Advice | One Method | Consequence, if ignored |
| Integrate open-ended questions, student responses, and instructor feedback into your instructional design | Cross-reference the different components of the course and design the course so that components support each other | Questions, responses, and feedback that are not experienced as an integral part of the course will be seen as "busy work" |
| Be Transparent: tell students why you are using quizzing, writing, and giving feedback | Include the rationale in your syllabus, before and after the first quiz, etc. Give the evidence, ask students to reflect -- whatever it takes to impress on them the importance of thinking | If students don't see that the extra work will benefit learning, they will resent it and not be attentive when they do it |
| Scaffold: Teach students how to answer the questions to meet your expectations (give them a structure and example) | Start by giving more structure - (See Kirkpatrick and Pittendrigh, 1984) and later it may not be necessary | Fear of failing can create resistance. Responses will be variable if they don't understand your expectations. For example, responses could be too long, not concise, not contain evidence |
| If team teaching, find an implementation scheme, an expectation that all can agree upon | Assign quiz on the same day, allow the same amount of time, agree on grading scheme, objectives for the questions, etc. | Students are confused when instructors use quizzing, writing, feedback differently |
| Make it easy for students to remember what they said for the open-ended question | Include a link to the quiz in the E-mail template so that students can see the feedback next to their own response | Students often can't remember what they said on the quiz, and the feedback is meaningless without that information. Students said they would look at their quiz if the link were included. |
Ideas
Some Question Types to Meet Objectives [+]
| Option | Student Gain | Instructor Gain | Instructor Effort | Case |
| Conceptual question linking concepts | Higher-order thinking | Gauge higher-order thinking; identify misconceptions in combining ideas | Writing questions is difficult. The Writing Lab's Brad Hughes (bthughes@wisc.edu) is a great reference. | All (Hopefully!) |
| Conceptional question asking for relevant examples | Engagement-relevance to life, good test of understanding | Gauge level of understanding by appropriate application | Less difficult writing questions, but may need to read more answers because of potential variation in examples | Zoo 460 |
| Factual plus conceptual (Two-part question with first part factual and second part asking "why," "how," etc.) | Understanding instructor expectations and relevance of fact to larger context | Gauge understanding beyond factual | Essay with template formats the response | |
| Fill in a table | See pattern or organization of content | Gauge understanding of student thinking | Essay with template formats the response | Biocore |
| Ask students, "what remains confusing, or if nothing is confusing, what did you learn that was most interesting?" | Thinking over the material, communicating with instructor | Knowledge of what students are thinking, what to clarify | Responses can be all-encompassing, ambiguous, or not specific. Suggested to show students what an appropriate answer would look like. | Intro Bio 151 - Section 1 |
Student Response Types [+]
| Option | Student Gain | Instructor Gain | Instructor Effort | Case |
| Writing text: short answer | Higher-order thinking, writing skills | Window into student thinking | Reading required | All |
| Draw: Graphing, generating diagram, filling in areas | Quantitative skills, express understanding visually | Window into student thinking | Easy to scan and tag | Chem, Physics 208, Intro Bio 152 |
| Filling in table, worksheet | Minimal writing, but helps students see pattern | See if students see pattern in content, different from the way it was presented in lecture | Easier to scan than writing, perhaps | Biocore |
Some Options for Reviewing Student Responses [+]
| Option | Student Gain | Instructor Gain | Instructor Effort | Case |
| Scan all | Interaction with instructor | Gauge class as a whole | High - depends on the question | |
| Scan subset (choose a section of class, students with lower grades, students that haven't received a message recently, etc) | Interaction with instructor, learning from their peer's answers | Evaluate a sample, representative of the whole | High, but not as high as scanning all, important to discuss common mistakes in class or via the report | Intro Bio 151 - Section 2 |
| TA scan and respond | Writing, even if not read by instructor, helps learning | Only if there is a mechanism for TAs to inform instructor | Little, instructor may read a subset to create a rubric for TA grading | Chem |
| Have students work in groups | Collaboration of ideas, learning from their peers | Ability to view all student responses without needing to scan all responses individually | How much lower than scanning all responses depends on the size of the groups | Biocore |
Some Options for Tagging Student Responses [+]
| Option | Student Gain | Instructor Gain | Instructor Effort | Case |
| Scan student responses, mentally create categories, create a tag for each category, tag student responses | Relevant feedback, interaction with instructor | Efficient way to give feedback, interaction with students, better understand major misconceptions | High, but depends on the number of student responses tagged | |
| Import tags/feedback from previous questions/years, tag student responses | Relevant feedback, interaction with instructor | Efficient use of previous effort, interaction with students | Less high because reusing tags and feedback | Intro Bio 152 (Abbott) |
| Generate a report by scanning student responses, create a tag and mark "typical" those responses that seem to occur most often | Students can compare their responses to a report to get feedback | Efficient way to sample student responses | Less high because fewer responses are read | |
| Search responses for specific words (Boolean search) to pre-group responses, tag student responses | Relevant feedback, interaction with instructor | Efficient way to categorize responses, give feedback | Less high because pre-grouping may makie it easier to tag, may be challenging to find search terms that distinguish groups |
Some Options for Giving Feedback [+]
| Option | Student Gain | Instructor Gain | Instructor Effort | Case |
| Use "Individual Feedback" field to create individual feedback | Very personal response, high satisfaction | Able to provide very specific feedback and help individuals | Extremely high | Biocore |
| Use the report function to show typical responses and associated feedback, e-mail link to report | Provides a rubric for students to gauge own performance | Summary of the range/frequency of student responses and can be used to compare instructional effectiveness from year to year | Moderate | |
| Use the report function to show in class | Provides a rubric for students to gauge own performance, and a verbal elaboration on some key points | Feedback need not be as detailed - just give students and idea of how they are doing - since elaboration will come in class | Moderate | |
| Create feedback for each response and send e-mail responses | Feedback for students, also interaction with instructor, feedback comes in an e-mail which is more likely read | Tags can be used and re-used in subsequent semesters; gains in student feelings of satisfaction with interaction | High | Hort 120, Zoo 460 |
| In-class | No personalized feedback, but they learn what the instructor expected | Less work | Low | |
| Filter responses by date-of-last e-mail feedback, grade on quizzes, etc and give feedback to a subset | Personalized feedback to a set of students | Target effort to students who might need feedback; equality in giving feedback | Less effort when feedback is given for a subset of students |
Some Options for Grading Student Responses [+]
| Option | Student Gain | Instructor Gain | Instructor Effort | Case |
| Grade a subset of student responses each time, eventually getting to all (low-stakes) | Motivation to do well | Frequent quizzing helps learning | Depends on the size of the subset graded | Intro Bio 151 - Section 2 |
| Grade a subset of student responses each time, and have the TA's grade the remainder, based on your "report" which is used as a rubric (low-stakes) | Motivation to do well | Frequent quizzing helps learning | Depends on the size of the subset graded | Chem |
| Give participation credit to all students for trying (low-stakes) | Helps the most self-motivated students only | Practice helps learning, but learning gain may not be as high | Low | Hort 120, Zoo 460 |
| Instructor grades all responses | Motivation to do well | Control over grades, feedback | Very High | Intro Bio 152 (Abbott) |
Ideas for Promoting Greater Effectiveness for Student Learning [+]
| Option | Student Gain | Instructor Gain | Instructor Effort | Case |
| Have students work in groups | Cooperative learning promotes understanding | Typed group responses | Less effort to respond to groups than individuals | Biocore |
| Base questions on required reading before class. Include a question: "what remains unclear." Read responses to guide the amount of time spent on different topics in subsequent class | Students have first exposure to topics prior to class, and therefore understand more | More time in class to spend on more difficult topics or activities | Respond only to questions of "what remeains unclear" or to common or major misconceptions uncovered in other questions | Eric Mazur |
Cases
Experiments with different ways of implementing aspects of Feedback Manager.Examples
- Dave Abbott's presentation: November 18, 2009
- Note: Wait until the image of the video appears in the upper left corner before clicking "play"