Assessment of Kinemage Authorship Projects

Kinemage Attitude Survey

The primary method will be the Kinemage Attitude Survey, a statistically reliable pilot-tested multiple choice instrument. This survey was developed by Dr. Deborah Booth. Instructors beta testing our authorship materials are requested to use this survey, which will probably require approval from their local Human Subjects oversight committee. Please contact Dr. Booth or Dr. Bateman for a copy of our human subjects materials. We will supply the survey instruments, analyze the results and send the instructor a copy of the analysis.

Performance Evaluation

If class size is not overly large and facilities are available, a second measure of student learning (a performance evaluation) can be done in conjunction with an exam, possibly the final exam. In a computer lab, students will be given a timed, supervised quiz asking questions about a kinemage they have not seen previously. The quiz will be graded by the instructor for their own use, and then the anonymous grades and a blank copy of the quiz sent to us for analysis. A sample kinemage final exam is available from Dr. Bateman upon instructor request. Students can bring their own laptop computers to this exam since they will not be given the exam until they arrive.

Laboratory-based Kinemage Authorship

If the instructor's course is accompanied by a laboratory, it may be an option to have students author kinemages as a laboratory exercise. This takes advantage of the blocks of time available to the lab and can be particularly valuable if the kinemage topics are related to the bench experiments in the balance of the lab. If the instructor uses premade kinemages as part of the lecture course and the laboratory students make up only a subset of the lecture student population, then the survey +/- performance evaluation could be given to all lecture students and would have a built-in control group (the non-laboratory students).

Lecture-only Control Groups

Instructors using kinemages in their classes (lecture and/or homework), but not interested at the present time in student authorship materials can still use our Kinemage Attitude Survey and performance evaluation. This is still very valuable as a control for our study.

Qualitative Evaluation Methods

At USM we have performed a number of formal interviews with student kinemage authors. This would obviously be difficult to perform on multiple remote sites. However, we are exploring to possibility of setting up an online interview form, which would be a more formal version of our current comment form.