Written and designed by the staff of the Center for Teaching and Learning. Reproduce with permission only.
March 1991
Last semester, in the spirit of pedagogical experimentation, Professor Edison decided to allow students an alternative to the traditional term paper. Students were encouraged to suggest innovative projects in which they could apply the concepts they had learned in the course. Edison felt that students had never delved very deeply into the course material in the term paper and was hoping that this innovation would foster more intellectual involvement on their part. Many students accepted the challenge, and Edison received proposals for three videotapes, two slide shows, a computer simulation, and two group presentations, all of which looked interesting and worthwhile. But at the end of the semester, sitting at a desk piled with videotapes, slides, computer disks, and term papers, Edison began to have second thoughts. How can one grade such a variety of work? Should some students automatically receive more credit for accepting the challenge and showing some creativity? Are the special projects in any way truly equivalent to the term paper? Is innovation really worth the trouble? Let's examine how Professor Edison got into trouble.
Edison was on the right track in thinking that some types of course material may be learned more effectively through field work, and may be more convincingly presented in a slide show or videotape than in a written paper. For example, a student in an Anthropology course on religious practices in the South might wish to compare the degree of congregational participation in services of different denominations. The student could gain invaluable insights into the subject by visiting churches, recording a variety of services on videotape, and then selecting sections of each to compare and contrast. In addition, role plays, class presentations, and interviews can be used to demonstrate the application of communication skills required in a broad range of courses. Such assignments call on students to present ideas actively, making clear how well they have been able to integrate material from readings, lecture and discussion. Moreover, this kind of exercise increases student involvement and motivation. As a result, they tend to learn more and remember it longer.
One other major advantage of non-traditional
assignments is the opportunity they afford to evaluate students on more than
one cognitive level. When students are required to exercise judgment and create
something new, they must operate on the highest cognitive levels (which, incidentally,
are the same levels at which we operate in our professional lives). Often, term
papers and similar exercises only approximate some of the important intellectual
skills that we expect our students to demonstrate. Of course, paper assignments
and tests are usually easier to design, evaluate, and administer than special
projects and performance exercises. The assignment of nontraditional projects
also requires clearly-defined criteria for evaluation, otherwise grading becomes
unacceptably subjective.
Once you have opted to assign a nontraditional project, decide whether you are interested in assessing a product (something tangible like a video), the process by which that product is created, or some aspects of both. In general, it is easier to concentrate on the product itself; however, processes can be successfully evaluated using journals in which students document the steps they took in creating their products. Assessing this process often provides the best opportunity to evaluate students' higher level cognitive skills. In the example of the student researching church services, you could restrict the evaluation criteria to the product, the mini-documentary itself. But you would have a more accurate idea of the student's ability to integrate ideas if you looked at earlier versions to see how the final material was selected. Does the process of editing footage of the services and interviews with church members show important insights into the attitudes that underlie the observed behavior? (A journal that describes the decision process would be helpful here).
In order to assess achievement accurately on any kind of test or assignment, you need to ask two questions: what are students expected to demonstrate in this assignment, and how confident can you be that you can accurately discriminate among different ability levels? Answers to these questions can be found in the next planning stage, when you try to define, as precisely as possible, the specific aspects of the product or process you intend to evaluate.
The first step is to break the assignment into sub-categories and then focus on specific points within each sub-category. For example, for the Anthropology videotape we might consider quality of presentation (sound quality, effective use of camera, lighting) and the presence of underlying ideas that explain observed behaviors (individual personalities, doctrinal differences between denominations, role of each church in the congregants' lives). The more detailed your set of criteria, the more reliable the evaluation: it will be easier to distinguish between students if you have specific points of comparison rather than vague feelings and intuitions about the performance as a whole. Specificity will also help increase consistency in grading while decreasing the chances that extraneous factors will influence your evaluation.
After preparing the list of criteria,
compare it to the list of objectives for the course or unit. The greater the
correspondence between the two, the more valid the exercise. For example, one
objective in an Anthropology seminar might be to demonstrate the ability to
convey research results to a broad audience of laymen. In this case, it would
be important to assess the tone of the video as well as the ideas present: does
the video effectively address the intended audience? Is it too technical? Is
there an obvious bias that would offend the viewers? On the other hand, in a
lower-level class the objective might be to gain experience in designing a project
and collecting and organizing data. In this case the evaluation criteria would
be different: does the video offer a plausible explanation for differences in
congregational behavior? Is that explanation based on the project, on ideas
learned in the course or on some combination of the two?
If Professor Edison had talked to colleagues in departments where applied skills are emphasized, they would have provided valuable advice about evaluating the course projects. In studio art, physical education, and musical performance, it is common to assess practical ability as well as theoretical knowledge. The three most common means of performance evaluation are anecdotal records, rating scales, and checklists.
Anecdotal records - the grader documents selective descriptions of performance. For example, to evaluate a class presentation, you would refer to the list of performance criteria you had constructed in planning the exercise. As the presentation takes place, you could make notes on overall organization and depth of coverage of various points. This system has the advantage of thoroughness, allowing the grader to distinguish carefully between various candidates. On the other hand, if the performance takes place as you grade, it is difficult to make detailed notes. Also converting notes into a grade is not easy and might decrease the consistency with which the assignments are evaluated.
Checklists - the grader chooses from two opposite fields such as right/wrong, yes/no, acceptable/unacceptable. Checklists can be used both quantitatively, to assess only the presence or absence of certain attributes, and qualitatively, to assess whether attributes meet certain standards of quality. The former are especially useful for process evaluation, where emphasis rests on the sequence of steps, and actions to be assessed occur in rapid succession, not leaving time for notes (e.g., a medical student's performance in making rounds). In the video example:
There was a narrator yes no
Sound quality was acceptable unacceptable
The disadvantage of checklists is
that they do not allow for gradation in scoring, often blurring distinctions
between students. Also, checklists don't take unexpected aspects of performance
into account, a fact which may unfairly penalize a creative student or reward
a poor one.
Rating scales - the grader chooses a point along a continuum that best describes the degree of correctness of each aspect of performance. The most common type of rating scale uses a numerical continuum between opposite fields, creating something like an expanded checklist:
Cuts in the video were effective in showing connections or contrasts.
never 1 2 3 4 5 always
How would you characterize the student's ability to use interviews to elicit pertinent information?
poor 1 2 3 4 5 excellent
Rating scales are particularly useful for qualitative attributes, for which the question is not whether an attribute is present, but the degree to which it corresponds to the grader's expectations. Potential drawbacks include the tendency to avoid the extremes of the scale and thereby assign a disproportionate number of grades in the middle of the scale. Also, aspects of a student's performance may seem to fall between points on the scale.
In constructing a rating scale, keep the following suggestions in mind:
Items in a rating scale can be graded separately, or you can establish minimum numbers of points necessary to achieve a certain grade on the assignment as a whole.
To increase grading flexibility and
raise the accuracy of the grades, it is a good idea to combine these systems.
Look over your list of criteria and decide which grading system would be most
appropriate for each item. You may wish to re-phrase the individual items to
better fit one or another system of evaluation. If there are severe limitations
(e.g., if you know that there is no time for anecdotal note taking) you may
need to plan out the evaluation before you write out the list of criteria.
One way to gain an overview in planning a nontraditional assignment is to develop a "blueprint" by writing out the important steps and specific attributes of each element of the exercise. Blueprint categories could include: objectives, explanation of the assignment, focus of evaluation (product/process, both), specific criteria to be assessed, method of assessment, how grades will be calculated. A blueprint will reveal how well the various aspects of the assignment correspond and will also make it easier for you to spot possible problems and correct them before students begin working on their projects. We have provided an example of a blueprint for the video project mentioned earlier, so you can see how the project might appear at the planning stage.
Unit: To learn about the variety of religious practices found in the American South and to understand the roots of that diversity.
Assignment: To experience a variety of church services and to apply ideas from the course to that experience by focusing on one aspect of the service.
Assignment: Visit at least four churches of different denominations and observe the extent to which the congregations participate in the service. Observations of services, as well as interviews and background material should be recorded to create a short documentary of 20 -30 minutes in length.
Focus:
Product: Final version of the documentary video, shown in class.
Process: Raw footage (before editing); journal describing the behaviors observed in church, reflections and insights about that behavior, and decisions made in editing the video.
Quality of presentation
- Narration
- Use of camera
- Editing
Content of presentation
- Assessment of individual attitudes of churchgoers.
- Accounting for doctrinal differences in behavior.
- Relating role of each church in the lives of the congregants.
Nontraditional assignments are not as common in academia as papers and tests, so students will have varying assumptions about them, none of which will necessarily correspond to the project you intend to assign. Therefore it is essential to explain the assignment fully to your students (your blueprint will make this task much easier). A handout of some kind is usually the most effective way to explain what you expect and it gives students something to refer to as they work on their projects. You may even wish to involve students in the development of the assignment and the criteria for evaluation. Doing so has three advantages. First, if students are involved from the beginning they will have more of a stake in the project and their motivation will increase. Second, they will begin to develop deeper insights into the material as they try to decide what is important about the project. Finally, by developing the project together there will be a greater correspondence between your expectations and theirs.
Returning to our original scenario,
it is obvious that Professor Edison had two problems: a lack of criteria for
judging non-traditional assignments and no clear idea of which course objectives
the exercises were supposed to reflect. With careful attention to the items
on the following checklist, you can avoid these pitfalls and create challenging
and successful projects for your students.
home / fyc index / publications / email