The Center for Research on Learning and Teachings (CRLT) standardized course evaluationsinstructor-designed questionnaires, or IDQsfilled out by students are widely used to evaluate the performance of professors, lecturers and teaching assistants. Item 2 on the IDQ form states Overall, the instructor is an excellent teacher, and students are asked whether they strongly agree, agree, etc., scored from 5 to 1.
Committees use median course scores in evaluating faculty for University awards, departments use them for salary merit increases, and individuals use them to evaluate their own performance. I have been involved in all of these, and with experience began to sense that course size might have an important effect on student responses.
CRLT recognizes that course size and other factors affect IDQ scores, cautions that scores must be interpreted with care, and warns against the very comparisons we all use IDQ scores to make (while supplying the quartile scores required to make the comparisons).
To investigate the effect of course size on IDQ score, I asked CRLT for a sample data set. James Kulik supplied median scores on IDQ item 2 for 2,411 courses taught during the fall term of 1993, and these course medians are plotted against course size in the accompanying graph.
Analyzing the scores, I was surprised by three things: (1) most courses are smaller than I expected, averaging about 25 responses; (2) student satisfaction with teaching is very high, with well over half of all evaluations lying above 4above agreement that the instructor is an excellent teacher; and (3) the effect of course size on median score is small. Standard regression of median score on course size (using number of respondents as a proxy) yields a coefficient of correlation r of 0.10 and a coefficient of determination r2 of 0.01. I expected a greater effect, but the observed effect cannot be ignored.
The quartiles reported by CRLT assume that median scores for courses are independent of course size: these have zero slope when scores are plotted against course size, as shown by heavy lines on the graph. Empirically, regression scores on course size have a slope of -0.20, indicating that scores are not independent of course size. Adjusted quartiles based on the regression slope and residuals are shown by heavy dashed lines.
What is the effect of such a small adjustment?
Instructors who teach large courses are systematically underrated in comparison to instructors who teach small courses.
In the sample considered, 82 out of 733 courses with 24 or more responses (11 percent) appear misclassified in the sense that they move up into a higher quartile when the quartiles are adjusted as shown to account for course size.
This is no cause for alarm, but misclassification is not fair to instructors who teach large courses either. Course size should be considered when evaluating instructors teaching large courses; at the very least we can feel a little better about the job we are doing in comparison to our peers.
Philip D. Gingerich, professor of geological sciences and director, Museum of Paleontology
As chairs and directors of departments and programs in the College of Literature, Science, and the Arts at the University of Michigan, we recognize that any administrators particular decisions and actions are always open to question, review and evaluation. It is important, though, that discussion of particular decisions and actions remain specific and fair.
In the context of a legitimate discussion of the particularities surrounding the Communication Departments past, present, and future, we feel that sweeping, inaccurate and unfair claims have been made about Dean Edie Goldenberg, some of them directly attacking her integrity. We feel that in our roles we have ample opportunity to observe Dean Goldenbergs personal integrity and commitment to undergraduate education. (Naturally we can only speak as individuals and do not here represent the views of others in our units.)
There can be no doubt that this dean has defined improvement of undergraduate education as a primary goal for the College. She has launched new programs to support undergraduate education aimed at both departments and individual faculty.
For example, she has worked to create a Quantitative Reasoning Requirement, and a new set of courses permitting students to use a second language across the curriculum. In addition, she has provided support for departments own efforts to improve undergraduate education, for example the math departments recent effort to provide a more interactive/cooperative mode of instruction (which has required increased funding of smaller classes.)
Perhaps most important, though, Dean Goldenberg has worked hard to clarify the special opportunities that arise for undergraduates in a major research university.
For example, the new program of first-year seminars helps ensure that students have an opportunity to discuss their ideas with senior faculty who are doing significant scholarship; the Undergraduate Research Opportunity Program provides a very different opportunity for students to work directly with faculty on their research.
By increasing the connections between faculty members scholarship and their work with undergraduate students, Dean Goldenberg has shown how faculty research and undergraduate educationfar from being opposite or alternative valuescan be complementary and integrated.
We look forward to considered discussions of complex College issues which take full account of the initiatives under way in the College under the strong leadership and broad vision of the dean.
Michael Awkward, director, Center for Afroamerican and African Studies; Wesley Brown, chair, Department of Biology; Alan Deardorff, chair, Department of Economics; Pat Gurin, chair, Department of Psychology; Rowell Huesmann, acting chair, Department of Communication; Don Kinder, acting chair, Department of Political Science; Ludwig Koenen, chair, Department of Classical Studies; Robert Kuczkowski, chair, Department of Chemistry; D.J. Lewis, chair, Department of Mathematics; Stuart McDougal, director, Program in Comparative Literature; William Paulson, chair, Department of Romance Languages and Literature; Abigail J. Stewart, director, Womens Studies Program; Maris Vinovskis, chair, Department of History; and Robert Weisbuch, chair, Department of English