Course evaluations to go paperless

Updated 10:00 AM September 24, 2007

Course evaluations to go paperless

By Laurel Thomas Gnagey

A trial run of a new online version of the University's Teaching Questionnaire (TQ) course evaluation system will be conducted at midterm in the College of Engineering (CoE) in preparation for a broader Universitywide shift to a paperless system during winter term 2008.

The change to online evaluations is in keeping with a University move toward more electronic processes, project leaders say, including those already in place for students, such as class registration, viewing of course schedules and paying of bills.

The online system will have many of the same features of the current TQ but will provide users quicker access to course evaluation data, while eliminating the costly and time-consuming paper process of creating forms, questionnaires and reports, says James Kulik, director and research scientist at the Office of Evaluations and Examinations (E&E), which administers the TQ system.

"Online collection of student ratings is more efficient," Kulik says. "Paper-based evaluation systems require the printing, mailing, sorting, and scanning of hundreds of thousands of forms each semester. Online systems eliminate the clerical bottleneck, and teachers will receive their results as soon as ratings are collected. They won't have to wait weeks for paper reports to arrive."

A long-term advantage of the Web-based system, Kulik says, will be greater freedom in the design of questionnaires. The current TQ allows users to select prepared questions from an organized catalog, but those seeking feedback still are constrained by the written page and usually must work to fit the evaluation on a single sheet of paper. The online evaluation system eventually will allow a greater level of customization for individual courses or instructors without the page limitations.

Additionally, like paper-based evaluations, online ones protect student anonymity. The technology insures that the student's identity is stripped from the evaluation as soon as he or she logs off. Although the program can keep track of who has filled out the form, allowing participation reminders to be sent, specific responses will be free of identification.

Similarly, comments on the evaluations that often are specific to the instructor can be separated from the quantitative data — an important part of safeguarding potentially sensitive personnel-related information. E&E will not retain copies of student comments once they are forwarded to instructors. It will be up to deans and department chairs, in consultation with their faculties, to determine who, in addition to individual instructors, will receive copies of ratings and student comments, Kulik says.

Course evaluation is an important tradition across the country that dates back to the 1920s. The move to online evaluations does not change what has become the norm: evaluations are used as tools by faculty to receive valuable feedback that can be used to improve the course or their teaching methods. Evaluations also can be useful to administrators during annual reviews or when considering promotion and tenure, and committees can use them when making teaching awards and preparing for accreditation reviews.

The move to a Web-based system was studied by a Task Force on Online Evaluations and Placement Examinations, appointed by Senior Vice Provost Lester Monts. It included faculty and staff, and was led by Gary Herrin, professor of industrial and operations engineering, CoE.

Herrin says successful implementation of online evaluations will require cooperation from all participants — unit leaders, staff members who prepare the TQ, faculty and students.

"One of the things learned from other universities that have implemented online evaluations is that considerable communication is needed to make this a successful transition," Herrin says. "It is going to require cooperation from the entire community."

The task force reviewed the findings of Northwestern University and Brigham Young University, two early adopters of online evaluations, in 2000 and 2002, respectively. Researchers at both universities confirmed the advantages of online rating systems, and those at Northwestern also found one more plus — students offered more written comments on online questionnaires than on paper.

The universities also found some concerns with online systems, including lower response rates and slightly less favorable ratings overall. Researchers at Brigham Young suggested, however, that faculty could encourage response rates by means such as making evaluations a course assignment (not necessarily a graded one).

The task force recommended, therefore, that considerable communication to students occur prior to the evaluation period to stress the importance of their feedback. Leaders plan to use targeted e-mail and other means to reach out to students prior to the trial and full implementation. They also will work with administrators and faculty to suggest ways to encourage participation.

CoE has been conducting online evaluations for two years, so it was a logical place to conduct the pilot, Kulik says. This week, the first communication to administrators and faculty in the college will go out, followed by notification to students.

Course evaluations to go paperless

More Stories