Purpose

While curricular innovation is most frequently beneficial, if only because of the discussion and reflection it generates among faculty, rigorous assessment is essential in order to ascertain whether the new course of study has brought about the intended improvements in student learning. When SUNY College at Fredonia instituted a skills-oriented general education program (the General College Program, or GCP) in the early 1980s, the College made a commitment to assess the curriculum once it had become well established. The project was designed to investigate whether students were learning what faculty thought they were teaching, and whether the learning that was occurring resulted from the instruction the students received.

Since the GCP does not require all students to take the same courses, ways had to be found to measure skill acquisition that would not depend on academic content. Moreover, Fredonia faculty and administrators were determined to ascertain whether gains in student growth could be attributed specifically to their teaching rather than to the normal process of maturation. Before work could begin on any of these issues, however, faculty had to define the skills and understandings that students were expected to develop in sufficient detail to allow measurement.

Innovative Features

Persuaded that the assessment of liberal education should be an exercise in it, the proposal to FIPSE granted the twelve faculty members of the Assessment Committee ample time to engage in extensive study and reflection on the topic of assessment. At the outset, none of the committee members was an expert in the field. Nevertheless, as liberally-educated individuals and the ultimate experts on their own campus and students, they confidently embarked on the exploration of assessment. As a result, the project generated a sense of intellectual discovery--and therefore commitment--among faculty that would have been impossible without the opportunity to undertake intensive study and discussion.

The group's first decision was to reject standardized tests. Designed for students from infinitely varied backgrounds and diverse types of institutions, the instruments did not adequately match Fredonia's curriculum, and faculty believed that the objective format would test recognition rather than original thought or the ability to manipulate ideas.

The Committee members concluded that they would develop their own short-essay tests of the skills addressed by the curriculum: reading, writing, reflexive thinking, scientific reasoning, mathematical problem-solving, and socioethical understanding (critical thinking was subsumed in the preceding categories). The tests would be administered to freshmen and upper classmen as well as to a comparison sample composed of freshmen and upperclassmen at Miami University and Miami's Western College, selected because their high school achievement levels were thought to be similar to those of Fredonia students, and because at that time Miami University did not have a skills-oriented general education program, but rather simply a distribution requirement.

Knowing students' proclivity to provide the answers they believe to be expected of them, Fredonia's Assessment Committee strove to elicit genuine responses rather than professions of faith. Thus the tests used open-ended questions and posed dramatic situations (a European exchange student criticizing certain aspects of American life, for example) to which students responded spontaneously and often passionately. The blind scoring system also examined essays for qualities not explicitly mentioned in the questions--such as ethnocentrism, or an exclusive focus on the present.

Recognition

The careful and unusual design of Fredonia's assessment project quickly attracted notice. Within the SUNY system, Fredonia has become a leader in assessment, participating in four regional SUNY assessment workshops. Members of Fredonia's Assessment Committee are in frequent demand as consultants; one faculty member served in the Exxon-funded 1989-90 portfolio project, described in the American Association of Higher Education's Time Will Tell: Portfolio-Assisted Assessment of General Education. The College maintains a mailing list of requests for information about the project, has made presentations at three AAHE assessment conferences and at regional conferences, and has conducted workshops for the South Carolina and Washington State assessment networks. A description of the project appeared in Assessment Update. Professor Karl Schilling of Miami University, currently director of the AAHE Assessment Forum, who collaborated with Fredonia and directed his own FIPSE assessment grant (see page 31), has integrated information about the Fredonia project in his own presentations and consulting.

Project Impact

The Assessment Committee produced nine tests, with scoring manuals, of reading, writing, reflexive thinking, scientific reasoning, mathematical problem-solving and socioethical understanding. These are instruments in which the faculty has confidence, that are sufficiently precise to allow for focused planning, and that can be used in the future to yield a longitudinal perspective on student progress. In the process of designing and administering these tests, faculty reflected on the purposes of the curriculum, learned about results of their program, and drafted recommendations for its improvement.

The findings indicate that Fredonia students make least progress in critical thinking--particularly with respect to discerning their own or others' assumptions and biases--and in problem solving. Students showed little awareness of themselves as learners, and exhibited poor scientific reasoning. They also remained quite ethnocentric in their views, and failed to establish connections among different courses as well as disciplines.

Although the Miami University and Western College populations turned out to be less comparable to Fredonia students than had initially been hoped, the results of comparisons between the two groups showed a consistent pattern. In general Miami freshmen scored at the level of Fredonia upperclassmen. However, there was a greater difference between the scores of Fredonia freshmen and upperclassmen than between beginning and advanced groups at Miami University.

This was cautiously interpreted at Fredonia as meaning that the curriculum, and the way it is taught, is responsible for the gains made by Fredonia students. This finding is all the more positive in light of research that indicates that better prepared students (in this case, Miami University freshmen) grow faster academically than their less-prepared peers.

Rather than suggest specific means of improvement, the recommendations on strengthening general education outlined by the Assessement Committee point to areas in need of attention. Two faculty study groups were formed to respond to these recommendations (see below).

Major Insights and Lessons Learned

The decision to dispense with outside experts and standardized tests not only produced custom-made instruments eminently suited to the campus and the curriculum, but resulted in a high sense of ownership of the project on the part of the faculty. This was enhanced by the visible commitment to assessment by administrators at the highest levels, and by the choice of highly respected faculty to serve on the Assessment Committee.

The project also yielded much incidental information on student learning, which, while often puzzling, nevertheless gave faculty a deeper insight into teaching and learning at Fredonia. Finally, the task of assessment itself was instructive, forcing faculty to give shape to their ideas of what constitutes effective teaching, and giving students the opportunity to reflect and write on a number of issues, and thereby increase their awareness of themselves as intellectual beings.

Participants in the project, having found standardized tests insufficient to their needs, welcome the national trend towards embedded assessment. Before this can succeed on a large scale, however, scoring techniques for long-term qualitative measures of portfolios, essays, and recorded interviews must be developed. The project findings, and more than likely the collaboration with Miami University as well, have caused project participants to reflect on the need to develop ways of measuring the correlation between academic and extra-curricular involvement.

Unanticipated Problems

It was learned belatedly that, while both Fredonia and Miami University freshmen had high school averages of B+, the numerical equivalent of that grade was 86 in New York and 91 in Ohio. Thus, the Miami University students were better prepared academically than their Fredonia peers, and this affected the assessment process.

In addition, although the purpose of the project was to find out the effects of the GCP specifically, it became impossible to distinguish whether the gains made by upperclassmen were due to the GCP courses alone or to other courses in the curriculum, especially since the College's entire program aims to develop the skills being measured.

Project Continuation

Two faculty study groups were formed to address the findings of limited student improvement in identifying biases and assumptions in reading and in their own thinking, and insufficient improvement in problem solving. As a result, workshops on problem solving have been instituted for faculty teaching introductory courses in mathematics.

Some changes have been made in the GCP itself. The departments of Chemistry, English, Foreign Languages, History, and Political Science experimented with portfolios as a way to help students improve reflexive thinking and make connections among courses. English and Foreign Languages have since made portfolios a permanent part of their programs. In general, the revised GCP emphasizes the attributes of thinking evaluated by the project, and increases the amount of attention to cross-cultural or international matters required in all fields.

To assess the effect of these changes, the College is repeating its study. All nine tests were given to incoming freshmen in summer of 1991, and will be repeated with a sample of these same students in spring of 1994.

Available Information

A substantial final report was filed with FIPSE at the end of the project. Two campus reports have been produced. The GCP and Student Learning: A Report To The Campus, appeared at the conclusion of the grant. The second, entitled GCP Study Group Reports, presents the conclusions of two faculty groups that met to examine the findings and recommendations of the Assessment Committee in the areas of critical thinking and problem solving. For information about these reports, contact:

Minda Rae Amiran
Department of English
State University of New York
College at Fredonia
277 Fenton Hall
New York, New York 14063
716-673-3125

[Miami University] [Table of Contents] [The State University of New York College at Plattsburgh]

FIPSE Home