EVALUATION OF PROGRAMS


OPE: Office of Postsecondary Education
Current Section
Lessons Learned from FIPSE Projects III - June 1996

Written in Collaboration with 31 Directors of College and University Reform Projects Who Tell What Worked, What Didn't Work, and Why


Table of Contents

Title Page

I. PREFACE

II. OVERVIEW

III. SCHOOL-TO-COLLEGE TRANSITION AND RETENTION

1.University of Michigan: Evaluation and Dissemination of an Undergraduate Program to Improve Retention

The Undergraduate Research Opportunity Program (UROP) pairs first- and second-year students with faculty to collaborate on original research projects. Students also participate in an intensive peer advising program, and a series of workshops and other activities. The evaluation of UROP sought to determine its effect on student academic performance and retention to graduation, on students' attitudes towards their intellectual development, and on faculty perceptions of students. The program has had an especially strong positive effect on student retention.

2. Rollins College: The Science Community Year--Addressing Attrition in Science .

This project attempts to stem the tide of attrition of students in science. First-year college students take science and math courses along with "master learners," non-science faculty members who are novices themselves and take the courses along with the students. Faculty act much like classroom ethnographers, learning together with students, teaching integrative seminars, and conveying to course instructors a sense of students' problems and frustrations. This feedback has resulted in substantial cur ricular improvements in chemistry, biology and calculus courses. It has been especially successful in doubling the number of declared majors in science among participants.

3. Eastern Washington University: The Single Parent Project.

To overcome the practical barriers that single parents face in attending and staying in school, a coalition of campus units and state agencies offers students on public assistance specially tailored academic and social services. Housed in campus dorms, students receive counseling in crisis intervention, financial aid, assistance in locating internships, tutoring, child care, food and clothing. Collaborative learning, using student study groups, induces most participants to achieve above-average grades and to continue to graduation.

4. Anne Arundel Community College: Supplemental Instruction With Mentoring Support.

The project explores some non-traditional ways to provide students in required science, mathematics, business and social sciences courses the support they need for academic survival and success. The strategies include study sessions conducted by student leaders, contact with faculty mentors who also function as "master learners," and visits from community mentors to provide a link between disciplinary study and professional practice. The evaluation shows that those who chose to participate in suppleme ntal instruction had markedly higher grades and course success rates than non-participants.

5. CUNY - The City College: Fluency First in English as a Second Language.

To reduce the high failure rate of non-native English speakers in writing courses, this project adopts the "whole language" approach to teaching English as a second language to a college-age population. Rather than confining discourse to a particular aspect of grammar in each lesson, the "whole language" approach stresses a more natural use of oral and written language, and leads students from fluency to clarity and eventually to correctness.

6. CUNY - The City College: Access to Science Study (PASS) .

This project is designed for students unprepared to enroll in science and math courses at the college level. It consists of a preparatory science course taught in tandem with a special counseling seminar, both of which help students develop the academi c habits and self-discipline to succeed in college and later in science-related careers. When compared to non-participating students, PASS students showed dramatic improvements in their math and science performance, better course retention, and higher grade point averages.

7. CUNY - College of Staten Island: Project Discovery.

The program, devised by college faculty working with two high schools, one in Staten Island and one in Brooklyn, helps to hold the interest of average students who might otherwise drift off the academic track. Discovery-oriented laboratories and exercises encourage students to learn for themselves, and a team of teachers from the basic disciplines works with a common group of students using an integrated curriculum. According to the evaluation, Discovery students in the Staten Island school performed significantly better than the non-participating students in math and science, and earned higher grade point averages. Students in the Brooklyn school showed no differences in performance.

IV. REWARDING EFFECTIVE TEACHING.

8. University of Nebraska at Lincoln: Improving Teaching at a Research- Oriented University.

Is it possible to enhance recognition of teaching excellence at a large research-oriented university, thereby strengthening the climate for teaching and learning? In 25 of 28 participating departments of agriculture, natural resources, and arts and sciences, faculty generated and adopted written plans and strategies for documenting, evaluating and rewarding teaching. Ten additional departments joined this effort after funding ended, for a total of 36 units currently engaged in changing the status of undergraduate teaching. Most measures reveal statistically significant improvements in the climate for teaching. By institutional policy, outstanding achievements in teaching and research are now being rewarded equally.

9. Rhode Island College: Investigating a Linkage Between Reward for Good Teaching and Improved Student Performance .

To explore the relationship between faculty incentives and student learning, seven departments--sociology, chemistry, social work, nursing, management, modern languages and special education--formulated and carried out plans to establish learning goals, assist students in reaching them, and assess the result of their efforts. Faculty whose students made significant learning gains obtained support for professional development activities such as travel or equipment purchases.

V. IMPROVING TEACHING AND LEARNING.

10. Mount Holyoke College: Increasing Access to Advanced Mathematics.

Mathematics Department faculty experimented with greatly reducing the requirements for certain advanced courses to make them available to non-majors. The creation of a Laboratory in Mathematical Experimentation and group problem solving strategies are key elements in the project's success.

11. Saint Anselm College: Using Computers for Collaborative Writing .

The inability to think critically while composing essays is often as problematic as the lack of mastery of writing mechanics. In this experiment, English faculty assumed that writing can be used as a way of learning. They formed co-authoring groups of three students to write common essays at the computer, and critiqued them on-line, in a continuing student- teacher dialogue. Even though co-authoring students were required to write less than students using traditional techniques, they improved more as writers and thinkers, and weaker students using co-authoring made greater progress than weaker students using traditional techniques.

12. University of Delaware: Computer Assisted Communication Within the Classroom--Interactive Learning.

The goal of this project was to allow students, regardless of their preparation, to follow step-by-step problem-solving procedures in an elementary astronomy course. The course was conducted in a classroom wired for student response/display pads connected to a central microcomputer operated by the instructor. This system enabled the instructor to provide individualized feedback to students during regular class sessions.

13.Washington State University: Enhancing Graduate and Undergraduate Training in Speech and Hearing Sciences .

The relationship between scientific theory and therapeutic practice is not always obvious to speech pathology students. At Washington State, faculty created instructional modules and software for microcomputer-based laboratories that help to clarify this connection. Students tested before and after the laboratory exercises and at the beginning and end of each semester showed strong learning gains. When compared to students taking the same speech science courses at another university not using the lab oratory exercises, Washington State students performed significantly better.

14. California State University at Northridge: Effects of Assistive Technology on Postsecondary Students with Learning Disabilities .

This project sought to determine the immediate effects of optical character recognition, speech synthesis, screen review and speech recognition on the reading comprehension, proofreading and written composition of learning disabled students. The project also investigated the long-term effects of assistive technology on the performance of learning disabled students in classes with heavy reading or writing components, in writing proficiency examinations, on retention and personal development.

15. Texas College of Osteopathic Medicine: Expert Systems Shell-Based Programs for Medical Education.

To strengthen the diagnostic capabilities of medical students early in their residency, osteopathic faculty employed the artificial intelligence-derived Knowledge-Based Inference Tool. First they determined that the tool was able to make highly reliable distinctions between the diagnoses of experts and novices and then developed expert systems that allow students to encounter presenting symptoms, make diagnoses, and compare their analyses with those of experts. A comparison of diagnostic results obtai ned from students using these computer-based instruments, from untrained students, and from students trained conventionally showed the validity of the artificial intelligence tool to improve diagnostic training.

VI. IMPROVING THE UNDERGRADUATE CURRICULUM.

16. The Community College of Aurora: Integrating Ethics Across the Curriculum .

Within a framework of instructional reform, 84 community college faculty from 20 departments learned how to infuse ethics into their individual courses as well as into the entire curriculum. Scores on the Defining Issue Test (a measure of moral growth) showed an overall favorable shift in the capacity of students in these courses for principled, moral thinking. Subsequently, the project linked ethics to service learning as a way to encourage civic responsibility among students. The college is dissemi nating this model through interdisciplinary ethics seminars and curricula to six other community colleges.

17. University of Minnesota: Ethics Education for Baccalaureate Nursing Students-- Multi-Course Sequential Learning .

Embedding a variety of modules within classroom and clinical courses throughout the curriculum represents a new approach to ethics education for nurses. The ethics faculty designed learning objectives and teaching procedures for each module so that students would consider ethical issues at all stages of their program. Student learning was evaluated in multiple ways, including take-home exams, course exams, term papers, small-group written exercises and graded group presentations. Most students performed better than average on these measures. They also showed significant improvements in their scores on the Defining Issues Test (a measure of moral growth) and in their moral reasoning abilities on clinical performance.

18. Kennesaw State College: Earth Algebra .

Mathematics faculty devised an introduction to college algebra that teaches all the relevant concepts of the course in terms of the problem of global warming. The course produced learning equivalent to regular algebra sections, much more positive student attitudes toward mathematics than standard sections, and greater gains in data analysis and mathematical modeling. A textbook for the course has been adopted at dozens of institutions.

19. University of Oregon: Workshop Biology for Non-Majors--Promoting Scientific Literacy Through Investigative Laboratories and Issue-Oriented Activities.

This three-term, laboratory-based introductory sequence seeks to enable students to make informed, critical decisions, consistent with their values, on science-related issues. The course acquaints students with scientific inquiry, critical thinking and decision-making, and the relationship of scientific knowledge to social issues.

VII. ASSESSMENT.

20. University of Connecticut: Assessing General Education Outcomes-- An Institution-Specific Approach

This project sought information about whether a student's performance in a new general education curriculum improves over time, and whether the number of courses taken in a general education area affects performance. The assessment process stressed the faculty's role in generating general education goals and deciding what constitutes evidence of success in achieving them. The project resulted in 16 locally-developed assessment instruments that match the new goals.

21. Miami University: Portfolio Writing Assessment in Student Placement .

In cooperation with high schools, the English Department developed a portfolio assessment process for new student placement in writing courses. The portfolio replaces the single piece of writing produced under timed conditions that is the usual basis for placement. Despite the added time that portfolio rating requires, the department has made it the preferred placement method.

22. University of Wisconsin at Madison: An Ability-Based Assessment Program at the Medical School .

This outcomes-based approach to evaluating medical students tests non-cognitive as well as cognitive abilities. Faculty identified nine generic abilities required for admission to residency training, and developed explicit criteria by which to judge these abilities throughout the curriculum. Faculty also created clinical vignettes using standardized patients that would allow them to teach as well as assess the desired skills.

23. Mathematical Association of America: Software for Computer-Generated Math Placement Tests.

The many colleges and universities that administer placement tests face the problem of devising multiple versions of the test that are equivalent in difficulty. The Mathematical Association of America has created a large bank of questions for several different levels of readiness that can be assembled into tests whose various versions are statistically equivalent.

VIII. TEACHER EDUCATION.

24. Baylor College of Medicine: Houston Elementary Science Alliance .

Believing that science education must begin early, the Houston Elementary Science Alliance (consisting of three research universities and a museum) provided extensive science content and hands-on activities to elementary school teachers. The intention was to provide a "cascade" effect: learning passing from the faculty trainers to teachers taking the workshops, to their colleagues in the schools, and from them to students. On a science knowledge test given before and after the training, teachers showed an increase of over 80 percent, and the evaluator noted that all of them were successfully employing the teaching strategies they learned in the training.

25. University of California at Berkeley, Lawrence Hall of Science: Enhancing Science and Mathematics Education for Child Care Providers and Preschool Teachers.

Staff of the Lawrence Hall of Science developed and piloted science modules for pre- school children. They trained child care workers in both public and private settings to teach these modules and have published curriculum materials and teaching handbooks, some in Spanish and English.

26. The Center for Applied Linguistics: Improving Elementary School Foreign Language Teacher Education .

When North Carolina mandated the teaching of foreign languages in elementary schools, state education leaders turned to the Center to devise a program to prepare college and university teacher trainers to offer the necessary K-12 certification. The college faculty, most of whom had no experience with elementary foreign language teaching, were mentored by experienced elementary school teachers. Both college and school teachers then worked together to devise a model curriculum, which has been full implemented. In the process, college and school faculty have gained a new regard for each other.

27. New York Hall of Science: The Science Teacher Career Ladder .

To increase the number of women and minorities in science, the New York Hall of Science teamed up with Queens College of the City University of New York in recruiting potential scientists and science teachers from museum docents. Queens College students were provided paid work as docents, a seminar connecting the job to academic work, and a tuition waiver for science and education courses, with the understanding that they would teach science in the schools for two years after graduation. Follow-up on 100 students in the program, the majority of whom were women and minorities, showed that 57 of them later engaged in a science career. The project received generous external funding from a variety of agencies, and to date 53 other science museums have adapted the science career ladder model.

28. Pace University: Case Studies for Teacher Trainers.

By developing case studies modeled after those used in business, Pace faculty hoped to bring examples of complex classroom problems to education courses. The case studies pushed students toward linking theory to practice, and increased their skills in structuring problems, raising questions, and proposing solutions. After testing and revising the completed cases, 28 were published for use, along with an instructor's manual. Both the project directors and Pace faculty noted improved student abilities in applying theory productively to actual situations. Pace University has underwritten the costs of a Center for Case Studies in Education that produces customized packages of cases to meet an individual instructor's preferences.

29. University of Oregon: Beyond Academics in the Preparation of Education Leaders.

Rarely does a preparation program for elementary and secondary school administrators stress the instructional leadership and school management skills needed in the principal's role. This program provides ample time for instructional collaboration between faculty and administrators by engaging them as co-teachers. Using the Assessment Center of the National Association of Secondary School Principals, teachers who aspire to become administrators identify the skills they need to develop in school-based internships the following year. Several evaluation studies showed that program participants used the skills acquired during their internship years in their jobs. Compared to a matched group of students in traditional training, they showed superior ability to apply skills and theories, and greater success in securing administrative positions after certification.

IX. POSTGRADUATE CURRICULUM AND INSTRUCTION.

30. California School of Professional Psychology at Alameda: Multicultural Education in Professional Psychology.

Students preparing to become clinicians often do not have the knowledge or sensitivity necessary to work with culturally diverse clients. In response, faculty augmented their curriculum based on a set of clinical competencies appropriate to the treatment of diverse ethnic identities. They then modified treatments they used with clients in light of these competencies. A comparison of course syllabi before and after the project revealed major changes in content to reflect the needs of various ethnic groups.

31. Georgetown University Law Center: Reform of the First-Year Curriculum for United States Law Schools .

This project addresses the shortcomings of the traditional first-year law curriculum, which usually fails to give students the economic, social, and political context of laws, and a sense of what lawyers actually do. Eight newly devised courses were taught to one fourth of the entering students for each of two years. Beyond surveying student reactions with questionnaires and "town meetings," a faculty oversight committee attended the new classes and read the course materials. They concluded that the experiment should be continued for five years, even though the grades of these students were no higher than those of non-participating students.



[Return to FIPSE Home Page] [Return to Publications Page]

 
Print this page Printable view Send this page Share this page
Last Modified: 03/08/2007