NAEP and Theatre: Framework, Field Test, and Assessment

Abstract: NAEP has developed its first set of assessment tasks for measuring student achievement in theatre. These tasks draw on a wide range of dramatic forms and use both paper-and-pencil and performance tasks.

The Consensus Project¹ identified objectives for arts education in general and individually for dance, music, theatre, and the visual arts, in the fourth, eighth, and twelfth grades. It also developed Assessment Specifications for each of the four arts, setting forth requirements for the content of the assessments in more detail. NAEP field-tested the theatre assessment for the fourth and eighth grades in 1995 and for the twelfth grade in 1997. In the spring of 1999, NAEP will publish a Field Test Process Report, on the development, administration, and scoring of arts tasks for the three grades. The report will cover all four arts, including theatre.

The next two sections of this Focus on NAEP summarize the framework for the theatre assessment, which can be found in full in the NAEP 1997 Arts Education Assessment Framework. Following these sections will be a discussion of the theatre field tests.

Theatre is rooted in the universal human impulse to play, imitate, create, and share ideas and feelings. A social art form, theatre reveals both the human condition and the human experience. In both its traditional forms and the newer media of television and film, theatre transports players and audience through time and space without leaving the here and now.

The NAEP Arts Education Assessment Framework is founded on a vision of a society that believes the arts, including theatre, are essential for every child's complete development and education. To be fully educated, students must be able to draw on artistic experiences and knowledge as a means of understanding both themselves and the world around them. The expectation is not that all students will become professional artists, although some will. What is expected is that they will have experienced enough of the discipline, the challenge, the joy of creating in different art forms to intimately understand the human significance of theatre, dance, music, and the visual arts (see table 1 for guidelines for theatre assessment).

The theatre assessment covers both content and processes. Content includes (1) knowledge and understanding of theatre and (2) perceptual, technical, expressive, and intellectual/reflective skills. Processes include (1) creating/performing, and (2) responding. While much of what students know, as well as what they can do, will be demonstrated through performance, the theatre assessment framework includes a written portion as well. Table 2 sets forth the theatre assessment framework in brief.

Knowledge means knowing about the art of theatre; its history, cultural, social, and personal contexts; its forms and structure, how it creates meaning, and its aesthetic qualities. Students understand the literary, visual, aural, oral, kinesthetic, and psychological aspects of a theatrical event. They are able to engage in self-criticism, taking into consideration form and structure, contexts, and aesthetic responses.

Skills are the abilities associated with the technical, perceptual, and expressive processes of theatre. Included in this category are such activities as skills required in creating a text, acting, staging, designing, and articulating a response. Abilities to create, perform, and respond in the theatre are predicated on the application of both knowledge and skills simultaneously.

Creating and Performing are closely linked in theatre. At the same time, different aspects of theatre call for different creative processes. All call for imagining, conceiving, and generating. Playwriting requires the use of all three processes to create character, story, dramatic actions, and dialogue. Acting involves discovering and developing emotions and creating a series of actions for a character. Designing requires discovering, developing, and organizing an environment. In order to direct, students must analyze a script and develop an interpretation by organizing the time, place, spaces and rhythms of a production. Creating in film making and television demand the development and organization of the elements of theatre as well as the special elements of the medium.

Performing is central to theatre studies. Performing is the evidence of creating; it is the process viewed and heard by an audience. Performing is a highly complex collaborative activity in which the performer is aware of the audience and fellow performers, responding and adjusting the performance accordingly. The ways of learning demanded by performance are equally complex, requiring the interplay of all the processes noted in creating. Performing is impossible to assess through any other means than through the processes themselves.

NAEP used two types of assessment tasks in the field tests: paper-and-pencil tasks and performance tasks. Assessment tasks were prepared by the Educational Testing Service, under the guidance of a committee of theatre education experts. The fourth-grade field test devoted about 60 minutes to the paper-and-pencil tasks and 15- 30 minutes to the performance task. The eighth-grade field test devoted 60 and 30 minutes, respectively, while the twelfth-grade field test devoted 60 minutes and 60- 90 minutes, respectively. In all cases, assessment times are approximate only, depending on the specific tasks involved.

The performance tasks assessed the creating and performing aspects of theatre. The paper-and-pencil tasks assessed students' abilities to respond to theatre, but also required them to make and defend a wide variety of creative decisions as well. To ensure coverage of all aspects of theatrical knowledge, and to lighten the burden on a given student, NAEP developed a number of different task "blocks," for both the paper-and-pencil and performance tasks. Students in the sample were assigned at random to several paper-and-pencil blocks and one performance block.

NAEP performance tasks are designed to be as authentic as time and resources permit. However, authenticity is limited by the fact that the assessment tasks must also offer the same opportunities and the same challenges, in the same circumstances, for all the students assessed. No comparison across students is possible without such standardization. NAEP plans to make available to the public all tasks, both paper-and-pencil and performance, that are not covered by copyright.

In addition to having students respond to theatrical performances, paper-and-pencil tasks asked students to address such issues as set design, decor, costume, casting, cinematic techniques, sound effects, and direction. (Examples are drawn throughout from the twelfth-grade field test.)

Students read excerpts from plays, looked at photographs, posters, and other visual stimuli, and saw video clips, both from films and from a tape of a live theatrical presentation. (See table 3 for a breakdown of the kinds of materials used in the field tests, taken from the Consensus Project's Assessment Specifications.) The tasks used multiple-choice, short constructed-response, and extended constructed-response questions (see table 4 for a sample scoring guide for an extended constructed-response task). Over two thirds were constructed-response rather than multiple-choice.

Performance tasks asked students to work together in small groups (usually two to four students) to do such things as improvise a scene on the basis of a few introductory lines of dialogue or to write and perform a brief play. Video cameras recorded student performances. In some of the tasks, students were asked to comment on their work what they were trying to achieve, and how well they did it.

Because of the collaborative nature of theatre, a student's performance in several of the tasks was a combination of an individual and a group score, unusual in NAEP. Both the paper-and-pencil and the performance tasks allowed students recognition for exceptional performance. Scoring guides for the performance tasks gave credit for students who performed their tasks in a "masterful" manner (see table 5 for a sample scoring guide for a performance task).

In the theatre field tests for grades four and eight there were many issues that NAEP had not dealt with before. Capturing student group performances (two to four students) on video for evaluation presented many challenges. A workable space had to be prescribed on the floor. A video camera, which would remain stationary throughout the performance, had to be properly placed to record the entire scene. Backgrounds were not always taken into account, and there was no special lighting, which meant that it was sometimes difficult to see facial expressions. Since the microphone was on the camera, it was sometimes difficult to hear actors' voices, particularly if the task took place in a noisy area. Facilitators had to have special training, so that they could make the students feel comfortable enough to perform without being "too helpful" unfairly contributing to the students' performance.

In the twelfth-grade field test, NAEP found it necessary to increase the amount of time allowed for performance tasks from 60 to as much as 90 minutes. Traditionally, NAEP has sought to use no more than one class period for an assessment task. However, NAEP found that some of the theatre performance tasks required more time, particularly the tasks that required students to create, perform, and then analyze their work.

The greatest difficulty encountered in the field tests was obtaining a full range of performance from the students sampled. This was particularly true in the eighth-grade field test. In the fourth- and eighth-grade field tests, which were given to random cross sections of students, it was difficult to obtain performances that went beyond the simplest level. To fairly assess the students and obtain results that resembled the breadth of the theatre standards, NAEP decided to sample only students who had had some instruction in theatre, for both the twelfth-grade field test and the eighth-grade assessment. The difference was significant. Student performance evidenced the full range envisioned by the scoring rubrics.

Creating and administering a national performance assessment was very challenging. The NAEP Theatre Framework, Field Test and Assessment provide useful models for future assessments, at the state and local level, as well as nationally. Several important problems were confronted during the development and scoring of the assessment. Among these were how to create tasks suitable for a variety of students with a variety of educational backgrounds, while also making distinctions among levels of student achievement. These issues, among others, will be explored in the upcoming arts reports.

Arts Education Assessment Framework for the 1997 National Assessment of Educational Progress. Single copies are available free from the National Assessment Governing Board, 800 North Capitol Street, NW, Suite 825, Washington, DC 20002-4233. Copies may also be obtained on the World Wide Web at http://www.nagb.org/.

National Standards for Arts Education. Copies are available for $20 from the Music Educators National Conference, 1806 Robert Fulton Drive, Reston VA 20191.

Arts Education in Public Elementary and Secondary Schools, NCES 95- 082, provides data obtained from questionnaires sent to the principals of nationally representative samples of public schools. It is not in print, but is available over the NAEP web site at /nationsreportcard/.

The following Focus on NAEP publications are also available free from the , U.S. Department of Education, Washington, DC 20208- 5653. Copies may also be accessed over the World Wide Web at /nationsreportcard/.

The NAEP Arts Education Framework, Field Test, and Assessment, NCES 98- 527

NAEP and Dance: Framework, Field Test, and Survey, NCES 98- 459

NAEP and Music: Framework, Field Test, and Assessment, NCES 98- 529

NAEP and the Visual Arts: Framework, Field Test, and Assessment, NCES 98- 526

The Focus on NAEP series briefly summarizes information on the ongoing development and implementation of the National Assessment of Educational Progress (NAEP). The series is a product of the (NCES), Pascal D. Forgione, Jr., Commissioner, and Gary W. Phillips, Associate Commissioner for Education Assessment. This Focus on NAEP was written by Alan Vanneman, of the Education Statistics Services Institute, in support of NCES, Claudette Morton, chair of the NAEP Arts Standing Committee and co-chair of the Theatre Planning Committee, and Lisa Beth Allen of the Sundance Institute, Theatre Program.

The NCES World Wide Web Home Page is //.