Evaluation of an Out-of-the-Window Air Traffic Control Tower Simulation for Controller Training
Preface
This investigation evaluated the effectiveness of an out-of-the-window tower simulation by studying
the first year of training conducted in the simulated tower. The author wishes to express gratitude
to the ten student controllers whose training was studied in this evaluation. Their many observations
provided an invaluable source of insight about the simulation.
Jon Bremseth, an O'Hare training specialist, operated the simulation. He previously developed a training
laboratory using videotapes correlated to flight progress strips, setting the stage for the current,
interactive simulation training facility. Jon served as the facility contact person for this evaluation.
He also assisted this evaluation by ensuring that one session was videotaped for each trainee each week
and by providing the corresponding flightstrips which were used in data reduction. He contributed useful
observations on the productive use of out-of-the-window tower simulation, many of which appear in this report.
Many other individuals at O'Hare deserve recognition for their outstanding contributions to this
evaluation. Ellen Jaeger is the Assistant Manager for Training at O'Hare. She and Roy Hillen, an
O'Hare training specialist, collected preliminary data on trainee errors during their simulation training
and provided information about the trainees' prior experience. Tower supervisors Bob Karnick and Tim
Fitzgerald, and area manager Kevin Markwell provided ratings of trainee skills. Matt Dunne, Acting Assistant
Manager of Traffic Management provided typical taxi delays, taxi times and aircraft rates during an outbound
push. The author also recognizes and appreciates the critical support provided by Tower Air Traffic Manager, Bill Halleck.
The author thanks three individuals for their help with technical aspects of this project. Mitch Grossberg
of FAA/ACT-500 contributed valuable suggestions on the format used to collect supervisor ratings. Patricia
Pilanen, a tower training specialist at Logan Airport, answered many of the author's questions about tower
training and drew the author's attention to the need for ground control trainees to learn to manage flight
progress strips without reducing their attention to the airport movement areas. Karl Hergenrother of RSPA/Volpe
Center created the data reduction programs used to reduce duration data from videotape, developed videotape data
reduction procedures, and contributed valuable observations on the simulation process.
Objective data reduced from videotape played a key role in this evaluation. This exacting work was accomplished
by Tufts University engineering psychology students, Ana Pons, Kathleen Kim, and Trudi Leone, while they were
employed at RSPA/Volpe Center. Bill Voss at ATZ-200 provided the requirement for this study. Larry Cole at
AAR-200 sponsored the effort. Their continuing interest and support were critical to the completion of this project.
Return to Top
Table of Contents
1. Introduction
1.1 Purpose
1.2 Background
1.3 Constraints on the Evaluation
1.3.1 Lack of a Concurrent Control Group
1.3.2 Ongoing Training
1.3.3 Sample Size
2. The Simulation
2.1 Description
2.2 Limitations of the Simulation
2.2.1 Partial Field of View
2.2.2 No Inbound Aircraft
2.2.3 Automated Controller Speech Recognition/Synthetic Pilot Speech
2.3 Precursor to Simulation Training at O'Hare
3. Out-of-the-Window Simulation Training Methods
3.1 Trainees and Schedule
3.2 Simulation Training Procedure
3.3 Modification of Training Methods
4. Evaluation Method
4.1 Measures of Effectiveness
4.2 Videotaped Training Sessions
5. Results and Discussion
5.1 Post-Simulation Training Measures
5.1.1 Days-to-Certification
5.1.2 Hours-to-Certification
5.1.3 Tower Supervisor and Area Manager Ratings of Ground Control Skills
5.2 Simulation Training Measures
5.2.1 Preliminary Data Considerations and Analysis
5.2.2 Taxi Delay
5.2.3 Taxi Time
5.2.4 Window and Stripboard Scan
5.2.5 Analysis of Instructor Assistance
6. Findings and Recommendations
6.1 Training Using the Simulator was Effective
6.2 Training Using the Simulator was Faster and More Effective then Traditional Training
6.3 Training Using Tower Simulation is Likely to Show Increased Benefits Upon Upgrading
6.4 Current Cost of Recommended Upgrading
6.5 New Applications of Tower Simulation Technology
Appendix A: Focus Group Summary
Appendix B: Baseline Development of Ground Control Skills
Appendix C: Outline of Recommended Requirements for a Tower System Assessment Simulator
Return to Top
Executive Summary
The objective of this evaluation was to gather evidence bearing on the potential usefulness of out-of-the-window
ATC tower simulation for training tower controllers. The FAA Research and Development Service (ARD) sponsored the
development of an out-of-the-window simulation as a proof-of-concept demonstration. It was about to be used for the
first time to train developmental controllers at O'Hare International Airport in Chicago. This simulation was a
prototype (beta) version of TowerPro (Aviation Simulations, Inc.). It had been developed specifically to train
outbound ground control at O'Hare. The data for this evaluation were collected on the progress and relative
post-training proficiency of all ten developmental controllers who completed simulation training at O'Hare during
1994, the first year of its use.
The simulation implemented at O'Hare included one controller position, outbound ground control. An out-of-the-window
view was projected onto three visual displays which approximated the size of actual tower windows.
Computer-generated aircraft appeared to move in three dimensions on these displays at rates which varied
with the type of aircraft. The aircraft were representative of the types and companies at O'Hare. A session
included 30 to 90 aircraft in 50 minutes.
The simulation was limited to three screens, so that only a 135 degree section of the airport could be displayed
at any one time. Controllers at O'Hare must observe aircraft movements on all sides of the tower. In the
simulation, instead of turning to look through a different window, the trainees used a hand-held control to rotate
or "pan" the displayed section to show the relevant part of the airport. The simulation was also limited to
outbound aircraft: No inbound aircraft were included in the simulation scenarios.
The evaluation was constrained by the lack of a concurrent control group consisting of
developmental controllers who would have received only traditional training. Instead, comparisons were made between
the trainees who used the simulation and those who had previously been trained at O'Hare, prior to the installation
of the simulation.
The primary measures of simulation effectiveness concerned how long (days and total hours) it took a developmental
controller to become certified on the outbound ground control position. The primary measures were supplemented by
ratings made by tower supervisors on specific ground control skills, following simulation training. These expert
ratings were compared to the supervisors' and an area manager's recollections of the baseline strength of
developmental controller skills after varying amounts of time.
Additional data were reduced from videotapes of the simulation training sessions. The measures obtained included
taxi delay, defined as the time it took the ground controller to provide taxi instructions to an aircraft that was
ready to taxi; taxi time, which began when an aircraft received taxi instructions and ended when it was instructed
to switch radio frequencies to a tower frequency; and visual scanning directed toward the "window" or toward the stripboard. Instructor comments made during the videotaped sessions were classified according to topic and tallied to provide additional indications of the development of specific ground control skills. Also, the first group of trainees' perceptions of the simulation training were solicited in a focus group session.
The data collected support the following conclusions:
The out-of-the-window tower simulation used at O'Hare:
- is an effective tool for training many ground control skills and knowledge, and appears particularly effective in increasing trainees' working speed.
- is more effective for training outbound ground control than more traditional training techniques.
The effectiveness of the simulation was indicated by the following:
I. Trainees' ground control skills increased noticeably during simulation training.
Taxi delay decreased consistently from the second week to the end of training.
Stripboard scanning began to consistently decrease after the second week of training, while window scanning began to
increase after the third week of training.
The amount of assistance needed for a trainee to properly handle a scenario decreased consistently throughout training.
II. Training using the simulator was more effective than traditional training.
The developmental controllers who were trained using simulation became certified on the outbound ground control position
in 25% fewer days than developmental controllers who were trained without simulation. However, using simulation, the
trainees needed about the same (only 5% fewer) hours to become certified on this tower position.
Expert ratings of eight ground control skills based on actual tower observations were all higher following simulation training than following traditional training.
The difference between the days-to-certification and total hours-to-certification results can be explained in terms of working speed:
The Working Speed Hypothesis: Simulation training increased developmental controllers' working speed, which enabled them to work in the actual tower under a wider range of conditions (i.e., under heavier or more complex traffic) and hence for more hours per day than with traditional training.
View complete document (MS Word, 788KB)
