III. DATA AND MEASURES

DATA

The data used to assess the program administration, rider education, and licensing practices implemented in the 47 States with rider education programs was collected from multiple sources, including published documents, the Internet, postal and e-mail correspondence, and telephone interviews. This data collection was conducted by the American Institutes for Research under a contract with NHTSA and is described in further detail in the publication Motorcycle Rider Education and Licensing: A Review of Programs and Practices (NHTSA, 2005). All data reflects rider education programs and licensing practices in calendar year 2001, the most recent year for which published data was available. Data collection began with an Internet search to compile a list of State rider education and licensing Web sites. The Web sites were carefully reviewed, and relevant data was classified into promising-practices areas organized within the larger three categories of program administration, rider education, and operator licensing.

Following the Internet search, project staff examined published documents containing information about State motorcycle programs and licensing procedures. This effort focused primarily on a review of material from the Motorcycle Safety Foundation (MSF) and the National Association of State Motorcycle Safety Administrators (SMSA), and from annual reports from State motorcycle rider education programs. All data collected from Internet searches and from the review of published documents was entered into a database. Using this database, project staff created draft State profiles that were sent to State program coordinators for review. The database was updated on the basis of changes made to the profiles by the coordinators and through additional correspondence aimed at resolving data contradictions and gathering missing data.

MEASURES

Table 1 presents the variables and scoring rubrics used to measure the promisingpractices model. The variables and scoring rubrics are organized by the three core dimensions of the model: program administration, rider education, and licensing. Of the 33 measures in the model, 30 were scored as dichotomous, with 1 indicating the presence of a practice consistent with the promising-practices model and 0 indicating the absence of a practice. The remaining three measures were scored on a 3-point scale, with 0 indicating no features consistent with the model, 1 indicating some features, and 2 indicating all features in place.

Table 1. Variables and Scoring Rubrics for Promising-Practices Model

Program Administration
Variables	Value Levels	Score Point*
Licensing agency same as rider education agency	No Yes	0 1
Ratio of State budget to number of operators	Values in the highest quartile coded as 1, all others as 0	0 1
Data available in electronic format	No data available Some data available Licensing data only Rider education data only All data available	0 1 2
Link between crash data and rider training and licensing data	No Crash data linked to licensing data only Crash data linked to rider education data only Crash data linked to both licensing and rider education data	0 1 2
Total possible score for program administration scale: 6.0

Rider Education
Variables	Value Levels	Score Point*
State offers novice MSF curricula	No Yes	0 1
State offers experienced MSF curricula	No Yes	0 1
Average waiting period to take course	None less than 1 month 1 to 3 months Greater than 3 months Variable	1 0
Total number of students on waiting list in 2001	Reverse Coded: Values in the highest quintile coded as 0, all others as 1	0 1
Courses per licensed operators	Values in the highest quintile coded as 1, all others as 0	0 1
Sites per 10,000 licensed operators	Values in the highest quintile coded as 1, all others as 0	0 1
Annual budget for public information and education	$0 $1 – $4,999 $5,000 – $19,999 > $20,000	0 1
Point reductions on license for rider education graduates	No Yes	0 1
No cost for rider education courses to student	No Yes	0 1
Reciprocity for rider education completed in another State	No Yes	0 1
Skills test waiver for successful completion of State rider education course	No Yes	0 1
Knowledge test waiver for successful completion of State rider education course	No Yes	0 1
Adult student cost for novice course	Reverse Coded: Values in the highest quintile coded as 0, all others as 1.	0 1
Rider licensed upon successful completion of rider education program	No Yes	0 1
Program conducts regular, scheduled evaluations	No Yes	0 1
Type of evaluation/evaluation process	Review of student/instructor evaluations Annual report Some other formalized evaluation None	1 0
State has formal quality control procedures	No Yes	0 1
Frequency of quality control implementation	Multiple times per year Annually Some other set schedule Intermittently, no set schedule	1 0
Instructors are State-certified	No Yes	0 1
State offers Instructor Preparation Course (IPC)	No Yes	0 1
Instructors complete internship/probationary period	No Yes	0 1
State has reciprocity for instructors trained in other States	No Yes	0 1
Total possible score for rider education scale: 22.0

Licensing
Variables	Value Levels	Score Point*
Graduated licensing	No Yes	0 1
Primary knowledge test	Modified MSF Local None MSF	1 0
Primary skills test	None Alternate MOST1 MLST2 Local off-street Local on-street M/C in traffic3	0 1 2
Primary operator’s manual used	Modified MOM4 Local MOM5	0 1
Agency responsible for training examiners	DOT/DMV DOE Law enforcement Other State agency Private contractor Rider Ed./ Safety Program	0 1
Riders under certain age must complete rider education course for license	No Yes	0 1
Reciprocal license waivers	No Yes	0 1
Total possible score for licensing scale: 8.0

Below we provide additional discussion of the variables used to measure the various components of a State’s practices along the areas of program administration, rider education, and licensing.

Program Administration

State program administration practices were assessed through four variables. Program integration was measured by a single dichotomous variable, and the adequacy of a State’s funding source was measured through the ratio of a State’s budget to the number of motorcycle operators. The last two measures under program administration represented a State’s capacity for collecting rider training and licensing data. The availability of data in an electronic format and links between crash data and rider training and licensing data were scored on a 3-point scale, with the highest scores assigned to States that maintained extensive data archives.

Rider Education

Twenty-two variables were used to measure a State’s rider education practices. Because all State programs used MSF curricula in their classes, States were assessed on the basis of whether they offered both novice and experienced courses. Effective training delivery was measured through four variables, two of which captured the speed at which students were able to enroll in classes. Waiting times for classes and the total number of students on a waiting list in 2001 were both recoded to dichotomous variables, with a 1 assigned to States that demonstrated some speed in meeting the demand for training. Two continuous variables, the ratio of courses to licensed operators and the ratio of sites to operators, were also rescored dichotomously. Positive scores on these variables indicate that a State was in the upper tier of the distribution across all States for offering courses and training sites.

The third feature of rider education promising practices, outreach, and information efforts, was measured by a dichotomous variable indicating whether the State program expended any funds to advertise its courses. Incentives encouraging operators to enroll in classes were assessed through six dichotomous variables: reductions in points for successful completion of a rider training course, no cost for courses, reciprocity for rider education completed in another State, skills test waivers and knowledge test waivers for successful completion of rider training, and the implementation of a “one-stop shop.” A seventh measure, adult student cost for a novice course, was recoded to a dichotomous indicator on the basis of the distribution of novice course fees across States.

Four variables were used to measure the implementation of regular program assessments and quality control efforts. States were assessed on the basis of whether they conducted regular, scheduled evaluations and, if so, the type of evaluation process they employed. Both variables were scored dichotomously. Quality control was assessed through two similar dichotomous measures, one for the implementation of a quality control program and the other for the frequency with which it was administered. The final feature of rider education practices, instructor education and certification, was measured through four dichotomous variables capturing State certification of instructors, the availability of training opportunities for new instructors, internship or probationary requirements, and reciprocity for instructors trained in other States.

Licensing

Features of licensing programs were assessed through six variables, beginning with a dichotomous indicator for whether the State had implemented a graduated licensing system for motorcyclists. Comprehensive testing was assessed through two dichotomous variables capturing the types of knowledge and skills tests used by a State. Because the content of the MSF examination is widely accepted and adopted, 1 point was awarded to States that used the MSF test over a local or modified MSF version. Regarding the skills test, NHTSA recommendations specify that the examination should be administered on-street so that riders can be evaluated in real-world conditions (NHTSA, 1997). Scores for promising practices in the administration of skills tests followed this guideline, awarding 2 points to jurisdictions that required on-street tests, 1 point for off-street tests, and no points for the absence of any skills test.

Practices related to obtaining and renewing a license were assessed through three variables. The adoption of the MSF’s Motorcyclist Operator’s Manual (MOM) is consistent with the NHTSA promising-practices model and was awarded credit over the use of any other type of operator’s manual. Examiners trained by a State rider education program should have familiarity with motorcycles, so States meeting this criterion were awarded 1 point. The final two variables under the licensing area, rider education requirements for minors seeking licensing and reciprocal license waivers, capture incentives for licensing and were measured with dichotomous variables.