National Fleet Safety Survey, 1996 Final Analysis Report Prepared for: Federal Highway Administration Office of Motor Carriers U.S. Department of Transportation Washington, D.C. 20590 Prepared by: Star Mountain , Inc. 3601 Eisenhower Avenue Alexandria, VA 22304 STAR MOUNTAIN project # 4284-016 March 25, 1997 TABLE OF CONTENTS EXECUTIVE SUMMARY I. INTRODUCTION II. BACKGROUND III. FINDINGS Distribution of Violations for All Trucks HazMat (HM) Data Distribution of Violations Differences Between Facility types, Road types, and Time of Inspection Supplemental Questions IV. CONCLUSION APPENDIX I: DESCRIPTION OF LEVEL I VEHICLE INSPECTION PROCEDURE APPENDIX II: SAMPLE DESIGN AND ESTIMATION PROCEDURES Selection of States Selection of Sites and Trucks Theory Estimation Methodology State Level Estimates HazMat-Specific OOS Rates LIST OF TABLES TABLE 1: UNWEIGHTED AND VMT WEIGHTED OUT-OF-SERVICE RATES BASED ON LEVEL I INSPECTIONS FOR ALL TRUCKS, BY STATE 7 TABLE 2: OUT-OF-SERVICE RATES FOR ALL TRUCKS BY STATE, WEIGHTED BY STATE VMT AD COUNTY AVERAGE TRUCKS COUNTS 9 TABLE 3: MOST FREQUENT VIOLATION CODES RESULTING IN AN OOS ORDER DURING LEVEL I INSPECTIONS FROM THE "ALL TRUCKS" SAMPLE 11 TABLE 4: DISTRIBUTION OF OOS VIOLATIONS PER INSPECTIONS FOR TRUCKS PLACED OUT-OF-SERVICE 12 TABLE 5: OUT OF SERVICE RATES FOR TRUCKS CARRYING HM, BY STATE, BASED ON LEVEL I INSPECTIONS 13 TABLE 6: MOST FREQUENTLY FOUND FEDERAL VIOLATION CODES RESULTING IN AN OOS ORDER DURING LEVEL I INSPECTIONS FOR TRUCKS CARRYING HM 15 TABLE 7: OVERALL OOS RATES BY FACILITY TYPE, ROAD TYPE, AND TIME OF DAY FOR "ALL TRUCKS" AND "HM" SAMPLES 17 TABLE 8: RAW OOS RATES BY DRIVER EXPERIENCE, TRAINING , AND TYPE OF LOAD FOR "ALL TRUCKS" AND "HM" SAMPLES 19 TABLE 1A: 1994 MCSAP LEVEL I OOS RATES FOR STATES SELECTED INTO SAMPLE, BY OOS STRATUM 28 TABLE 2A: NUMBER OF INSPECTIONS REQUIRED AND ACHIEVED, BY STATE BY ROAD TYPE FOR THE "ALL TRUCKS" SAMPLE 30 TABLE 3A: 1994 TRUCK VMT (MILLIONS) BY STATE BY ROAD TYPE 30 EXECUTIVE SUMMARY This report describes both the methodology and findings from the 1996 National Fleet Safety Survey, conducted by the Federal Highway Administration's (FHWA) Office of Motor Carriers (OMC), in conjunction with the Research and Special Programs Administration (RSPA) of the Department of Transportation (DOT). In this study, data were collected from over 10,000 random Level I truck inspections and were used to estimate overall, driver-specific, and vehicle-specific out-of-service (OOS) rates for the general population of commercial vehicles on the road and for the subpopulation of commercial vehicles carrying hazardous materials (HM). These estimated rates represent weighted estimates, using a weighting scheme based on State vehicle miles traveled (VMT) for trucks. The key findings of the study are as follows: * The National overall OOS rate, based on Level I inspections, is estimated to be 32 percent. National vehicle-specific and driver-specific OOS rates are estimated to be 29 and 5 percent, respectively. These rates are slightly lower than similar rates obtained from FY 1996 MCSAP data and these differences appear real (i.e., statistically significant). * For the subpopulation of trucks carrying HM, the National overall OOS rate, based on Level I inspections, is estimated to be 27 percent. National vehicle-specific and driver-specific OOS rates are estimated to be 25 and 4 percent, respectively. * For the general population of trucks, the most frequently found OOS-related violations pertain to brakes. Approximately 49 percent of all OOS-related violations found in the survey were brake-related, the most common of these violations involving Federal Violation Code 396.3A1BA (brakes out of adjustment). For the subpopulation of trucks carrying HM, the results were similar. Approximately 51 percent of all OOS-related violations found were brake-related, and the most common of these violations, again, involved Federal Violation Code 396.3A1BA. * The study did not find appreciable differences in OOS rates between types of inspection facilities (fixed vs. mobile), times of inspection (day vs. evening), types of load, driver years of experience, or driver years of training. For the subpopulation of drivers with little experience (three years or less), however, the survey found that those with no training had a higher OOS rate when compared to those drivers receiving some form of training (51 percent vs. 35 percent). * The overall OOS rates for the general population of trucks inspected at urban sites are slightly lower than for those inspected at rural sites (30 percent for urban interstate vs. 33 percent for rural interstate; 29 percent for urban noninterstate vs. 34 percent for rural noninterstate). * 5.6 percent of all randomly sampled trucks in the survey were found to be carrying HM. Based on these data, the weighted national estimate of the percentage of operating trucks carrying HM is 7.2 percent. Seventy-eight percent of those trucks sampled requiring HM registration were found to have the correct documentation onboard. HM carriers with appropriate registration documentation had an overall OOS rate of 24%, compared to 31% for those lacking such documentation. I. INTRODUCTION This report presents findings from the 1996 National Fleet Safety Survey, conducted by the Federal Highway Administration's (FHWA) Office of Motor Carriers (OMC), in conjunction with the Research and Special Programs Administration (RSPA) of the Department of Transportation (DOT). In this survey, a stratified random sample of over 10,000 trucks was used to assess the level of compliance with the Federal Motor Carrier Safety Regulations (FMCSRs) and with the Hazardous Materials Regulations (HMRs). Specifically, this study estimates Out-of-Service (OOS) rates (both driver-related and vehicle-related) for the general population of commercial vehicles on the road and for the subpopulation of commercial vehicles carrying hazardous materials (HM), based upon random Level I inspections performed during June and July of 1996 in 11 states. For the purposes of this analysis, the OOS rate is defined as the proportion of operating commercial motor vehicles having at least one FMCSR or HMR violation severe enough to require the vehicle to be placed out-of-service for a prescribed period of time (in the case of a driver-related violation) or until the vehicle defects are corrected (in the case of a vehicle-related violation). II. BACKGROUND FHWA, in conjunction with State enforcement authorities, has routinely inspected trucks in operation on the highways since the mid-1950s. These inspections are currently performed under FHWA's Motor Carrier Safety Assistance Program (MCSAP). MCSAP inspections take place at permanent and portable truck weigh stations, at special inspection pads, at port of entry facilities and carriers' terminals, at hazardous waste stations, at rest stations, and at other locations. Inspections are conducted in accordance with standards developed by OMC in cooperation with the Commercial Vehicle Safety Alliance (CVSA). These standards allow for national uniform inspection procedures as well as provide criteria for identifying out-of-service violations (violations of the FMCSR or HMR severe enough to place the vehicle out-of-service). Five different kinds of inspections are performed under MCSAP, the most comprehensive of which is the Level I-type inspection used in this study. Level I inspections involve a full examination of the vehicle (by conducting a walk-around and by examining underneath the vehicle) as well as an inspection of all driver records. A detailed description of the vehicle component of the Level I inspection procedure is given in Appendix I. Trucks generally are targeted for inspection under MCSAP on a selective basis: if a cursory auditory and visual examination of the truck reveals potential violations, it is inspected. Otherwise, vehicles and drivers are randomly selected. As a result, OMC's Motor Carrier Management Information Systems (MCMIS) data, containing information on all inspections conducted every year, cannot be considered a representative sample of commercial motor vehicles and their drivers on the roads. As far back as 1969, representatives of the trucking industry alleged that OMC's (at that time, the Bureau of Motor Carrier Safety) published data on roadside inspections were not representative of the true state of vehicles on the road. As a result of this claim, the Bureau conducted a controlled study to determine whether its published out-of-service rates differed substantially from what would be obtained from a statistical random sample of trucks. This sample was implemented during the summer of 1971. For the 1971 study, a sampling frame of 156 fixed inspection sites was developed. A sample of sites was then selected from four geographic regions across the country. Prior to sample selection, all sites on the sampling frame were further classified within each region as "urban" or "rural" (based on traffic volume). One rural and one urban site were then selected into the sample from each region (using a "probability proportional to size selection procedure" based on traffic volume). At each selected site, time intervals were then randomly selected to inspect trucks. The total sample consisted of 1,172 trucks. Based on the available documentation from this study, it is not clear, however, whether all inspections performed for this investigation were Level I inspections, although this seems most likely. The 1971 survey found that 28 percent of the trucks inspected were in an "out-of-service" condition. This compares to approximately 40 percent of trucks being ordered out-of-service from the regular inspections for 1970. Based on these results, FHWA concluded that targeting for inspection those vehicles appearing the least well maintained did not produce a significant bias in their published data.1 Since many inspections under MCSAP are still performed selectively and not randomly, OMC decided in 1995 to reassess the baseline out-of-service rate (percentage of trucks operating on the highways in an out-of-service condition) by means of a random sample. A total of 17 States initially expressed an interest in participating in the 1996 study. Given manpower and other constraints, OMC decided that 12 States could be accommodated by the new study. One State, Georgia, later withdrew from the study. For the 1996 study, an overall minimum sample size of 5,000 trucks was decided upon. To ensure that the sample had a sufficient number of trucks carrying hazardous materials to allow for the production of separate OOS rate estimates for such vehicles, the random truck selection procedure was supplemented with an oversampling procedure for trucks with HM: States were instructed to set aside one day where only trucks carrying HM would be sampled. Fixed and non-fixed inspection sites were selected within each State from each of the following road-type categories: (1) Rural Interstate roads, (2) Rural Non-interstate roads, (3) Urban Interstate roads, and (4) Urban Non-interstate roads. The selection of these sites was left to each State and was not random but purposive, although states were instructed to select as many different kinds of sites as possible, including at least one non-fixed site per road-type category. The States were given only the minimum number of sites to be selected within each road-type category; the maximum number of sites to be selected in each of these categories was not specified. Although, the required minimum number of inspections was not always achieved in each road-type category in all States (see Appendix II), the overall sample size achieved turned out to be quite large, exceeding 10,000 inspections and including over 1,300 inspections of trucks carrying HM. To ensure that the minimum number of HM-related inspections was achieved, the States were requested to set aside one day during the data collection period for collecting only HM-related data. This suggested procedure was followed in nine States. In some States, the final sample of HM-specific inspections was too sparse in particular road-type strata to allow weighting by road type in the production of HM-specific estimates. As a result only national estimates were weighted for HM specific data (see Appendix II). At each inspection site, trucks were randomly selected for Level I inspections. Upon completing an inspection, the inspector would select another truck by choosing the third truck passing through the site. The volume of trucks passing each site was measured by taking a five minute truck count prior to the beginning of the inspectors' work shift. The survey sampling design and the estimation procedures used are discussed in more detail in Appendix II. III. FINDINGS Table 1 presents both weighted (by VMT) and unweighted estimates of OOS rates for each State and for the total U.S., for the overall population of trucks. Similar information based on the weighting procedure incorporating the truck counts from the sites (see Equation 3, Appendix II) is presented in Table 2. For the State level estimates, the VMT weighted estimates represent a weighted average (weighted by VMT) of the raw OOS rates found in each road-type category. The national estimate represents a VMT weighted average of the OOS estimates obtained for the States (see Appendix II). The total number of inspections available for analysis was 10,167. Two hundred eight of the inspection data records, however, did not have the proper road-type stratum identification necessary for the production of weighted estimates. As a result, the VMT weighted estimates are based on a total sample size of 9,959 inspections. As shown in Table 1, the raw unweighted OOS rate for all States combined is 34 percent. When the data are adjusted for VMT, the estimated rate is 31 percent for the 11 States combined, and 32 percent for the Nation as a whole. National estimates of driver and vehicle OOS rates are 5 and 29 percent, respectively. Note that the driver and vehicle OOS rate estimates in the table do not necessarily sum to the overall weighted OOS rate. This stems from the fact that the categories are not mutually exclusive. That is, both a vehicle and its driver may be placed out-of-service at the same inspection. Table 1 indicates that, for the individual States, the estimated overall OOS rates (weighted) range between 30 and 39 percent, with the exception of California, whose overall OOS rate is 22 percent. State level vehicle OOS rates (weighted) range from 21 percent (California) to 35 percent (Maryland). Also presented in Table 1 are driver and vehicle OOS rates based on MCSAP Level I inspection data for Fiscal Year 1996. Comparing the Fleet Survey OOS rate estimates with the FY 1996 MCSAP data, one notes that the weighted National estimates for both driver and vehicle OOS rates from the survey (5% and 29%, respectively) are lower than the FY 1996 Level I MCSAP rates (6% and 32%, respectively). Table 1 Unweighted and VMT Weighted Out-of-Service Rates Based on Level I Inspections for All Trucks by State State Inspections Unweighted OOS Rate VMT Weighted OOS Rate VMT Wgt. Driver OOS Rate VMT Wgt. Vehicle OOS Rate MCSAP 1996 Lev. I Driver OOS Rate MCSAP 1996 Lev. I Vehicle OOS Rate Total U.S. -- .32 .05 .29 .06 .32 All 11 States 10,167 .34 .31 .05 .28 -- -- CA 1272 .23 .22 .02 .21 .03 .26 CT 438 .37 .31 .07 .27 .10 .51 IL 466 .32 .33 .12 .26 .05 .31 KS 417 .35 .35 .07 .30 .05 .33 MD 944 .39 .37 .04 .35 .05 .36 MO 2,565 .37 .37 .07 .34 .10 .46 NC 786 .35 .36 .05 .32 .06 .32 NM 370 .37 .39 .10 .34 .07 .41 OH 970 .36 .38 .09 .34 .12 .43 WA 944 .31 .29 .04 .27 .05 .32 WI 995 .29 .30 .02 .29 .08 .37 These differences between the Fleet Survey estimates and the 1996 MCSAP rates, although small, are real (i.e., if the survey were to be replicated, one would expect the Fleet Survey estimates to be lower again). That the survey estimates are slightly lower is consistent with the fact that MCSAP inspections are not always random and frequently target trucks that appear to have problems. This pattern is generally true for the individual States as well, although in some States the MCSAP Driver OOS rates are lower than those from the Fleet Survey (Illinois, Kansas, and New Mexico). In addition to the fact that MCSAP inspections are frequently targeted, differences between the MCSAP and Fleet Survey estimates may also be attributable to the fact that the MCSAP OOS rate estimates are not weighted. Table 2 presents similar estimates based on a weighting procedure that incorporates the truck counts obtained from the inspection sites. The road-type category OOS rates used to produce the estimates in this table represent a weighted average of the raw OOS rates associated with each county sampled in the road-type category, using the sample's average truck count for the county as the weight (The road-type category OOS rates used to produce the estimates in Table 1 are raw rates. In both Tables 1 and 2, the OOS rates for the road-type categories are then averaged by each category's VMT to obtain overall estimates -- see Appendix II). Table 2 Out-of-Service Rates for All Trucks by State Weighted by State VMT and County Average Truck Counts State No. Inspections Weighted OOS Rate Weighted Driver OOS Rate Weighted Vehicle OOS Rate Sample Size For Weighted Estimates Total U.S. -- .32 .05 .29 9,959 All States 9,058 .31 .04 .28 9,939 CA 1,250 .20 .01 .19 1,210 CT 425 .33 .07 .28 437 IL 342 .29 .12 .19 444 KS 287 .36 .06 .31 417 MD 929 .41 .04 .38 902 MO 2,332 .41 .06 .38 2,561 NC 553 .36 .05 .33 760 NM 352 .34 .05 .31 358 OH 828 .36 .06 .34 951 WA 844 .33 .03 .31 935 WI 916 .34 .01 .33 984 The weighted OOS rate estimates in Table 2 for all States combined, as well as the National estimates, are almost identical to those found in Table 1. At the level of individual State estimates, however, small discrepancies can be found between the two tables. For example, for Illinois, the weighted vehicle OOS rate is 26 percent in Table 1 and 19 percent in Table 2. This particular discrepancy may be at least partially attributable to the different sample sizes used in the two approaches: for the Illinois estimates shown in Table 2, 82 inspections had to be excluded because they were associated with counties that did not have the required minimum number of inspections as outlined in Appendix II. For the most part, however, the differences in the State level estimates between the two tables are slight. Distribution of Violations for All Trucks Table 3, below, presents the most frequently found OOS-related violations for the "all trucks" sample. Approximately 49 percent of all OOS-related violations found in the survey pertain to brakes (this number cannot be derived from the table since the table does not include all brake-related violations found in the survey). The most common of these violations involves Federal Violation Code 396.3A1BA (brakes out of adjustment). Although not shown in the table, only two percent of all OOS-related violations found in the survey pertain to hours-of-service rules (10 hour, 15 hour, or 60/70 hour rule violations) and seven percent of all OOS-related violations pertain to hours of service in general. (It should be noted, however, that the relatively low frequency of OOS-related violations pertaining to hours-of-service and log sheets may be more a reflection of the difficulty in detecting these violations than a true measure of the extent of their occurrence.) Table 3 Most Frequently Found Federal Violation Codes Resulting in an OOS Order During Level I Inspections From the "All Trucks" Sample FED. VLTN CODE PROBLEM TOTAL OOS VIOLATIONS PERCENT OF ALL OOS VIOLATIONS 396.3A1BA Brakes out of Adjustment 1455 26.4 % 396.3A1 Inspection, Repair&Maint. 268 4.9 % 393.48A Inoperative Brakes 244 4.4 % 393.19 Defective Turn/Hazard Lamp 236 4.0 % 396.3A1B Brakes (General) 218 4.0 % 393.47 Inadequate Brake Lining 204 3.7 % 393.25F Stop Lamp Violation 163 3.0 % 393.207 All Suspension Violations 146 2.6 % 396.3A1BH Damaged Brake-Hose 143 2.6 % 393.51 Defective Brake Warning Device 123 2.2 % 393.207C Defective Leaf Spring Assembly 121 2.2 % 393.9 Inoperable Lamp 109 2.0 % 392.9 Driver Load Secure 93 1.7 % 393.100 Improper Load Securement 90 1.6 % 395.8F1 Driver Duty Status not Current 88 1.6 % 396.3A1BL Brake Reserve Pressure Loss 88 1.6 % Table 4 shows the distribution for the number of OOS violations given for trucks placed out-of-service from the survey. The table shows that, for the 3625 trucks ordered out-of-service, 59 percent had only one OOS violation, 23 percent had two OOS violations, and nine percent had three OOS violations. Nine percent of trucks ordered out-of-service had four or more OOS violations and roughly one percent had ten or more such violations. Table 4 Distribution of OOS Violations per Inspection for Trucks Placed Out-of-Service Total OOS Violations per Inspection Frequency Percent Cumulative Frequency Cumulative Percent 1 2134 58.9 2134 58.9 2 834 23.0 2968 81.9 3 331 9.1 3299 91.0 4 166 4.6 3465 95.6 5 72 2.0 3537 97.6 6 31 0.9 3568 98.4 7 14 0.4 3582 99.1 8 9 0.2 3591 99.1 9 12 0.3 3603 99.4 10+ 22 0.7 3625 100.0 HazMat (HM) Data 5.6 percent of all randomly sampled trucks in the survey were found to be carrying HM. Based on these data, the weighted national estimate of the percentage of operating trucks carrying HM is 7.2 percent. Table 5 gives unweighted State level estimates and weighted National estimates of OOS rates for trucks carrying HM. As discussed in Appendix II, because of the small number of inspections involving HM in several road-type strata, weighting was only deemed appropriate for National estimates. The table indicates that 28 percent (unweighted estimate) of the Level I inspections performed on trucks carrying HM resulted in an OOS order (i.e., the truck being placed out-of-service). Twenty-six percent of these inspections resulted in a vehicle-related OOS order and four percent resulted in a driver-related OOS order. Table 5 Out-of-Service Rates ** For Trucks Carrying HM, by State Based on Level I Inspections State Number of Inspections Overall OOS Rate Driver OOS Rate Vehicle OOS Rate Total U.S. (weighted) 1,320 .27 .04 .25 All 11 States(unweighted) 1,320 .28 .04 .26 CA 36 .14 .06 .08 CT 81 .37 .05 .36 IL 62 .31 .03 .29 KS 83 .29 .02 .28 MD 69 .32 .01 .32 MO 278 .30 .05 .27 NC 117 .26 .02 .24 NM 73 .29 .04 .27 OH 233 .21 .03 .21 WA 118 .25 .06 .21 WI 170 .31 .03 .31 ** unweighted for State estimates, weighted for National estimate. Comparing these findings to the data from the "all truck" sample (Table 1), one notes that for all States combined, both the raw and the weighted OOS rates are slightly lower for the "HM" sample. The unweighted overall OOS rate for these trucks is 28 percent (compared to 34 percent for the "all trucks" sample) and the weighted National estimate is 27 percent (compared to 32 percent in the "all trucks" sample). Measuring the statistical significance of the difference between the estimates is difficult, since the actual variances of the estimates are not known. Considering the weighted estimates, however (where there is a smaller difference between the results of the two samples), and treating the estimates as though derived from a simple random sample, the difference is statistically significant at the alpha=.05 level. This would suggest that the differences, although small, are real (i.e., they actually exist in the population at large) and are not merely a product of random sampling. Comparing the raw State level overall OOS rates in Table 5 with the unweighted State level rates in Table 1, one notes that, with the exception of Wisconsin, the raw rates for trucks carrying HM are consistently lower (although only slightly in some cases) than those obtained from the "all truck" sample. The most dramatic difference occurs in the case of Ohio, where the raw unadjusted overall OOS rate is .21 for trucks carrying HM, compared to .36 for the "all trucks" sample. It should be pointed out, however, that in several States, the sample sizes are rather low. As discussed in Appendix II, the weighted National estimates from the "HM" sample were obtained by weighting the raw State level OOS rates by State VMT. Table 5 shows that these weighted nationally representative estimates differ very little from the raw estimates. Distribution of Violations Table 6 shows the most frequently found OOS-related violations for trucks carrying HM. As can be seen from the table, the distribution is very similar to what was found with the "all trucks" sample. Again, Federal Violation Code 396.3A1BA is the most common OOS-related violation (constituting roughly 25 percent of such violations). Approximately 51 percent of all OOS-related violations found for trucks carrying HM pertained to brakes (again, number cannot be calculated from data shown in table). Only 11 percent of the OOS-related violations in the HM sample were HM-specific violations (Federal Violation Codes 171 through 180). Table 6 Most Frequently Found Federal Violation Codes Resulting in an OOS Order During Level I Inspections Found For Trucks Carrying HM FED. VLTN.CODE PROBLEM TOTAL OOS VIOLATIONS PERCENT OF ALL OOS VIOLATIONS 396.3A1BA Brakes out of Adjustment 143 24.6% 396.3A1B Brakes (General) 27 4.6% 393.19 Defective Turn/Hazard Lamp 24 4.1% 393.48A Inoperative Brakes 24 4.1% 396.3A1BH Damaged Brake-Hose/Tube 24 4.1% 396.3A1 Inspection, Repair,&Maint. 21 3.6% 393.47 Inadequate Brake Lining 20 3.4% 393.25F Stop Lamp Violation 19 3.3% 393.207C Defective Leaf Spring Assembly 16 2.7% 393.207 All Suspension Violations 13 2.2% 393.207A Defective Axle Positioning Parts 13 2.2% 392.9 Driver Load Secure 12 2.1% 393.45 Brake Tubing Aid Hose Adequacy 11 1.9% 396.3A1BL Brake Reserve Pressure Loss 11 1.9% 393.51 Defective Brake Warning Device 9 1.5% Differences Between Facility types, Road types, and Time of Inspection Table 7, below, presents OOS rates and corresponding sample sizes for both the "all trucks" and the "HM" sample, broken down by facility type (fixed site or roadside inspection site), road type (urban interstate, urban non-interstate, rural interstate, rural non-interstate), and time of inspection (daytime vs. evening). For the OOS rates in the table broken-out by type of inspection facility and time of inspection, the rates represent unweighted estimates, since the sample design cannot accommodate the production of accurate weighted estimates at this level of detail (the sample sizes become too small in particular road-type strata). These unweighted estimates are nonetheless useful for exploratory purposes. Furthermore, the analyses already presented have shown that the differences between weighted and unweighted estimates are frequently not substantial (see Table 1). For the estimates broken-out by road-type, the estimates are weighted. The table shows few appreciable differences between the raw OOS rates within most of the breakouts listed. The table does show, however, slightly higher OOS rates for the rural road-type categories in the case of the "all trucks" sample (i.e., urban interstate vs. rural interstate, and urban non-interstate vs. rural non-interstate). Although these differences appear real (if the estimates were treated as having been obtained from a simple random sample, the differences would be statistically significant at the alpha=.05 level), the table does not show this same pattern with the HM data. Table 7 Overall OOS Rates by Facility Type, Road Type, and Time of Day for "All Trucks" and "HM" Samples All Trucks (inspections) HM (inspections) Category Facility Type:* Fixed Site .33 (7322) .28 (942) Roadside .35 (2845) .26 (399) Road Type: ** Urban Interstate .30 (1850) .27 (181) Urban Non-Interstate .29 (856) .27 (114) Rural Interstate .33 (4671) .27 (678) Rural Non-Interstate .34 (2582) .27 (347) Time of Day:* 6am to 5:59pm .34 (9167) .27 (1245) 6pm to 5:59am .31 (1000) .32 (96) * numbers are unweighted. **numbers are weighted. Supplemental Questions In the National Fleet Survey, the five following supplemental questions were added to the inspector's data collection form: (1) Who loaded the materials into the vehicle? (2) How much experience have you had driving a commercial vehicle? (3) How much experience have you had transporting hazardous materials? (4) What type of entry level commercial motor vehicle training have you received? (5) Do you have a copy of the RSPA registration number onboard? (for HM only)? Table 8 presents raw unweighted OOS rates broken-out by categories suggested by these questions (as in the case with Table 7, the level of detail required by the table is too fine to allow for the production of reliable weighted estimates). The table shows that slightly less than half of the trucks inspected in the "all trucks" sample have drivers with 12 or more years of experience. For drivers of trucks carrying HM, more than half (58%) of those sampled have at least 12 years of general experience, and a little less than half (45%) have at least eight years of experience specific to transporting HM. For the "all trucks" sample, the table does not show much difference in overall OOS rates among the various general experience categories (none of the differences is statistically significant at the alpha=.05 level). When these same drivers in the "all trucks" sample are classified according to years of experience transporting HM, however, the differences become somewhat more pronounced. For example, the overall raw OOS rate for drivers from the "all truck" sample with zero to three years of general experience is .34, compared to an OOS rate of .28 for drivers with 15 or more years of experience transporting HM. This same pattern is found for both the driver and vehicle OOS rates in addition to the overall OOS rate. Considering only trucks carrying HM, the table again shows a decrease in OOS rates as driver experience increases. Note that these patterns do not necessarily point to a cause and effect relationship between increased driver experience and lower OOS rates. For example, it may be that more experienced drivers are assigned newer vehicles which are more likely to pass an inspection. Table 8 Raw OOS Rates by Driver Experience, Training and Type of Load for "All Trucks" and "HM" Samples All Trucks HM Only Ovrl. Ovrl. # Drv. Veh. Ovrl Ovrl. # Drv. Veh. OOS Inspcts. OOS OOS OOS Inspcts. OOS OOS Driver Experience: 0 to 3 yrs .35 (1963) .08 .31 .35 (177) .06 .33 4 to 7 yrs. .35 (1738) .07 .31 .28 (184) .04 .25 8 to 11 yrs. .33 (1490) .05 .30 .28 (202) `.03 .27 12 to 15 yrs. .32 (1054) .06 .29 .24 (156) .03 .22 15 + yrs. .33 (3900) .05 .29 .26 (616) .03 .25 Driver HM Experience: None .36 (4604) .07 .32 .38 (127) .09 .34 Less than 3 yrs. .34 (2258) .06 .31 .31 (332) .04 .29 4 to 7 yrs. .33 (1167) .06 .29 .25 (261) .04 .23 8 to 11 yrs. .29 (732) .04 .27 .23 (196) .02 .22 12 to 15 yrs. .29 (370) .04 .26 .27 (124) .03 .25 15 + yrs. .28 (784) .04 .26 .25 (287) .03 .24 Driver Training: Training .34 (9713) .06 .30 .27 (1291) .04 .26 No Training .35 (403) .06 .31 .36 (39) .03 .33 Type of Load: Driver Loaded .32 (1872) .06 .28 .25 (456) .02 .24 Shipper Loaded .34 (5429) .07 .30 .28 (607) .04 .26 Freight For. Loaded .34 (607) .06 .30 .31 (87) .05 .29 Other Loaded .31 (635) .04 .29 .27 (84) .01 .27 RSPA Registration Required and Onboard NA NA NA NA .24 (807) .02 .22 Req. and Not Onboard NA NA NA NA .31 (228) .04 .30 For the "all trucks" sample, Table 8 suggests that driver training has little impact on OOS rates. These results, however, may stem from the fact that the majority of drivers encountered in the survey had several years of general experience (the table indicates that 81 percent of drivers in the "all trucks" sample had four or more years of general experience). When only those drivers are considered having three years or less of experience, the raw OOS rate is 51 percent for those without any training versus 35 percent for those with some form of training (not shown in table). Although the sample size for the subpopulation of drivers having both three years or less of experience and no training is rather small (65 drivers), this difference is statistically significant at the alpha=.05 level (treating the survey as a simple random sample). Most of this increase is attributable to driver-related OOS violations: the subsample of the 65 drivers with three years or less of experience and no training have a raw driver-related OOS rate of 17 percent (not shown in table). In the case of trucks carrying HM, the sample size for trucks driven by individuals with no training is too small to make meaningful comparisons with trucks driven by trained drivers. When the raw OOS rates are broken-out by type of load, Table 8 suggests that, for the "all trucks" sample, the load category has little effect. When only examining trucks carrying HM, however, one does note small differences in rates among the load categories. In particular, driver loaded trucks tend to have slightly lower raw OOS rates (.25), and freight forwarder loaded trucks slightly higher rates (.31). Reasons for such a difference, however, cannot be determined from the data. Carriers are required to be registered with RSPA if they ship or transport radioactive, explosive, or extremely toxic (by inhalation) materials; if they ship or transport HM in bulk packages with a capacity greater than 13,248 liters (for liquids or gases) or 13.24 cubic meters (for solids); or if they ship or transport HM in non-bulk packages greater than 2,268 kilograms. Of those inspected vehicles carrying HM and requiring RSPA HM registration, 78 percent (807 out of 1035) had registration documentation onboard. As shown in Table 8, both driver and vehicle OOS rates were lower for the subsample of trucks requiring RSPA registration and having it onboard, when compared to trucks requiring such registration and not having it onboard. Because of the small sample sizes for these subpopulations, however, these differences can only be shown to be statistically significant in the case of the vehicle OOS rate. IV. CONCLUSION Based on the data collected and the estimation procedures used, the National OOS rate (overall) is estimated to be 32 percent for the general population of trucks on the highways. This estimate can be interpreted as indicating that if the general population of trucks on the highways were subjected to a Level I inspection, 32 percent of them would have at least one Federal code violation severe enough to place them out-of-service. The National driver and vehicle OOS rates are estimated to be five and 29 percent, respectively. When considering only trucks carrying HM, the overall OOS rate is estimated to be 27 percent. Driver and vehicle OOS rates for this same population are estimated to be four and 25 percent, respectively. For both the general population of trucks, as well as the subpopulation of trucks carrying HM, approximately half of the OOS-related violations found in the survey were brake-related. Differences in OOS rates according to facility type, road type, time-of-day, driver experience, driver training, and type of load carried were also examined. With the exception of road-type category, weighted estimates, however, were not generated, as the overall sample size of the study cannot support the production of reliable weighted estimates at these levels of detail. APPENDIX I DESCRIPTION OF LEVEL I VEHICLE INSPECTION PROCEDURE APPENDIX II SAMPLE DESIGN AND ESTIMATION PROCEDURES For the 1996 study, a stratified random sampling plan was developed for collecting data on Level I inspections. Individual States constituted the primary sampling units. At the second stage of selection, a stratified sample of locations (inspection sites) was selected within each State. Finally, at the third stage of selection, individual trucks were sampled at the inspection sites. An overall minimum sample size of 5,000 trucks was decided upon. In the case of a simple random sample, such a sample size would ensure that the estimated OOS rate from the sample would be within one percentage point of the true value, at the alpha=.05 level of confidence. The alpha=.05 confidence level indicates that if the study were to be replicated, one would expect, on average, the estimated out-of-service rates from the sample replicates to be within one percent of the true value in 95 out of 100 samples. To ensure that the sample had a sufficient number of trucks carrying hazardous materials to allow for the production of separate OOS rate estimates for such vehicles, the random truck selection procedure was supplemented with an oversampling procedure for trucks with HM: States were instructed to set aside one day where only trucks carrying HM would be sampled. Since the actual sample design used in the study is a stratified sample rather than a simple random sample, the accuracy of the sample estimates may be somewhat higher than what is specified above. The sampling design is discussed in more detail below. Selection of States All States were first classified into one of four primary sampling strata based on their past overall Level I out-of-service rates (25-31.9%, 32-38.9%, 39-45.9%, and 46% and above), using 1994 MCMIS Level I interstate inspection data (these strata are subsequently referred to in this document as out-of-service strata). The States were then notified about the study and were asked about their interest in participating. A total of 17 States initially expressed an interest in participating in the study. Given manpower and other constraints, OMC decided that 12 States could be accommodated by the study. These States were then purposely (nonrandomly) selected in such a manner that all of the four strata, defined above, were represented in the sample, and that participating States were geographically dispersed. One State, Georgia, later withdrew from the study because of manpower shortages resulting from holding the Olympic Games. Table 1A, below, lists the remaining 11 States selected in the sample, the out-of-service strata to which they belong, and the OOS rates in these States from the 1994 Level I interstate MCMIS inspection data. Table 1A 1994 Interstate Inspections Level I OOS Rates for States Selected into Sample, By OOS Stratum 94 MCMIS 94 MCMIS 94 MCMIS MCMIS Selected Overall Lev. I Driver Lev. I Vehicle Lev. I Sample Stratum State OOS Rate( OOS Rate( OOS Rate( Size 25%-31.9% CA .26 .04 .24 63,691 25%-31.9% IL .28 .05 .25 10,395 25%-31.9% NC .32 .04 .29 10,871 32%-38.9% KS .36 .05 .33 8,510 32%-38.9% WA .36 .04 .34 22,528 32%-38.9% MD .38 .04 .36 16,710 38%-45.9% WI .40 .10 .34 10,067 39%-45.9% NM .44 .04 .43 4,858 46% + OH .46 .13 .40 15,149 46% + MO .56 .09 .52 32,565 46% + CT .57 .07 .55 4,976 (rates have been rounded to two decimal places. *from MCMIS. The 5000 sample units (trucking inspections) were allocated to the 11 States following loosely the rule of optimal allocation. This rule from sampling theory suggests that the overall sample be allocated to the sampling strata in a manner proportional to the size and variability found in each sampling stratum. That is, sampling strata having more population units or more variability among the units for the characteristic in question receive proportionally more sample units. In the case of this study, however, both the population variability for trucks being out-of-service as well as the number of trucks operating (i.e., the population units) in each stratum were unknown. Hence, the population variability was assumed equal in all States and strata, and the number of trucks operating in each State was assumed correlated to truck Vehicle Miles of Travel. Optimal allocation was then loosely applied by simply allocating more truck inspections to the States having more VMT. (It should be noted that, strictly speaking, the States are primary sampling units and not sampling strata. The end result is the same, however. The sample units are allocated more heavily to States with more population units.) Further adjustments to the sample allocation had to be made based on the manpower capabilities of individual States. Selection of Sites and Trucks In the second stage of selection, fixed and non-fixed inspection sites were selected within each State from each of the following road-type categories: (1) Rural Interstate roads, (2) Rural Non-interstate roads, (3) Urban Interstate roads, and (4) Urban Non-interstate roads. The selection of these sites was left to each State and was not random but purposive, although states were instructed to select as many different kinds of sites as possible, including at least one non-fixed site per road-type category. The States were given only the minimum number of sites to be selected within each road-type stratum; the maximum number of sites to be selected in each of these strata was not specified. As in the case of allocating the overall sample to the States described above, more truck inspections were allocated to the road-type strata having more truck VMT. The number of inspections requested of each State as well as the actual number of inspections performed is given in Table 2A by road-type category. One notes from Table 2A that, although the total number of inspections actually performed in each State usually far exceeds the State's requested minimum, the requested minimums for the road-type Table 2A Number of Inspections Required and Achieved by State by Road Type for the "All Trucks" Sample Rural Urban Interstate Other Interstate Other Req. (Ach.) Req. (Ach.) Req. (Ach.) Req. (Ach.) State CA 180 (175) 180 (77) 100 (731) 100 (227) CT 170 (180) 140 (174) 56 (34) 40 (49) IL 150 (209) 180 (164) 85 (17) 85 (54) KS 110 (112) 150 (221) 40 (31) 50 (53) MD 140 (441) 140 (134) 80 (225) 90 (102) MO 210(2005) 150 (512) 140 (20) 150 (24) NM 140 (197) 110 (62) 70 (98) 70 (1) NC 110 (216) 150 (201) 110 (192) 100 (151) OH 140 (512) 140 (267) 100 (106) 100 (66) WA 140 (219) 180 (435) 70 (171) 80 (110) WI 100 (405) 120 (335) 50 (225) 70 (19) Table 3A 1994 Truck VMT (millions) by State by Road Type (source:FWHA) Rural Urban Interstate Other Interstate Other State CA 2,309 3,664 2,443 11,502 CT 164 173 484 385 IL 1,957 1,002 2,747 1,610 KS 473 786 183 317 MD 565 621 764 963 MO 1,621 1,850 878 1,193 NM 213 656 160 268 NC 1,370 4,184 808 3,212 OH 1,852 2,704 2,152 1,839 WA 517 1,095 704 1,616 WI 426 1,945 338 1,135 strata were frequently not met. Notwithstanding these difficulties, the overall sample size achieved turned out to be quite large, exceeding 10,000 inspections and including over 1,300 inspections of trucks carrying HM. Table 3A gives 1994 truck VMT totals for each of the road-type categories. To ensure that the minimum number of HM-related inspections was achieved, the States were requested to set aside one day during the data collection period for collecting only HM-related data. This suggested procedure was followed in nine States. In some States, the final sample of HM-specific inspections was too sparse in particular road-type strata to allow weighting by road type in the production of HM-specific estimates. As a result only national estimates were weighted for HM specific data (see below). At each inspection site, trucks were randomly selected for Level I inspections. Upon completing an inspection, the inspector would select another truck by choosing the third truck passing through the site. The volume of trucks passing each site was measured by taking a five minute truck count prior to the beginning of the inspectors' work shift. Both weighted and unweighted estimates of OOS rates were produced in this study. Estimates pertaining to the general population of trucks on the road were generated using all sample inspection data, with the exception of data collected from "HM only" sampling days. Separate estimates for trucks carrying HM were generated from data from "HM only" sampling days as well as from HM-related data contained in the general sample. Because the number of HM-related inspections was extremely low in several road-type strata for some States, separate weighting procedures had to be developed for estimates of HM-specific OOS rates. For the weighted estimates, several approaches were considered for weighting the data, involving the use of standard statistical techniques with some modifications. These modifications became necessary since many of the population parameters required by standard estimation formulas were unknown and could not be estimated. This section begins by examining the feasibility and appropriateness of applying various statistical estimation techniques to the data in this study. This is followed by a discussion of the estimation procedures actually implemented. Theory Based on standard statistical estimation procedures, the general formula for estimating a percentage from a single stage stratified sample is given by, (1) P = / , where ph is the percentage of sample units in the sampling stratum having the characteristic of interest, and Nh is the number of units existing in the stratum population. In this study the ph values represent the percentage of trucks estimated to be out-of-service in stratum h. The challenge in using this approach for this study stems primarily from the fact that the values for Nh (the total number of trucks operating on the highways in sampling stratum h) are unknown. Hence, to use this formula in this study, proxies must be substituted, which are correlated with these Nh values. In this study, truck VMT is used as the proxy. It should be pointed out, however, that Equation (1) applies to a single-stage sample design, whereas the National Fleet Safety Survey is actually a multi-stage design, involving the selection of States, and within States, the selection of inspection sites and trucks. Traditionally, in the case of a multi-stage design, the percentage of units in the population having a particular characteristic is obtained by using a ratio estimator. Using the ratio estimator, the estimated out-of-service rate would take the form, (2) P = / , where, Wgt1 through Wgtn represent the sampling weight components corresponding to each stage of sample selection, Xi represents the total number of sampled trucks found in an out-of-service condition at a sample site, and Yi represents the total number of sampled trucks at the site. Each sampling weight component, Wgt, corresponds to the inverse of the probability of selection for the unit at that particular sampling stage. For this study, however, the ratio estimator as given by Equation (2) becomes problematic because of the difficulty in developing appropriate sampling weights for some of the sampling stages, particularly the stage associated with the selection of inspection sites. These sites were selected purposively and do not have selection probabilities associated with them from which sampling weight components could be derived. Had the sample been limited to fixed inspection sites, the sites could have been quite easily selected by means of a probability sample. In the case of the non-fixed sites (corresponding to roadside inspections), however, such a procedure becomes considerably more complex: all road segments in each sampling stratum would have to have been defined, enumerated, and assigned a selection probability prior to sampling. Since these procedures were not performed, Equation (1), with some modifications, becomes a more practical option for estimation. Estimation Methodology Representative weighted sample estimates of OOS rates for all trucks were obtained from Equation (1) by using the formula iteratively, beginning with the strata associated with the final sampling stage (road-type strata), and then working backward. Thus all the sample data in each State were initially collapsed to the road-type stratum level and the percentage of inspections resulting in out-of-service violations (ph in Equation 1) was calculated for each road-type stratum in the State. Then, substituting the total 1994 truck VMT for the total number of trucks operating in each road-type stratum (Nh), a weighted estimate of the OOS rate was obtained for each out-of-service level stratum (from which the States were selected) by the formula, (3) Poos = / , where Poos is the estimated out-of-service rate for a particular out-of-service level stratum (one of four), and ph and VMThVMTh are, respectively, the raw out-of-service rate and total truck VMT for road-type stratum h in State s. The inner summation in the formula is over all four road-type strata within a given State s belonging to the out-of-service stratum, and the outer summation is over all States belonging to the out-of-service stratum. The road-type stratum's out-of-service rate, ph , in Equation (3) was calculated directly from the raw sample data in each road-type stratum. In addition, these percentages were alternatively calculated using a weighted average based on average county truck counts obtained from the sample. With the alternative approach, ph was calculated by the formula, (3a) ph = / , where Yc is the average truck volume (averaged over all inspection sites) for a given county, based on the five minute sample truck counts; Xc is this same average truck volume multiplied by the raw OOS rate for the county obtained from the sample; and the summations in the numerator and denominator are over all counties contained within road-type stratum h. Xc represents the sample estimate of the number of trucks contained in the truck count, Yc , that are in an out-of-service condition. Originally, the intention had been to calculate the parameters X and Y at each inspection site rather than at the county level. However, due to lack of consistency in the labeling of site identifiers in the survey data base, it was not possible to identify uniquely all inspection sites (for example, a location along Route 50 in Illinois might have been labeled as RT50, or I50, or R50, etc.). Hence, the micro data were aggregated to the county level, and individual counties found in the sample were treated as inspection locations. Even after the data were aggregated to the county level, it was found that some counties still had too small a number of inspections to produce a reasonable estimate of Xc, above. Thus, to ensure that Xc was based on a reasonable sample size, counties with fewer than ten total inspections were excluded from the calculations (this resulted in the exclusion of 250 counties comprising a total of 553 inspections). For counties in Illinois and Wisconsin, the required minimum number of inspections was relaxed from ten to six. This became necessary when it was found that keeping the required minimum at ten in these States resulted in several road-type strata not being represented in the sample. Once OOS rate estimates were obtained for each out-of-service Level stratum from Equation (3), a weighted estimate of the National OOS rate was obtained by, again, applying Equation (1) to the out-of-service strata. Thus, (4) Pntnl = / , where poos is the estimated out-of-service rate for a given out-of-service rate stratum (obtained from Equation 3) and VMToos is the total truck VMT in the same stratum. State Level Estimates Weighted representative estimates of out-of-service rates for individual States were obtained using a similar approach as shown by Equation (5), below. (5) Ps = / . This formula is the same as Equation (3), except that the summation here is only over the four road-type strata in one particular State (hence, the outer summation sign in Equation 3 is missing). HazMat-Specific OOS Rates As mentioned above, the number of HM-related inspections was often very low for particular road-type strata in several States. If the weighting procedures outlined above had been implemented to estimate an OOS rate specific to trucks carrying HM, some values of ph (the percentage of trucks in the road-type stratum found to be in an out-of-service condition), used in Equation (3) would have been based on a sample size too small to ensure any reasonable accuracy. Hence, for State level estimates, HM-specific OOS rates were generated from the raw survey data without applying any weighting procedures. For a National estimate of a HM-specific OOS rate, however, the unweighted OOS rates for each State were weighted by VMT using the following procedure. First, a weighted estimate of the OOS rate in each OOS level stratum was obtained using the formula, (3b) Poos = / , where ps is the raw OOS rate for State s. The values of Poos obtained from Equation (3b) were then plugged into Equation (4) to obtain a National level estimate. 1 "Report of Controlled Sampling Survey from Roadside Inspections on Units of Property Motor Carriers," Bureau of Motor Carrier Safety, FHWA, December 1971. Final Analysis Report 6 March 25, 1997 Final Analysis Report 1 March 25, 1997 Final Analysis Report 37 March 25, 1997 Draft Analysis Plan 8 March 25,1997