Evaluation

Specific Evaluation Methods and Results

Exposure and Knowledge Survey

The WTSC conducted a paper-and-pencil survey in cooperation with the DOL. Four driver licensing offices servicing the areas of interest— Bellingham, Kelso, Lacey and East Spokane—served as sampling sites. The Bellingham and Lacey offices were the only ones close to the intervention corridors on which counter­measures were being applied. Kelso had only a single DOL office. Spokane has multiple DOL offices. The East Spokane DOL office was selected to represent this comparison locale because of its size and closeness to the highway on which the behavioral measures were taken.

Four cycles or “waves” of survey data were collected in each office. Each wave covered approximately two weeks. The data collected in Wave 1 represented the baseline or “before” data since they were collected prior to the implementation of the TACT countermeasures and media campaigns. Subsequent waves were collected during the initial period of countermeasure activity, just after the peak of the countermeasures and after countermeasures had been in place for several months.

The survey process placed a low burden on both the respondents and the office staff. DOL agents in each office handed copies of the one-page, 16-item survey (see Appendix C) to licensed drivers and asked them to complete the information while they were waiting for their driver license or other transaction to be completed. Respondents either handed the anonymous completed survey back to the DOL employee or dropped it in a designated box. At the end of each wave, the completed questionnaires were sent to Dunlap and Associates, Inc. for key entry and analysis.

Overall as shown in Table 2, 6,155 people responded to the survey in the four DOL offices. The primary purposes of this survey were to determine if people in the intervention sites had read, seen, or heard the media and enforcement campaigns and whether they reported that they changed their behavior in response to the countermeasures. For analysis and presentation purposes, data were combined into intervention and comparison groups and pre and post campaign periods. Thus, Lacey and Bellingham were aggregated to form the intervention sites, while Kelso and Spokane were combined as the comparison sites. The pre exposure period consisted of Wave 1 of the survey which was conducted between May 3 and May 14, 2005. The post campaign period was comprised of Waves 2-4 collected during July 19 – 30, 2005, August 16 – 27, 2005, and September 20 – October 1, 2005, respectively.

Table 2.  Sample Sizes of DOL Survey Respondents

The driver licensing office survey technique has been widely used to assess trends and changes in public awareness of traffic safety programs, such as Click It or Ticket. It is not intended to be a representative survey of all Washington State drivers, but the technique is good at measuring changes over time. It is the magnitude of change across the TACT waves that is of interest, not the precise measurement. Customers who arrive at a driver licensing office are the target audience of interest in traffic safety programs – they own or drive vehicles on the public highways.

Demographics. Demographic information included age, gender, ethnicity, annual mileage, and type of vehicle driven most often. Demographics characterize the people who responded to the survey and are checked to assure that generally the same types of people completed the survey in each of the waves. There were no noteworthy differences in these demographic variables among the various waves. Table 3 below shows that the ages of survey respondents appeared to be a reasonable representation of the driving public when compared to the age distribution of licensed drivers in Washington State.

Table 3. Age Distribution of Department of Licensing (DOL) Public Awareness
Surveys vs. Licensed Drivers in Washington State

* Of those who reported age on the survey
** Drivers with valid Washington licenses and residences, DOL, 2004

Safety Belt Use and Other Driving Habits. Because of the general interest in safety belt use and as a further method of characterizing the survey sample, respondents were asked how often they used their safety belts when driving. Approximately 93 percent of all respondents said that they “always” use their safety belt. This is consistent with the Washington statewide observed safety belt use.

There were no remarkable patterns seen in basic information regarding driving habits, such as type of car driven most often and miles driven per year, among the four offices. Of particular interest was the finding that relatively few (1.3% in the total sample) of the respondents drove a semi truck as their most frequent vehicle. Thus, it is a reasonable conclusion that the survey sample had little if any first hand experience with respect to the operational characteristics and limitations of a semi truck.

Media Exposure. The media and enforcement campaigns were successful in creating meaningful exposure levels among drivers at the intervention sites. Based on the total sample, the percentage of people who said that they heard or saw something that was clearly related to TACT increased significantly from 17.7 percent in the pre period to 67.3 percent in the post period at the intervention sites (Chi Square = 924.851, p < 0.001, N= 3,828).¹ There were no significant changes at the comparison sites, where percentages stayed low (17.0 % pre and 19.9 % post, N.S.). Figure 9 shows these findings for pre and post periods and by wave. Exposure levels jumped from Wave 1 to Wave 2 and remained at the higher level throughout the subsequent waves of data collection.

It is not surprising that approximately 17 percent of the respondents reported exposure in the pre period. The specific question asked was “Have you recently read, seen or heard anything about giving semi trucks more space when you pass them?” This prompt is sufficiently general that it can be expected that some people will respond in the affirmative. This could be based on hearing some other recent highway safety message or simply on traffic safety messages remembered from any time in the past. It also could be the result of trying to give what they perceived to be the “correct” answer. Regardless of the origin of the baseline values, however, the change in reported exposure to messages about leaving more room for semi trucks is large and clearly only at the intervention sites. This demonstrates that the TACT media exposure was reaching its intended audience of licensed drivers and that they recalled the messages when prompted.

Figure 9. Percent who saw/heard media about giving semi trucks space

Since the TACT program resulted in high exposure, it was of interest to determine which forms of media were most successful in reaching the public. The survey asked where a person had seen or heard the information about driving behavior around semi trucks. Choices included newspaper, radio, TV, road sign, brochure, police, billboard, poster and banner. The media campaign had engaged in all of the above methods of information dispersion, with the exception of using billboards. However, billboard was included in case people thought that the large road signs constituted billboards.

Five of the media forms showed a statistically significant and operationally meaningful increase in the percentage of people who said they had heard or seen the message via that particular medium. These media forms were road signs, billboards, radio, television, and newspapers. Although billboards and road signs separately showed a significant exposure, their data were combined since it is virtually certain that people responding with respect to billboards were actually making reference to the large road signs.

As seen in Figure 10, the percentage of people seeing the signs in both the intervention and comparison groups increased significantly from the pre to post periods.² However, the increase for the intervention sites was substantially higher. Out of all of the people surveyed at the intervention sites, only 4.4 percent in the pre period reported seeing road signs or billboards, while nearly 39.9 percent in the post period reported seeing them (Chi Square = 634.631, p < 0.001, N = 3,828). A dramatic increase is seen between Wave 1 and Wave 2 at the intervention sites and remains steady between Waves 2 and 3, with a small drop-off between Waves 3 and 4.

At the comparison sites, 1.2 percent of all people surveyed in the pre period said they saw the signs, while nearly 3.7 percent in the post period claimed to have seen the signs (Chi Square = 15.142, p < 0.001, N = 2,327). Although statistically significant, the magnitude of the increase in exposure at the comparison sites is not operationally meaningful compared with the large jump at the intervention locations. The wave-by-wave data in Figure 2 reveal that exposure increased for comparison sites at a steady rate across waves but still remained well below the levels of exposure seen at the intervention sites. Also, virtually all of the increase at the comparison sites came from Kelso. Since Kelso is relatively close to Lacey and on the same I-5 corridor, Kelso residents might have been exposed to the signs near Lacey. In order to investigate this possibility, a follow-up intercept survey was mounted. Its results are presented later in this report.

Figure 10. Percent who saw the road sign

Exposure to the radio message at the intervention sites followed a pattern involving a large initial increase between Waves 1 and 2 and smaller increases between Waves 2, 3 and 4, while exposure at the comparison sites remained low and fairly constant (See Figure 11). Respondents at the intervention sites reporting having heard the radio message showed a significant and operationally meaningful increase in exposure rising from 3.4 percent in the pre period to 17.6 percent in the post period (Chi Square = 183.673, p < 0.001, N = 3,828). There was low exposure to radio at the comparison sites and no significant change from pre to post. This clearly suggests that the radio messages were effective.

Figure 11. Percent of people who heard radio message

Exposure to the earned media on television followed a similar pattern to the radio messages (see Figure 12). Respondents at the intervention sites who said they saw a message on television showed a significant and operationally meaningful increase in exposure rising from 4.8 percent in the pre period to 14.9 percent in the post period (Chi Square = 100.00 p < 0.001, N = 3,828). The primary increase was once again between Waves 1 and 2, with a smaller increase at the intervention sites between Waves 3 and 4. There was low exposure to messages on television at the comparison sites and no significant change from pre to post, although by Wave 4 there does appear to be a small increase in exposure.

Figure 12. Percent of people who saw the television messages

The survey also showed an increase in exposure to the newspaper materials at the intervention sites, however the change was not as large as for the other media forms (See Figure 13). Exposure rose from 3.9 percent in the pre period to 9.4 percent in the post period ( Chi Square = 42.90 p < 0.001, N = 3,828). The increase occurred between Waves 1 and 2, and exposure levels remained steady through the subsequent waves. There was low exposure to the newspaper placements at the comparison sites and no significant change from pre to post.

The other media, including brochures, banners, and posters showed no meaningful increase in reported exposure for the intervention or comparison sites over time. It is interesting that the radio messages, which were well produced and carefully distributed with paid media, did not produce as large an exposure increase as did the road signs. It is possible that this is an artifact of the survey process. Respondents were given a list of media forms and asked to check all that they had seen or heard. Some people, however, may only have checked the one form they remembered first or only those media that they had been exposed to most recently or most repeatedly. The road signs had the ability to produce many repeated exposures especially for anyone commuting along the I-5 corridor near Lacey or Bellingham. Thus, they may have been the most compelling presentation of the TACT message due to repetition and because they presented the TACT message at the point of behavior for drivers. Or they may have been compelling because they were an “official” sign placed by the transportation department.

Figure 13. Percent of people who saw the newspaper messages

Recall of Program Name. After determining that people experienced increased exposure to the message of giving semi trucks more space when passing, it was also of interest to determine whether people remembered the name of the program. The official program name of Ticketing Aggressive Cars and Trucks (TACT) was virtually unknown in both the intervention and comparison sites. Only 1.5 percent of respondents in both the pre and post periods at the intervention sites, and 1.3 percent and 2 percent for the pre and post periods, respectively, at the comparison sites, said they knew the name TACT as a program relating to safety around semi trucks. This is not surprising since the name was purposely not publicized as part of the program and therefore served as a distractor question in the survey.

A much larger percentage of people said that the names “Give Big Rigs Big Space” and “Leave Room When Passing” were programs that did pertain to safety around semi trucks. Although these were not the actual program names, they are central themes in the TACT messages, especially on the road signs and in the radio messages. If these three names are combined for purposes of analysis, there is a significant increase in recall for the intervention sites but not for the comparison sites (Figure 14). Awareness went up from 14.2 percent in the pre period to 40 percent in the post period for intervention sites ( Chi Square = 302.345, p < 0.001, N = 3,828). Similar to results discussed earlier, a large jump in exposure is seen between Waves 1 and 2 with a smaller increase between Waves 3 and 4. There was no significant change at the comparison sites.

Figure 14. Percent of people naming TACT/Big Rig/Leave Room

Self-Reported Behavior Change. Since the survey confirmed that much of the driving population in the intervention sites had been exposed to the TACT message, it was of interest to examine the extent to which this exposure affected the target behaviors. The survey asked people to self-report if they had changed their driving behavior around semi trucks in the past two months. Results indicated that significantly more people reported having changed behavior in the last two months during the post period than the pre period for the intervention sites, but not for the comparison sites. People reporting that they changed behaviors rose from 25.9 percent to 33.8 percent at the intervention sites ( Chi Square = 27.382, p < 0.001, N = 3,828). Figure 15 demonstrates the gradual increase at the intervention sites across Waves 1, 2 and 3, with a minor reduction during Wave 4. Comparison sites showed virtually no change across the four waves.

Figure 15. Percent who said they changed behavior in last 2 months

Of the three choices presented for changes in behavior, the one selected most frequently was “I leave more space when passing.” As shown in Figure 16, respondents reporting that they left more space when passing rose significantly from 16.4 percent in the pre period to 24.3 percent in the post period at the intervention sites (Chi Square = 35.431, p < 0.001, N = 3,828). Increases are seen at Waves 2 and 3 that level off at the higher level during Wave 4. There was no significant pre/post change at the comparison sites. The other two survey responses of “I don’t follow as closely” and “I stay out of the truck driver’s blind spots” showed no significant change between pre and post periods for either the intervention or comparison sites. These results are consistent with the TACT campaign’s message.

Figure 16. Percent leaving more space when passing semi trucks

Another way to measure behavior change in response to the TACT interventions is with respect to specific actions motorists take when passing semi trucks and cars. Separate survey items asked people to indicate how many feet or car lengths they left when they passed a car and when they passed a semi truck. Based on the responses, a ratio of semi truck to car distance was computed to determine if people left less, the same, or more space when passing semi trucks than when passing cars. A significant increase in the number of people saying they left more space for trucks than for cars was found at the intervention sites with percentages rising from 58.5 percent in the pre period to 68.0 percent in the post period,(Chi Square = 31.323, p < 0.001, N = 3,310).³ The increase is slow and steady at Waves 2 and 3 and levels off at Wave 4. Comparison sites showed no significant change. As shown in Figure 17, these results provide further support that people self-reported that they were leaving more space for trucks after exposure to the TACT campaign. As will be seen later, these self reports were confirmed by the observational data.

Figure 17. Percentage who leave more space for semi trucks than for cars.

Strictness of Enforcement. Respondents were also asked how strictly they thought the Washington State Patrol enforces unsafe driving around semi trucks. Analysis showed a significant increase in the percentage of respondents at intervention sites saying that they thought the WSP enforcement was somewhat strict to very strict. Percentages went from 52.2 percent in the pre period to 56.4 percent in the post period ( Chi Square = 5.907, p = 0.015, N = 3,422). Overall, more respondents at comparison sites (approximately 61 percent) thought that enforcement by the WSP was strict, but the percentage did not change significantly from pre to post.

Another item asked if the respondent had ever been stopped by the police for tailgating or cutting off a semi truck. The number of “yes” responses was extremely small at all sites, and no significant effects were found.

Summary of Survey Results. Overall, the DOL public awareness survey demonstrated that people at the intervention sites were seeing or hearing the TACT messages and remembering the core message of leaving more space when passing trucks. Interestingly, in the present study road signs were the best method of relaying the TACT safety messages to drivers, with radio ads also effective but a distant second. This is not surprising since both media types are point of behavior countermeasures, and drivers likely have more repeated exposures to the road signs than to the radio spots. A noteworthy fact is that people reported changing their behaviors around semi trucks, especially when it comes to leaving more space when passing. Results also indicated that people at the intervention sites felt that the WSP was being stricter about unsafe driving around semi trucks after the TACT program was implemented. Whether this perception came from the publicity about increased enforcement or the higher visibility of the WSP cannot be determined from the survey. Overall, the DOL survey results suggest that both the media and enforcement campaigns had the desired effects on public awareness of the messages and enforcement and self reported behaviors.

¹ Statistical significance was tested using the Chi Square statistic which is a measure of association. Chi Square analyses examined the actual versus expected frequency of responses at intervention and comparison sites or between pre and post periods. A pre to post or intervention to comparison effect was considered statistically significant if the probability that it could have arisen by chance as calculated from the Chi Square statistic was less than 5 percent (p < 0.05). In the remainder of this section, significant associations are reported together with their Chi Square value and the associated exact probability that they could have arisen by chance. Differences that were not significant by the Chi Square test are simply listed as “(N.S.)” or reported as not significant when described. The sample size or “N” value on which each Chi Square is based is also shown when the results are reported as percentages.

² All of the percentages reported for the survey are based on the total number of respondents. This includes everyone who did not see any of the media (about one third of the sample). If the percentages had been based only on those respondents who indicated they had been exposed to some TACT campaign component, the percentages would have been higher. As presented, the results are the best estimate of the extent of the total intended audience that each media form reached.

³ Note that N is based on the number of people who provided viable data for analysis. Missing or uninterpretable responses were eliminated.