Evaluation

Specific Evaluation Methods and Results

I-5 Corridor Intercept Survey

Examination of the videotaped violation rates and media exposure for each site individually led to the realization that one of the comparison sites, Kelso, may not have been as isolated from the TACT program as intended.  Kelso showed a steady decline in observed violation rates similar to what would be expected at an intervention site (Table 3 in Appendix F).  Reviewing the public awareness survey data suggested a possible media spillover.  For example, the percentage of respondents at Kelso’s driver licensing offices who said they had heard or seen any TACT related message increased from 19 percent in Wave 1 to a high of 28 percent in Wave 3.  These findings were masked in the general analyses because many more violations were observed at the other comparison site, Spokane, than at Kelso.  Spokane’s larger numbers biased the combined site violation rates towards the higher levels (seen at Spokane).  In turn, results of the DOL survey in Spokane showed lower levels of media exposure compared to Kelso.  This served to lower the overall levels of exposure for the comparison sites when data from Kelso and Spokane were combined.

The WTSC conducted a follow-up intercept survey after the conclusion of the TACT program in October to explore whether to which drivers in the Kelso I-5 corridor comparison site had been exposed to the TACT message. Survey results from the Kelso DOL office indicated a significant increase in the number of people reporting that they had seen or heard something about safety around semi trucks. Also, violation rates, as seen on the video recorded by troopers, dropped at the Kelso site from pre to post periods even though no increased enforcement or media had intentionally been implemented in the area. The focus of the intercept survey was therefore to determine if the TACT message had actually penetrated into Kelso.

WTSC conducted the intercept survey at two sites, one at the Kelso rest area on the I-5 corridor in Kelso, and another at the Maytown rest area on the I-5 corridor near Lacey/Olympia. At each of these sites, the WTSC contractor surveyed 100 drivers. Drivers were asked about where their trip began and would likely end, what type of vehicle they drive, how often they drive in the Olympia area, how often they listen to Olympia/Seattle radio stations, whether or not they knew any of the TACT messages and if they had seen the TACT road signs. See Appendix C for the complete intercept survey.

Responses were analyzed to determine the extent to which people in the Kelso area indicated that they had seen or heard the TACT message compared with people at the Maytown rest area. It was expected that people at the Maytown rest area would be aware of the TACT program because the rest area was so close to Lacey/Olympia where TACT signs and radio messages had been deployed. The effect of interest was whether or not the possible routine movements of the Kelso population along the I-5 corridor had exposed them to TACT, particularly the signs and radio messages.

Intercept Survey Results. Results indicated that there was a noteworthy spillover of the TACT message to drivers using the I-5 corridor around Kelso. Similar percentages of respondents said that they heard the TACT message or saw the road sign. A non-significant difference suggests that people at the Kelso and Maytown rest areas were similarly exposed to the TACT messages; 68 percent of the people at Kelso and 76 percent at the Maytown rest area said they saw the road sign. These results suggest that the TACT message reached beyond the intended target area. Kelso is 70 miles south of Lacey/Olympia and yet people were saying that they had been exposed to the TACT message. This intercept survey was a convenience sample of very small size intended to suggest possible explanations that could be explored in future studies. 

Implications of the Intercept Survey Findings. There are several implications of the spillover of the TACT program into Kelso, a designated comparison site. From the standpoint of planning an intervention project, a lesson to be learned is that interstate interventions may have a more far reaching effect than initially considered. When the TACT program was devised, it seemed reasonable to assume that most Kelso drivers would be isolated from the signs 70 miles to their north. In fact, these findings suggest that mobility along the I-5 corridor may be greater than anticipated.

The effect of these findings was to suppress the magnitude of the positive shifts observed in the intervention corridors. In other words, categorizing Kelso as a comparison site when it displayed an intervention effect in the DOL survey and in violation rates made it less likely that a significant intervention/comparison difference would be detected.

Limitations of Study. The study design included before, during, and after public awareness data, unsafe driving observations, and law enforcement and media activity data along two intervention corridors and two comparison corridors.  This is a reasonably powerful evaluation design because it tracks before and after measures while simultaneously assessing whether those changes might have occurred naturally at the comparison sites.

Analyses conducted after the TACT project suggested that motorists near one of the comparison sites (Kelso) may not have been completely isolated from TACT’s intervention activities, which is not so unusual in field demonstration projects. The 100-person intercept interview surveys conducted in October gathered information about possible spillover of the media messages in Kelso, which was 70 miles away. The intercept interview technique is a convenience sample and is not intended to be representative of Washington’s driving population and further testing would be necessary to confirm the preliminary suggestions of the intercept interviews. If, however, Kelso’s data were removed as one of two comparison corridors, the effect would be to increase the magnitude of TACT’s positive effects in increasing motorist’s awareness about leaving more space around trucks and in reducing the instances of unsafe driving around large trucks.