![]() David Weil Acknowledgments Funds for this study were provided by a grant from CPWR – Center for Construction Research and Training (CPWR). I am grateful to Pete Stafford and James Platner of CPWR for their interest in this project, which builds directly on a prior study of OSHA enforcement also supported by CPWR, and to Jane Seegal for her assistance in editing this manuscript. Thanks also to Bruce Swanson, H. Berrien Zettler, and Richard Rinehart of the OSHA Directorate of Construction for their input and access to OSHA personnel, to Joseph DuBois of OSHA's Office of Statistics, and to the OSHA Construction Targeting Task Force for provision of survey data. Dr. William Schriver of the Construction Industry Research and Policy Center at the University of Tennessee generously provided me access to data as well as insight into the Dodge/CRA system. Don Cotchen and Anita Gryan of F.W. Dodge Inc. were extremely helpful in providing information on the Dodge system and feedback on issues in the proposal. Finally, I am grateful to Alison Morantz and Marcus Stanley for comments at early stages of the project and to Nao Valentino for her ongoing research support. The analysis and conclusions of this paper reflect my own views and not those of any of the above individuals or institutions. Abbreviations
Contents
Construction targeting procedures and the Barlow's decision Crafting alternative policies for targeting in construction
The effects of current targeting procedures The view of targeting performance from OSHA area offices Implications for future targeting efforts
Projectlevel targeting Riskbased targeting
An alternative method for assessing prospective risk Correlating project characteristics and risk levels
Alternatives outside of the Dodge/CRA system
Targeting based on project stage Using local emphasis programs for targeting
Classic experimental design A proposed experimental design for evaluating targeting Selection of area offices Treatment groups Dodge/CRA list treatments Other treatment groups Treatment procedure Control groups Outcome measures Evaluating program success Costs and benefits of the pilot experiment approach References Figures
2. Effects of site and contractor level inspections on predicted compliance with key construction safety and health standards, 198793 3. Effects of recent contractorlevel inspections on predicted compliance with key construction safety and health standards, 198793 4. Evaluation of Dodge/CRA targeting methods 5. Cited problems of Dodge/CRA targeting methods 6. Organization of construction projects general contractor versus construction manager coordination 7. Top 15 violation rates per contractor, by major project type 8. Bottom 15 violation rates per contractor, by major project type
2. Value of projects by region, Dodge/OSHA IMIS combined sample, FY19992001 3. Number of contractors on projects in Dodge/OSHA IMIS combined sample, FY19992001 4. Major project types in Dodge/OSHA IMIS combined sample, FY19992001 5. Correlates of violation rates by project characteristics, FY19992001 6. Violation rate correlates with project characteristics and types with high violation rates: region 2 vs. region 3, FY19992001 The problem of targeting in construction Construction job sites are some of the most dangerous workplaces in the United States. In 2002, there were 1,153 deaths from injuries in construction more than in any other industry and 163,700 injuries and illnesses involving days away from work1(Bureau of Labor Statistics 2004a, 2004b). Although the rate of reported injuries fell substantially over the past decade, the rate for construction remains well above that for the private sector as a whole. 1.Illnesses make up only about 2% of the figure for construction. 2. The OSHA Directorate of Construction has commissioned several evaluations of targeting, including the study by Ringen in 1999. In late 2002, the Assistant Secretary of Labor for OSHA, John Henshaw, appointed a Construction Targeting Task Force to study this problem. The author has worked in consultation with Richard Rinehart, chair of the task force, in the present study and has used data generated by their work. The views and recommendations found in this report reflect only my conclusions and not those of the task force or OSHA. Construction targeting procedures and the Barlow's decision There are millions of active construction projects at any time. In 2002, OSHA had 1,123 inspectors at the federal level, devoting roughly 40% of their time monitoring safety and health conditions in the construction sector. ("OSHA" throughout this report refers to the federal agency, whose jurisdiction includes 29 states; the 21 other states have separate state OSHA plans.) Given the limited number of inspectors, selecting sites for inspection is an enormously difficult task for OSHA, but remains critical to the agency's ability in carrying out the intent of the Occupational Safety and Health Act (OSH Act) of 1971 to improve workplace safety and health. Ideally, OSHA would focus its resources on those projects that subject the largest number of workers to the most severe safety and health risks. However, with limited information about schedules for construction activity in a given region and about safety and health conditions in construction, scheduling inspections remains a major problem. Worker complaints and referral inspections can provide some information about risks on current work sites, but this information is not systematic3. OSHA inspections triggered by a workplace death or major accident can provide information on major problems that may have value for targeting, but the information is not often used for this purpose. When the agency began, in the early 1970s, OSHA offices used a variety of formal and informal methods for targeting inspections, ranging from reliance on complaints, to use of information available from local permitting and public agencies regarding construction activity, to drawing on the knowledge of areaoffice OSHA compliance officers about upcoming construction. The targeting system now used by OSHA arose from a Supreme Court ruling in Marshall v. Barlow's Inc. (429 U.S. 1347, 97 S. Ct. 776) in 1977. In that decision, the Supreme Court required that OSHA use an objective and documented basis for selecting targets for inspections. Specifically, in selecting workplaces for programmed (planned) inspections, OSHA needed to demonstrate use of "specific neutral criteria" to prevent arbitrary or abusive behavior by personnel of the agency. Though the definition of "specific neutral criteria" is complicated, it rests on the idea that all establishments within a defined universe of employers covered by OSHA bear similar chances of being inspected. As a result, the definition of the relevant universe of employers becomes critical to the targeting mechanism. Subsequent to the ruling, OSHA formally adopted in its Field Operations Manual a procedure for targeting construction work sites: A list of all known construction worksites that are or will be active during the forthcoming scheduling period shall be prepared using sources such as Dodge reports, local building permits, and CSHO [compliance safety and health officer] sightings of construction activity Field Operations Manual, Chapter II.E.2.b(2)(a), cited in U.S. Department of Labor 1987 3. Workers exercise their rights under OSHA unevenly. Because many workers are uninformed about their rights under the OSH Act or fear discrimination, many workers (such as immigrants, younger workers, and those working on nonunion, smaller sites) do not complain even when faced with dangerous conditions ( see Weil 1992 and 2001). A 1987 audit by the Office of the Inspector General of the U.S. Department of Labor concluded that several OSHA area offices had not come into adequate compliance with the Barlow's decision requirements on establishing a neutral and documented system of construction targeting. The Inspector General noted 6 of 8 area offices audited failed to have documented lists of all potential construction sites for the area (U.S. Department of Labor 1987, pp. 4546). Coinciding with the Inspector General's investigation, OSHA initiated a project to create a computerbased targeting system to be introduced the following year in order to automate the process of selecting projects from a universe of all potential sites. The targeting procedure created in response to Barlow's and the Inspector General report uses construction permitting data collected by F.W. Dodge Inc. and statistical models that predict construction starts and estimated durations to establish the universe of active construction projects for a given geographic area corresponding to each OSHA area office. For each OSHA area office, a list of projects is randomly selected from the estimated universe of active construction projects. The OSHA area office must then inspect contractors and subcontractors on all sites provided on the list during the course of the month (or carried over to the next month if there is insufficient time for completion). In this way employers (contractors and subcontractors) are identified on the basis of projectlevel activity. As a byproduct of the unique structure of construction, this procedure of employer identification therefore differs markedly from the fixedsite/fixed employer model that OSHA generally uses. Crafting alternative policies for targeting in construction The current construction targeting system's ability to help OSHA achieve its central mission of improving workplace safety and health has been a topic of debate since the system was instituted in the late 1980s. Proponents of the system argue that it fulfills the Barlow's requirement of a neutral and documented method for identifying construction sites, drawing upon arguably the most comprehensive data available nationally regarding construction activity. Current Targeting Practice and Performance Operation of the Dodge/CRA system The performance of the current targeting system is a function in part of the method that has been employed to create a representative universe of construction required by Marshall v. Barlow's, Inc.
F.W. Dodge Inc. (a division of McGrawHill Companies) employs a network of field reporters across the country to collect information from local permitting agencies, public bidding systems, companies, and end users regarding planned construction activity. Information on planned activity is used to create a comprehensive database (Dodge reports) on planned construction projects, which includes commercial projects valued at more than $100,000. Contractors, subcontractors, and other construction material and service providers planning to bid for private and public work are the primary users of Dodge reports. Beginning in 1988, OSHA contracted with Dodge and a group of researchers at the University of Tennessee Construction Resources Analysts (CRA) to use these data to generate a universe of construction activity for geographic areas across the country4. The resulting construction universe created by the Dodge/CRA system serves as the basis for selecting construction sites for OSHA inspection on the "neutral basis" required by Barlow's5. 4. Construction Resources Analysts (CRA) has been headed since its inception by William R. Schriver, PhD. Although the group recently changed its name to the Construction Industry Research and Policy Center, it is referred to as CRA throughout this report. 5. The focus of targeting procedures discussed here are OSHA programs, which are under the jurisdiction of federal OSHA. Although stateadministered programs often draw on Dodge data that are provided to them at no charge, states that administer their own OSHA programs 18(b) states can establish their own targeting procedures and protocols. 6. The Dodge system remains the most comprehensive national database on construction available in the United States. Its coverage of construction activity is extensive for projects above $100,000 in most sectors outside of residential construction. The logic of using this database as the underpinning of targeting therefore is strong. See Meridian Research 1992 and Ringen 1999 for earlier assessments of the adequacy of the Dodge data 7. Barlow's does not apply to all OSHA inspection activity in construction. Only programmed inspection activity is germane, rather than inspections initiated by a complaint, death or major accident, or by inspector referral. Planned programmed activity is governed by a systematic and neutral selection procedure that usually falls into one of two categories scheduled inspection using the Dodge/CRA system or a national or local emphasis program where a different procedure has been developed that is directed at a particular problem (falls) or sector (bridges). Although the targeting procedure is based at the project level, OSHA enforcement activity in all other respects is focused on the contractor(s) active on each site at the time of an inspection. That is, OSHA personnel record, track, and report on inspection activity on the basis of contractors, rather than projects, inspected. Thus, two projects of equal scale would be counted differently if only one contractor was present at the time of one inspection and eight on another. Because the number of inspections conducted is an important metric for OSHA, both in terms of its internal procedures and its reporting to the White House, Congress, and the public, the incentives for the agency have always pushed toward visiting construction projects with a large number of contractors present. Thus, area offices can provide CRA with project characteristics (called "deletion criteria") that limit the universe from which random sites are selected. The most common form of these is specifying a minimum project size (measured in dollar value) for inclusion in the list. In order to maximize the number of contractors inspected during a visit to a site, area offices tend to prespecify that only larger construction sites are included in the sampling universe8. Because most area and regional OSHA offices provide minimum dollar volumes in their deletion criteria, the resulting lists are biased toward larger projects. Because of this emphasis on larger work sites, OSHA enforcement for programmed inspection tends to be skewed toward monitoring the safety and health activities of large construction contractors and subcontractors, with annual revenues well above $5 million. The effects of current targeting procedures The targeting procedure developed in response to Barlow's, which was biased toward large sites, made sense in OSHA's first few decades of operation when it was reasonable for the agency to try to move as many contractors as possible toward compliance in a world of widespread noncompliance with newly promulgated safety and health standards. Studies of the effects of OSHA inspections in the early period of regulation show a high level of responsiveness to enforcement and therefore the rationality of an approach focused on larger employers (Bartel and Thomas 1985; Scholz and Gray 1990; Gray and Mendeloff 2001; Jones and Gray 1991; Stanley 2000; Weil 1996, 2001). 8. The universe of potential targets can be modified by general criteria selected by the area office, although offices are obligated to then inspect all those randomly selected from that list in order to comply with the Barlow's standards (as interpreted by OSHA practice). The tendency for larger projects to have better safety and health performance can be vividly seen in comparisons of the incidence of violations of a subset of core OSHA standards relating to physical hazards between the sample of major national contractors inspected by OSHA and all other construction inspections conducted during this period (fig. 1). Large national contractors had a lower percentage of inspections with violations compared to inspections conducted at all other construction establishments. In 1993, for instance, while 51% of the inspections of large national contractors found at least one violation, almost 69% of the inspections conducted at all other construction establishments found violations9. The percentage of inspections with serious violations shows a similar gap between the two groups that grew from 5.6% in 1987 to 16% in 1993 (Weil 2001). In addition to maintaining higher levels of compliance, large contractors tend not to improve compliance further, even though their projects receive a great deal of scrutiny. In the national contractor sample of 2,060 firms, the average number of inspections per contractor was 19.4 over the 6 years studied, with some contractors receiving more than 100 inspections during the period. This meant that these large contractors had about a 5050 chance of receiving an inspection on at least one project in any given year. Yet the study found that repeated inspections did relatively little to improve compliance. Figure 2 displays the impact of site and contractorlevel inspections on predicted levels of OSHA compliance10. The upper line displays the effect of sequential inspections of a contractor by OSHA on one site; the lower line indicates the effect of sequential inspections on predicted compliance given inspections conducted on any site where the contractor was active. The upper line shows that contractors started on a given site at high states of compliance (74% in compliance at the time of the first site inspection) and changed little, even given repeated inspection on the same site. Even after eight inspections of a contractor at the same site, predicted compliance rose only to 80.5%. Similarly, contractors responded more, but still modestly, to the effects of inspections conducted on any of their project sites, with predicted compliance rising from 61% on any site at the time of the first inspection in the time period to 76.6% at the time of the eighth inspection on any site. Figure 3 underscores these modest enforcement effects on large contractors by focusing on the effect of an additional inspection conducted within one year on compliance behavior11. One would imagine that recent contractor inspections are more likely to result in changes in compliance behavior. Yet even when limiting the predicted effects to more recent inspections, large contractors showed limited responsiveness to OSHA. There are several explanations for the recalcitrant behavior of large contractors. Large contractors tend to begin in and maintain relatively higher states of compliance (fig.1) than other, smaller contractors targeted by OSHA. This lack of response to OSHA enforcement implies that contractors have decided on some level of safety and health activities and practices factoring in the risk (and cost) of being inspected and penalized and choose not to alter them. The behavior might imply also that contractors cannot make the changes required to come in full compliance with standards, despite repeat inspections, because the changes would require responses of other parties at the construction site, including those involved in overall project management activities. 9. The percentage of all inspections where any violation was cited is used as a broad measure of compliance in these comparisons. Figure 1 includes violations of any OSHA standard for both groups, including but not limited to the subset of standards used in the rest of the 2001 Weil study for examining compliance. Serious violations of standards include those classified as "serious," "willful," or "repeat" by inspectors. 10. Compliance is defined here as not violating any of a set of 100 core OSHA standards associated with physical hazards. The methodology and results are discussed in detail in Weil 2001. 11. Compliance is measured in figure 3 in two ways: the lower line measures compliance as a contractor not violating a key OSHA standard regardless of severity; the upper line measures compliance as a contractor not being cited for a serious violation of standards. Ringen (1999) discovered related problems in his study of OSHA enforcement targeting procedures: There is a tradeoff between neutrality and inspection effectiveness as measured in terms of violations and penalties. The planned programmed inspections, based on neutral selection of inspection targets, are bound to produce "less inspection bang" than the unprogrammed inspections that are based on cause.(pp. iiiiv) Ringen focused on the fact that inspections triggered by worker complaints or death/accident investigations are more likely to result in violations than any programmed procedure not directly linked to potential problems. The view of targeting performance from OSHA area offices In spring 2003, the OSHA Targeting Task Force undertook a survey of federal OSHA offices regarding targeting practices. The survey was distributed to all federal area offices and to California, which has a state OSHA plan. The task force received 69 responses, including some from 3 regional offices and 4 federal area offices in state plan states. Eleven of the responses from the California agency, CalOSHA, were not included in the analysis, however, because of systemic differences between the California system and others. The OSHA Targeting Task Force provided the survey data for review for this study. 12. Of the 40 using an alternative system either in part or in lieu of Dodge more than half, 23, say the alternative system is very useful. The problem of the Dodge/CRA system most commonly cited by survey respondents concerned the level of construction activity found on sites at the time of inspections (fig.5). Despite the efforts of CRA to improve prediction of project start times and durations, half of the respondents partially or completely agreed with the statement "construction sites are either already completed or not yet started at the time of inspection." Onethird of respondents agreed with, "the same contractors are selected over and over again." About the same percentage agree with the statement that construction sites on the Dodge/CRA lists " cannot be found or do not exist." Few area offices view the mix of project types that is, their end uses (such as factories or hotels) identified in the lists as a problem, while only 13% of respondents agreed with the statement that "the site selected are not representative of the work in the area. Implications for future targeting efforts Results of the OSHA Targeting Task Force survey suggest that the system adopted in response to Barlow's has been substantially adopted in federal OSHA offices. Yet the problems just discussed are widely recognized. 13. CRA has undertaken a series of studies and revisions of techniques to forecast construction starts since the beginning of its contract with OSHA and the Department of Labor. These include studies and modification of procedures in 1989, 1992 and 1997. See Schriver (1997) for a discussion of the most recent analysis of estimation of construction starts. Targeting Alternatives: Principles If one began with a blank slate to design a policy for allocating OSHA resources across the construction industry, it would reflect that the work process is inherently dynamic because the geographic location of work changes over time, the conditions on any given site are shaped by multiple employers, and working conditions are in constant flux as a project proceeds. An ideal targeting policy would follow three major principles that derive from these characteristics (each one will be discussed in detail below):
Data used for this section This section, Targeting Alternatives, draws on a unique data set created by the OSHA Office of Statistics for analysis by William Schriver, the director of CRA (see Schriver 2003). The data set combines information from the F.W. Dodge system regarding projectlevel characteristics (used for assembling targeting lists) and from OSHA's Integrated Management Information System (IMIS) on inspections of companies operating on those projects. 14. Prior to the incorporation of Dodge project IDs, the only way to group contractors together on a common site was to use other identifiers in the IMIS system address, zip codes, dates to link the contractor files back to a common project location. This method was both extremely time consuming and prone to errors, for example because a single construction site might be listed under multiple street addresses. 15. Unfortunately the Dodge project identification number is not yet uniformly provided in IMIS. Some inspection files might not include the Dodge number because the inspection was nonprogrammed (that is, triggered by a complaint, accident, or referral). In other cases, OSHA personnel might not have provided the data for programmed activities. I was not provided with an estimate of the number of contractor records that were excluded from the sample because of a missing Dodge identifier, nor characteristics of the excluded records. 16. The results presented here represent my own analysis of the data and all conclusions are solely my own and not intended to represent those of William Schriver, CRA, or OSHA. The projects analyzed were large, with an average value of $7.1 million (table 1). That average masks the scale of many of the projects inspected by OSHA, with almost 10% of the projects having a value above $15 million and the largest project in the sample an estimated value of $373 million. In contrast, less than 5% of the projects had a value below $500,000. By region, the average value of projects ranged from a low of about $5.0 million in region 10 to a high of $8.5 million in region 1 (table 2). The number of contractors inspected on projects varied widely in the sample. The average number of contractors on a project was 1.4, with the largest number of contractors inspected at a single site at 26. Only one contractor was inspected at 74% of the projects (table 3). A considerable percentage of this group might reflect cases where inspection records did not contain Dodge record identifiers and could therefore not be linked to a project for the analysis. However, the cases where there was only one contractor inspected per project may also reflect the ongoing problem of accurately predicting peak construction activity at sites (identified in the OSHA Targeting Task Force survey as a major problem)17.Projectlevel targeting A projectlevel focus for OSHA activity makes sense given the central role that coordination plays on most construction sites. A construction project of any size requires synchronization between many separate business enterprises and workers, with varied responsibilities, skills, and roles. It is therefore not surprising that the industrial organization and industrial relations/human resource systems in construction are extremely complex, as well as decentralized (see Dunlop 1961 for a classic discussion of this issue). 17. Unfortunately, it is impossible, given the present data, to estimate the portion of cases involving a failure to match Dodge identifiers with IMIS records from the case where only one contractor was inspected on a project. However, comparing construction industry classifications (SIC code) of contractors receiving inspections on projects with only one contractor inspected versus multiple contractors inspected provides a glimpse into the prevalence of the problem of timing on construction sites. If the projects where only one contractor has been inspected represent a case where the Dodge/CRA system led an inspector to a site too early or too late in the construction cycle, one would expect those sites to have a relatively high prevalence of subcontractors rather than general contractors at the time of the single inspection. A review of 250 records in the data set found that 38% of projects with only one contractor inspected consisted of employers classified as subcontractors versus 22% for projects where multiple contractors were inspected. The owner, in turn, typically hires a firm to coordinate construction. Historically, this role was filled by a general contractor (GC) who served two functions: managing the construction project and being the direct employer of the occupations that tend to remain throughout a construction project (the "basic trades," laborers, carpenters, and operating engineers). The general contractor would be responsible also for overseeing and coordinating the work of subcontractors associated with specialty trades, such as electrical, plumbing, sheet metal, roofing, and other contractors. The larger and/or more complex the project, the more subcontractors typically would be on a job. 18. See Weil 2003 for a more detailed discussion of these changes in industry structure and their implications. 19. The Dodge system uses a moredetailed system of classification consisting of 109 individual project types. To simplify this system, I have followed a project enduse classification system developed by Schriver in his recent analysis that aggregates these into the 36 categories listed in table 4. Riskbased targeting The notion of targeting construction projects on the basis of risk seems selfevident: If a construction site has a higher potential level of injuries and deaths than another, one would want to direct OSHA enforcement there. OSHA has over the years drawn on this notion generally in targeting industries (including construction) with higher reported injury and illness rates. Various efforts have also been made to do so at the establishment level within industries, particularly in the manufacturing sector (GAO 2002)20. 20. The GAO (2002) indicates that, even in manufacturing, there have been a number of difficulties in implementing enforcement procedures that draw on establishmentlevel injury data for targeting. 21. Other studies point to a related problem of relying on administrative information involving selfreporting of injuryrelated outcome. Ruser and Smith (1988) found consistent evidence that inspection targeting based on firmreported injury rates (the "record check" method used by OSHA during the Reagan administration) not surprisingly leads firms to underreport injuries. This means that any system relied on for risk information must draw on data where the incentives to understate risk are low or kept in check by countervailing procedures. Even if one could wave a magic wand and obtain accurate injuryrate information for contractors, it is still not clear that this would provide the best prospective measure. One reason is the differing interactions of contractors with one another and with the GC/CM and project owner on a given project. Anecdotal information suggests that one construction company can act differently on two separate projects. Once again, the complexity of construction project organization becomes important (fig. 6). Is the best predictor of future behavior on any site the average past performance of the contractor on a range of completed projects or the contractor's performance on sites with comparable construction management or other characteristics? This is a particularly tricky issue because significant lostworkday injuries and especially deaths from injuries often occur after a breakdown in a system of factors involving multiple parties rather than failure of a single player. Major accidents generally occur when a complex system has been pushed too far and several failures occur simultaneously (Perrow 1999). An alternative method for assessing prospective risk Given the myriad of substantive and administrative problems regarding injury and illness rates as risk predictors, what other measures of potential risk could be applied in setting regulatory targets in construction? 22 There are other explanations for these results as well. One is that they reflect the phenomenon known as "regression to the mean" that arises in statistical processes involving sampling. An observation that is significantly above the mean in one period will tend to fall back toward the mean subsequently. Another explanation offered by Ruser for his findings is the presence of unobservable characteristics associated with injury rates. The more one knows these correlates, the better the characteristic is as a targeting instrument. The flip side is that studies like Ruser's indicate that the "hidden" covariates may be hard to find, particularly a priori. One potential resource is OSHA's Integrated Management Information System, with its historical record of programmed, complaint, accident, and referralbased inspection reports. IMIS cannot, however, be treated as a scientific, random sample of workplaces. Correlating project characteristics and risk levels A revised targeting process that uses prospective risk to workers as a criterion for selecting projects requires connecting construction project characteristics with risk measures. The projectlevel data set described earlier (pg.9) provides a method for doing so by using OSHA inspectionbased measures of risk from IMIS and information on project characteristics from the Dodge database. 23. This method was used as the basis for analysis of OSHA enforcement performance in Weil 2001 and described in this report, in Current Targeting Practice and Performance, above. Methodology: For each project identified in the data set, the total violations found across all contractors on that project are tabulated. Although we do not construct the more refined measures of key standards described above, the methodology and analysis that follow could be easily applied to a more refined measure of violations. Here, we create a standardized measure of violation rate equal to the total violations found on the project as a whole divided by the total number of contractors inspected (average number of violations per contractors per project). For the whole sample, the average value for this variable was 1.3, varying from 0 to a high of 34 violations per contractor per project. 24. For example, one problem in this analysis is the large number of projects where only one contractor has been identified as having had an OSHA inspection. Because we cannot assess how often other contractors were not linked to the site because of a failure to include the Dodge identification number in the IMIS file, it is not possible to estimate how representative the sample used here is of the population of project types. We discuss this problem further below. 25. The estimates are based on a Tobit regression model. A Tobit model is used for estimation purposes because of the large number of projects where there were zero violations cited by OSHA. Because the value of the dependent variable cannot go below 0 and there are a large number of these observations, the dependent variable is leftcensored leading to biased estimates for the independent variable using ordinary least squares procedure. In the Schriver (2003) study of the same data, he models violation rates as a dichotomous (0/1) variable, where the outcome is defined as the presence or absence of any violations. The results might not hold in a changed analysis. A number of results from the model command attention. First, the dollar value of the project is negatively (and significantly) related to the violation rate that is, as the value of the project increases, the estimated violation rate decreases, holding other factors constant. This is consistent with the notion that larger projects tend to have better overall safety and health practices than smaller practices. A second variable that measures the squared value of projects is included to determine if the size/violation rate effect changes with scale. The positive and significant value of that variable implies that the size effect diminishes as projects become very large (that is, the size effect on violation rates becomes less negative as project size becomes very large), but the magnitude of this effect is quite small. Several different estimation techniques affected other variables, but the negative relation of project size and violation rates remained in most models. 26. Results are available from the author. Other potential project factors might be associated with violation rates in future analyses, once merged samples like the one used here have been assembled and analyzed. One factor of particular potential use is the association between certain project managers (GCs and CMs) and end users (public or private) in a region and violation rates. It is possible in principle to create projectlevel variables for ownership type or project managers in a region (for example, classifying public projects by the type of bidding procedure used to allot work or private users by whether a project was intended for their own use or for resale/leasing) and gauge their relation with violation rates. Or, it might be possible in certain regions to even capture the relation between a particularly dominant end user or project manager and violation rates on projects owned or managed by that party. Given the importance of coordination, examining these relationships could provide further insight into prospective project risks, based on past experience, and would rely on data already captured in the Dodge and IMIS systems. Efficacybased targeting The final targeting principle focuses on the willingness of contractors to change behavior once they have been targeted for inspection. Measures that capture regulatory efficacy address the question that, once a problem has been identified in an industry or with an employer, exactly how much resistance would OSHA face in trying to solve it? Targeting Inside and Outside the Existing System Alternatives using an amended Dodge/CRA system Any realistic modification to the existing targeting system must do so with the recognition that the Dodge/CRA universe will remain the starting point for the foreseeable future. As a result, any procedure for targeting must draw on information that can be found prior to identification of projects for inspection and using criteria that could be identified using projectlevel fields available in the Dodge system. 27 See PBS Frontline, A Dangerous Business, January 2003 (www.pbs.org/wgbh/ pages/frontline/shows/workplace/). See also David Barstow and Lowell Bergman, At a Texas Foundry, An Indifference to Life, The New York Times, January 8, 2003, p. A1, www.nytimes.com. Sizebased targeting. Studies conducted using a variety of data sets (including the analysis presented in this section) suggest that large projects tend to have lower safety and health risks. Large projects remain an area of concern for OSHA because of the large numbers of workers in each of those projects. Yet the evidence strongly indicates that some type of reverse weighting procedure that would raise the probabilities of project selection for smaller projects seems appropriate. Dodge/CRA could therefore experiment with a project weighting scheme that incorporates some form of inverse weighting based on size (within the Dodge sample)28. 28. Ringen advocated a similar modification based on his analysis of regional data in 1999. Alternatives outside of the Dodge/CRA system There are several additional ways to change the targeting system to better attain the principles of targeting described at the outset. As these methods involve the use of systems outside of Dodge/CRA, they are probably not feasible in the short term. Nonetheless they should be considered as alternative procedures to Dodge/CRA (or as adjuncts to that system) in the longer term. Targeting of residential housing Starts of singlefamily homes reached 1.27 million in 2001, amounting to a total value of $206 billion (Joint Center for Housing Studies 2002). The scale and extent of the residential housing sector and OSHA's historic lack of attention to it have been an area of concern in and out of the agency. An effort mediated by the late John T. Dunlop and involving OSHA, the National Association of Home Builders, and the Building and Construction Trades Dept, AFLCIO, identified OSHA standards and training programs applicable to residential construction (Coulton 2003). The current programmed inspection procedure, however, virtually ignores the sector in establishing inspection priorities. This, in part, arises from the spotty inclusion of the residential sector in the Dodge system, as well as the difficulty in efficiently inspecting residential projects from the point of view of OSHA. Targeting based on project stage As noted, different stages of construction entail different risk levels. An alternative method for setting regulatory targets would be to focus enforcement activity at those stages of a given construction project when safety and health risk exposures are highest. For example, this would be when roofing activities are being undertaken in residential work or steel erection in multistory, commercial building. The OSHA projectlevel data provide some information relating to correlates of technical aspects of construction and the number of violations found during contractor inspections. Using information from the Dodge/CRA system on project start dates and OSHA/IMIS data regarding when inspections were conducted, some evaluation could be done to find correlations between violation rates and different stages of construction. This analysis could be done for end uses consistently associated with higher rates of violations. Using local emphasis programs for targeting A wide range of local emphasis programs in OSHA area offices are attempting to use other means outside of the Dodge/CRA system for identifying safety and health problems in construction (see Current Targeting Practice and Performance, above). The 2003 survey found that area offices using a variety of LEPs have found them a useful supplement and in some cases a substitute for Dodge/CRA. As such, LEPs bear further scrutiny and evaluation (see pg. 7, above). The OSHA Targeting Task Force is cataloging local emphasis programs in use. OSHA should consider expanding successful LEP approaches to other areas and regions as part of a larger evaluation of new methods of undertaking construction enforcement targeting. Implementing the targeting options The three potential alternative methods for targeting inspections, with some additional analytic work, could be incorporated into the existing system of inspection targeting. Other targeting alternatives focusing on homebuilding and on particular phases of construction projects could be developed in the mediumtolong term. In all cases, these alternative forms of targeting could be tested against the existing system as part of a systematic evaluation process. Designing a Pilot Study to Evaluate Targeting Alternatives Rationale for pilot studies The desirability of alternative methods for targeting construction inspections rests on their ability to improve safety and health at construction work sites relative to the present system. At issue is, How might OSHA design pilot studies that could assess the performance of different methods of targeting programmed OSHA inspections? Classic experimental design Experimental designs are most commonly used in medical research where they are used to evaluate the performance of new drugs, treatments, and other interventions on health outcomes (for instance, Cook and Campbell 1979; Boniface 1995; Selwyn 1996). The classic experimental design is a doubleblind clinical trial. In this design, the effects of a treatment (for instance, a new pharmaceutical product) are assessed by testing it on a sample of patients, representative of the population that would ultimately use the drug. In the sample, patients are randomly assigned to a treatment group or a control group. The treatment group receives the drug during the experiment, while the control group receives a placebo that looks identical to the treatment but has neutral medicinal value (such as a sugar pill). Random assignment to control and treatment groups is critical so there is no interaction between who receives the treatment and who does not that might influence the health outcome. 29. Even in medical research, conducting true doubleblind clinical trials has proven difficult, partly because of the costs involved and because of ethical controversy regarding random assignment of individuals to control and treatment groups. 30. There are many examples of using randomized experiments to test social programs. Examples include welfaretowork and job subsidy efforts (such as, Dubin and Rivers 1993; Katz, Kling, and Liebman 2001; Bloom, Hill and Riccio 2003), job training (LaLonde 1986; Bloom and others 1997), and educational vouchers (Howell and Peterson 2004). A proposed experimental design for evaluating targeting The focus for pilot studies would be OSHA area offices. Rather than simply piloting new methods of targeting inspections in any area office that chooses to apply them, an experimental design would set up the pilot efforts in such a way that the effects of the new methods could be compared to the existing system. The "treatment" would be the new types of targeting methods described in the previous section, particularly new methods of using the Dodge/CRA system. The "control" group would be area offices using the existing Dodge/CRA targeting system. Selection of area offices The first problem that arises in applying an experimental design structure to pilot efforts involves the choice of area offices. The survey results described earlier indicate that most area offices use the Dodge/CRA system at least as a component of programmed inspection planning. Only about 11% reported not using the system in any way. On the basis of discussions with OSHA staff in area offices and in Washington, D.C., it is evident that gaining the upfront cooperation of area officials and staff would be crucial to any pilot effort; thus there might be a bias in the relative enthusiasm of offices that agree to participate. While the group of survey respondents might not be representative of the population of area offices, they can be thought of as the sample frame for the pilot effort31. For control and treatment groups, one could use the area offices that indicated that they use the Dodge/CRA system in some form, if they were willing to participate in the pilot effort. Treatment groups Dodge/CRA list treatments The treatment groups for the study would fall into two categories. First would be offices that are currently using the Dodge /CRA system and would continue to do so, albeit in a modified form. That is, this group would continue to receive their Dodge lists as under the present system, but would use a targeting method drawing on one or more of the procedures described above. Ideally, area offices would not be able to detect whether selection criteria used in generating their lists had been altered. By masking the method used to generate monthly inspection lists, the pilot would more closely conform to the "doubleblind" structure of experimental design. 31. It is likely that area offices that responded to the Construction Targeting Task Force survey were more interested in the targeting issue (as shown by their willingness to fill out the survey) and therefore more likely to use the existing system. Depending on the number of area offices and the length of the pilot efforts, the study could use a number of combinations of targeting criteria (discussed earlier).
The other type of treatment would use methods for selecting projects outside of the present Dodge/CRA system. One set of treatments are some local emphasis programs already under way that target on the basis of specific sectors (such as, residential), contractors, or types of problems (for instance, falls). The LEPs chosen for the pilot evaluation might be established rather than newly instigated LEPs in participating area offices. By using LEPs that have already been in place for some time, the pilot evaluation would capture effects related to the protocol rather than startup or learningcurve effects. 32. As noted, targeting based on project phase is the most speculative form of targeting at this time and would require further evaluation of both the relationship between worker risks and construction phase by project type as well as refinement of CRA estimation methods of project completion. As a result, this type of targeting effort might not be tractable in the near term, but could be tested at some future time using similar procedures. Treatment procedure The area offices chosen as treatment sites would follow existing Dodge/CRA or LEP protocols throughout the pilot period. These offices and those in the control groups would not use their own deletion criteria, such as projects below a given dollar value, during the pilot study. Following the existing protocols would ensure a common basis of comparison and once again mask for those using Dodge/CRA lists their status as either a treatment or control group area office participant. The control groups for the pilot effort would be some area offices that have already been using the Dodge/CRA system for programmed targeting. These offices would continue to receive periodic Dodge lists, although they would not know whether those lists had been altered by some of the new targeting methods and thus whether they were control or experimental groups. They would therefore continue to conduct programmed inspections as in the past. Outcome measures Given OSHA's purpose to improve workplace safety and health it would seem that injury/illness rates should be used as an outcome measure for the pilot study. Changes in injury and illness incidence could be charted for the projects inspected, and the effects of different targeting methods on injury and illness incidence charted. Evaluating program success Using outcome measures from the inspectionbased surveys, OSHA could calculate the average incidence of violations of key construction standards on projects in area offices in the various treatment groups and in the control area offices. As noted, incidence levels would be measured at the beginning and the end of the trial period for participating offices. Program effect would be measured as the change in incidence in treatment groups relative to change in incidence among controls. 33. This is related to the approach taken to measuring the impact of construction inspections retrospectively in my prior studies of OSHA enforcement (Weil 2000, 2001). 34. Contractors found in violations of the Fair Labor Standards Act during these inspections are still subjected to penalties and additional followup by Wage and Hour. However, the inspectionbased surveys are administered and tracked separately from the main enforcement program (Wage and Hour Division 2001). In those cases where the treatment consists of inspections focused on particular sectors (such as, residential), treatment and controlgroup measures would be taken for a representative sample of those types of construction projects only. Nonetheless, crossproject comparisons could be made based on percentage rather than absolute reductions in incidence over the study period. By evaluating performance at the area office or regional level (rather than only at sites that have been inspected) the trial would be measuring the direct and indirect effects of OSHA on injury and illness risk exposure. This is important because a welldesigned inspection targeting system should provide incentives to construction contractors and project coordinators to not only reduce safety and health risks retrospectively (via direct inspection impacts) but more importantly prospectively (through deterrent effects). The results of the proposed study would need to be evaluated carefully in terms of their application to nonparticipating area offices, particularly those that have resisted use of the present Dodge system. The pilot design described above uses the subset of OSHA offices currently participating in a Dodge or LEP program for targeting. Although the results might be applicable to other offices that have shown such an interest, OSHA would need to understand that study results might be less applicable to offices that are unwilling to use even the existing system. The agency would need to overcome institutional resistance and barriers that go beyond the scope of an experimental pilot program. Costs and benefits of the pilot experiment approach An experimental approach to pilot targeting programs can reap benefits. Through careful assignment of area offices to treatment and control groups, the effects of different forms of targeting on outcomes can be discerned from other factors that might influence safety and health outcomes, but are unrelated to targeting. The experimental approach would therefore provide much more clear guidance on which targeting methods are most desirable on a national level than would be available from simply piloting different methods without an overarching experimental design. Given the prevalence of severe injuries and deaths in the construction sector, improving safety and health in the construction industry must be an important component of future efforts to address workplace conditions in the United States. OSHA policy is only part of the overall system that produces safety and health outcomes. Yet this study argues that, for its part, OSHA could contribute more by addressing its procedures for selecting work sites and targeting its limited resources. Bartel, Anne and L. Thomas. 1985. Direct and Indirect Effects of Regulation: A New Look at OSHA's Impact. Journal of Law and Economics, 28:125. Bloom, Howard, Carolyn Hill, and James Riccio. 2003. Linking Program Implementation and Effectiveness: Lessons from a Pooled Sample of WelfaretoWork Experiments. Journal of Policy Analysis and Management. 22(4): 55175. Bloom, Howard, Larry L. Orr, Stephen H. Bell, George Cave, Fred Doolittle, Winston Lin, Johannes Bos. 1997. The Benefits and Cost of JTPA Title IIA Programs. Journal of Human Resources, 32(3): 54976. Boniface, David. 1995. Experiment Design and Statistical Methods for Behavioral and Social Research. London: Chapman & Hall. Bureau of Labor Statistics. 2004a. Fatal Occupational Injuries by Industry and Event or Exposure, 2001. Census of Fatal Occupational Injuries, reported at www.osha.gov. . 2004b. Occupational Illness and Injury Rates by Industry, United States, reported at www.bls.gov. Cook, Thomas and Donald Campbell. 1979. QuasiExperimentation: Design and Analysis Issues for Field Settings. Boston, MA: HoughtonMifflin. Coulton, Kent. 2003. Housing in the Twentyfirst Century: Achieving Common Ground. Cambridge, MA: Wertheim Publication Series/Harvard University Press. Dubin, Jeffrey and Douglas Rivers. 1993. Experimental Estimates of the Impact of Wages Subsidies. Journal of Econometrics, 56(2): 21942. Dunlop, John T. 1961. The Industrial Relations System in Construction. In: Arnold Weber, ed., The Structure of Collective Bargaining. Chicago, IL: University of Chicago Press. GAO, U.S. General Accounting Office. 1989. Tax Administration: Information Returns Can be Used to Identify Employers Who Misclassify Workers. GAO/GGD89107. . 1996. Tax Administration: Issues in Classifying Workers as Employees or Independent Contractors. GAO/TGGD96130. . 2002. Workplace Safety and Health: OSHA Can Strengthen Enforcement Through Improved Program Management. GAO0345. Gray, Wayne and John Mendeloff. 2001. The Changing Effects of OSHA Inspections Over Time: A Review of Three Data Sets from 1979 to 1998. Manuscript, University of Pittsburgh, August. Howell, William O., and Paul Peterson. Uses of Theory in Randomized Field Trials. American Behavioral Scientist 47(5): 699717. Joint Center for Housing Studies. 2002. The State of the Nation's Housing, 2002. Cambridge, MA: Joint Center for Housing Studies of Harvard University. Jones, Carol, and Wayne Gray. 1991. Longitudinal Patterns of Compliance with Occupational Safety and Health Administration Health and Safety Regulations in the Manufacturing Sector. The Journal of Human Resources, 36: 62353. Katz, Lawrence, Jeffrey Kling, and Jeffrey Liebman. 2001. Moving to Opportunity in Boston: Early Results of a Randomized Mobility Experiment. Quarterly Journal of Economics 116(2): 60754. LaLonde, R. 1986. Evaluating the Econometric Evaluation of Training Programs with Experimental Data. American Economic Review 76(3): 60420. Meridian Research. 1992. Review of the Coverage Issues Related to the Universe Field Used by OSHA for Construction Targeting. Washington, DC: Report to OSHA, Office of Statistics. Morantz, Alison. 2001. Assessing the Impact of Regulatory Devolution: The Case of OSHA Enforcement. Manuscript, Harvard University. Nelson, Nancy, Joel Kaufman, John Kalat, and Barbara Silverstein. 1997. Falls in Construction: Injury Rates for OSHAInspected Employers Before and After Citation for Violating the Washington State Fall Protection Standard. American Journal of Industrial Medicine, 31: 296302. OSHA, Occupational Safety and Health Administration, U.S. Department of Labor. 1993. The 100 Most Frequently Cited OSHA Constructions Standards in 1991: A Guide for the Abatement of the Top 25 Associated Physical Hazards. Washington, D.C. Perrow, Charles. 1999. Normal accidents : living with highrisk technologies : with a new afterword and a postscript on the Y2K problem. Princeton, NJ: Princeton University Press. Planmatics. 2000. Independent Contractors: Prevalence and Implications for Unemployment Insurance Programs. Report for US Department of Labor, Employment and Training Administration. Ringen, Knut. 1999. A Critical Review and Recommendations. Report prepared for the Directorate of Construction, Occupational Safety and Health Administration. Roth, Rachel. 2003. "Consolidation in the Professional Distribution of Residential Building Products." Working Paper, Joint Center for Housing Studies/HCTAR, July. Ruser, John. 1995. SelfCorrection versus Persistence of Establishment Injury Rates. Journal of Risk and Insurance, 62: 6793. Ruser, John and Robert Smith. 1988. "The Effect of OSHA RecordsCheck Inspections on Reported Occupational Injuries in Manufacturing Establishments. Journal of Risk and Uncertainty. 1: 41535. Scholz, John and Wayne Gray. 1990. OSHA Enforcement and Workplace Injuries: A Behavioral Approach to Risk Assessment. Journal of Risk and Uncertainty, 3: 283305. Schriver, William. 1997. Estimating Construction Duration. Draft Final Report for the Office of Statistics, Occupational Safety and Health Administration, J9F50018. . 2003. An Analysis of the Relationship between Construction Project Characteristics and Rate of Contractor Violations Cited during Administrative Inspections. Report prepared for the Office of Statistics, Occupational Safety and Health Administration, DOL. Selwyn, Murray. 1996. Principles of experimental design for the life sciences. Boca Raton, FL : CRC Press. Stanley, Marcus. 2000. Testing Cooperative Regulation: A Case Study of the Maine 200 Program. Manuscript, Harvard University, July. Thomason, Terry, Timothy P. Schmidle, and John F. Burton, Jr. 2001. Workers' Compensation: Benefits, Costs, and Safety under Alternative Insurance Arrangements, Kalamazoo, MI: Upjohn Institute. U.S. Department of Labor, Office of the Inspector General. 1987. Special Review of OSHA Enforcement Activities. Report no. 02602810105. Wage and Hour Division, U.S. Department of Labor. 2001. 19992000 Report on Initiatives. Washington, D.C. Weil, David. 1992. "Building Safety: The Role of Construction Unions in the Enforcement of OSHA. Journal of Labor Research, 13(2): 12132. . 1996. If OSHA Is So Bad, Why Is Compliance So Good? The RAND Journal of Economics, 27(3): 61840. . 2000. OSHA Enforcement and Regulatory Compliance in the U.S. Construction Industry. Proceedings of the 52nd Annual Meetings of the Industrial Relations Research Association, 8393. . 2001. Assessing OSHA Performance: New Evidence from the Construction Industry. Journal of Policy Analysis and Management, 20(4): 65174. . 2003. Building Comprehensive Market Recovery Strategies for the Construction Industry. Working USA, 7(1): 2648.
N=9,312 projects.
N=9,312 projects.
Source: Analysis by the author of OSHA IMIS data at the project level; based on data provided by William Schriver, June 2003. Table 4: Major project types in Dodge/OSHA IMIS combined sample, FY19992001
Source: Analysis by the author of OSHA IMIS data at the project level; based on data
provided by William Schriver, June 2003.
N=9,312 projects
N=9,312 projects. |