II. Research Methodology

This section presents the methodology of this research. It is divided into two subsections. The first introduces the contents and format of the research questionnaire and outlines the protocol used in the questionnaire administration process. The second presents the response rate.

The primary method of data collection is a survey of employers who have declined to participate in the mediation program. The field offices distributed the surveys for a two-month period beginning in June and ending in August. We also use a secondary source, internal documents from the EEOC, to collect information regarding the characteristics of the charges.

The Questionnaire

1. Contents and Format

A medium length, multi-part questionnaire was designed to measure the reasons for the lack of employer participation in the EEOC mediation program. It contained both open-ended and close-ended questions. Three of the questions, which measured attitudes and perceptions, used the Likert-scale. This flexible format was used to solicit the most accurate and complete responses from the employer representatives completing the survey (“representatives”).

The research team formulated the questionnaire, which was then sent to the EEOC for feedback. Once the questionnaire was finalized, it was pre-tested to measure its reliability and validity with the respondents and to evaluate the implementation of the data collection methodology based on the feedback of the EEOC coordinators who assisted the team.⁶

The questionnaire consisted of 20 questions designed to elicit information specific to the charge, the employer, and the employer representative. In the succeeding discussions and tables the basis of analysis is either the total number of charges, the employers, or employer representatives, depending on the specific question being analyzed.

These questions are organized into four parts. The first part, which was comprised of 10 questions, asked for basic information about the employer, the employer representative, and the charging party. This was designed to collect data on the background of the employer representatives completing the survey, their prior experience with the EEOC and its programs, and the employer’s prior participation in the EEOC mediation program. Three of the questions dealt with the employer representative’s knowledge of the charging parties and their representatives.

The second part of the questionnaire contains six questions. These questions pertain to the following: (a) internal investigations, if any, that the employer representatives and/or employers undertook to investigate the charge prior to their decision to decline to mediate; (b) their familiarity with the mediation program; (c) their reasons for declining to mediate; and (d) their general perceptions about the EEOC. The third section contains questions for external attorneys and consultants about their experiences and perceptions of factors related to the decision to decline mediation.

The last section contains one close-ended question and two open-ended questions. The first open-ended question invited feedback about whether the EEOC could have done anything differently so that the employer would have mediated the charge. The second solicited general comments regarding the EEOC.

As noted above, our questionnaire captures information specific to the charge, employer, and employer representative. Since the primary emphasis of the survey is WHY employers elected not to participate in the EEOC mediation program for a PARTICULAR CASE, the survey is designed to cover every charge for which the respondent (employer or employer representative) declined mediation. Given the fact that some of the employers are repeat participants, it became pertinent that we develop a shortened version of the questionnaire. As discussed in the next section, the need for a shortened questionnaire was a key improvement derived from our pretest. Hence we developed two versions of the questionnaire: the full version (Questionnaire A) and the short version (Questionnaire B). (These are shown in Appendix A and Appendix B, respectively.) Questionnaire B was a modified survey designed for employers with multiple charges. It omits certain employer and representative questions that were already captured in our database from the employer's initial response to Questionnaire A. The full version was sent to employers, unless they indicated that they had already participated in the survey before.

2. Pretest

We pretested the questionnaire and the protocol. The objectives of the pretest were to test the reliability of the survey instrument (“the questionnaire”) and the effectiveness of the data collection methodology in generating responses. Six EEOC district offices—Charlotte, Chicago, Houston, Chicago, Milwaukee, and Denver—participated in this pretest. In order to examine the effectiveness of the two different data collection methodologies—faxing and mailing—three of the offices were asked to fax the surveys and the other three to mail the surveys. We decided to add the option to receive the completed surveys by fax to “strike while the iron was hot” to maximize feedback.

The pretest was successful. The pretest results and the feedback from the coordinators of the participating district offices provided valuable insights into the conduct of the survey. Based on our analysis of the pretest responses, questions were added, rephrased, or deleted from the questionnaire. Based on the coordinators’ feedback, the protocol was modified to address the concerns of the field coordinators and the respondents. The development and use of the shortened questionnaire (Questionnaire B) was a by-product of the pretest. Some field coordinators relayed the “exasperation” that some employers felt in having to complete more than one questionnaire. In order to maximize our coverage, we opted for a shorter questionnaire to avoid fatigue.

An analysis of the pretest response rates showed that faxing yielded a better response rate than did mailing. This, combined with the suggestions from the field, led to our decision to use fax as our primary survey delivery method. Field coordinators were instructed to offer to immediately fax the questionnaire once the respondent expressed their willingness to participate in the survey. In summary, the successful pretest enabled us to make the modifications that needed to be made to the survey instrument, data collection process, and protocol.

3. The Survey Protocol

All EEOC district offices participated in the survey. A protocol was established to ensure the standardization of data collection (Appendix C). It relied on the field office personnel to immediately identify those employers who declined to participate in mediation and offer their representatives an opportunity to participate in the study. If the representative agreed, the field office coordinators immediately offered to send a questionnaire by fax or mail.

We requested that the field office mediation staff keep a weekly log of their activities regarding the survey distribution. The staff were instructed to record the following information in their weekly logs: the charge numbers of employers who were offered to participate in the study, the type of questionnaire sent (A or B) and the mode of transmission (fax or mail) among employers who agreed to participate in our survey, and a short description of why employers chose to decline the offer of mediation. These logs form the basis of the computation of our response rate.

Response Rate

1. Overall Response Rates

Based on the weekly logs, a total of 2,365 respondents were contacted concerning participation in the survey. Of these, 2,085 agreed to participate in the survey. A total of 185 organizations refused to participate in the survey. Another 95 organizations did not participate due to the coordinators’ inability to contact them, their failure to return calls, or some other actions other than a direct refusal. (We have reason to believe that the number of respondents who were contacted and, more importantly, the number who agreed to participate in the survey are higher than the above figures. This is discussed in detail below.)

Questionnaires were sent to the 2,085 employers who agreed to participate. Of these 2,085 employers, 2016 (97%) were sent Questionnaire A, which was the full survey. Questionnaire B was sent to 69 (3%) employers. Most of the questionnaires were sent by fax. Table I shows the survey delivery breakdown.⁷

Table I
Questionnaire Type and Mode of Transmission
For Those Who Agreed to Participate in the Survey (N = 2,085)

We received a total of 653 questionnaires: 629 Questionnaire A surveys and 24 Questionnaire B surveys. An inspection of the charge numbers reveals that a total of 135 completed questionnaires had charge numbers that were not listed in the weekly logs. For example, of the 629 version A questionnaires we received, only 502 are actually listed in the logs (see Table II.A below).

Table II.A
Number of Questionnaires Received

* Since some are not listed in the logbooks, the exact numbers cannot be determined.

While a few (nine) of the completed questionnaires cannot be traced because they had no charge numbers, a majority of these charges were not recorded in the weekly logs. This implies that the number of employers who agreed to participate in our survey is greater than 2,085.

This has bearing on the computation of the response rates. The response rate is the ratio between the number of questionnaires received and the number of questionnaires sent out. In the case of the full-version questionnaires, the proportion of questionnaires received to the number sent as recorded in the weekly logs (629 divided by 2,016) overstates the actual response rate since we suspect that the actual number of questionnaires sent out is greater than 2,016. Furthermore, because information regarding how many were actually sent by fax or mail is incomplete, we are unable to compare the difference between faxing and mailing.

The approach we took is to base the computation of the response rates on charges that are listed in the weekly logs. Additionally, we included only those that also appear in the EEOC database: Of the 2,085 listed in the weekly logs, only 2,034 have matching entries in the EEOC database.⁸ We attribute this discrepancy to recording errors, e.g., illegible handwriting and incomplete charge numbers. Thus, the response rates were computed based on charges listed in the weekly logs and appearing in the EEOC database. These response rates are based on a subset of the actual number of questionnaires sent out—a subset of verifiable charge numbers as they appear in the weekly logs and the database.

The resulting numbers and response rates are shown below. The results show that faxing results in a higher response rate than mailing.

Table II.B
Overall Response Rates

As mentioned earlier, we received 629 Type A questionnaires (the complete or full-fledged version) and 24 Type B questionnaires (short version). Since the number of Type B questionnaires is relatively small relative to the number of Type A questionnaires, we base our analysis on the 629 completed questionnaires. Using just one data set (629) makes the presentation of results simple and streamlined. We have ascertained that the exclusion of the 24 questionnaires will not significantly affect our analysis and resulting conclusions.

3. Field Office Participation

Table III presents information about the EEOC field office participation. We identify the total number of surveys received from each office and the respective percentage of the overall data.

Table III
EEOC Field Office Participation

In survey research one of the key questions that needs to be answered is the “representativeness of the sample.” In other words, we need to ascertain that the sample is representative of the whole population. In this study, this consists of charges that came before the EEOC encompassing the same fiscal year (2003) when this study was conducted. To do this, we present a profile of our employer respondents with respect to certain characteristics and the profile of all the charges filed from October 1, 2002 to August 15, 2003 (the period corresponds to the start of the current fiscal year and the end of the survey period).

First, we present a comparison between the distribution of the survey charges and all EEOC charges with respect to the originating district or field office. This is shown in Table IV below. While the distribution of the survey respondents does not closely mirror that of the entire population, we believe that this discrepancy will not materially affect the validity of our conclusions. As discussed in the succeeding discussions (Sections III.A and III.C), we have strong indication that our sample is representative of the population with respect to employer and charge characteristics. Furthermore, since the study is primarily concerned with the nationwide evaluation of the program, the most important consideration in this regard is that all district offices are represented in our survey, albeit the distribution does not perfectly mirror the district-by-district distribution.

Table IV
Comparison of Field Office Participation

Previous | Start | Next

This page was last modified on December 2, 2003.

Return to Home Page