The U.S. Equal Employment Opportunity Commission

V. Research Methodology

A. Sampling Plan and Response Rate

The purpose of this study is to report the survey results of participant satisfaction with the EEOC's mediation program. These participants are the charging party, an employee or ex-employee, and the respondent, the employer or ex-employer. The population for this research includes all participants in the EEOC mediation process that was conducted under the supervision of the 50 EEOC field offices from approximately March 1 to July 31, 2000. The comprehensive survey design means that sampling techniques were not used.

The researchers surveyed all EEOC district offices. On or about March 1, 2000, all field offices were instructed to provide the survey to all mediation participants for completion and to return these completed surveys in a sealed envelope to the researchers. The number of surveys received from each EEOC district office is set forth in Appendix A.1.

The response rate for these field offices for charging parties was at least 46.25%. The authors received a total of 2,209 surveys completed by the charging parties. Of these, 526 responses were not used in this study (while some were not filled out or only partially completed, most rejected surveys either did not indicate the case charge number or did not have a matching case number in the EEOC database) resulting in a total of 1,683 usable surveys. Thus, the effective participation rate was at least 35%.

The response rate for respondents was approximately 50%. Of the 2402 surveys received from the respondents, only 1,572 were in a usable state resulting in an effective participation rate of 33% by the respondents. The combined effective participation rate of the total sample was approximately 34%. However, our sample is representative of the overall population, as demonstrated in the "profile section" later.

The reason that we state our effective participation and response rate as "at least" figures is that the EEOC district offices did not distribute the survey to participants exactly on March 1. While we derived our figures by looking at all the EEOC mediations held in all 50 field offices between March 1 and July 31, it is clear that not all mediations were surveyed due to this staggered rollout. Also, we have not included those surveys (for mediations held by July 31) which we received after our July 31 cutoff date.

Regarding these surveys that were not used, a review of the responses indicates that they are, in all ways, similar to the results that we report below. Also, our reported response rate is reduced due to our use of numerous cross-tabulations. Finally, where parties at the site of the mediation declined to fill out surveys, a "nonresponse log" was kept. Reviews of these nonresponse logs indicate that there is no probative evidence that the reasons for nonresponse would affect our reported results.

B. Data Collection Procedures

Data collection was performed through a survey. To maximize feedback, the parties were asked to fill out the survey at the conclusion of the mediation. The surveys were given to the mediators, for distribution at the conclusion of the mediation session. A protocol addressing the handling of the surveys was carefully constructed. This protocol was distributed to all district office ADR coordinators and to all mediators in advance. According to the protocol, mediators were asked to inform the parties at the beginning of the session about the survey and to distribute the survey to the parties at the conclusion of the session; then, they had to leave the room while the surveys were being completed. Whenever possible, the participants were separated, allowing them to complete the surveys in two different rooms. After the participants received the survey, they were asked to complete it, place it in an envelope, and seal it to ensure confidentiality. Mediators were then asked to forward the surveys in the sealed envelopes to the local ADR coordinator along with the other required documentation about the mediation. The local ADR coordinators were asked to mail the surveys once a week directly to the research team. As discussed earlier, the mediators or field office ADR coordinators were also asked to indicate when participants did not fill out the survey. This was done to allow the researchers to measure the non-response bias. The complete protocol can be found in Exhibit 2.

C. The Participant Satisfaction Survey

A survey was developed to measure the satisfaction of the parties with the EEOC mediation process. This survey was based on the prior dispute resolution system research performed by Dr. McDermott (one of the authors), feedback from the EEOC National ADR coordinator Steve Ichniowski, and the literature review of the field of ADR program evaluation (section II of this report). Care was taken to ensure that we measured the EEOC's goals (where applicable) with regard to the fairness of its mediation program, which were to provide adequate information about the process, the opportunity for assistance, knowing and voluntary participation, neutrality, confidentiality, and enforceability.

The survey is a 22-item survey that included 14 five-point Likert-type responses ranging from strongly disagree (1) to strongly agree (5). A Likert scale was used because there is evidence that it is superior to other measurement formats in measuring attitudes, and its underlying factorial structure is more stable across situations and cultures.<117> Of the remaining eight questions, five were primarily "yes or no" questions, two were open-ended questions, and one was a multiple-choice question. One open-ended question gave participants who did not resolve their claim a chance to explain why they thought there was no resolution of their charges. The second open-ended question gave participants the opportunity to offer suggestions for improving the mediation process. The multiple-choice question sought to identify the mediation status of the participants. A pretest established that, for most participants, the survey took about 5 minutes to complete. A copy of the survey can be found in Exhibit 3.<118>

Since the survey is an original measure, there are no prior reliability and validity figures. It has construct validity since its items measured the various elements of participant satisfaction with the mediation process. The survey was carefully constructed to ensure that the questions were properly worded. Mr. Steve Ichniowski, National ADR Coordinator and an ex-EEOC administrative judge, Dr. Pat McDermott, an employment law litigator, and Dr. Mollie Bowers, a professional neutral and member of the National Academy of Arbitrators, used their professional expertise to ensure the proper wording of these questions. We also used the results of the pilot study to modify questions.

D. Pilot Study

Researchers recommend pilot testing the survey to ensure that the questions are well understood by the target population.<119> In order to validate the survey, the following was done: (1) a panel of experts validated the survey. The panel members included not only the research team, but also persons familiar with the EEOC mediation process;

(2) the survey was pretested on the mediation participants to determine the time required to complete the survey, whether the participants understood the questions, and whether the questions elicited the information for which they were designed. Also, the authors used the pretest to ensure that the questions did not contain any biases or other errors. Based on the feedback received from the respondents of the pilot study, modifications were made to the survey in order to improve its construct validity.

E. Data Analytical Techniques

The data was analyzed using Microsoft Excel and SPSS. The data analytical techniques of this study were primarily summary statistics, such as frequencies, percentages, and averages. We also used ANOVA and Chi-Square Tests of Independence to investigate whether participant responses varied according to variables, such as the characteristics of the mediation sessions and the nature of the charges. The appropriateness of these tests as the data analytical techniques comes from the nature of the study, which measured participant satisfaction with the mediation program.


previous | contents | next

This page was last modified on October 2, 2000.

Home Return to Home Page