|
Methodological Challenge
Using Qualitative Research Methods
One important methodological option in conducting
management research is the use of qualitative methods for data
collection and analysis. Qualitative research, with its emphasis
on understanding complex, interrelated and/or changing phenomena,
is particularly relevant to the challenges of conducting management
research. Qualitative methods combined with quantitative ones can
provide particularly rich and robust inquiries. Either alone or in
combination, qualitative research must be conducted with methodological
rigor.
This section does not attempt to provide a primer
on qualitative methods. The role, benefits and appropriate use of
qualitative research have been discussed extensively in the literature.
Several references to excellent articles can be found in the references
section and links to references.
Our more limited aims here are:
- To offer, for those who are not familiar with qualitative methods,
a brief overview of how they are used and what value they offer,
drawing heavily from articles by Shoshanna
Sofaer. 6
- To propose the use of qualitative methods in hypothesis testing.
Qualitative methods are often used inductively, for exploration,
theory building and description. Less attention has been given to
their use in deductive hypothesis testing. The white paper,
prepared by Brian Mittman 7
discussed in this section explores those potential uses.
Specially this section addresses four questions:
What are the uses and value of qualitative research?
Qualitative research is characterized by an emphasis on describing,
understanding, and explaining complex phenomena - on studying, for
example, the relationships, patterns and configurations among factors;
or the context in which activities occur. The focus is on understanding
the full multi-dimensional, dynamic picture of the subject of study.
Its approaches contrast with quantitative methods that aim to divide
phenomena into manageable, clearly defined pieces, or variables.
Quantification is good for separating phenomena into distinct and
workable elements of a well-defined conceptual framework.
But when we focus research on what we already know how to quantify,
(e.g., what can be reliably quantified), we may miss factors that are
key to a real understanding of the phenomena being studied. The
downside of quantification is that it does not always support
(as well as qualitative methods) understanding of complex,
dynamic, and multi-dimensional wholes.
Qualitative methods are useful, not only in providing rich descriptions
of complex phenomena, but in constructing or developing theories or
conceptual frameworks, and in generating hypotheses to explain those
phenomena.
What are the methodological challenges in qualitative research techniques?
Key challenges to conducting rigorous qualitative research
range from instrument development through data collection to data analysis.
In addition, results need to be documented and reported using formal accepted
methods.
For example, typical deficiencies are unfocused
instrument development and lack of supporting theory. Rigor related
to instrument protocol development requires attention to validity,
intrusiveness (the Hawthorne effect) and triangulation. In addition,
attention must be paid to distinguishing between collecting subjective
and objective data, information on the formal vs. the informal organizational
structures and processes and the differences between collecting facts vs.
opinions vs. interpretations.
Planned, systematic, comprehensive data collection
requires variable definitions and measures, document coding form
protocols, administrative database specifications and survey instrument
question libraries. In the data collection phase, problems can be minimized
through pilot-testing and pretesting, validity/quality checks, triangulation
and monitored flexibility. Sole reliance on subjective data, self-reports,
etc. can reduce validity. Some tips to insure rigor in data collection
management include training of all data collection staff and conducting
immediate post-collection coding for time/memory sensitive data. Other
methods to ensure the validity of data include tape recording interviews,
performing real time data entry and editing, using paired interviewers,
and implementing quality assurance fo each instrument. And, to avoid
further problems, incomplete, missing or unusable data should be corrected
immediately.
Pitfalls related to data analysis include using ad hoc,
emergent, exploratory, informal analyses that may lead to inappropriate
conclusions and unpublishable results. Rigorous analysis requires an a
priori theoretical model and hypothesis, a formal framework guiding data
collection and analysis and adherence to the formal framework and research
best practices.
Finally, reporting requires results structured
by hypotheses and an analysis plan. Reports need to include data
syntheses and summaries with a focused analysis of the data.
Conclusions must have a documented basis and systematic formal
analysis methods, and validity must be documented.
What are some key qualitative research methods?
A wide range of tested qualitative research
methods are available to address these challenges. The selection
of method, or combination of methods, will be tailored to the
questions being studied and the setting for research. Typical
methods include:
Naturalistic inquiry, or ethnography,
has its roots in anthropology and sociology and involves long-term
exposure to a setting or a group of people. Extensive use of unstructured
observations and conversations documented by detailed field notes form the
basis for this type of research, often considered the purest form of
qualitative research. Naturalistic inquiry is used when situations
are unique or complex, when the level of uncertainty about the
questions to ask is high and when there is little or no theory to
direct the investigator.
A subset of this type of inquiry involves
participant observation in which the investigator becomes a part of
the setting or the process being studied. (Sofaer) reports that she
was able to learn more from attending a few group meetings in a particular
setting than she could have by using more structured qualitative methods
such as interviews or surveys.
Case studies are the preferred strategy
when 'how' or 'why' questions are being posed, when the investigator
has little control over events, and when the focus is on a contemporary
phenomenon within some real-life context. The case study is especially
appropriate when the boundaries between phenomenon and context are
not clearly evident. The case study copes with the technically
distinctive situation in which there will be many more variables
of interest than data points, and as one result relies on multiple
sources of evidence, with data needing to converge in a triangulating
fashion.
The case study approach can involve a single event
or multiple cases and can be short or long term. However, rather than
requiring total immersion in the setting or culture, sampling of sites,
experiences and/or informants is typical. The methods used in case
study research is similar to those of naturalistic inquiry. However
the data collection is often more structured, using key informant
interviews, structured observations of events and interactions and
the collection and content analysis of relevant documents (e.g., to
help establish the facts, the assumptions, values and priorities,
or to illuminate differences in perceptions). Case studies often
also include quantitative data for background or to help generate
questions to ask informants (e.g., data on demographics, heath
status, utilization, finances, etc.).
Structured Observations of meetings
This involves attending meetings of the group that you wish to research.
This can also be extended to observation of individuals in their daily
work routine or on special tasks. The purpose of observing is to learn
what is going on at the meeting and witness the group dynamic in process.
This can be a rich information source as it can give researchers insight
into the group.
Content analysis of documents
This is a non-intrusive form of research. This involves reviewing
documents, memos or other pieces of written information for content
and themes. By examining written word, the researcher is studying one
type of communication that occurs in the selected sample.
Collection and analysis of other archival,
administrative and performance data
This method also is non-intrusive. Information that has been previously
collected, or secondary data, is reviewed to gain a better understanding
into the topic. This information is part of the organization’s history
and can be a valuable key to understanding the past.
Focus groups usually explore specific issues.
The focus group brings together individuals chosen to meet a specific profile.
They may be homogenous along some dimensions and heterogeneous along others
and a structured, yet informal, setting is used to explore a limited number
of questions. Focus groups, unlike individual interviews, provide the added
dimension of the interactions among members. Focus groups are often combined
with more quantitative approaches such as surveys that can be administered at
different points in the group discussion and even used as grist for additional
discussion.
Cognitive interviews are typically used in survey
development. One-to-one interviews are conducted (with people meeting the
criteria for completing a particular survey) as the individuals complete the
instrument being tested. This method helps investigators understand how people
perceive and interpret language and their own experiences as they refine the
survey instruments.
Mail and telephone surveys are a method of collecting
information by sending surveys via email or postal mail. Participants return
completed forms to the researcher or an outside vendor. Surveys may ask
respondents to rate items on a scale (e.g., Likert scale of 1-5). Some
surveys also allow respondents to write their feelings or attitudes about
a particular event or to elaborate in more detail on an item, or to
express suggestions, etc.
What is the role of qualitative research in
hypothesis testing?
The origins and development of qualitative research
methods and their close association with inductive, interpretive and
historical research have led many researchers to associate these methods
exclusively with these forms of research and to fail to recognize their
value in conventional deductive empirical research.
Some investigators, however, contend that
hypothesis-testing, deductive research can benefit from the use of
qualitative research methods - and that these methods can be used in
a manner consistent with accepted standards of rigor and validity.
In particular they believe that the acknowledged strength and unique
contribution of qualitative methods in developing insights into actors'
values, beliefs, understandings and interpretations of events and other
phenomena, or in explaining historical occurrences, can enhance
"conventional" forms of empirical research.
Brian Mittman, in a white paper prepared for
the MDRC workshop on Management Research in VA, argues for the use
of qualitative methods in hypothesis testing, and outlines the key
components of the rigorous approach needed to use these methods
successfully. His paper is motivated by two interests: first,
convincing researchers not experienced in qualitative methods
that they can enhance their empirical, deductive work, and, second,
minimizing the misuse of qualitative methods in ways that threaten
the validity of studies.
Dr. Mittman's
paper is linked here.
top | next
|
