Methodological Challenges to Management Research
External Review Considerations
The previously mentioned challenges confronting
management research have important implications for efforts to obtain
funding to conduct management research in the VA or elsewhere. In
general, study sections or other peer review panels responsible for
funding decisions are very focused on factors that threaten a
proposed study's internal validity. As such, study sections are
likely to have a decided preference for experimental designs since,
as noted, these are the gold standard. The proposed use of non-experimental
designs, while certainly legitimate and common in social science research,
will inevitably lead to heightened concerns among reviewers about the
internal validity of the proposed study.
Similarly, proposed management research entailing complex variables and
primary data collection will also heighten concerns among study section
members no matter how favorable their orientation to management research.
The proposed study of complex variables such as culture and coordination
raises concerns about possible poor reliability and validity of measures.
Primary data collection raises concerns about the
possibility of low response rates to surveys. In addition, because
management research is conducted in organizational settings, some
logistical issues in conducting a study cannot be fully appreciated
and addressed until the study is actually underway. However, this
also presents an element of uncertainty when the inclination of
study section members is to have all research procedures specified
completely in advance as part of the protocol.
To counter these concerns, study designs for
management research should include:
- Strong conceptual or theoretical frameworks for the questions being studied; and
- A well-specified methodology that anticipates threats to validity
and problems in data collection, and details how they will be addressed.
* The official guide to research proposals
in the VA system can be found at
http://www.research.va.gov/funding/
top | next
|