|
Converging evidence indicates that researchers should not solely rely on self-report but rather supplement it with gold standard objective measures. However, if self-reports are the only practical option, one should utilize strategies that we know will improve validity. In the past two decades, a multitude of attempts have been made to improve the accuracy of self-reports (Huber & Power, 1985 xClose
Huber, G.P., & Power, D.J. (1985). Retrospective reports of strategic level managers: Guidelines for increasing accuracy. Strategic Management Journal, 6, 171-180.; Loftus, Fienberg, & Tanur, 1985 xClose
Loftus, E.F., Fienberg, S.E., & Tanur, J.M. (1985). Cognitive psychology meets the national survey. American Psychologist, 40(2), 175-180.). In this section, we offer some practical considerations for using self-report methods for cancer-related variables. In selecting measures, one should explore the extant literature on the accuracy of self-report for the variables of interest, determine whether valid and reliable measures exist for the study population, and whether they are content valid for the assessed constructs. Researchers should also consider the use of retrieval aids (e.g., timeliness, landmarks; Loftus & Marburger, 1983 xClose
Loftus, E.F., & Marburger, W. (1983). Since the eruption of Mt. St. Helens, has anyone beaten you up? Improving the accuracy of retrospective reports with landmark events. Memory and Cognition, 11(2), 114-120.; Smith, Jobe, & Mingay, 1991 xClose
Smith, A.F., Jobe, J.B., & Mingay, D.J. (1991). Questions induced cognitive biases in reports of dietary intake in college men and women. Health Psychology, 10(4), 244-257.), increased response time for participants (Hammersley, 1994 xClose
Hammersley, R. (1994). A digest of memory phenomena for addiction research. Addiction, 89(3), 283-294.), bounded-recall interviews (Babor et al., 1990 xClose
Babor, T.F., Brown, J., & Del Boca, F.K. (1990). Validity of self-reports in applied research on addictive behaviors: Fact or fiction? Behavioral Assessment, 12, 5-32.), and motivational boosts (Baker & Brandon, 1990 xClose
Baker, T.B., & Brandon, T.H. (1990). Validity of self-reports in basic research. Behavioral Assessment, 12, 33-51.). Finally, convergent evidence can be obtained from others (e.g., spouse, physician, caregiver) who have access to the domain of assessment.
Alternatives to retrospective self-reports (time based assessments)
Experience sampling methodology (ESM) & ecological momentary assessment (EMA). An alternative to dealing with the associated problems of retrospective self-reports is to reduce the interval of recalled assessments to more immediate or momentary judgments. In a typical ESM or EMA study, participants are asked to carry a data entry device, such as a Palmtop computer, with a built in alarm that signals them to record their current location, activity, and feelings multiple times throughout the day.
EMA can improve the accuracy of self-reports for several reasons (see also Smyth & Stone, 2003 xClose
Smyth, J., & Stone, A. (2003). Ecological momentary assessment research in behavioral medicine. Journal of Happiness Studies, 4, 35-52.). First, it enables assessment in participant’s naturalistic life settings, which rules out the possibility that findings are an artifact unique to unnatural experimental settings. Second, the reports of EMA are momentary and not reconstructive. Third, depending on the nature of the research question, the timing of assessment can be random or event contingent. In addition to avoiding recall biases, EMA allows for detailed analysis of within-subject effects. EMA data are also suitable for examining complex diurnal cycles and interactions or between-effects of environmental and psychosocial factors.
EMA is not a perfect assessment method, however. Given the intense nature of the assessments, the response burden is higher than typical for both researchers and participants. Although a preliminary study has reported otherwise (Cruise, Broderick, Porter, Kaell, & Stone, 1996 xClose
Cruise, C.E., Broderick, J., Porter, L., Kaell, A.T., & Stone, A.A. (1996). Reactive effects of diary self-assessment in chronic pain patients. Pain, 67, 253-258.), the possibility that intensive monitoring can affect daily experiences (i.e., reactivity) needs further scrutiny. Special equipment and knowledge are required to develop and conduct the study, and analysis of the overwhelming number of multivariate data entries may seem initially overwhelming (see Schwartz & Stone, 1997 xClose
Schwartz, J.E., & Stone, A.A. (1997). Data analysis for EMA studies. Health Psychology, 17, 6-16.). Furthermore, EMA is appropriate for more frequently occurring events or experiences but is less well suited for collecting self-report information about uncommon events. Also, given the technical and logistic requirements of its use, EMA is difficult and costly to implement for use in very large samples (e.g., National surveys).
Day reconstruction method (DRM). DRM combines a time-use study with a technique for recovering affective experiences (Kahneman, Krueger, Schkade, Schwarz, & Stone, 2004 xClose
Kahneman, D., Krueger, A.B., Schkade, D., Schwarz, N., & Stone, A.A. (2004). A survey method for characterizing daily life experience: The Day Reconstruction Method (DRM). Science, 306(3), 1776-1780.). DRM asks participants to revive memories of the previous day by constructing a diary consisting of a sequence of episodes. They are then asked to describe each episode by answering questions about the situation and about the feelings that they experienced (comparable to the experience sampling method). This process provides an accurate picture of the experience associated with activities and circumstances. Evoking the context of the previous day is intended to elicit specific and recent memories, thereby reducing errors and biases of recall (Robinson & Clore, 2002 xClose
Robinson M.D., & Clore, G.L. (2002). Belief and feeling: Evidence for an accessibility model of emotional self-report. Psychological Bulletin, 128, 934-960.). Although DRM is still being tested, it has been shown to reproduce the information collected by ESM with less response burden and no disruption to daily activities. Additionally, DRM also provides information about the duration of each experience.
Cancer researchers should consider these two approaches as ways to improve the accuracy of information gained from self-reports. EMA is ideal for the study of rapidly changing states and frequently occurring events, whereas DRM is more suited for studies of extremely rare events, when the duration of a particular event is in question, and/or studies involving very large numbers of respondents. Thus, in addition to continued development of assessment strategies, it is important to clarify which strategies would best serve to fit the focus and the emphasis of the study at hand.
|
