The Impact of Instructions on Survey Translation: An Experimental Study
Brian Kleiner, Jerelyn Bouic, Yuling Pan
KEY WORDS: Survey translation, translation quality, translation instructions
ABSTRACT
In calling for faithfulness to intended meaning as a guiding principle of survey translation, Harkness and Schoua-Glusberg (1998) argue further for the need to provide translators with documentation so that they can better understand the intended readings and research aims of survey items. The authors also point out that translators should be given guidelines regarding an acceptable degree of adaptation. In actuality, however, translators are rarely provided with such documentation and guidelines, and while doing so seems reasonable in theory, there is currently little empirical work that lends support to the utility of this practice. The experimental study described in this paper examined the impact of providing special instructions and supporting material to translators. Specifically, it addressed whether translators provided with explanatory text and guidelines were able to produce translations that were more faithful to the intended meaning of source survey items and more culturally appropriate and natural sounding compared to translators who received no such guidance.
To assess the impact of different types of instructions, 27 professional translators translated an English source instrument into three target languages—Mandarin Chinese, Spanish, and Canadian French (9 translators per language)—following one of three sets of instructions (translators were randomly assigned to three instructional subgroups). The first subgroup received a core set of instructions. The second and third subgroups received the core instructions as well as question by question explanations of the intended meaning of source items, and were told to translate in a way that was faithful to the intended meaning of the source items. The third subgroup received an additional instruction to translate in a natural sounding way with respect to the cultural and linguistic norms of the target population.
Once the translations were completed, 15 professional survey researchers (5 per language) who were native speakers of the target languages conducted blind evaluations, with each examining three translated versions of survey items (that followed the three sets of instructions). The evaluation involved rating each of the translated survey items on Likert scales along three dimensions—overall quality, faithfulness to intended meaning, and cultural appropriateness. Telephone interviews were subsequently conducted with the evaluators to better understand their beliefs and practices regarding survey translation and how they approached the evaluation task.
Study findings indicate that while the provision of special instructions and documentation to translators had a significant impact on their translations, the direction of the impact (positive or negative) differed across the target languages, according to ratings of the survey researchers. We attribute these differences to the beliefs of the survey researchers and their level of commitment to two conflicting general types of equivalence in survey translation: equivalence of stimulus and equivalence of effect. Our research suggests that the issue of providing translators with documentation and guidelines is more complex than assumed, and that researchers should consider carefully in advance whether this is necessary, given the nature of the survey as well as their own beliefs about adaptation and the primacy of equivalence of stimulus or equivalence of effect.
CITATION:
