ACF Healthy Marriage - Measuring Program and Marital Satisfaction Among Hispanic Couples

Measuring Program and Marital Satisfaction Among Hispanic Couples

Hispanic family researchers and national marriage researchers convened at national forums to address
the challenges of assessments utilized to measure aspects relevant in parenting, family, relationship
satisfaction, and marital quality in Hispanic families. The recommendations summarized in this memo
are drawn from discussions at a research meeting held in conjunction with the Hispanic Healthy
Marriage National Conference, May 11, 2006, in San Antonio, Texas, and a research meeting held in
conjunction with the Center for Latino Family Research on September 15-16, 2006, in St. Louis,
Missouri.

Given the wide scope related to outcome measures, this memo will not be comprehensive and will only
summarize the discussion of the main themes in relation to Hispanic marriage and families as raised by
researchers attending the forums.

Introduction
Public policy and social programs providing marriage enrichment services are relatively new fields.
Historically, the dynamics relevant in marriage and relationships have fallen outside the scope of
government programs and social services. Instead, couples and families have privately struggled with
internal and external stressors impacting their relationships. Friends, family, marital and family
therapists, clergy, and religious institutions have been the only sources of support for couples who turn
towards the community for direction. Many couples, however, choose to struggle in isolation and turn
away from traditional sources of support because of stigma, lack of knowledge of services available,
financial hardship, insurance limitations, transportation, and resistance towards opening up to others.

Evidence of the negative repercussions of coping with stressors in isolation is observed in social science
research reporting that 50% of marriages in the United States end in divorce. In contrast to traditional
reasons for divorce – e.g., nonsupport, abuse, or alcoholism – couples today are more likely to say they
are separating because of communication problems, a lack of fulfillment, or incompatibility (Sigelman
and Schaffer, 1991). The effects of divorce on children including increased aggressiveness,
noncompliance, acting-out behaviors, disruptions in peer and other interpersonal relationships, academic
difficulties, and poverty have increased awareness of the importance of family structure and attitudes
about marriage and relationships (Hetherington and Clingempeel, 1992). As research reveals the cost
generated as a consequence of disintegrated families, marriage and couple relationships are now viewed
as social constructs that influence social issues such as child well-being and poverty.

As government and social service providers implement programs designed to help individuals who
choose marriage for themselves form, sustain, and maintain healthy marriages, researchers are
challenged to adequately measure program and individual outcomes. Developing and implementing
valid and reliable outcome measures of marriage and family formation dynamics is a complex process
that requires a multifaceted approach. Apart from obtaining nominal measures depicting frequency of
participation, participant satisfaction, outreach and delivery format, and adequacy of facilities where
programs were rendered, observing the effects of the programs on marital satisfaction and distress
requires quantitative and qualitative measures. The social sciences have struggled with successfully
capturing the ultimate criterions of relationship dynamics such as: affect and context communication,
problem solving, child-rearing, friendship and sexuality, nuclear and extended family, religious
influence, aggression and anger. Accurately converting these ultimate criterions into actual measurable
criterions is a challenge. Even when social scientists succeed in achieving this with some statistical
significance, additional limitations arise.

Limitations can range from internal to external factors that threaten the validity of the measure. Is the
measure written at a reading-level appropriate for the couple taking the assessment? Are there cues in
the environment that influence, either negatively or positively, the manner in which the couple
responds? Are there external or internal events occurring in the life of the individual or couple that are
not accounted for that may influence their response? Does a self-report provide an adequate depiction of
the nature of a relationship? Researchers developing valid and reliable measures to observe the
outcomes of programs for couples are plagued with these and many other nuances of measurement
construction.

Moreover, the challenges of developing outcome measures for programs developed for couples are
complicated further when influencing factors of personal values, personality, diversity, and social and
economic factors are also considered. Individual values are influenced by culture, religious and ethnic
background, environmental context in which the person was raised, social context, and belief system.
All of these aspects interact in forming aspirations and expectations towards marriage, which in turn
impact negative and positive interactions in relationships.

This is particularly important as our attention is turned towards developing and implementing program
outcomes related to marriage and family in the Hispanic community. Given the growth of the Hispanic
population in the U.S., providing adequate programs that serve this community has become a concern of
public policy. Thus, as local programs and federal initiatives develop that encourage healthy marriage,
evaluators and researchers are challenged by the question “what works best for Hispanics living in the
United States?” To answer this question, investigators must have the tools to measure these programs.
Similar processes that have been undertaken by social scientists to develop tools to assess marital
enrichment programs for the mainstream community in the United States will also need to be undertaken
for the Hispanic community. One exception to this process is that in order to create valid outcome tools
for Hispanics residing in the U.S., social scientists will also need to focus attention on the interaction of
the dimensions of culture as influencing latent traits of marriage and relationship dynamics.
The purpose of this memo is to initiate a conversation among the research community on how to
measure programmatic and individual success in healthy marriage programs that are culturally relevant
and statistically valid among Hispanic couples in the U.S.

What Do We Know?
At least three processes are involved in administering measurement tools to Hispanic marriages and
families: 1) development of culturally relevant tools, 2) format and contextual administration of the
measurement tool, and 3) analysis of the results. Although the available research conducted in the
arena of marriage and Hispanic families provides limited information with regards to these three
processes, additional lessons can be gleaned from research on other aspects relevant to Hispanic families
that have been investigated more widely.

Development of Culturally Relevant Tools
Although universal factors can be found across cultures, such as facial expressions (e.g., sadness,
happiness, surprise, anger, fear, contempt, and pain), interpretations of experiences and modes of
experiences may differ from culture to culture. Individual or relational constructs or concepts may
interact with cultural and social factors. For example, in the United States, low-context communication
is prevalent, meaning it is highly dependent on verbal cues and content. Whereas in Latin America,
high-context communication or communication that relies on non-verbal body language is most
prevalent.

Cross-ethnic and cross-cultural research can be approached from an etic or emic perspective. The etic
perspective is comparative and the emic perspective is intracultural. The etic perspective attempts to
explain common and similar phenomena across different cultural contexts, while the emic perspective
seeks to understand the elements from within the culture and its context. The emic perspective aims to
study the internal logic of culture, its singularity, and interrelationship with other intracultural elements.
Cross cultural researchers are challenged to reconcile these two approaches as they adopt and translate
measures.

The role of culture is key in integrating universal similarities and contextual differences. Culture
provides categories and lexicons for emotional experience and expression, sets limits of tolerance for
specific emotions and affect, shapes social responses to distress and disability, influences beliefs about
the etiology of distress, provides a context for making meaning out of suffering, and sets variations in
family structure, composition, and interactions (Zayas). Care must be taken, however, to hold the
tension between inherent human characteristics and culture and individual and/or relational elements as
a result of our common humanity. Culture has a significant impact on how individuals understand and
negotiate daily activities and relationships. Cultural factors do not eliminate individual or relational
constructs but inform their operation. When conducting research with Hispanic and other ethnic
families, these constructs provide the language needed in order to identify areas of behavioral change
(Reyes).

If Hispanics immigrating to the U.S. preserved aspects of the culture in a dynamic form, researchers
would readily isolate cultural variables and translate these into measurement tools. Even though
Hispanics tend to preserve culture, culture does not remain static. It is dynamically interacting and
evolving and can be explained by theories of acculturation.

The process of acculturation for Hispanics varies from individual to individual. The process may
encompass adhering to the dominant culture and maintaining the culture of origin. The acculturation
process is assessed by two separate models each containing limitations (Cabassa). The acculturation
process may be described as either one-dimensional (acculturation process moves along a single
continuum – from immersion in culture of origin to immersion in new culture) or bi-dimensional (the
person opts to either neglect or maintain the culture of origin while participating in the dominant
culture). However this process is described, in addition to developing operational constructs of culture,
measures also need to address the process of acculturation. The challenge in developing culturally
relevant tools for marriage programs and couples is incorporating dimensions of the Hispanic culture
and assessing the role of acculturation while holding individual and marriage universals constant.

Given this tension, it makes sense that translating existing measurement tools has proven to be
inadequate. Translations ignore the premise that culture is an influencing agent in marriage and families,
and assumes that aspects that have been found to be relevant in one culture can be directly observed and
applied in another culture. Equally important to recognizing differences between Latino and non-Latino
cultures, is the diversity within the Latino community. Hispanics residing in the United States are
rooted in different cultures with different colloquialisms and nuances in language. This requires that the
translation be as general as possible. Measurement instruments should be comparable before they are
administered by utilizing several different groups (Mexican, Puerto Rican, Cuban, etc.) to evaluate
whether or not they are understandable. Translations of instruments should also be relevant for recent
immigrants and respondents of differing literacy levels. Since literal translations closely follow the
grammatical and lexical forms of the source text, a preferred method when using translations is to move
beyond literal translations.

When translating instrument tools, a more rigorous approach, back-translations, is recommended.
Backtranslations occur when a second translator translates the document back into English once it has been
translated into Spanish. The back translator must have excellent command of both languages and
perform a more literal translation than normal. The original translation is then compared side-by side
with the back translation by the authors of the instrument tool. Even though back-translations are
deemed better than literal translations, cross cultural research also reveals limitations to this approach
since cultural equivalency is still not obtained (Hilton and Skrutkowski, 2002).

Translation of an existing instrument not only runs the risk of losing the original intent of the message,
but it also assumes that marriage dynamics described in the tool are occurring similarly for Hispanics.
For example, an assessment tool may ask the rater to respond either positively or negatively to an item
that states, “my partner sometimes screams or yells at me when he/she is angry.” Upon translation, the
person may interpret the question as, “my partner is abusive when he/she is angry.” Based on the
respondent’s cultural experience he/she may respond negatively as this second interpretation may be
associated with a negative impression.

A cultural equivalence model for translating and adapting instruments may be more appropriate. The
cultural equivalence model allows for multiple linguistic and sociocultural factors, such as determining
whether the construct that the original instrument evaluates is pertinent to the target culture. It evaluates
the instruments in several dimensions: semantic (similar meaning in an item), content (culturally
relevant content of items), technical (original and translated version yield comparable data), criterion
and conceptual equivalence (similar evaluation of results and similarity of the theoretical construct being
evaluated in the different cultures) (Chavez and Canino, 2005).

Another proposed method would be to construct measurement tools that begin with the premise of
culture by identifying key concepts/constructs in a culture and generating measurement based on these
constructs. In this approach, the task is to understand the phenomenon in the local culture, integrate the
known universals of marriage dynamics, and then produce a package of questions that are meaningful
for the Hispanic culture. This approach ideally would create valid and culturally appropriate
measurements. Although this approach is more demanding, requiring investment of more resources and
the involvement of both researchers and practitioners in the field, it would address some of the
previously discussed barriers to translating available tools.

A review of the measures that have been created in Latin America is another approach to create
culturally appropriate measurements. These measures incorporate values of the Latino culture which are
upheld by many Latinos residing in the United States. Subsequently, researchers may work with
developers of the instrument to make adjustments for the population the measurement would be applied
to in the United States.

When creating assessment tools to measure program effectiveness, researchers should note the
mediating effects for Hispanics in participating in an educational group. Extraneous variables that may
need to be accounted for are the benefits that are inherent in participating in a group, regardless of the
content that is delivered. Some of the benefits of participating in a cohesive group, which often are
formed in educational programs, include: instillation of hope, connection, universality, and support.
These factors can in themselves be powerful driving forces of change. Thus, in developing tools to
measure outcomes of a marital enrichment program, a researcher would want to control for these as
well. Hispanics generally assume a world-view of community and interdependence. Upon migrating to
the United States, Hispanics increasingly isolate as adaptation and assimilation occurs. With isolation
comes lack of support which combined with added stress from the acculturation process (i.e., new
language, culture, community, jobs, schools), leads to increased risk of mental health problems such as
depression and anxiety. Groups, even educational groups, provide a medium by which Hispanics find
the support they once relied upon in their country of origin. Controlling for these factors presents a
challenge when measuring program effectiveness.

Researchers also need to account for social desirability, or the tendency to respond favorably in order to
give a positive impression. This is particularly relevant for Hispanics, a culture that places high value on
preserving a positive image and respecting authority. Researchers and practitioners will most likely be
highly respected by Hispanics. A Hispanic research participant may assume that he or she will be
respectful by responding in a positive way to the items. Consequently, measures that exclusively include
closed-ended and dichotomous prompts may result in skewed results. Measures developed and
implemented for Latinos, will be strengthened if scales of internal validity are included. Furthermore,
measures developed for Hispanic couples will be strengthened if they include influences and motivators
that prompted participation in programs, exploration of what constitutes healthy marriages for
Hispanics, and ways stressors such as discrimination and economic constraints impact couple
relationships and child-rearing.

In developing valid instruments to measure program effectiveness, a researcher may choose to utilize
outcome measures that focus on the content of the information delivered. That is, tools that answer the
question, “How much did the participant cognitively learn about the material presented?” A pre- and
post- assessment may be developed and the mean results compared to determine whether learning
occurred in a statistically meaningful way. However, a researcher would ultimately want to know
whether the individual and/or couple applied the concepts learned to their relationship. To answer this
question, a researcher would need to develop an instrument that assesses the strengths and weakness in a
couple’s relationship and measure if there are any observed differences before and after the program. A
clear understanding of what are considered strengths and weaknesses in Hispanic relationships would be
necessary to develop an appropriate instrument to make a comparison of this kind.

Furthermore, it would be assumed that the marriage education program delivered included components
that would be helpful for the marital relationship. This may also present challenges as many of the
curricula used in marriage education programs are designed with the dominant culture in mind. The
relevancy of the concepts to Hispanic participants’ relationship and experiences would greatly effect
whether or not they view the program as being helpful. This may effect whether or not the concepts and
skills are applied to the couple relationship.

Another challenge faced by researchers assessing program effectiveness is the reliance on self-reports
from participants. Although, there are some promising marital self-report assessments available such as
the Marital Satisfaction Inventory-Revised, a main drawback is that they still rely on the report of the
individual. A more accurate assessment may be obtained through direct observation although this
approach is costly and more intensive.

Format and Contextual Administration of the Measurement Tools
Determining the type of tool to utilize for Hispanic populations depends on whether the study is
quantitative or qualitative. It is not a matter of distinguishing quantitative and qualitative as opposites
since all methodologies are crossed in research (Díaz-Loving). Qualitative interviews may be more in
tune with a culture that likes to communicate verbally. However, quantitative methodologies are equally
valuable. It is important that researchers take advantage of both methodologies (Díaz-Loving). Each
methodology provides meaningful information in its own way.

Researchers may experience challenges when using quantitative instruments with Hispanic respondents
residing in the United States. One of these challenges is related to language proficiency and literacy.
Examples of ways to address these challenges are: adjusting the reading level of quantitative measures
making them understandable to more members of the population, providing revised, brief versions of the
measures as opposed to longer versions and providing training on test-taking strategies. Furthermore,
when administering quantitative measures in groups, it may be beneficial to walk through items as a
group. The proctor can prompt and encourage the group to follow along as opposed to having
individuals work independently.

Qualitative interviews may be less threatening and more personable then a formal assessment measure.
Focus groups are a very effective format for obtaining qualitative data when relatively little theoretical
or empirical knowledge is available. Focus groups are useful since respondents feel that researchers
want to learn from their experiences. Moreover, participants may be more open to responding when they
are surrounded by their peers and when they perceive that they share similar experiences. This may
make focus groups particularly attractive when working with Latino couples. Other listed advantages of
focus groups include: developing trust and increasing participation due to direct contact with
researchers, interpreting and adding depth to responses to quantitative methods, obtaining a large
amount of information in the participant’s own words, and obtaining data from individuals with literacy
limitations (Umaña-Taylor and Bámaca, 2004).

Qualitative measures of acculturation and culture can provide some guidance in terms of the type of
variables (such as respeto) that could be measured via quantitative measures. For example, a qualitative
study by Skogrand, et al revealed cultural values, practices, and strengths evident in strong Latino
marriages. The study found components of strong marriages such as children, communication, and
religion for Hispanics that may have otherwise been overlooked if the study had originally been a
quantitative one. The study revealed that for the couples interviewed, parenthood (not marital
happiness) was the primary goal of marriage. For many Hispanics, family and parent-child relationships
take precedence over marital relationships.

Given that the family is central for Hispanics, exclusively providing assessments to the marital dyad is
limiting. The depth and breath of the results may be strengthened by incorporating measures for the
extended family. Exploring ways the family has observed and experienced change on the constructs
measured may enhance the researcher’s understanding of a given issue.

Analysis of the Results
The influence of culture and values is also addressed during analysis of the results. In order to make
meaningful interpretation of raw scores derived from quantitative measures, they must be converted into
standard scores that can then be compared to others in the standardized sample. These norm-referenced
interpretations allow for comparison of individual differences. This again presents challenges since very
few assessment instruments include substantial diversity in their sample to which comparisons of
Hispanics can be made. Hence, the results generated from the Hispanic groups are compared to the
results of a sample that is usually skewed towards Anglo, middle-class North Americans. The
assumption is made that there are sufficient universals present in the values assessed that comparison of
the results should be valid. Of the quality marital assessment instruments available in the market, the
Marital Satisfaction Survey-Revised, is one of the only instruments where the developers have
conducted norm-based studies on Mexican-American populations.

Qualitative measures also need to be coded in order to identify common constructs and make descriptive
statements of the results generated. Coding these statements is highly dependent on the rater’s
objectivity and ability to follow the protocol provided.

Individual, subjective biases may easily influence the manner in which these results are interpreted and
coded. Since the process of analyzing results is not a culture-free process, some feel that investigators
should reveal their ethnic background so as to inform the reader of the biases they may impart in the
interpretation of the results (Umaña-Taylor and Bámaca, 2004).

When researchers are performing analysis, they should consider the attitudes of the research
participants. Many communities are hesitant towards investigators “invading,” “studying” and
“analyzing” them in order provide a list of recommendations on how to “fix” them. Anecdotal stories
reveal that communities feel betrayed when investigators temporarily enter their neighborhoods with the
pursuit of finding out what is wrong with them and then depart without debriefing or providing a
summary of the findings. Failing to return to the community to provide feedback with regard to the
findings is concerning as it may create greater distance between communities and investigators. When
this occurs in Hispanic communities, Hispanics may feel as if they were disrespected. Integrity and
respeto (respect) are cited cultural values assumed by many Hispanics. A means by which researchers
may show a sign of respect to the community is by returning and providing specific feedback on the
results, limitations of the study, and recommendations.

What Do We Still Need to Know?
The interest in better understanding marriage and relationships within the Hispanic community has
accelerated exponentially in recent years. This interest has made evident the glaring gaps in knowledge
in this area and the rigorous work that is still needed. Culturally relevant tools that integrate values of a
culture in movement still need to be developed. To do this, a framework for research in the Hispanic
community must be established to answer a plethora of questions. Some of these questions include:
How can outcome tools be developed so that they satisfy the demands of quality research while at the
same time remain flexible and adapt to the growing Hispanic community? What are the recommended
guidelines available for investigators developing instruments for Hispanics? How can investigators
avoid common errors in the development of contextualized measures?

What Should We Do With What We Know?
In order to create contextual measurement tools that are valid for Hispanics, the research community
should assess measurements to more systematically evaluate how outcomes translate. Focusing on
creating and implementing valid and reliable measurement tools for Hispanics would help to increase
awareness of the existing need. It would also create a forum to address this need and would synergize
the effort and energy needed to implement strategic proposals toward this endeavor. Given the limited
knowledge of how to adequately measure program and individual outcomes for Hispanic couples, it
would be of great benefit for researchers to evaluate the processes of newly funded healthy marriage
programs. Many of these programs will serve a large proportion of Hispanics, and studying these
programs and participants would further the knowledge base of Hispanics living in the United States.
Evaluation of programs serving Hispanic couples should include back translation teams, rely on Latino
researchers to review measurements created, and include both qualitative and quantitative measures with
multiple levels of analysis including the family, couple, and individual.

Since time and funding for more in-depth studies may not always available, researchers should make
efforts to develop measurements that are valid and reliable with regards to Hispanic couples. Literal
translations of existing instruments compromise the validity of the tool since the audience may interpret
items differently than what was originally intended to be measured. Since cultural sensitivity in research
increases its accuracy researchers are encouraged to assume a model that falls more in line with the
derived etic approach (Chavez and Canino, 2005). This approach attempts to integrate both emic and
etic perspectives in order to incorporate cultural flexibility into the adaptation of the instrument and to
retain cross-cultural generalizability of the findings. Strategies for the derived etic approach include
using a cultural equivalence model for translating and adapting instruments, focus groups, and a
multinational bilingual committee. The reader is encouraged to review the Toolkit on Adapting and
Translating Documents prepared by Chavez and Canino and developed by the Human Services Research
Institute and the U.S. Department of Health and Human services, which describes the derived etic
approach, lists 13 steps for adapting and translating instruments, and provides specific field examples.

Conclusion
Opponents of efficacy studies argue that the strict conditions created in clinical trials limit the degree to
which the results can be generalized. Individuals and participants in the population are faced with
stressors that are not observed with volunteers in experiential groups in a study. Alternative
effectiveness studies, which are quasi-experiential are recommended instead. Both positions would
agree that even the strictest methodological research study has limitations one way or another. The
measurement tool, heavily relied upon for recommendations made towards promotion or elimination of
programs, is usually given the utmost care in its construction precisely because of its weight and
significance in the study.

For this reason, a myriad of quality assessment instruments are available on the market for assessment of
relationships. Even more tools are found for measuring program outcomes in terms of knowledge
learned and participant satisfaction. The investment given to development of quality tools has resulted
in a menu of options that meet various needs – at least for the majority culture. As expressed by several
participants of the research meetings and seemingly a consensus among all present, reliable and valid
measurement tools for Hispanics must be created. “Packaged” questions cannot continue to be imported
and made to fit the Hispanic population. Further, the statement “we already have an instrument for that”
can no longer be comfortably accepted, but instead, researchers must consider cultural factors and the
concept of values as they play a significant role in measuring program and individual outcomes.

References
Chavez, L. & Canino, C. (2005). Toolkit on Translating and Adapting Instruments, Human Services
Research Institute and U.S. Department of Health and Human Services., Retrieved on September 22,
2006 from www.tecathsri.org.

Hetherington, E.M. & Clingempeel. W.G. (1992). Coping with marital transitions. Monographs of the
Society for Research in Child Development. 57 (2-3, Serial No. 227).

Hilton, A.,Skrutkowski, M. (2002). Translating instruments into other languages, development and
testing process. Cancer Nursing, 25, 1-7.

Jones, P.S., Lee J.W., Phillips, L.R., Zhang, V.E., Jaceldo, K.B. (2001). An Adaptation of Brislin’s
translation model for cross-cultural research. Nursing Research, 50(5), 300-304.

Sigelman, C.K. & Shaffer, D.R. (1991). Life-Span Human Development. Belmont, CA: Brooks/Cole
Publ. Co.

Umaña-Taylor, A.J & Bámaca, M (2004). Conducting Focus Groups with Latino Populations: Lessons
from the Field. Family Relations, 53(3). 261-272.