3. Creating and Validating an Index of Socioeconomic Status (continued)

3.4 Validation of the SES Measure

Before proceeding to use the four category SES measure in the tabulations and multivariate regression models, we undertook to validate the index score. To validate this measure, we needed to have a large sample similar in characteristics to the one on which we intend to run the tabulations and regression models. In addition, the sample needed to include data that we expected to be related to SES.

We used the national probability sample of Medicare beneficiary respondents to the three Medicare fee-for-service CAHPS surveys for 2002-2004 as the basis for validation of the SES measure. This was part of the sample on which we had developed the surname imputation algorithm. It happened that we had requested and received some income-related information for the respondents to those years of the survey from the Social Security Administration (SSA) for use by CMS for special analyses. We requested and received permission from CMS to use these data. The two variables from Social Security were the indexed monthly earnings (IME) that were taxed for Social Security purposes while the beneficiary was paying the Social Security tax, and the monthly benefit amount (MBA) that Social Security is currently paying beneficiaries. The former is an indicator of the beneficiary's past earned income level, while the later is a measure of the beneficiary's current benefit payments which are partly tied to past earned income level.

The first step in the validation process involved computing the SES index scores for the full validation sample. The geocode available from our previous work with these data was used to link to the Census block group data needed to calculate the SES index scores for 381,429 CAHPS Medicare fee-for-service survey respondents in the three survey years. The distribution of SES index scores was partitioned into fourths according to the groupings of scores (see categories in Table 3.5) used to create the quartiles for the analysis sample (the sample to be used in the tabulations and regression analysis for the present study). The distribution of the validation sample according to the quartile score ranges of the analytic sample is presented in Table 3.6. A comparison of the four category distribution of the validation sample to the analytic sample shows them to differ slightly, with the former composed of slightly more higher SES beneficiaries.

Table 3.6 Quartile Distribution of SES Categories: 2002-2004 CAHPS Survey Respondent Validation Sample Person-Level Data (N = 381,429)

SES	Category	%
1	(0-49)	30.6
2	(50-52)	24.5
3	(53-56)	25.5
4	(57-100)	19.4

We next computed the means of the two SSA supplied variables within each level of SES and we also cross tabulated the two SSA supplied variables with SES scores for those beneficiaries in the 2002-2004 CAHPS surveys for whom we were given the SSA variables. For the cross tabulations, both the SES and SSA variables were partitioned into four levels.

The top half of Table 3.7 shows the mean value of the indexed monthly earnings (IME) that were taxed for Social Security purposes within each of the SES quartile categories. Clearly the mean IMEs increased as the SES level rose, and the test of significance of mean differences across the four levels of SES is highly significant. The distribution of beneficiaries across the four categories of SES according to the four categories of their IME is also presented in the top half of Table 3.7. The test of significance for the joint distribution is also highly significant, indicating that, proportionately more beneficiaries with lower IME are classified in lower SES categories, and proportionately more of those with higher IME are classified in higher SES categories. The lower half of Table 3.7 analyzes the relationship between the mean monthly benefit amount (MBA) and SES quartile category, as well as the cross tabulation of the four category SES measure with the four level measure of MBA. A very similar pattern to the IME is present, and both tests of significance are very highly significant as well. The mean MBA increased as the SES category went from the lowest to the highest. The cross tabulation of the two categorical measures (MBA and SES) showed the same kind of association, with proportionately more low MBA beneficiaries in the lowest SES category and proportionately more high MBA beneficiaries in the highest SES category.

In addition to the two SSA variables, we had several other variables from the CAHPS survey and one from the EDB that we believed should be related to a measure of SES. These include whether or not a beneficiary is simultaneously eligible for both Medicare and Medicaid. This EDB measure is affirmative mostly for low income beneficiaries. The remaining variables are from the CAHPS survey. They include: having additional insurance (not including Medicaid), having private insurance to cover prescription drugs, reporting health status to be fair or poor, and achieving educational status no higher than high school graduate. We have presented the distributions of the five dichotomous categorical variables for respondents to the 2002-2004 CAHPS surveys according to the SES index quartiles in Table 3.8.

Table 3.7 Monthly SSA Earnings/Benefits of CAHPS Medicare Fee-for-Service Survey Respondents by SES Index Categories

Variable	SES Index Categories				N	p-value
	1 (0-49)	2 (50-52)	3 (53-56)	4 (57-100)
Indexed Monthly Earnings (IME)
Mean in dollars ($)	1,450.75	1,604.31	1,743.74	1,968.68	177,427	< .0001
$0-$645	28.9	24.7	22.7	20.4	177,427	< .0001
$646-$950	28.2	25.3	22.2	18.8
$951-$1192	25.3	27.1	26.3	23.7
$1193 or more	17.6	22.9	28.8	37.1
Monthly Benefit Amount (MBA)
Mean in dollars ($)	861.83	921.48	970.17	1,056.92	222,977	< .0001
$0-$762	30.3	25.5	23.3	20.1	222,977	< .0001
$763-$1513	28.6	24.7	21.7	19.4
$1514-$2341	24.5	28.7	27.9	23.1
$2342 or more	16.5	21.0	27.2	37.3

Note: Disabled beneficiaries are excluded from these analyses. Significance of mean differences is tested using analyses of variance. Significance across percentages is tested using chi-square tests.

Table 3.8 Percentage of CAHPS Medicare Fee-for-Service Survey Respondents by SES and Selected Demographic Characteristics

Variable	SES Index Categories				N	Chi-Square
	1 (0-49)	2 (50-52)	3 (53-56)	4 (57-100)		p-Value
Dually eligible (from EDB)	21.5	9.5	6.2	4.6	293,312	< .0001
Have insurance in addition to Medicare (excluding Medicaid)	62.2	78.9	84.5	87.8	292,027	< .0001
Have other insurance to cover prescription costs	50.2	56.6	61.8	65.0	281,438	< .0001
Fair or poor self-reported health	44.7	35.1	30.0	24.2	287,270	< .0001
High school graduate or less	74.7	67.5	57.6	40.0	281,332	<.0001

The tests of the joint distributions of each variable with SES are very highly significant, and the directions of the joint distributions are as expected: larger percentages of dually eligible beneficiaries, persons in poor or fair health, and persons who had no more than a high school education are in the lower SES categories, and fewer persons with other insurance (not including Medicaid) and prescription drug coverage are in the lower SES categories.

Rank-order correlation coefficients between each of the five dichotomous measures from the CAHPS sample and four level SES index as well as for between the four levels of the two SSA variables and SES are presented in Table 3.9. As with all of the previous validation measures, the direction of association is always as expected. While the magnitude of the associations is moderate, all of the associations are very highly statistically significant.

Table 3.9 Spearman Correlations of SES Index Scores with Demographic and Insurance Characteristics of CAHPS Medicare Fee-for-Service Survey Respondents

Variable	N	Correlation with SES Index
Self-reported health status	287,270	0.18***
Dual eligibility	293,312	-0.21***
Have insurance in addition to Medicare (excluding Medicaid)	292,027	0.24***
Have other insurance to cover prescription costs	281,438	0.12***
Highest grade completed	281,332	0.31***
SSA IME	177,427	0.15***
SSA MBA	222,977	0.17***

***p < .0001
Note: Variable for self-reported health status is recoded, so higher values represent better health.

Return to Contents

3.5 Association of Clinical Measures with SES Index and SSA Measures

While not a part of the planned validation analysis, at the request of the AHRQ project officer, we examined the association between the SES index we created (as well as the two SSA variables) and a series of clinical measures derived from Medicare claims. We cross tabulated several clinical measures with the SES index we created and with the two SSA measures we had for the 2002 CAHPS survey respondents. These tabulations are presented in Appendix A along with a brief discussion.

Return to Contents
Proceed to Next Section