A. Cecile J.W. Janssens, PhD
Center for Medical Decision Making
Department of Public Health
Erasmus University Medical Center Rotterdam, the Netherlands

Educational objectives

After reading this case study, you should be able to:

• interpret gene-disease associations in terms of clinical validity of a genetic test
• calculate basic indicators of clinical validity, such as sensitivity, specificity and area under the receiver operating characteristic curve (AUC)
• discuss the potential clinical or public health impact of genetic testing multiple disease susceptibility genes.

Introduction

Type 2 diabetes mellitus represents a significant health problem in the United States. According to the most recent assessment by the Centers for Disease Control and Prevention (CDC), the prevalence of type 2 diabetes is nearly 6% nationwide; prevalence exceeds 10% in some ethnic subgroups, including African-Americans, Hispanics and Native Americans. Although the risk of type 2 diabetes increases with age, incidence has been rising in all age groups and the increase in children is of particular concern. In 2000, diabetes was the sixth leading cause of death.

In type 2 diabetes, the body’s impaired production and use of insulin results in chronically elevated blood glucose levels and long-term tissue damage. Evidence for genetic susceptibility to type 2 diabetes comes from studies of population admixture, familial aggregation, mono- and dizygotic twins, and genetic association and linkage. Diabetes is also a major feature of certain monogenic disorders including maturity-onset diabetes of the young (MODY). Genome-wide linkage studies have identified susceptibility loci for type 2 diabetes on many chromosomes and associations with numerous candidate genes (including mitochondrial genes) have been reported. Most of these associations are small (OR 1-2) and inconsistent among studies.

You may choose to view this quiz in an interactive Flash format.

Case study

In December 2005, Lyssenko et al. reported significant associations of the PPARG PP and CAPN10 SNP43/44 GG/TT genotypes with type 2 diabetes (T2D). They suggested that genetic testing for these variants might offer a future approach to identifying individuals at risk of T2D.

Lyssenko V, Almgren P, Anevski D, Orho-Melander M, Sjogren M et al. (2005) Genetic prediction of future type 2 diabetes. PLoS Medicine 2: e345 DOI: 10.1371/journal.pmed.0020345

Lyssenko, et al., analyzed data from a prospective cohort study to investigate the value of genetic factors in predicting type 2 diabetes (T2D). They examined the predictive value of six loci and found that PPARG and CAPN10 genotypes were significantly related to T2D. One of their most striking findings was a 21.2-fold increased risk for T2D in obese carriers of the PPARG PP and CAPN10 SNP43/44 GG/TT genotypes who have elevated fasting plasma glucose (FPG).

1.  What are the possible results of a test for PPARG P12A?

2.   What are the possible results of a test for CAPN10 SNP43/44?

3.  The authors combined PPARG and CAPN10 SNP43/44 in one test.

I. This composite test has 27 possible results
II. The PPARG PP and CAPN10 SNP43/44 GG/TT combination is considered the positive test result.

What is the correct answer?

4.  Calculate the hazard ratios (HR) for the single genetic tests and the combined tests. Which of the following answers is NOT correct?

In the abstract the authors report a HR of 21.2.

5.  How was the HR of 21.2 calculated? (Give your interpretation of this HR. What does it tell you?)

This HR is the ratio of T2D risk in PPARG PP / CAPN10 SNP43/44 GG/TT carriers and

6.  What percentage of the study population was in the high-risk subgroup (with HR=21.2)?

7.  What does the HR of 21.2 tell you about the value of genetic testing of the aforementioned genotypes for the prediction of T2D?

Because the HR of the PPARG / CAPN10 SNP43/44 composite test was 3.3 and that of BMI and fasting plasma glucose was 6.0, a test for this genotype does not replace BMI and fasting plasma glucose for the prediction of T2D. We are interested in the extent to which PPARG / CAPN10 SNP43/44 testing can improve the prediction of T2D. For this purpose, we can examine the area under the receiver operating characteristic (ROC) curve for the prediction of T2D based on fasting plasma glucose and BMI, with and without the results of the genetic test. We can do this under one assumption, namely that follow-up time did not differ by genotype.

ROC analysis
The usefulness of composite tests can be evaluated by the area under the receiver operating characteristic (ROC) curve. The ROC curve presents the combinations of sensitivity and specificity for each possible cut-off value of a continuous test result that can be considered to define positive and negative test outcomes. The discriminative accuracy, quantified as the area under the curve (AUC), is determined by the distribution of disease risks in those who will develop the disease and those who will not. The AUC can be interpreted as the probability that the test correctly identifies the diseased subject from a pair in which one is affected and one is unaffected. An AUC of 0.95 means that 95% of the pairs are correctly classified, whereas a test with an AUC is 0.50 is non-discriminative—as accurate as tossing a fair coin. The AUC ranges from 0.5 (total lack of discrimination) to 1.0 (perfect discrimination) and is independent of the prevalence of disease. The magnitude of the AUC indicates whether a test is useful for identifying individuals who are at increased risk of disease (screening; e.g. AUC ~ 0.80) or to diagnose a disease before the onset of symptoms (presymptomatic diagnosis; e.g. AUC >> 0.99).

8.  What is the sensitivity and specificity of the combined testing of fasting plasma glucose, BMI and genotype when only the combination of elevated fasting plasma glucose, high BMI and PP/GG/TT is regarded as a positive test result, and all other combinations are considered negative?

The figure shows the ROC curves of the prediction of T2D based on fasting plasma glucose and BMI alone and when genetic testing was added.

The size of the AUC was 0.68 for the combination of BMI and fasting plasma glucose, 0.68 when the composite genetic test was added, and 0.69 when the prediction model included all possible interaction effects.

9.  What is your conclusion about the value of testing PPARG and CAPN10 SNP43/44 genotypes for the prediction of T2D?

10. How do you explain the difference between your conclusion and that of the authors?

The results of this exercise have been published as a letter to the editor in PLoS Medicine, including a response of the authors.

Janssens ACJW, Gwinn M, Subramonia-Iyer S, Khoury MJ. Does genetic testing really improve the prediction of future type 2 diabetes? PLOS Medicine 2006; 2:E114.

Link:
http://medicine.plosjournals.org/perlserv/?request=get-document&doi=10.1371/journal.pmed.0030114 (last accessed 1/2008)

Author’s reply:
http://medicine.plosjournals.org/perlserv/?request=get-document&doi=10.1371/journal.pmed.0030127 (last accessed 1/2008)

Correct Answers:

Question 1:
Answer c is correct:
The test has three possible results, namely the genotypes of PPARG at the P12A locus: PP, PA and AA. The PP genotype was associated with increased risk of T2D and was thus considered the positive test result.

Question 2:
Answer d is correct:
This test is a composite test of two SNPs. The possible results at SNP43 are GG, GA and AA, and at SNP44 are TT, TC and CC. The composite test has nine possible answers, namely all possible combinations of the single tests: GG/TT, GG/TC, GG/CC, GA/TT, GA/TC, GA/CC, AA/TT, AA/TC, AA/CC. The GG/TT combination was considered the positive test result.

Question 3:
Answer c is correct:
This test is a composite of three tests: PPARG P12A, CAPN10 SNP43 and CAPN10 SNP44. This composite test has 3*3*3=27 possible results, namely all possible combinations of the single tests: PP/GG/TT, PP/GG/TC, PP/GG/CC, PP/GA/TT, etc. Only the PP/GG/TT combination was considered as the positive test result.

Question 4:
All of the hazard ratios are correct except d. The hazard ratio for PPARG PP CAPN10 SNP43/44 GG/TT is 3.3, not 21.2. The hazard ratios can be found in Table 3 in the column of univariate effects.
PPARG:                                     HR = 1.7
CAPN10 SNP43:                          HR = 1.1
CAPN10 SNP44:                          HR = 1.5
CAPN10 SNP43/44:                     HR = 2.1
PPARG / CAPN10 SNP43/44:        HR = 3.3

Question 5:
Answer d is correct:
The HR of 21.2 was obtained by comparing individuals who test positive on the PPARG / CAPN10 SNP43/44 (i.e., who have the PP/GG/TT genotype) AND who have BMI>30 AND who have elevated fasting plasma glucose compared with those who do NOT carry the PP/GG/TT genotype AND have BMI<30 AND have normal fasting plasma glucose. Thus, the HR of 21.2 includes the T2D risks associated with high BMI and fasting plasma glucose.

Question 6:
Answer a is correct:
The multivariate column in table 3 reports the HR of 21.2. There were 38 individuals who had the PP/GG/TT genotype AND BMI>30 AND elevated fasting plasma glucose. Percentages reported in the text suggest that there were few missing values. Therefore, we estimate that only 38 out of 2293 persons (2%) were in the subgroup with a HR of 21.2.

Question 7:
Answer c is correct:
The HR of 21.2 includes risks associated with high BMI and elevated fasting plasma glucose in addition to genotype. To support the predictive value of a genetic test, the data analysis would need to show that 1) the genetic test by itself is a better predictor of T2D than the combination of BMI and fasting plasma glucose, or 2) the prediction of T2D can be improved by adding a genetic test to measurement of BMI and fasting plasma glucose.

Question 8:
Answer b is correct:
In all, 132 individuals developed T2D; 17 of these had elevated fasting plasma glucose, high BMI and the PP/GG/TT genotype (Table 3). The sensitivity is 17/132 = 0.13 or 13%, which means that the test would have identified only 13% of persons who eventually developed T2D. Of the 2293 persons in the study, 2161 did not develop diabetes. Of those 2161, 21 had a ‘positive test result’ and 2140 had not. The specificity is thus 2140/2293 = 0.99 or 99%.

Question 9:
Answer a is correct:
The AUC of the T2D prediction model did not increase when the composite genetic test was added. Hence, genetic testing for PPARG and CAPN10 SNP43/44 does not improve the prediction of T2D. Overall, the AUC is 0.69, which is generally insufficient for screening purposes.

Question 10:
Answer c is correct:
The authors focused on the comparison between the two most extreme subgroups: those with the PPARG and CAPN10 SNP43/44 risk genotypes AND BMI>30 AND elevated fasting plasma glucose compared to those without the risk genotypes with BMI>30 and normal fasting plasma glucose.