The Careers and Professional Activities of Graduates
of the NIGMS Medical Scientist Training Program
September 1998

Appendix II - Methods

Prior to conducting the study, a small pilot study was initiated to examine the feasibility of the planned study design in terms of identifying comparison groups, obtaining current addresses, soliciting curricula vitae, and extracting the necessary information. This effort involved selecting a random sample of MSTP-supported M.D.-Ph.D. recipients, stratified by Ph.D. cohort, and locating a comparable sample of Ph.D. recipients who had received NIH research training support and who matched each MSTP graduate on relevant characteristics. The results of the pilot study guided the final study design in several ways; for example, the diversity of available specializations included in the broad field of economics (labor economics, microeconomics, etc.) suggested disciplines whose graduates should not be included in the study due to the difficulty of identifying through extant data an appropriate match for an MSTP graduate with an M.D. and a Ph.D. in health economics.

The full study group population included all individuals who had been MSTP trainees and who met certain criteria: 1) they received at least 12 months of MSTP support, 2) their first year of MSTP support occurred no earlier than July 1969, and 3) both the M.D. and Ph.D. degrees were awarded by June 1990.³¹  In addition, other criteria were applied in order to aid in the construction of appropriate comparison groups. First, in order that the M.D.-Ph.D. comparison groups consisted of individuals graduating from dual-degree programs, both degrees had to be awarded by the same institution and be coterminous (i.e., the Ph.D. had to be awarded no more than 3 years prior to the M.D. or no more than 2 years after the M.D.). Finally, individuals whose Ph.D.s were earned in disciplines that accounted for very small numbers of dual-degree recipients were excluded from the study.³²  

Because a large number of curricula vitae were successfully obtained for the individuals chosen for the pilot study, those who had provided their curricula vitae were retained in the final samples of MSTP and Ph.D. graduates. Additional individuals were then selected (randomly or by matching) until the desired sample size was achieved.

Comparison Groups

Four comparison groups were constructed as follows:

MSTP M.D. Only (n = 269). This group consisted of all MSTP participants who earned the M.D. but did not complete the Ph.D. degree. Because the number in each cohort¹³ was relatively small, all identified individuals were included in the study. Comparison of this group to MSTP graduates provides an assessment of the relationship between Ph.D. completion and career outcomes.

Ph.D. Graduates (n = 398). This group included Ph.D. recipients who had received at least 12 months of non-MSTP predoctoral research training support from NIH. For each of the sampled MSTP graduates, a Ph.D. recipient was selected who matched the MSTP graduate based on: 1) Ph.D.-granting institution, 2) year of receipt of Ph.D. degree, 3) field of Ph.D. degree, and 4) gender. Because it was not always possible to locate an exact match, the following strategy was used:

1) Based on the study design and previous literature on careers, the matching variables of the Ph.D. institution and gender were assumed to be of primary importance and were the first criteria to be satisfied in locating a match for a specific MSTP graduate.

2) When a match was available by institution, gender, and Ph.D. year (the academic year from July through June) but not by Ph.D. field, an individual in a closely related field who matched on all other criteria was sought first. This required some "clustering" of fields: 1) cellular biology, molecular biology, and biochemistry, 2) microbiology and immunology (and cellular/molecular biology if no matches were found), 3) general biological sciences and other biological sciences, 4) general health sciences and other health sciences, 5) and physiology and biophysics.

3) When all criteria matched except the date of receipt of Ph.D. degree, the next step was to identify an individual whose Ph.D. degree was awarded "just outside" the academic year of the MSTP graduate. To illustrate, if the MSTP graduate received a Ph.D. degree in biochemistry in June 1985 and there were no corresponding individuals in the Ph.D. graduate population from that institution, a search was conducted for a person who received the Ph.D. in another academic year (but still within 11 months of the date of the MSTP graduate's degree). This approach was not used when relaxing of this criterion would cross Ph.D. cohorts, although it was impossible to avoid in a very small number of cases. In addition, for another few cases, extending the 11-month criterion was necessary in order to locate a match on all other variables.

4) If all the above procedures yielded no matching candidates, an individual who matched on all variables except Ph.D. field was chosen. For example, the match for a female MSTP graduate with a Ph.D. degree in pathology from Institution A in 1984 was another female who earned her Ph.D. degree in 1984 from Institution A, but in the field of genetics.

For the most part, this strategy resulted in locating a "match" Ph.D. for the large majority of MSTP graduates in the sample (97 percent). Those MSTP graduates for whom a match could not be found were all members of the 1971-75 cohort.

An examination of the quality of the matches--prior to any reassignment based on information contained in the curricula vitae--revealed that 69.2 percent of sampled MSTP trainees had a Ph.D. who matched on all criteria. Another 26.8 percent had a match on all variables except field of Ph.D. degree, 2.3 percent did not match on year of receipt of Ph.D. degree, and 1.8 percent did not match on Ph.D. field or year of receipt of degree.

This comparison group was intended to permit the examination of similarities and differences in outcomes for individuals whose research training was similar to that of MSTP graduates in terms of disciplinary focus and institutional environment, but was dissimilar because it did not include clinical emphases and skills.

Non-MSTP M.D.-Ph.D.s from MSTP Institutions (n = 314). This group included all M.D.-Ph.D.-degree holders who graduated from the same institutions as the MSTP graduates, with coterminous degrees, but who did not receive MSTP support. Comparison to this group was intended to assess whether selection and participation as an MSTP trainee is associated with a different career progression than that of individuals with similar training experiences but no MSTP support. All individuals meeting these criteria were included in the study. No individuals from the earliest cohort (1971-75) were identified.¹⁷

M.D.-Ph.D.s from Non-MSTP Institutions (n = 314). This group included individuals with coterminous M.D. and Ph.D. degrees from institutions without MSTP training programs. This group was used to compare outcomes of MSTP graduates with dual-degree recipients at institutions without MSTP training programs. No individuals were identified from the 1971-75 cohort. The study included a random sample of the 380 individuals who were identified.

Some reassignments of individuals among study groups were necessary as a result of more complete information on degree status contained in the curricula vitae collected from study participants. For example, 25 individuals originally in the MSTP M.D. only group were found to have received Ph.D. degrees and were either reassigned to the MSTP graduates group or were excluded from the study for not having received coterminous degrees by 1990.

Methods of Curricula Vitae Collection

Several mechanisms were used to locate the MSTP and comparison group members selected for the study. First, the NIH database of applicants for grant support was used to locate a large number of individuals. Second, all current MSTP program directors were contacted for assistance in locating the selected graduates of their institutions. In addition, the program directors were asked for help in locating the individuals from the comparison groups from their institutions. With the exception of two programs, all MSTP institutions supplied the most recent phone numbers and/or addresses they had on file for the graduates. The directors of a number of the M.D.-Ph.D. programs at non-MSTP institutions supplied location information for the selected graduates of those institutions. Individuals were also located using a number of membership databases available online (American Medical Association, Federation of American Societies for Experimental Biology, American Society for Microbiology, Association of American Medical Colleges, and Society for Neuroscience), publication information available through MEDLINE, and "people" search sites on the Internet.

Data Analyses

The conclusions in this report were based on pairwise comparisons of the MSTP graduate group to each of the four comparison groups. Two principal methods of analysis were used to perform these comparisons: multiple regression was used when the dependent variable was continuous (e.g., number of publications), and logistic regression was used when the dependent variable was dichotomous (e.g., receipt of research grant support). The independent variables used in each analysis were cohort and group (the latter being a dichotomous variable indicating whether an individual was an MSTP graduate or a member of one of the comparison groups). Group differences in the time-to-degree data were analyzed using the Mann-Whitney U statistic. All conclusions were based on a 0.05 level of statistical significance.

When there were between-group differences in demographic or other variables (e.g., length of NIH training support) that might also have some relationship to an outcome variable, these other factors were controlled for in the analyses using a hierarchical regression approach. In general, such factors had little effect on the outcomes of the analyses.

Back - Next - Contents