Developed by the MCH Model Indicators Working Group:

Charlyn Cassady, Johns Hopkins University

Anita Farel, University of North Carolina

Priscilla Guild, University of North Carolina

Joan Kennelly, University of Illinois at Chicago

Mary Peoples-Sheps (Chair), University of North Carolina

Patricia Potrzebowski, Pennsylvania Dept. of Health

Cheryl Waller, University of North Carolina

Coordinated at the School of Public Health,

University of North Carolina

November 14, 1997

Development of the MCH Model Indicators was greatly dependent upon the generous and gracious contributions of many experts in the field, especially those who participated in the development meetings, held in October, 1995 and June, 1997, and those who commented on the preliminary health status indicators in September, 1996. Their contributions are sincerely appreciated. The important contributions of Sarah Teagle to development of indicators for adolescents, of Cerise Knowles to collecting and organizing the information on data sources in Appendix B, and of Ruth Petersen, Ruth Benedict, Eleanor Locklin and Randahl Kirkendall for research assistance at various stages of the project are gratefully acknowledged. The MCH MI database was developed by Jane Stein, DrPH, whose talent, perseverance, and attention to detail are sincerely appreciated. Finally, the assistance of Bill Poole in preparing many of the materials used by the Working Group and of Lori Whitehand and Sarah Pfau in organizing the national meetings and supporting the project in innumerable other ways is acknowledged with thanks and applause.

Developed under a subcontract with the Public Health Foundation, Maternal and Child Health Information Resource Center, No. 240-94-0047.

Part 1: Introduction 1

Overview of the report 2

Purpose of the project 3

Guiding principles 4

Part 2: MCH Model Indicators 6

Conceptual Model 7

Rationale for a model 7

Domains 7

Relationships among domains 8

Categories of indicators within domains 8

Database 11

Descriptions of the indicators 14

Using the Indicators 155

Data sources 155

Rates based on small numbers of events 155

Model-based (synthetic) estimation 156

Using the conceptual model 156

Using the MCH MI in conjunction with

other sets of MCH indicators 164

Part 3: Process of Indicator Development 166

Background 167

General approach 168

Stratification by race and ethnicity 170

Generating concepts 171

Health status indicators 171

Risk/protective and health services indicators 173

Health systems and contextual indicators 174

Recommendations for further development

and ongoing revisions 175

References 177

Appendices (following page187)

A Several views of the MCH Model Indicators

B Descriptions of data sources for health status

indicators

C Special issues regarding data on injuries

D Rates based on small numbers

E Model-based (synthetic) estimation

F Existing MCH Data Sets, Spring, 1995

G Survey of MCH Leaders, Spring, 1995

H Model Data Set Consensus Conference,

October, 1995

I MCH Information Resource Center Subcontractors

Meeting, January, 1996

J Review of preliminary health status indicators,

September, 1996

K Model Indicators Stakeholders Meeting, June, 1997

In the following pages, a set of Maternal and Child Health (MCH) Model Indicators (MI) is presented. The Model Indicators are intended to be used as a barometer of present conditions of mothers and children in the United States and as a guide to the future development of MCH data sources. As a barometer of the present, the MCH MI reflect a wide range of key health problems and related factors that affect mothers and children. They were developed within a conceptual model that promotes analysis of health problems and development of interventions to address those problems. To allow for timely application, many of the indicators can be constructed from existing data sources. In their role as a guide to the future, the Model Indicators include some measures that are important to monitor but for which no available data sources could be located. These indicators are in the MCH MI to emphasize their importance and to encourage development of data sources that will permit routine surveillance of them. For many of these indicators, some movement towards developing or refining data bases to produce the necessary data elements is already underway.

The conceptual model used to guide development of the MCH MI has five domains: health status, risk/protective status, health (and related) services, health systems capacity and adequacy, and contextual characteristics. Indicators in three of the domains-- health status, health systems capacity and adequacy, and contextual characteristics -- lend themselves to routine surveillance. Indicators in the remaining two domains -- risk/protective status and health services -- should be examined closely, and perhaps monitored routinely, if related indicators in the other domains reach unacceptable values. Each of the indicators fits into one of the following levels of importance for routine surveillance:

Core indicators: 20 health status indicators that should definitely be monitored on a routine basis in order to provide a current snapshot of critical aspects of the health of mothers and children.

Recommended indicators: 55 health status indicators, 9 contextual indicators, and 16 health systems indicators that provide a more detailed view of the health of mothers and children and should be monitored on a routine basis, if possible.

Optional indicators: 49 indicators of risk/protective status and 68 indicators of health services that should be examined when related indicators in the other domains reach unacceptable levels.

This report is presented in three parts. Part 1: Introduction includes general background information about the report and the overall project. Part 2: MCH Model Indicators includes detailed descriptions of the indicators, technical information about the database in which they are stored, discussions of issues involved in calculating them, and guidance on how to use the indicators in MCH practice situations. Part 3: Process of Indicator Development describes the processes involved in developing the indicators and presents recommendations for continuing development and revisions.

The MCH Model Indicators represent a large step forward in a process that started in the early 1980s to develop measures of MCH with standard definitions that would support comparisons across jurisdictional and organizational boundaries. Developed with input from numerous people involved in Maternal and Child Health, including local, state and federal program personnel, advocates for MCH population groups, professional organization staff members, academicians, and researchers, this set of Model Indicators is offered to the MCH community by the MCH community as the next step in this ongoing process.

The need for MCH Model Indicators is evident on many fronts:

• Requests for identification of key conditions and services that should be monitored on a regular basis, and for guidance on the preferred content of MCH data bases come regularly to the Maternal and Child Health Bureau (MCHB).

• Needs for new and different types of information, such as indicators to monitor MCH program performance and the effects of managed care, continue to grow. While such changing demands often require new developmental efforts, it is widely believed that they should emanate from a single core set of MCH indicators.

• An important ongoing function of MCH data is to identify differences in the health of population groups across geographic areas. Efforts to standardize definitions of MCH indicators so that such comparisons can be made accurately have been successful at the regional level (e.g., multi-state) but have rarely been approached at the national level.

Existing sets of indicators do not meet the needs described above. For example, Healthy People 2000 includes many indicators of maternal and child health and related factors, but the list of indicators is extensive and it is not organized to focus on the specific concerns of the MCH community. Other prominent sources of indicators at the national level, such as the new MCHB Title V Performance and Outcome Measures, are deliberately limited in scope because of the reporting requirements associated with them.

In response to evolving demands and long-term fundamental needs for data and information, the Division of Systems, Education and Science in MCHB (recently renamed Division of Science, Education and Analysis) developed an Overall Strategic Plan for Data Utilization and Enhancement. The strategic plan is a dynamic document which is revised frequently and provides a comprehensive set of strategies for enhancing data availability and use. One of those strategies is development and promulgation of a model set of indicators for MCH.

The MCH Model Indicators have a broad mission: to provide a panoramic view of the health of mothers and children, within a framework that encourages problem-solving. They include indicators capable of serving such diverse functions as needs assessment, policy and program development, evaluation, resource allocation, program and policy monitoring, quality assurance, and accountability. These indicators are not intended to be promulgated as requirements for any reporting or funding mechanism, although some of them are identical to required items in the Omnibus Budget Reconciliation Act of 1989 and to the new MCHB Performance and Outcome Measures.

To establish boundaries for the project and to promote consistency in the selection of indicators, the following principles were developed and applied:

1. The MCH Model Indicators are a collection of measures that, taken together, give an indication of the health of mothers and children and corresponding needs for specific actions.

2. The indicators should build on existing work (e.g., regional projects of MCHB, Healthy People 2000), and be consistent with reporting requirements (e.g. OBRA ’89, MCHB Performance Measures).
3. The indicators as a group should reflect the health of the full population of women of reproductive age and children of all ages.
4. Women’s health indicators should be limited to reproductive conditions or conditions of women that are likely to have a major impact on the family (e.g., mental illness).
5. The indicators should fit into a framework that encourages useful analysis and interpretation.
6. To be immediately applicable, most of the indicators should be amenable to construction from existing data sources.
7. To encourage development of new data sources and refinement of existing ones so that they meet current needs for data in MCH more effectively, a small number of critical indicators should be recommended regardless of data availability.
8. To provide guidance for users who cannot produce all of the indicators, they should be categorized according to level of importance for routine surveillance.
9. The indicators are for the use of MCH practitioners in carrying out their assessment, policy development, and assurance roles. While the indicators may also be applied to research purposes, they were not developed for research.

10. Decisions about indicator content and measurement should begin with state-of- the-art research findings, modified to reflect the realities of MCH practice and data availability.
11. Participation of a broad array of MCH and related professionals is essential to satisfy many of these principles and to encourage use of the indicators in the future.

To ensure that the indicators represent a studied and complete review of options, a broad framework for development was essential. Selecting a framework reflective of the work of many MCH practitioners was also a high priority because it would encourage useful interpretations of the indicators. The model shown in Figure 1 met both of these criteria. It is a variation of a model typically used in program and policy planning.

The model was created after reviewing several other frameworks, beginning with regional models developed by members of the MI Working Group, including those of the Region IV Network for Data Management and Utilization (RNDMU) and the Midwest Maternal and Child Health Data Improvement Project (MMDIP), and then expanding to other relevant work (e.g., the Child and Adolescent Health Policy Center, 1995; Health Systems Research, 1996).

The model includes five domains selected to reflect health status problems of mothers and children and factors that contribute to those problems.

The primary relationships among the domains in the model are indicated by arrows in Figure 1. While specific indicators may have many other associations, the relationships shown in Figure 1 represent the rationales for selecting these particular domains to guide development and interpretation of the Model Indicators. The health status domain represents the primary focus of all interventions in MCH. The health status of a population, in turn, is influenced by characteristics that place the population at risk for, or provide protection from, a health condition. These characteristics are affected by the health services the population uses. As the model suggests, use of health services also affects health status independent of its effects on risk/protective characteristics. In the environment of these three conceptual domains are two others that have important influences on health services and maternal/child and family characteristics: the capacity and adequacy of the health system and the context in which the population lives.

Within each of the domains, categories of indicators were identified to ensure that no relevant group of indicators was overlooked . In essence, the categories and subcategories served as an outline for the indicator development process. The first set of categories was derived from the MCH and CSHCN directors’ questions, the sources noted above for selection of domains, and the MCH Overall Strategic Plan for Data Utilization and Enhancement. Other categories evolved as the work unfolded. Table 1 is a list of categories and subcategories.

The MCH Model Indicators are stored in a Filemaker Pro database. Filemaker Pro is a versatile program that is available for both IBM-type and Apple Macintosh computers. It has the capability to prepare self-contained databases that can be opened by a user who does not own the program.

The database presently provides several views of the data, some of which are presented in this part of the report, while others are in Appendix A. Specific characteristics and response options included in the database are described below.

Indicator ID: Alphanumeric code assigned to each indicator

Indicator name: Descriptive name

Formula: Numerator, denominator and multiplier. Formulas are presented as rates, proportions, or frequency counts. Indicators of extremely rare events are presented as rates or proportions if one of these is the conventional manner of reporting. Otherwise they are presented as frequency counts.

Domain: One of the following:

• Health status
• Risk/protective status
• Health and related services
• Contextual characteristics
• Health system capacity and adequacy

Category: Category within the selected domain, as listed in Table 1

Subcategory: Subcategory within selected category, as listed in Table 1

Population groups: Standard age groups for children are <1, 1-4, 5-9, 10-14, 15-19, and 0-19. Standard groups for women are 15-17, 18-19, 20-34, and 35-44. Non-standard age groups and gender strata (for infants, children and adolescents) are recommended when research findings and/or data availability suggest that they should be. The phrase in specific population is used in many formulas and refers to the age/gender categories specified for the indicator.^*

Recent values: The most recent values for each health status indicator stratified by the recommended age/gender groupings are presented whenever available. In many cases, the data available in published sources do not correspond exactly with recommended strata. For a few indicators, values could not be located in published materials, even when a data source was identified (e.g., sickle cell hospitalization rates).

Reference for recent values: Reference for the recent value.

Existing data source, numerator: One or more data sources required to calculate the numerator of the indicator. When more than one source is given, all of them are required. However, an understanding of the information that precedes this entry is important to accurate interpretation. In some cases, data from more than one source should be combined to produce the indicator. In other cases, different data sources are required to calculate the indicator for different population groups.

Existing data source, denominator: One or more data sources required to calculate the denominator of the indicator. When more than one source is given, all of them are required. However, an understanding of the information that precedes this entry is important for accurate interpretation. In some cases, data from more than one source should be combined to produce the denominator. In other cases, different data sources are required to calculate the denominator for different population groups.

If the indicator is based on survey data, both numerator and denominator are from the same data source. To use the values derived from national or state-level surveys in smaller areas, model-based (synthetic) estimation must be used. See Appendix K for more details.

• Routinely available at the local level (e.g., vital records)
• Sometimes available at the local level (e.g., birth defects registries)
• Sometimes available at the local level and collected through periodic national surveys (e.g., hospital discharge data)
• Routinely available at the state level, but not at the local level (e.g., Behavioral Risk Factor Surveillance System)
• Sometimes available at the state level, but not at the local level (e.g., PRAMS, YRBS)
• Only available from periodic national surveys (e.g., NHANES)
• Not routinely or periodically available at any level, requires a special survey

Importance:

• Core: Key health status Indicators that can be generated from existing data sources, and are representative of the major categories of health status (listed in Table 1), the range of age/gender population groups, and the levels of data source availability (e.g., all but the last category above). Core indicators should definitely be monitored on a routine basis in order to provide an ongoing picture of key aspects of the health of mothers and children.
• Recommended: Indicators of health status, contextual characteristics and health systems capacity and adequacy that provide a more detailed view of the health of mothers and children and should be monitored on a routine basis, if possible.
• Optional: Indicators that should be examined when related indicators in the other domains reach unacceptable levels.

Consistency with HP 2000 or OBRA ‘89:

• Same formula
• Same concept but uses a slightly different formula or population group
• Same concept but very different formula or population group
• Not in standards (either HP 2000 or OBRA ‘89)

Rationale for including in the MCH MI: Reason for inclusion

Other references: References for general reading about the importance of the condition represented by the indicator and relevant measurement issues.

Comments: Statements about noteworthy aspects of the indicators, including substantive definitions of conditions, discussions of measurement issues, and observations about any of the characteristics listed above.

Relationships: Associations among indicators. Relationships are noted for health status, risk/protective and health service indicators. In most cases, these relationships constitute a rationale for including the risk/protective and health service indicators in the MI. Other relationships that may be documented in the literature are not noted here because a systematic search for all other relationships was not conducted. The contextual and health systems indicators are extensively interconnected with the others. These relationships are not listed because the lists would be lengthy and repetitious and the relationships are fairly well-known.

-------------

The database presently contains the complete information on indicators for this stage of the project, and it is designed to communicate its potential to users. The database structure is flexible and future-oriented. Additional information is easily added; additional indicators are easily appended; and the information can be easily retrieved, sorted, and formatted for a variety of printed reports. The computer files, themselves, can be polished and distributed to users.

As presently designed, the Filemaker Pro system has four main files:

1. a home file (Home.fm) that orients the user, contains information of the model indicators database and on references, and links to the data files;
1. a health status indicators file (HStatus.fm) that contains all basic information on the health status indicators;
1. a risk/protective factors and health services file (RP_HSrv.fm) that contains all basic information on these factors;
1. a contextual and health system file (Cxt_HSys.fm) that contains all basic information on these factors.

In addition, the database contains an auxiliary file (HStatRel.fm) that holds information on the linkages between the health status indicators and the risk/protective and health services indicators. This file contains one record for each linkage—the indicator ids and indicator names for each pair of linked health status and risk/protective or health services indicators. Thus there are multiple entries for those health status indicators that are linked to more than one risk/protective or health services indicator and vice versa. The ids are entered into this file and the names are added using a lookup function that goes to HStatus.fm or RP_HSrv.fm for the names. If any names are changed in that original file, the change will not be automatically made in HStatRel.fm. To incorporate the change, it is necessary to open HStatRel.fm, select any related id, and then use the relookup command to update this information. The information from this link file is then used within both HStatus.fm and RP_HSrv.fm to incorporate the indicator ids and indicator names of linked variables (on the fly) in the standard report and can be used for other screens and reports as desired.

Finally, there is a small work file (Merged.fm) that brings together some information from the three data files, including indicator id and name, importance code, domain, and category.

Filemaker Pro is also commonly used to prepare data for access via the Worldwide Web. The current version of the database does not take advantage of this potential, but a Filemaker Pro programmer could make the necessary modifications. Since data presentation is best designed with a full understanding of user needs, more extensive development of this database should occur when the future uses and users have been specifically identified.

The MCH Model Indicators are presented in two different ways in this section. First, the indicators are listed by importance, domain, and category in Table 2. This is followed by printouts of all the information in the database for each indicator. The amount and type of information varies by domain. Health status indicators have the most complete descriptions, followed by contextual and health systems indicators. These indicators have importance codes of core or recommended for routine surveillance. Indicators of risk/protective status and health services include relatively less information. They are presented as optional for routine surveillance or investigation of health status indicators that reach levels of concern. For a complete discussion of the rationales for assigning importance levels, see Part 3 of this report

Descriptions of most of the data sources recommended for health status, health system and contextual indicators are in Appendix B. Some of these data sources will also support generation of risk/protective indicators and of a few health services indicators. Most of the health services indicators, however, depend on local data sources. Since the availability of data varies across indicators and geographic areas, users are encouraged to work with vital and health statistics staff at state or local levels to identify which data sources are available to them.

Injury morbidity data are especially difficult to locate at the present time. However, efforts are underway to address this shortcoming. See Appendix C for a description of current initiatives to improve the availability of data on injuries.

Many of the indicators require denominators derived from census data. Since census data are collected every ten years, they become increasingly inaccurate as each decade progresses. States vary in the frequency, complexity, and methodology of their incensal population updates, often based on data from the Current Population Survey. The best method for obtaining good quality population estimates in the age/gender/racial groups required for the MCH Model Indicators would be through continual estimation of the census. A bill to support continual estimation is currently under consideration by Congress.

Several of the MCH Model Indicators represent rare events, which present unique problems when they are used to calculate rates or proportions. This is a particularly prevalent problem in sparsely populated geographic areas and in age, gender or race-specific population groups. In the MCH MI, rare event indicators are stated as rates, percentages or frequency counts, depending on which of these modalities is conventionally used. When the indicators of rare events are stated as rates or percentages, users should take the following precautions, as recommended by the National Center for Health Statistics: The rate or percentage should not be calculated if the numerator is based on fewer than 20 cases. If the numerator is between 20 and 100 cases, the rate or percentage can be calculated but it should be accompanied by a cautionary statement regarding interpretation (Ventura, et. al., 1994). Another approach to dealing with small numerator frequencies is to combine data across time periods and geographic areas. For more information on these topics, see Appendix D of this report, Problems with Rates Based on Small Numbers (Buescher, 1997), and the Technical Appendix of the annual report, Vital Statistics of the United States, Volume I, Natality.

For a number of the health status and risk/protective indicators, the only available data sources are national or state surveys. To use data from these sources to calculate rates or percentages in smaller geographic areas, model-based (synthetic) estimation techniques must be applied. Model-based (synthetic) estimation is a process of applying the rate or percentage of a condition or risk/protective factor derived from a survey to the population in a geographic area of interest, thus producing an estimate of the number of cases of the condition or risk factor within the area. The estimate can be refined by estimating the number of cases with the condition for relevant strata (e.g., age/gender/racial groups) of the population. The resulting estimates for each stratum are then summed to produce an estimate of total cases in a larger segment of the population. An estimated rate or percentage is generated by dividing the synthetic estimate of the number of cases for the population by the true number of people in the population and applying the appropriate multiplier in the formula (e.g., 100, 1000).

The key to using this process correctly is to select strata that are relevant to the condition or risk and available within the local population data base (usually census data for MCH Model Indicators). Model-based (synthetic) estimates should be used with care. It is always better to use data derived from the population of concern. Synthetic estimation is only appropriate when no other option is available, as is the case for some important child health indicators (e.g., anemia, overweight).

Additional guidance and examples of model-based estimation are in Synthetic Estimation Process (PA Department of Health, 1994) and Standardization of Rates and Synthetic Estimation (Rosenberg, 1997) both of which are reproduced in Appendix E. For a more technical discussion, see Small Area Estimation: An Empirical Comparison of Conventional and Synthetic Estimators for States (NCHS, 1979)

The MCH Model Indicators were developed within the framework of a conceptual model (Figure 1) that also serves as a guide to interpretation. To use the model effectively, the core health status indicators should be monitored on a routine schedule, such as annually or semi-annually. If possible, the remaining health status indicators, especially the ones that can be calculated from the same data sources as the core indicators, should also be under surveillance on a routine basis. Also recommended for routine surveillance, if circumstances permit, are the contextual and health systems indicators. To carry out these surveillance activities, users may wish to obtain two other products developed through the MCH Information Resource Center. Indicator Templates developed by the Family Health Outcomes Project provide for easy data entry and straightforward analysis of trends in rates and proportions (FHOP, 1996). Methods for trend analysis and a discussion of related issues are in Trend Analysis and Interpretation: Key Concepts and Methods for Maternal and Child Health Professionals (Rosenberg, 1997).

When any of the routinely monitored indicators achieve levels of concern^*, the associated risk/protective and health services indicators should be reviewed. That review may, in turn, lead to a broader examination of other factors that influence the health status indicator.

Figures 2 through 7 demonstrate how Model Indicators fit together within the conceptual models for six of the core health status indicators. The relationships can be interpreted as follows, using Figure 3 as an example. The neural tube defects (NTDs) prevalence rate is a core Model Indicator that should be monitored on a regular basis. An unacceptably high rate would trigger assessment of indicators of risk/protection for NTDs (e.g., knowledge about folate therapy on the part of women of childbearing ages and providers of primary care) and use of appropriate interventions or services (e.g., periconceptional folate therapy, preconceptional risk assessment). Results of this assessment may lead to questions about the health system in which services are offered (e.g., use of MCH/CSHCN standards of care, mortality reviews, provider availability) and the larger context in which the problem exists (e.g., levels of poverty and educational achievement). With this understanding of several dimensions of the problem of NTDs, decisions can be made about what to investigate further (e.g., why women are not aware of the importance of periconceptional folate therapy) and which specific interventions to implement or modify in order to address the NTD problem.

The close relationships between health status indicators and the risk/protective and health service indicators associated with them are apparent in Figures 2 through 7. These figures also show that indicators of contextual characteristics and health systems capacity and adequacy have broader applications, as demonstrated by the extent to which these indicators are repeated across the figures. Unacceptable levels of these indicators, therefore, may be associated with many risk factors, health service deficiencies and, ultimately, health conditions. For example, a shortage of primary care providers, detected through monitoring indicators of health systems, could affect use of such services as preconceptional risk assessments, folate therapy, smoking cessation, immunizations, primary care for children, and counseling and screening for STDs. Poor utilization of these services may, in turn, influence levels of risk for a variety of health status conditions (e.g., knowledge about folate therapy, smoking during pregnancy, use of condoms) and the extent of the health conditions themselves (e.g., low birth weight at term, NTDs, measles, gonorrhea, asthma hospitalization). This type of review of the influence of a shortcoming in the health system (e.g., shortage of primary care providers) on risks, services, and health status can be used to generate support for addressing the system level deficiency.

 

 

 

Using the MCH Model Indicators in

Conjunction with Other Sets of MCH Indicators

The MCH Model Indicators were developed to satisfy needs for measures that are not met by other sets of indicators. Thus they are intended to complement the other sets and, when appropriate, to be used in conjunction with them. Four highly related sets of indicators were considered carefully in the MCH MI development process. When using the MCH Model Indicators, an understanding of how they complement the other sets of indicators will be useful.

MCHB Performance and Outcome Measures. The Performance and Outcome Measures were under development during the last year of the MCH MI Project. The goals of the two development efforts were different but there is a great deal of consistency between the results of the two projects. Indicators of newborn metabolic screening, up-to-date immunizations at age 2, protective sealants, CSHCN with a medical home, uninsured children, infant mortality, neonatal mortality, and perinatal mortality are essentially the same in the MCH MI and the MCHB Performance and Outcome Indicators. Three other indicators measure the same concepts but use slightly different formulas:

• Very low birth weight rate: MCHB uses <1500 grams; MCH MI uses <1000 grams.

• Adolescent (15-19) birth rate: MCHB uses live births; MCH MI uses reported pregnancies.

• Very low birth weight infants delivered at risk-appropriate facilities: The definitions of risk-appropriate facilities differ.

There are many more indicators in the MCH MI than there are MCHB Performance and Outcome Measures. From the MCH MI, states may choose to select additional ("negotiated") indicators for routine surveillance and reporting. Also, by using the conceptual model as a framework, MCH Model Indicators can be used to explain values on Performance or Outcome Measures.

MCHB Systems Indicators. An MCHB Working Group was developing indicators of health systems relevant to MCH prior to December, 1995. This group produced two documents on System Indicators. One of those documents, System Indicators: Development of Community Performance Measures, recommended development of systems indicators within five categories (called primary measures). The MCH Model Indicators, while not satisfying every concept identified in that document, are very responsive to the following four of the five categories: 1) early identification and referral system, 2) primary and specialized health service network, 3) family satisfaction and quality of care, and 4) assessment, development, and coordination of primary, specialized, and related services. In the MCH MI, indicators in these categories are found in both the health systems and health services domains. The fifth category identified by the Systems Indicator Working Group, family participation, is not addressed in the MCH Model Indicators. The Model Indicators that correspond with the community performance measures may be used as leading indicators of system performance. When problems are identified by monitoring these leading indicators, additional community performance measures should be examined.

Healthy People 2000 and OBRA’89. Many of the indicators in the MCH MI also correspond with indicators in Healthy People 2000 and in the Omnibus Budget Reconciliation Act of 1989. Other Model Indicators are similar but reflect new developments in conceptualization, measurement and/or data source availability. Correspondence between MCH MI and these sets of indicators is coded in the Filemaker Pro database. Tables showing correspondence between the indicator sets are included in Appendix A.

In October, 1994, identification of the components of the MCH MI began under the leadership of the Maternal and Child Health Bureau through a contract with the Public Health Foundation, the Maternal and Child Health Information Resource Center (MCHIRC), and subcontracts with selected universities and organizations represented by the MCH MI Working Group. The first issue addressed by the Working Group was whether to focus on: 1) identification of specific data elements to be collected or 2) indicators to be reported. Either approach would satisfy the broad language in the Overall Strategic Plan for Data Utilization and Enhancement. While there were good reasons to go in either direction, resources were available to pursue only one. To resolve this issue, initial work on the project involved three activities:

• Review of recent and concurrent MCH data development projects. Numerous data development projects, many of them supported by the Maternal and Child Health Bureau, were undertaken in the decade before this project began. Projects were identified through MCHB publications and by self-identification in response to a solicitation placed in relevant newsletters. Each project was requested to submit information describing the nature of its work and the products developed (e.g., data set, data elements, indicators). This solicitation and its results are described in Appendix F.

• Identification of a preliminary set of questions/issues that indicators should address from the perspective of MCH leaders. A survey of MCH and CSHCN directors was conducted in Spring, 1995. These leaders were asked to identify the five most important questions they would like to have answered by a model data set. Thirty-eight responses to this survey were received, representing all regions of the country and a wide range of interests/concerns. A summary of the results and a list of respondents are in Appendix G.

• Consensus Conference, Arlington, Virginia, October, 1995. Fifty individuals representing a diverse range of interests participated in this two-day meeting which focused on the possibility of using the project to specify elements for data collection. A summary of the results of this meeting and a list of participants are in Appendix H.

The results of these three activities suggested that the Working Group should focus on indicators. Other projects like the National Committee on Vital and Health Statistics’ Core Health Data Elements Project (NCHS, 1996), were focusing on data elements. The focus of that committee was not specific to MCH. However, it was clear that before specific data elements in MCH areas could be identified, it was necessary to specify the concepts to be measured and to determine whether the elements required to construct measures of them were available in existing data sources. This could be done while developing indicators. Also, a national level comprehensive effort to develop MCH indicators, had not been conducted for more than a decade (Miller, et. al., 1986). No nationally-accepted set of MCH indicators that was broad enough to characterize the MCH population and could be used to guide development of data bases while also providing a frame of reference for interpretation of maternal and child health data was found.

In January, 1996, during an MCHIRC subcontractors meeting, the decision to proceed with development of indicators for reporting data rather than elements for collecting data was reached (Appendix I). The MCH Model Indicators are the result of this decision. They are a collection of measures that taken together give an indication of the health of mothers and children, and corresponding needs for specific actions. The measures in the MI are primarily rates and proportions, supplemented by sentinel indicators for important but rare events.

The process for development of the MCH Model Indicators had five major components:

As development of the indicators proceeded, it became apparent that each component could not be fully developed for each indicator because of the time-consuming nature of the development process which was complicated by the different functions of the domains in the conceptual model. Identifying concepts in all domains involved a significant amount of time, partly because identification of some concepts was dependent upon identification of others (e.g., risk/protective concepts were derived from health status concepts). Once identified, another investment of time was required to select appropriate measures of the concept, especially if there were many options, and then locate a data source, if one existed. As a result, it was necessary to establish priorities for time and other resource allocation within the limits of the project. Since health status indicators are the foundation of the entire set of indicators, they were accorded highest priority; that is, they were developed first and all of the associated components were completed.

Risk/protective and health service indicators presented a different set of challenges. The conceptual model calls for them to have very close links to the health status indicators. As a result, at least one indicator in each of these domains was identified for each health status indicator, thus producing a large number even though some are associated with more than one health status condition. Also, because of the close linkages with specific health conditions, the extent to which risk/protective and health service indicators need to be monitored regularly as part of a routine surveillance system depends to some extent upon whether the associated health status indicator reaches a level of concern. Finally, the data for indicators in these two domains is, in general, dependent on local data bases which vary greatly across the country. These peculiar circumstances of risk/protective and health service indicators led to a decision to develop them last and to focus greatest attention within these domains on identifying concepts.

The indicators of contextual characteristics and health systems capacity and adequacy are intended to have broader application than the risk/protective and health services indicators. Each of them affects many risk/protective and health service factors and, in turn, health conditions. Moreover, many of them can be constructed from existing, widely available, data bases and should be monitored on a routine basis to assure an acceptable health system infrastructure and to assess demographic changes in population groups that may affect needs for services. Indicators in these two domains were developed after the health status indicators. All of the components were addressed, although additional work in some areas is still warranted.

All of the indicators were coded according to importance using the following scale:

Only indicators with fully developed components were considered for inclusion in the core set of MCH Model Indicators. Thus, the core group consists exclusively of health status indicators. Health status indicators that are not core are recommended. Also in the recommended category are the contextual and health systems indicators. The optional category includes risk/protective and health service indicators. With further development, some of the indicators in the latter four domains would probably be reclassified as core. For example, a risk/protective indicator like unintended pregnancy and a health service indicator like immunization status of 2 year olds may be considered essential for routine surveillance. Thus, continued development of indicators in the latter four domains would lead to some expansion of the core set of 20. However, expansion of the core set beyond a reasonable number (30-40 total) is not recommended.

As indicated above, the indicator development plan included specification of age/gender categories for each of the indicators. However, race and ethnic categories are not specified for each indicator. The indicators should be reported by race and ethnicity: 1) when there is evidence that they would provide information on current or emerging disparities across racial/ethnic groups; 2) if they are being used to monitor potential discriminatory practices; or 3) if there is another good reason to stratify.

To be able to construct indicators by race and ethnic groups, data must be collected in the appropriate categories. In support of the Public Health Service Task Force on Minority Health Data’s recommendation to include the Office of Management and Budget (OMB) standard item on race and ethnicity in all uniform health data sets developed or sponsored by the Public Health Service, the MCH MI Working Group recommends that at a minimum, the racial and ethnic categories soon to be promulgated in the revised OMB Directive 15, be included in all data sets needed to produce the model indicators. A final version should be available in October, 1997. Users who need more refined racial or ethnic groupings are encouraged to collect them in a way that will allow aggregation up to the categories in the revised OMB Directive 15.

This recommendation is made to serve the interests of public health for monitoring population differences in health and related indicators, including emerging and changing disparities, as well as facilitating resource allocation and targeted interventions. It supports, for example, measurement of the disparity between black and white infant mortality rates, one of the MCHB outcome measures. At the same time, the Working Group is mindful of the problems with using race and ethnic-specific indicators. Since racial and ethnic categories are social constructs which have no scientific or anthropological meaning, interpretation of disparities and health differentials by race and/or ethnic groups requires the simultaneous analysis of additional socio-economic indicators (e.g., income, education, social status and class) to further elucidate the meaning of race and ethnicity in society.

One of the major components of an indicator is its underlying concept. Several steps were required to produce the initial set of concepts for the MCH Model Indicators. The first draft of concepts in all five domains was developed by the working group to correspond with key questions identified by MCH directors and other staff members through the Spring, 1995 survey (Appendix G). These concepts were modified with input from MCHB and other MCH leaders who participated in the Model Data Set Consensus Conference in October, 1995 (Appendix H). From these early concepts, working formulas were constructed.

In the Spring, 1996, a series of comparisons were made in order to assure that the concepts and formulas were consistent with key reporting requirements and previous work. First, comparisons were made with indicators derived from OBRA ‘89 and Healthy People 2000 to promote consistency of content and measurement, when appropriate. Subsequent comparisons were made with the following sets of indicators that are currently in use:

• Region IV Network for Data Management and Utilization (RNDMU), University of North Carolina
• Midwest Maternal and Child Health Data Improvement Project (MMDIP), University of Illinois at Chicago
• Family Health Outcomes Project. Outcome Indicators for Public Health Surveillance in a Changing Health Care Environment, University of California at San Francisco
• Arizona’s Population-Based MCH Surveillance System
• Child Health, USA

This step illuminated a number of concepts that were not previously identified and it highlighted differences in measurement of similar concepts. Based on these comparisons, revisions were made and a very large pool of preliminary indicators (>500) was completed. Using a large group of preliminary indicators minimized the probability of omitting a critical concept in any domain. These indicators served as the basis for subsequent refinements and revisions.

The preliminary health status indicators were critically reviewed in September, 1996 by a select group of stakeholders. The reviews were conducted in large part by mail, but some were completed by telephone, and responses from many MCHB staff were obtained during a meeting at the Bureau in October, 1996. Stakeholders were asked to consider the severity and modifiability of each condition, and the likely reliability of the proposed indicator over time and place, as well as to comment on proposed age categories. The original 100 participants in this process included representatives of federal, state, and local health agencies, relevant professional organizations, and selected committees with data development as a primary goal and MCH data as a component of that activity. Participants were encouraged to solicit input from colleagues who were more familiar with some of the concepts and/or formulas than the selected participant. Ninety-nine (99) written responses to the request for review and comment were received. A copy of the request for input and a list of respondents is in Appendix J. Responses to this solicitation played a major role in shaping the final set of health status indicators.

Health status indicators are intended to represent actual health conditions, as measured by incidence or prevalence in a population. For a number of concepts, this intention was not realized. In some cases, no data base with the necessary incidence or prevalence elements could be located. Sometimes, this led to a decision to measure the concept for a subset of the population for which data were available usually representing the most serious form of the condition (e.g., hospitalizations for prenatal complications instead of prevalence of prenatal complications). In other situations, this approach was not feasible, but the indicator of incidence or prevalence was considered sufficiently important that it is recommended without a source of data (e.g., several indicators of childhood nutrition).

A few of the indicators in the health status domain are actually health behaviors. These are behaviors (e.g., smoking, drug and alcohol use in adolescence) that are clearly associated with numerous health conditions, many of which may not appear until later in life (e.g., cancer, cardiovascular disease). These behaviors represent such important aspects of adolescent health that the working group decided to include them among health status indicators in the MCH MI. If they were excluded, key risk factors and service-based interventions for adolescents could also have been omitted from the MCH MI.

In addition to formula and data sources, age and gender population groups are specified for each indicator. Most of the indicators are recommended for one or more of the following groups: children <1, 1-4, 5-9, 10-14, 15-19 and women 15-17, 18-19, 20-34, 35-44. Variations from these standard age groups are recommended if justified from recent data (presented for each indicator) or from research studies. Gender stratification is only recommended if justified in the literature.

The final set includes 75 model indicators of health status. Of these, some are related conditions that can be drawn from the same data source (e.g., 8 vaccine preventable diseases and 9 injury-related hospitalization rates). This feature limits the number of data sources required to calculate the indicators while allowing a full range of MCH conditions to be included.

From the large set of 75 indicators, 20 were chosen for the core set. Indicators in the core set represent important conditions of each MCH age/gender population group. All of these indicators can be calculated from existing data sources and most from sources available at the state or local level. Users are encouraged to calculate and monitor all indicators that can be constructed from readily available data (e.g., all indicators based on vital record data).

The process for refining and revising indicators of risk/protective status and health services differed from the one described above for health status. The preliminary indicators in these domains were generated from questions posed by MCH leaders and reviews of MCH indicators used for other purposes, as described above for health status indicators. However, the preliminary indicators were not linked to specific health status indicators, as prescribed by the conceptual model.

Identifying indicators in these two domains was the focus of one of the two days of the MCH MI Stakeholders meeting in June, 1997. The participants in this meeting were carefully selected to represent a variety of MCH population groups and service modalities and they came from all geographic areas of the country. Local, state and federal levels were represented. A participant list, meeting materials, and summaries of the products of the meeting are in Appendix K.

On the first day of the meeting, participants were divided into four groups, each of which was assigned 15-20 health status indicators. For each of their assigned indicators, participants were asked to identify no more than one indicator of risk or protection and one of health and related services which exerted the greatest influence on the health condition, and was potentially modifiable and measurable. The resulting indicators were reviewed by the full group of participants and a few suggestions for additional indicators were made. Some of the additional indicators do not link directly to a health status indicator but were considered important enough to stand on their own (e.g., newborn metabolic screening).

Following the meeting, these indicators were compared to the preliminary indicators in the same domains. As necessary, relationships identified at the meeting were verified in the literature. Many of the indicators in these domains relate to more than one health status indicator. However, no systematic efforts were made to identify additional relationships between risk/protective or health service indicators and health status indicators, beyond those identified by meeting participants.

Since some risk/protective and health service indicators are associated with more than one health status indicator, developing recommendations for age/gender strata was complicated. As a result, these indicators are currently linked to broad categories of women (15-44), children (including infants and children) and/or adolescents.

While participants in the June, 1997 meeting were charged with developing formulas for the risk/protective and health service concepts they recommended, most time was devoted to debating the value of the concepts. Many of the formulas were developed later, with input from the preliminary indicators. For some, the concept was too vague to develop any further. In all cases, data sources were not specified and, in many cases, more specific definitions of terms in numerator, denominator, or both remain to be defined. As indicated, many of the risk/protective and health services indicators will require local data sources which vary greatly in content and availability In the absence of further development, indicators in these two domains are considered optional in the MCH MI.

Indicators in the health systems and contextual domains are also related to the health status indicators but they do not have the close one-to-one relationships that many of the risk/protective and health service indicators have. The approach to refining these indicators took yet another turn. Preliminary indicators in these domains were examined to identify a small number that would represent population demographics and five characteristics of the health system: availability, acceptability, accessibility, scope, and coordination. Other criteria that were used in selection of these indicators were: 1) relationships with more than one indicator in the other domains and 2) data availability. In addition to the preliminary indicators developed by the Working Group, two other sets of indicators, the Kids Count Data Book (Annie E. Casey Foundation, 1997) and State-Level Data Book of Health Care Access and Financing (Robert Wood Johnson Foundation, 1995) were reviewed to identify indicators in these two domains. Finally, experts in specific areas (e.g., rural health) were consulted when existing resources were not sufficient.

Selecting preliminary indicators in these two domains was challenging. The contextual domain was intended to include both demographic indicators and indicators of environmental conditions that affect risk, health services and health status. However, indicators in the latter category, with fairly available data sources, were not included in the preliminary group. As a result, no indicators of environmental conditions are currently in the MCH MI. With regard to health system indicators, there were many options but all of the categories were not well-represented. In some cases, surrogate indicators were selected to represent a category. For example, the percent of the population who reside in a primary health care professional shortage area is an indicator of availability of health services. It is included in the MCH MI as a surrogate indicator of accessibility because professional shortage areas can exist within geographic areas with acceptable population to provider ratios. In these cases (e.g., low income areas of inner cities), the indicator reflects access as well as availability. The coordination category under health systems presented a different challenge. Coordination of services can be represented by documented agreements among agencies or by the utilization patterns of mothers and children. The latter approach was considered more consistent with the overall MCH MI. Indicators selected to represent service coordination in the health systems domain are also included in the health services domain.

On the second day of the MCH MI Stakeholders meeting in June, 1997 (Appendix K), the proposed contextual and health systems indicators were examined by participants. Again, participants worked in small groups focusing on a few (5-10) indicators. They recommended that each indicator be accepted as written, revised, or rejected. Participants were also invited to recommend indicators that were not already under consideration.

Most, but not all, of the contextual and health system indicators have working definitions and existing data sources. Since some developmental work remains to be done on them, however, none of the indicators in these domains are currently included in the core set of indicators. However, all of the contextual and health systems indicators are recommended for routine surveillance.

The process used to develop the set of model indicators involved a great deal of participation from vested and knowledgeable individuals. It is important to acknowledge, however, that there were many defensible paths the Working Group could have taken and that any of those paths would have led to different decisions about the indicators in the MCH MI. Nevertheless, the indicators recommended are representative of the conditions faced by MCH populations in 1997 and factors that contribute to those conditions. Many of the model indicators can be calculated from existing data sources. Yet some require new data sources or changes in existing ones. By including indicators that cannot at present be constructed from available data sources, the MCH MI offers direction for the future.

The Model Indicators in this report represent a major step forward in the long-term process of MCH data utilization and enhancement. As such, they can and should be disseminated and implemented extensively by health and related units with interests in maternal and child health. At the same time, however, the next steps in improving the MCH MI should be undertaken so that while the field gains experience

with a model set of indicators, the set itself is being improved. The following recommendations would accomplish both of these ends.

1. Complete the remaining steps for development of the indicators of risk/protective status, health services, health systems, and contextual characteristics (e.g., standardize all formulas and definitions, recommend data sources, specify age/gender groups for risk/protective and health services indicators).
2. Select core indicators in these domains (e.g., risk/protective status, health services, health systems, and contextual characteristics) and revise recommended and optional categories accordingly.
3. Consider expanding contextual characteristics to include indicators of environmental factors that affect maternal and child health, and health systems indicators to address family participation in children’s health services, as well as other concepts, suggested by participants in the stakeholders meeting, that remain undeveloped.
4. Complete the identification and coding of all relationships among indicators.
5. Finalize the Filemaker Pro database so that it is readily useable by a defined group of users.
6. Complete the conceptual models for all core indicators, at a minimum.
7. Develop a package of materials and an educational plan for dissemination of the MCH MI.
8. Select a few organizations that represent different aspects of MCH services (e.g., state, city, and rural health departments, health centers, insurance plans) and work with them to develop a pilot test of the MCH MI. The pilot should include collection, interpretation, reporting (written) and presentation (oral) of the Model Indicators.

The MCH Model Indicators are intended to be responsive to the information needs of the MCH community. To this end, two additional recommendations are offered:

1. Provisions should be made to ensure regular reviews and updates of the MCH MI.
2. The MCH MI should be recognized and promoted by MCHB as the foundation for development of other sets of indicators intended to provide more detail on specific aspects of maternal and child health (e.g., effects of managed care on maternal and child health).