Patient-Reported Outcomes Measurement Information System (PROMIS)

PROMIS - Sexual Function

Subdomains

• Perceived sexual attractiveness
• Sexual Function
  • Activities
  • Therapeutic aids
  • Sexual response
    • Desire
    • Erectile function
    • Vaginal function
    • Anal discomfort
    • Orgasm
• Interference from cancer-related symptoms
• Satisfaction

Measure Development

• Item review (1136 items)
• Sixteen focus groups (n=109), 30% minority
• Cognitive interviews (n=39), 35% minority, 40% low literacy
• Web or telephone survey (n=500, about 80% complete as of 12/15/2008), 15% minority
• All samples included men and women with a diagnosis of breast, lung, colorectal, prostate, gynecological, or other cancers, in or before treatment for early and late stage cancers as well as recent and long-term survivors.

PROMIS - Sleep/Wake Function

Subdomains

• Factors affecting sleep
• Sleep parameters
• Daytime functioning
• Satisfaction

Measure Development

• Item review (670 items)
• Ten focus groups (n=67), 30% minority
• Cognitive interviews (n=18), 35% minority, 40% low literacy
• Web or telephone survey (n=500, about 80% complete as of 12/15/2008), 15% minority
• All samples included men and women with a diagnosis of breast, lung, colorectal, prostate, hematological, or other cancers, in or before treatment for early and late stage cancers as well as recent and long-term survivors.

PROMIS Illness Impact Item (II)¹ Bank

Banks

• Negative Illness Impact (n=40)
• Positive Illness Impact (n=46)

Subdomains

• Self-Concept
• Social
• Stress Response/Coping
• Meaning & Spirituality

Bank Development

• Expert review of items (n=89) from prior item banking grant (“SyMon 1”)²• Domain-level cognitive interviews (n=40), 45% minority
• Two focus groups (n=12), 50% minority
• Additional item review & item writing sessions to fill content gaps, augment subdomains, and balance item count across subdomains
  • 52 total items added
• Item-level cognitive interviews (n=20), 75% minority
• Item format change based on cognitive interview data
• Additional cognitive interviews (n=20) to determine best item format for field testing, 10% minority
• Field testing, web or telephone survey (n=500, 91% complete as of 12/15/2008), 13% minority
• All samples included men and women with a diagnosis of breast, lung, colorectal, prostate, gynecological, or other cancers, in or before treatment for early and late stage cancers as well as recent and long-term survivors.

PROMIS Perceived Cognitive Function Item Bank

Banks

• Cognitive impairments (Negative PCF) (n=42 items)
• Cognitive abilities (Positive PCF) (n=36 items)

Content areas

• Mental acuity
• Concentration
• Memory
• Verbal fluency
• Multi-tasking/interruption
• Interference
• Functional change
• Comments from others
• Impact on QOL

Bank Development

• Expert review of items (n=53 items) from prior item banking grant (“SyMon 1”)
• Domain-level cognitive interviews (n=40 participants), 45% minority
• Item-level cognitive interviews (n=15 participants), 67% minority
• New items written (n=25 items) based on analyses from previous field testing, qualitative data, and expert input
• Item-level cognitive interviews on newly-written items (n=5), 100% white, non-Latino
• Field testing, web or telephone survey (n=500 participants, about 90% complete as of 12/15/2008), 14% minority
• All samples included men and women with a diagnosis of breast, lung, colorectal, prostate, gynecological, or other cancers, in or before treatment for early and late stage cancers as well as recent and long-term survivors

¹II = Illness Impact
²SyMon1 refers to a project led by Dave Cella (funded through a NCI R01 grant) to develop a symptom (“Sy”) monitoring (“Mon”) system for cancer patients.

PROMIS Pediatrics Item Bank Development: Brief Summary Report

Qualitative Assessment Pre-Large-Scale Testing

Aims:

• Focus Groups: To identify themes identified by children to describe their experiences in physical, mental, and social health.
• Cognitive Interviews: To test initial items for content validity and clarity among the general pediatric population and children with asthma
  • Finalize items for large-scale testing

Methods:

• Focus groups
  • Cognitive interviews (2 rounds – second round of CI’s completed after initial item assessment and revision)
    • Total of 318 items tested; each respondent answered approximately 30 items
      • 6 domains: Physical Functioning, Pain, Emotional Health, Social Health, Asthma, Symptoms and Fatigue
      • All items were reviewed by at least 5 respondents
    • Results analyzed by researchers and 35 items were revised and re-tested in additional cognitive interviews

Recruitment:

• University of North Carolina at Chapel Hill, The Children’s Hospital at Scott and White Hospital in Temple, Texas
• Assessed medical records at general pediatric clinics and pediatric pulmonary clinics
  • In-person (focus groups)
  • Mailed letters to patients to recruit for (cognitive interviews)
• Ages 8-17
• Speak and read English

Focus Group Sample:

• • n=41, 8 focus groups total
• • 20 from general clinical population, 21 children with asthma
• • Racially and ethnically diverse (for details, see reference below)

Cognitive Interview Sample:

• • n=77
• • Racially and ethnically diverse (for details, see reference below)

Reference:

• Walsh, TR, Irwin, DE, Meier, A, Varni, JW, DeWalt, DA. (2008). The use of focus groups in the development of pediatrics item bank. Qual Life Res, 17(5), 725-35.

Independent Pediatrics Project – Large Scale Testing

Aims:

• Confirm factor structure of the domains
• Evaluate items for local dependence and differential item functioning (DIF)
• Create item calibrations for each domain
• Estimate profile scores for children with asthma

Methods:

• Cross-sectional administration of the computerized survey
• Tested pool of 293 PROMIS items, legacy questionnaires, and asthmas-specific items
  • The 293 items were divided among 6 testing forms: each respondent answered a maximum of 76 items on the forms

Recruitment:

• University of North Carolina at Chapel Hill, Duke University, The Children’s Hospital at Scott and White
  • In-person recruitment at general pediatrics and specialty clinics
  • Over-sampling of asthma population
• Mailed recruitment letters to children enrolled in after-school programs and health classes in the Chapel Hill Carrboro Public School System
• Child participants must speak and read English and be able to complete a computer administered survey

Population:

• n=4,129 children
• Ages 8-12: 55%
• Ages 13-17: 45%
• White: 58%
• Black: 23%
• Other: 10% (Asian/Pacific Islanders, Native American and other races)
• Hispanic ethnicity: 17%
• Chronic illness diagnosis in past 6 months: 28%
• Ever diagnosed with asthma: 29%
  

Findings

• Item banks: Physical Function Mobility, Physical Function Upper Extremity, Social Relationships with Other Children, Depressive Symptoms, Anxiety symptoms, Anger, Pain, Fatigue, and Asthma Symptoms
• Analyses of item banks are almost complete and it is anticipated that items, short forms, and CAT will be available at the Assessment Center by March or April 2009
  

Parent Proxy-Report for Children

Goal: to develop and calibrate a parent-proxy report version for the PROMIS pediatrics item banks for measuring child health ages 5-17.

Aims:

• Convert PROMIS pediatric items into parent proxy-report items and confirm clarity and understandability with cognitive interviews
• Administer the parent proxy items to a large sample of parents while their children simultaneously complete the PROMIS pediatrics items
• Perform psychometric analysis to understand item characteristics and identify items with inconsistent responses between parents and their child

Methods:

• Cognitive interviews with parents were conducted to elicit qualitative data to develop items for the 5 to 7 year-old age group
• Cross-sectional administration of the survey instrument of parent-child dyads

Domains Tested:

• Physical Function, Fatigue, Pain, Emotional Distress, Social Health and Asthma
• Parents and children answered approximately 80 items each

Recruitment:

• University of North Carolina at Chapel Hill, Northwestern, University of Washington, University of Texas, Duke University
• In-person recruitment: pediatrics and pediatric asthma clinics
• Participants must speak and read English and be able to complete a computer administered survey

Sample:

• 25 parents (cognitive interviews)
• 1748 parent-child dyads (large scale testing) as of 1.2.09
  • 369 children with asthma
  • 1379 children presenting for general health care visits
  • Ages 5-7: n=340
  • Ages 8-17: n=1408

Current Status:

• Data collection will conclude early 2009
• Psychometric analysis will begin in early 2009

Supplement to Expand PROMIS Testing in Pediatric Chronic Disease Patients

Goal: To enrich PROMIS pediatric item banks, test items among a clinically diverse population, and obtain factor, calibration, and validity data

Aims:

• Evaluate the results of analyses from the existing Wave 1 pediatrics item bank testing and determine banks that need enrichment with new items
• Perform testing of the PROMIS Pediatrics item banks in selected populations to confirm factor structure and item calibrations obtained in Wave 1 testing (using whole bank administration) and establish preliminary validity data in a diverse group of pediatric chronic disease populations using short forms across all domains
• Integrate documentation of the PROMIS Pediatrics project into the PROMIS website and Assessment Center

Methods:

• Conduct up to 20 cognitive interviews to enrich the current item banks
• Cross-sectional computerized surveys to be completed by 1900 children with chronic illnesses (see populations below)

Domain Item Banks Tested:

• Physical Functioning (mobility and upper extremity), Pain, Fatigue, Depression, Anxiety, Anger, Social Relationships.
• Children will complete a unique combination of whole banks and short forms to provide additional data for whole bank analysis and profile scores across all illness groups.

Recruitment:

• Cognitive interviews: Performed at University of Washington site
• Surveys: procedures will vary based on recruitment site
  • 10 Collaborative sites, over 20 sub-contracted data collection sites across the country
• Participants must speak and read English

Populations:

• Large-scale survey:
  • Ages 8-17
  • Rheumatic diseases (500)
  • Chronic kidney disease (500)
  • Sickle cell disease (300)
  • Cancer (300)
  • Obesity (200)
  • Long-term physical rehabilitation (100)

Current Status:

• Data collection to begin April 1, 2009

Development of PROMIS Physical Function banks appropriate for users of Assistive Technology

In 2008, the University of Washington Center on Outcomes Research in Rehabilitation received supplemental funding from the NIH Roadmap Initiative to assure that the PROMIS Physical Function (PF) banks are appropriate and relevant to users of wheelchairs, mobility aids and other assistive technology (AT). Items in existing PROMIS PF banks were reviewed by experts and divided into two sets: those that were relevant and appropriate to users of assistive technology and those that were not. For instance, items that reference ability to walk or run have been flagged as potentially inappropriate for wheelchair users. New items were added to assure that all aspects of PF that measure concepts most relevant to users with disabilities were included. These items went through several rounds of reviews starting with expert panels and ending with cognitive interviews with individuals with disabilities.

The expert panels were composed of physical therapists, occupational therapists, and psychometricians, and discussed conceptual issues related to clinical and self-report measures of physical function in individuals with disabilities. Expert panels defined four qualitatively different groups of users of assistive technology for data collection purposes (users of power chairs, manual chairs, scooters, and upper body assistive aids) and agreed to define physical function in AT users as a function of a person using the appropriate technology. Recommendations of expert panels were used to make decisions about retaining, changing or deleting items before conducting cognitive interviews with users of AT.

Cognitive interviews were conducted with users of AT living with multiple sclerosis (MS), spinal cord injury (SCI) and arthritis to evaluate functioning of the items. Two rounds of cognitive interviews were conducted when items needed to be modified. In the first round, each new item was reviewed by at least eight individuals, including at least one person with MS and one person with SCI, one user of lower body AT devices, and one user of upper body or trunk devices. If changes were proposed, two to three individuals reviewed each modified item in the second round. Based on the results of the interviews, the candidate item bank for data collection was developed.

The set of new/modified items reviewed by expert panels and tested in cognitive interviews, and a set of unchanged (original PROMIS PF bank) items will be administered to 200 participants with MS, 200 with SCI, and 200 with Arthritis. The data collection will finish by February 2009. Out of this sample of 600 participants, we targeted at least 200 users of assistive technology and 200 who do not use AT. A subset of original/unchanged items (those that do not exhibit differential item functioning) was also administered and will be used to calibrate the new items to the same metric as the original PF items. The final product of this supplement will be a set of alternate items that are meaningful in the context of AT use while providing a PF score for users of AT on the same continuum as the PF score for users without disabilities. Psychometric analyses will follow the PROMIS statistical plan and will include evaluations of IRT assumptions (dimensionality and local independence), differential item function, and fit to the graded response model. Short forms targeted for users of AT will be available in Fall 2009. Modified banks will also be available for computerized adaptive testing via the PROMIS Assessment Center.