Step 2: Fostering a Research Culture

In Step 1, you assessed your organization's current needs and capabilities. In Step 2, you will learn about fostering a culture of health services research by:

• Obtaining departmental and institutional leadership support.
• Establishing formal documents and policies in support of health services research.

A wide variety of indicators demonstrate improvements or shifts in organizational research culture. Organizational culture is defined by formal documents and policies, as well as informal traditions and attitudes. Examples of indicators we found in our research are shown in Figure 4.

Figure 4. Indicators of a shift in research culture

Types of Indicators
Changed performance evaluation system to recognize the importance of collaborative health services research Increased institutional support of health services research through Supervisor support Release time Incentives and rewards for research participation Institutional funds to support a health services research center, program, or core office	Established internal partnerships to conduct health services research Increased staff/faculty skills and involvement in health services research Held regular meetings (for project and other interested faculty/staff/students) to discuss health services research topics Established new educational curricula related to health services research Established a central research oversight committee

Obtaining departmental and institutional leadership support for your capacity-building project and its activities will facilitate future initiatives. When seeking internal or external funding, inform the organizational leadership about your activities. For external funding, include a letter of support from an administrative leader to illustrate organizational commitment to your project. Keep administrators involved by providing updates on your progress and engaging them in the infrastructure-building process. This may facilitate support that is conducive to health services research. Support may include physical space, institutional funding of infrastructure, additional release time for conducting research, and changes to organizational policies and practices.

Creating a culture of research can be difficult and often takes time. One of the key approaches to creating a shift in culture is through policy and leadership. Significant policy changes that can affect culture include:

• Formal documents such as mission statements and strategic plans at departmental or institutional levels to support health services research.
• Committees or staff positions that explicitly support research (such as a staff research council or VP of research).
• Policies and practices regarding release time for conducting and preparing proposals and completing research projects.
• Recognition or reward for impact in the community (policy changes, improved health care delivery systems).
• Performance evaluations and rewards (raises, promotions, and/or tenure) that place value on obtaining outside research funding, conducting research, and disseminating research in the community and through scientific journals.

Return to Contents

Maximizing Success

There are common challenges that arise when developing an organizational culture that encourages health services research. They include:

• Lack of organizational knowledge — Provide an engaging summary of what was found in your assessment to organizational leadership.
• Skepticism about health services research — Provide information to show how the department or organization's existing experiences can be used to leverage and build an interdisciplinary team in the field of health services research. Share current articles from prominent journals. A list of the leading health services research journals can be found in the sidebar.
• Problems related to a teaching university becoming more research oriented — Examples of problems include needing to work out research release time issues and needing to educate senior administrators about the appropriate use of research funds. Use monies from research to leverage release time and share the budget and work plan with administrators. Set a schedule for research and non-research activities in advance and work hard to stick to it.

You should begin to think about how you can overcome these barriers in your own organization. One step you can take is to share articles from top health services research journals with leadership.

Return to Contents

Step 3: Planning the Infrastructure Support

Cookeⁱⁱ suggested six principles that must be in place for research capacity-building to be effective. These six principles have been incorporated into the Guide to ensure that the activities you plan to implement are successful. Figure 5 maps the crosswalk between Cooke's six principles and the associated steps in this Guide. We have included additional steps in this Guide that we believe are of comparable importance.

As previously discussed, it's important to tailor activities for your organization. The goals you set out for developing infrastructure support will help identify activities to prioritize in planning the expansion of your health services infrastructure capacity. However, your goals may need to be modified to fit the requirements of a specific funding announcement.

When setting out goals be sure that they are:

• Achievable and realistic, given your assessment of your organization and planned activities.
• Observable and measurable.
• Meaningful to your organization.

Figure 5. Crosswalk of effective research capacity-building principles and steps described in the guide

Cooke's Six Principlesii	Coverage in the Guide
1. Investment in infrastructure	Step 3—Planning the Infrastructure Support, Creating a Research Center
2. Building skills and confidence	Step 3—Planning the Infrastructure Support, Building Research Skills Step 3—Planning the Infrastructure Support, Assembling an Advisory Group
3. Developing linkages and partnerships	Step 3—Planning the Infrastructure Support, Establishing and Sustaining Partnerships Step 3—Planning the Infrastructure Support, Assembling an Advisory Group
4. Ensuring research is applicable and “close to practice”	Step 1—Assessing Your Organization's Needs and Capabilities Step 3—Planning the Infrastructure Support, Building Research Skills: Developing and Supporting Individual Research Projects Step 5—Evaluating the Infrastructure Support Initiative
5. Developing appropriate dissemination of research	Step 4—Communicating and Reporting
6. Building elements of sustainability and continuity	Step 6—Planning for Sustainability

Based on a review of the literature, our research on two AHRQ programs and interviews with staff from Federal research capacity-building programs, we suggest engagement in each of the following four activities:

1. Creating a research center.
   1. Developing a thematic focus.
   2. Selecting a principal investigator or research director.
   3. Acquiring research support staff.
   4. Developing management support staff.
   5. Attaining research facilities and equipment.
2. Building research skills.
   1. Providing training and educational opportunities.
   2. Mentoring.
   3. Developing and supporting individual research projects.
3. Establishing and sustaining partnerships.
4. Assembling an advisory group.

It will be important to monitor progress and periodically conduct goals- and process-based evaluations to identify any needed adjustments or changes to goals or activities. Monthly internal monitoring via meetings and annual reports to the funding agency are the most common means of monitoring progress.

Return to Contents

Creating a Research Center

A research center is defined as a group of investigators who collaborate on research in a common topic area, operate as a formal entity within the structure of the organization, and generally share physical space. A research center may be a less formal or decentralized collaborative of investigators. It is the infrastructure support that a research center or collaborative offers that is fundamental to the success of a health services research program. The research center houses both administrative and technical support that can serve multiple projects.

In developing your research center, there are five basic areas to consider:

1. Thematic focus.
2. Principal investigator or research director.
3. Research support staff.
4. Management support staff.
5. Research facilities and equipment.

Developing a Thematic Focus

Ideally, a health services research center will have a thematic focus: the research should feature a specific topic area, population, or research method. A thematic focus allows you to develop a critical mass of staff with expertise in similar content and methods. Further, commonality allows you to target training opportunities that can serve everyone and mentors that can serve multiple mentees.

Selecting a Principal Investigator or Research Director

If this initiative or project is sponsored by an outside funding institution, that institution will set the requirements for a principal investigator (PI). Funding agencies require that the PI have appropriate credentials to demonstrate that they have the requisite knowledge, skills, and abilities to carry out the proposed work.

It may be beneficial to select an experienced PI because experienced PIs have the most success at garnering additional funding, publications, and presentations. An experienced PI can be someone at mid-to-senior level of his or her career, with 5 to 30 years of experience in research. It is not necessary that their field of expertise be within a specific area, but it is important that the PI have experience leading projects and have a modest rate of publications. Less experienced staff can work towards becoming a PI. Consider structuring the initiative to be supported by a senior PI and a more junior Co-PI.

Another particularly important aspect to consider when selecting a PI is to choose someone who intends to stay with the funded project throughout its entirety. While other project staff turnover has minimal effect on the success of a capacity-building project or initiative, PI turnover can cause great turbulence and has a much greater impact on the success of a project.

Acquiring Research Support Staff

Research support staff may include biostatisticians, research assistants, graduate students, and technical writers. Biostatistical support is one of the most critical factors to the success of a project designed to build health services research capacity. Typically, organizations hire a new staff member or use project funding to support an existing staff member's time. The biostatistician may provide input on study design and training or assist in managing electronic data, conducting analyses, and writing results. It's beneficial to have a biostatistician who is skilled in the craft and can effectively communicate complex ideas to fellow researchers. Further, since research is project-based, the biostatistician should be team-oriented.

Research assistant and graduate student support is generally provided by part-time help that is hired or by existing staff and whose time is supported by project funding. This support is important because it creates a broader interest in health services research. The research assistants and graduate students gain new knowledge and skills. The researchers and the center reap the benefits of affordable, skilled labor. In many cases, research support staff members go on to build expertise and help cement a foundation of health services research at the organization.

Technical writing support can facilitate dissemination of research findings and development of proposals and grants. Technical writing support can come in the form of a science writer, grant writer, or editor. If a designated writer is unavailable, identify someone to serve in this role on a case-by-case basis. Consider experienced faculty with an extensive publication history or a consultant with similar capabilities. The technical writer may: help develop a hypothesis, assist in outlining the document, review the document, edit the document, write grant applications, and pull together sections of a report, article, or proposal.

Developing Management Support Staff

Staff within the organization should have explicit responsibilities to help manage the research center and program. Staff support is needed to assist in management of funding efforts, project performance, and finances.

Managing Funding Efforts

Management staff should identify funding opportunities and disseminate information about these opportunities throughout the organization or center. Staff should also be available to assist with all aspects of the grant/proposal preparation process. Preparation includes a number of activities.

• Creating boilerplate. Boilerplate is text that can be reused in new contexts or applications without the need for substantial change from the original. For example, a useful piece of boilerplate might include background information on your organization and its capabilities.
• Accessing past proposals and grant applications. Management staff should develop a system to facilitate access to sections of past proposals and grants that might be useful in subsequent attempts at funding. Consider developing a database that will help identify relevant sections from your ongoing funding efforts. Relevant sections can include sections discussing specific research methodologies or content areas.
• To facilitate access, all proposals and grants should be electronically accessible by project staff.
• Managing the proposal/grant process:
  1. Arrange release time for staff to participate in proposal development.
  2. Clarify roles of proposal staff and create a staffing plan:
     • Who will lead the proposal?
     • Who will manage the proposal?
     • Who will be responsible for reviewing the proposal?
     • Who else is on the proposal team: writers, editors, production team, accounting team?
  3. Create a proposal timeline. Set deadlines for drafting materials, formatting, review, production, and delivery. Use a Gantt chart, list, or calendar format. Assign specific staff to each activity.
  4. Notify all staff of upcoming efforts. Notify writers, reviewers, and staff included in the proposal, budget staff, editors, production staff and any other staff responsible for preparing or approving the final proposal about key dates in the timeline.
  5. Contact the funding agency for answers to questions and to obtain input on your proposal development before beginning to write, if possible.
  6. Share a proposal outline and writing assignments with staff. As appropriate, include page limits for each section.
  7. Coordinate subcontractors and partners. Provide details and examples of what materials are needed. Set deadlines in advance of internal organizational deadlines.
  8. Gather material for references, biosketches, résumés, CV's, and other appendixes. Collect, appropriately format, and reference all appendixes in the proposal.
  9. Draft the proposal sections. When final editing is complete, the proposal should read as if it is the work of a single author.
  10. Prepare the budget. Format the budget as requested in the procurement.
  11. Arrange for review of the proposal. Address any issues that come up in the review process and finalize the proposal.
  12. Print and assemble the proposal. Gather all materials (binding materials, etc.) and prepare labels for the delivery boxes. Be sure to double-check everything.
  13. Clarify deadlines, submission method, and submission details. Plan for contingencies and obtain a record of submission.

Project and Performance Monitoring

Management staff and systems are necessary to help keep projects running effectively. Management staff will use tools such as timelines, organizational charts, and work plans to achieve their goals. In particular, staff must be able to manage:

• Organizing meetings and training. Schedule and handle the logistics such as advertising training sessions, acquiring space, setting up for training sessions, and paying trainers.
• Serving as coordinator of staff communications to ensure flow of activities.
• Monitoring progress of individual research projects and the overall research center program, including the gathering of information about progress and challenges. This can be done informally through staff meetings or more formally by requiring submission of regular status reports.
• Setting deadlines. Some deadlines will be set in advance as determined by your organization's management or the funding agency. However, it may be important to set intermediate deadlines.
• Ensuring that reporting requirements are being met.
• Managing finances. Systems may need to be put in place to ensure that:
  • Invoices are sent with the timing and format required.
  • Payment is addressed to and received by the proper office.
  • Payments received are timely and in the correct amount.
  • Invoicing does not exceed project funding.
  • Staff are appropriately billing their time.

Attaining Research Facilities and Equipment

• In addition to obtaining personnel support for the research center, there are physical needs that must be met.
• Space and facilities are needed for getting together as a team of researchers.
• Computer equipment could include both hardware and software necessary for collecting, storing, and analyzing data.
• Data protections and security policies and procedures must be in place for managing data.
• Literature search capabilities are needed for accessing library databases and journals for literature reviews and environmental scans.

Maximizing Success

Creating a research center can be one of the most difficult activities to fund and sustain.

• Consider applying for funding from foundations to establish a research center.
• Use funds from multiple projects to support the staff and activities of the research center.
• Obtain institutional funding to support the research center.
• Ensure organizational commitment to the program.
• Select a thematic focus for the center.
• When choosing a PI or research director, consider selecting someone who is likely to stay on the project, has previous experience as a senior investigator on research projects, and has established a strong publication track record.
• Ensure that the PI will be provided ample release time to serve the program

Return to Contents

Building Research Skills

To develop a cadre of health services researchers, you will need to build their skills through training and education, mentoring, and experience on research projects.

Providing Training and Educational Opportunities

A key to developing enthusiasm for health services research is to stimulate staff and student interest in the topic through training and educational opportunities. Share opportunities to learn more about health services research widely with others in your organization. You will need to create a training and educational plan. Factors to be considered include:

1. What kind of training is needed and for whom? How will you decide on the topics? Use the assessment to target and tailor training resources to what trainees need. Training can be varied depending on the needs of individuals and can potentially include areas such as:
   • Health services research, the health care delivery system, and policy (background about the field).
   • Content specific areas (e.g., access, cost and financing of care, quality of care, patient safety, health information technology, outcomes of care, comparative effectiveness research).
   • Study design and research methods (e.g., survey design, large-scale database analysis, retrospective and prospective quasi-experimental designs, statistical analysis, qualitative methods).
   • Scientific and grant writing.
   • Research ethics and protection of human subjects.
   • Research software.
   • Managing grants.
2. How will you communicate training opportunities to those in your organization?
3. Who will provide the training (internal experts, advisory group members, mentors, partners)?
4. How will the training be conducted?
   • On-site, on-line, or off-site? Participation in conferences, workshops, short-term training programs?
   • Formal or informal?
   • Didactic or discussion format?
   • Classes, seminars, or journal clubs?
5. What is the frequency and duration of the training needed?
6. How will you evaluate the appropriateness of the training for conveying needed skills and competencies?

Mentoring

Our research, as well as work by Zea and Belgrave,ⁱⁱⁱ found that mentoring was an integral part of successful health services research capacity-building. We recommend that you include mentoring as part of the overall initiative and its component research activities. A mentoring plan should include information about the nature of the mentoring arrangement(s) that will be used. Specifically, a plan should include a description of skills and characteristics you are looking for in a mentor, expectations of mentors/mentees, the setting for meetings (one-on-one, pairs, groups), and the frequency/duration of the meetings.

We found that it was useful for mentees to have access to a day-to-day on-site mentor and an expert scientific mentor. The day-to-day on-site mentor provides guidance on managing the organization-specific culture and administrative issues, helping with time management, and overcoming challenges. The expert scientific mentor, who may be located off-site, provides guidance and instruction on research design and methodologies, dissemination of work via presentations and publications, and opportunities to obtain additional research funding.

Mentoring can occur one-on-one, in pairs, or in groups. One method we observed had individual investigators meet with the PI, another senior advisor, or an outside expert consultant on a one-on-one basis. Other mentoring plans served mentees in pairs or groups. Each mentee was mentored by two or more senior or technical staff (such as the PI, a co-PI, biostatistician, and/or scientific writer). There is a “second level” of mentoring that often occurs on research projects whereby mentees provide mentoring to junior faculty, staff, and students.

Mentoring should not be considered as an incidental activity. It is wise to formally train mentors and to acknowledge their importance. Adequate support for both mentors and mentees, whether paid or in-kind, demonstrates its value and importance to participants. The support can also provide an incentive for prioritizing the activity.

Developing and Supporting Individual Research Projects

The goal of your research infrastructure is to facilitate the development and support of research projects.

Selecting Individual Research Projects

Promote commonality in the content area of the individual research or small team projects and, to the extent required, in the research methods. This allows you to develop a critical mass of staff with expertise and similar interests, which strengthens your team.

Managing Successful Research Projects

The following factors contribute toward the success of individual research projects:

• Instituting specific strategies to promote communication and networking among project staff.
• Increasing individual investigator research capacity rather than developing it anew.
• Developing large secondary databases and expertise in working with these databases.
• Obtaining protected release time.
• Maintaining commonality of content and/or methods across projects.

One of the challenges in managing research teams can be staffing changes. Organizations should be cognizant and plan for contingencies accordingly. Turnover, when properly managed, does not impede success.

Maximizing Success

• Training and educational opportunities:
  • Develop training plans based on the needs of the staff and the financial and human resources available to you.
• Mentoring:
  • Appropriately match the skills and knowledge of mentors and mentees. In our experience, when mentors and mentees were mismatched in terms of their overall experience, seniority levels, or areas of expertise, the success of the project and career development were jeopardized.
  • Structure the mentoring relationship as best fits the personality and organizational culture of those involved. Consider one-on-one, pairs, and group mentoring settings. More structured mechanisms (such as formally scheduled meetings) may ensure project progress.
  • The time commitment for a mentor is often greater than it might seem initially. Take into account the time commitment of mentoring activities and provide financial support to both the mentor and mentee.
• Individual research projects:
  • Provide opportunities for staff to participate in research projects. Exposure to research projects will build practical experience for your staff.
  • Select projects with a thematic focus — either topic area or methods — to reap the advantages of specialization.

Return to Contents

Establishing and Sustaining Partnerships

In our research, partnership development was the most frequently occurring project activity and also was viewed as the most important project activity. Multiple partnerships are encouraged. Partnerships can be intra- or inter-organizational. Yanagihara, Chang, and Ernst^iv suggested that smaller organizations “form collaborative partnerships with strong research groups at majority institutions” in order to gain access to the equipment and research personnel necessary for increased capacity.

The support of partners can be effective in the immediate future and in the long-term. For example, without state-of-the-art equipment, emerging institutions struggle to attract senior-level talent. Research shows that “at least three RO1-level funded investigators with expertise and track records in the proposed research are required to demonstrate a genuine need for such equipment.” Partnerships offer the immediate benefit of access to needed equipment. In the long-term, a resource-sharing strategy may strengthen an emerging institution's ability to attract future research talent, increasing its own capacity for funding.^iv

Partners may provide support in different ways:

• Collaborating on the conduct of research and manuscript production.
• Providing training and mentorship.
• Providing access to resources.
• Offering access to data or a study population.
• Offering clinical or technical expertise.
• Serving on advisory groups.

As mentioned in Step 1, there are different types of partners that can fill different needs.

Challenges to Maintaining Partnerships

• Resource-sharing inequities. It is important that PIs closely monitor equitable distribution of funds and tasks among partners.
• Accountability. It is important to set clear expectations for the work products and processes.
• Diverging priorities. Partnering organizations are likely to have different goals, priorities and cultures. For example:

Faculty incentives to perform even similar activities may be quite different at each institution. A teaching-intensive institution may value and reward the number of courses taught, teaching awards, and number of students advised. Research-intensive institutions may value publications, dollar value of grants brought, in and percentage of salary covered by external funds. All faculty wish to develop and succeed, but the criteria for success may differ between institutions.^v

It is important that each organization recognize those goals and priorities at the outset and find ways where all partners can both benefit from the collaboration.

• Data ownership. It is important to assure that all partners have some ownership of the data and that authorship of journal articles is determined in a fair and generous manner.

Keys to Sustaining Partnerships

• Additional funding. Search for funding opportunities where you and your partners can continue your relationship with additional funding.
• Incentives for partnerships. For example, position yourself within partnerships by bringing some expertise, equipment, access to research populations, or materials to the table that is unique to your organization. Promote opportunities for shared journal article authorship.

Maximizing Success

• Keep an inclusive perspective when it comes to issues of data ownership and authorship with partners.
• Establish expectations of partners.
• Encourage future partnerships by:
  • Including your partners in funding applications.
  • Providing a unique skill or capability.

Return to Contents

Assembling an Advisory Group

Advisory groups can serve multiple purposes. They are particularly valuable for seed projects and can provide advice and counsel through:

• Providing mentoring and guidance on the development of the overall health services research program.
• Providing mentoring to the PI, research center director, and others.
• Assuring the conduct of rigorous research by providing input on project design.
• Suggesting linkages for partnerships.
• Suggesting opportunities for disseminating findings in publications and at conferences.
• Reviewing publications and conference presentations.
• Recommending relevant funding opportunities.
• Reviewing proposals.

Based on our research, we believe that these groups should each consist of 4-8 experts. The Advisory Group should be large enough to provide a variety of perspectives, yet small enough to facilitate meetings. Members should be available to respond to ad hoc inquiries and attend quarterly meetings in person or via teleconference. The members should have expertise in managing research programs, procuring funding, and publishing in leading peer-reviewed journals. The group can include internal and external experts. In addition, members with relevant methodological or subject matter expertise may offer additional mentoring and technical support.

The Advisory Group can be a resource for evaluating the success of the project, through the provision of feedback in the form of systematic, periodic assessments.

Maximizing Success

Advisory groups should be established to provide different types and levels of support. Groups of four to eight experts should meet quarterly to review progress and provide guidance. Ideally, members should:

• Be available to respond to ad hoc inquiries.
• Have expertise managing or developing research programs.
• Have a proven research funding record.
• Have published extensively in top peer-reviewed journals.
• Possess relevant subject matter or research methodology expertise.

Consider using the Advisory Group as a source of feedback about your program's progress.

Return to Contents
Proceed to Next Section