Detailed Information on the
Rural Distance Learning and Telemedicine Loan and Grant Program Assessment

Implement periodic independent reviews, focusing on how well the program is accomplishing its mission, and meeting its long-term goals.

Review program performance and adjust assumptions used to develop budget estimates of loan program costs.

Collect grantee performance information and make it available to the public.

Measure: Percentage of rural counties that receive telemedicine facilities through DLT funds.

Explanation:This measures the impact of grant funding in rural communities. The number of counties obtained from the loan or grant application is divided by the number of rural counties in the U.S. from the U.S. census.

Measure: Number of rural healthcare providers receiving telemedicine facilities through DLT funds.

Explanation:This measures the impact of funds on providing services through rural healthcare providers. This data is obtained from the loan or grant application and verified by staff. The number of healthcare providers receiving telemedicine facilities is determined by the nature and size of projects proposed by applicants.

Measure: Percentage of rural counties that receive distance learning facilities through DLT funds.

Explanation:This measures the impact of grant funding in rural communities. The number of counties obtained from the loan or grant application is divided by the number of rural counties in the U.S. from the U.S. census.

Measure: Percentage of medically underserved counties receiving telemedicine facilities through DLT funds.

Explanation:Rural counties are more likely to be medically underserved (as designated by the Department of Health and Human Services). This measure indicates the coverage of these counties by grant funds. The counties served by loan and grant funds are identified in the application and verified by staff. Staff also verifies the Medically Underserved Area (MUA) status of each county. The number of counties served by DLT funds that are MUAs is then divided by the total number of MUA counties in the U.S.

Measure: Percentage of rural school districts offering distance learning classes.

Explanation:This measure is an indicator of the growth in distance learning services tied directly to the strategic goals of the program. Growth in overall distance learning industry is one positive impact that can result from the Rural Development DLT Program. The numerator of the equation is the number of rural K-12 school districts offering distance learning classes, whether funded by Rural Development or not. This number is obtained from the Department of Education National Center for Education Statistics. The denominator is the total number of rural K-12 school districts, also obtained from the National Center for Education Statistics.

Measure: Number of rural educational providers receiving distance learning facilities through DLT funds.

Explanation:This measures the impact of funds on providing services through rural educational providers. This data is obtained from the loan or grant application and verified by staff. The number of educational providers receiving distance learning facilities is determined by the nature and size of the projects proposed by applicants.

Measure: Percentage of rural public K-12 schools receiving distance learning facilities through DLT funds.

Explanation:This measure indicates how many rural public schools are actually benefiting from services provided through the DLT grant program. The number of K-12 public schools receiving distance learning facilities is obtained from the loan or grant application and verified by staff. The number of rural public schools is obtained from the Department of Education National Center for Education Statistics.

Measure: Cost per grant/loan.

Explanation:

Is the program purpose clear?

Explanation: The Distance Learning and Telemedicine (DLT) loan and grant program has a clear purpose as laid out in the authorizing legislation, 7USC 950aaa. The purpose of this program "is to encourage and improve telemedicine services and distance learning services in rural areas through the use of telecommunications, computer networks, and related advanced technologies by students, teachers, medical professionals, and rural residents.' These funds are used to provide the access necessary to have working distance learning and telemedicine facilities.

Evidence: The programs purpose is laid out in 7USC 950aaa.

Does the program address a specific and existing problem, interest, or need?

Explanation: Obtaining specialized medical treatment without traveling large distances is very difficult in rural America since this type of expertise is usually found in urban or university hospitals. Telemedicine helps rural residents meet their specialized medical treatment needs without traveling large distances. Providing telemedicine in rural areas has several challenges, including: small rural health facilities may have little or no IT support, very often there is no business case for connectivity, no aggregate buying power, and there is low patient volume in rural healthcare facilities. This USDA program provides grants and low cost loans which help rural areas finance telemedicine. Distance learning is today's fastest growing segment in education and training, according to the US Distance Learning Association. Distance learning encompasses all technologies and supports the pursuit of life long learning for all. Providing courses and electronic field trips are among the principal applications for distance learning in Pre-K through grade 12 education. Distance learning is also used to support rural classes with student enrichment, student courses, staff development and in-service training for teachers and administrators. Distance learning is providing undergraduate and advanced degrees to students in offices, at community colleges and at various other sites. A GAO report provided an estimate from a 1996 RAND Corporation study that 'estimates that providing a 'technology-rich' learning environment in every school would cost $10 billion to $20 billion per year. State and local government resources alone can not meet the needs for technology in schools.' A second GAO report cited a study, by McKinsey & Company, Inc., that indicated that 'placing one networked computer laboratory in each school nationwide would cost $11 billion up front and $4 billion in annual costs.' States are unable to provide this level of funding. The USDA program is able to fill some of the funding gaps.

Evidence: The Office of Rural Health Policy (ORHP) recently highlighted health IT activities at ORHP. In September 2006, ORHP held the first National Rural Health IT meeting to identify the barriers and the challenges facing rural healthcare today. There are plans to have a second National Rural Health IT meeting in the fall of 2007. US Distance Learning Association, www.usdla.org. A Department of Education report, Distance Education Courses for Public Elementary and Secondary School Students: 2002-2003, noted that "Distance education is more prevalent in rural districts, where distance learning may be the students' only available option." National Library of Medicine Telemedicine Information Exchange, "The Capitol Hill Steering Committee on Telehealth Discusses Rural IT Programs;" 3/2/05. GAO reports HEHS-99-133 Telecommunications Technology Federal Funding for Schools and Libraries and HEHS-98-35 School Technology Five School Districts' Experiences in Funding Technology Programs.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: There are no programs that provide the same assistance. USDA's program provides grants and loans to fund fully operational DLT facilities. Other potentially similar programs normally fund only specific aspects of providing DLT services. The Department of Health and Human Services has a telehealth program which is designed to help communities build the human, technical and financial capacity to develop sustainable telehealth programs and networks. GAO identified 35 programs supporting telecommunications and information technology for schools and libraries of which only four programs target schools, libraries and technology. GAO 'did not identify instances where two individual programs were providing identical services to identical populations ' that is, had the same goals, the same activities or strategies to achieve them, and the same targeted recipients.' A second study showed school districts did not rely on a single source of funding. The districts used local, state, federal and private funding to provide for school technology needs. Only one district relied heavily on federal funding. The Federal Communications Commission recently initiated a pilot grant program for telemedicine and the Telecommunications Program staff is currently working with the FCC to identify how the programs might complement each other.

Evidence: GAO report HEHS-99-133 Telecommunications Technology Federal Funding for Schools and Libraries and GAO report HEHS-98-35 School Technology Five School Districts' Experiences in Funding Technology Programs. FCC Order 06-144, dated 9/24/06.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: There is no strong evidence that there are major design flaws. The program has operated successfully since the early 1990s, essentially under the same criteria. One potential flaw that was corrected was the requirement of applicants to provide a 30% match in order to be eligible for funding. While matching funding is an important part in achieving project sustainability and local community involvement, it had the unintended affect of excluding poorer communities that could not achieve the required 30% match. The matching contribution was lowered to 15% for eligibility purposes in 2002. However, potential beneficiaries are not interested in obtaining loans. They prefer to receive grants. Due to this applications for less than 10% of loan funds are approved each year.

Evidence: 7 CFR Part 1703, Subparts D and E.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: In providing financial assistance, RUS gives priority to rural areas that have the greatest need for distance learning and telemedicine services. In general, the need is greatest in areas that are economically challenged, costly to serve, and experiencing outward migration. Points are given for comparative rurality, the need for and benefits from the service, economic need, ability to leverage, innovativeness, cost effectiveness, and participation of EZ/EC and champion communities.

Evidence: 7 CFR 1703, Subparts D and E; Distance Learning and Telemedicine Application Guide

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The performance of this program is linked with USDA's strategic goal to support increased economic opportunities and improved quality of life in rural America. Increasing access to educational and health services can increase economic opportunities available to rural businesses and improve the quality of life of rural residents. USDA has developed long-term performance measures that will measure how well the program is meeting its end purpose of facilitated access to improve educational and health services by residents in rural communities. The first measure is the percentage of counties designated as Medically Underserved Areas by the Department of Health and Human Services that are provided telemedicine services under the DLT grant or loan program. Medically Underserved Areas and Medically Underserved Populations have shortages of primary medical care, dental or mental health providers and may be geographic (a county or service area), or demographic (low income, Medicaid-eligible populations, cultural and/or linguistic access barriers to primary medical care services). This measure directly ties performance to the strategic goals of the program. The second measure is the percentage of rural school districts offering distance learning classes. A primary goal of the program is to provide funding for rural school districts to provide distance learning classes so that rural students are not disadvantaged by the limits of their particular school's educational opportunities. This measure is an indicator of the growth in distance learning services tied directly to the strategic goals of the program. Therefore, growth in the overall distance learning industry is one positive impact that can result from the Rural Development DLT Program.

Evidence: USDA Strategic Plan; Long term measures are the percentage of Medically Underserved Area counties served by telemedicine grants and the percentage of rural school districts offering distance learning classes.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: Ambitious targets and timeframes have been developed. The targets are ambitious because, if the targets are met, there would be significant increases in the provision of distance learning and telemedicine over the next 3 years.

Evidence: Percentage of medically underserved counties receiving telemedicine facilities was 8% in 2005; target is 25% by 2010. Percentage of rural school districts offering distance learning classes was 46% in the 2002-2003 school year, the most recent data available, which serves as the baseline. The target for 2010 is 60%.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: USDA has developed new annual measures that reflect the impact the program has on improving rural distance learning and telemedicine. USDA has developed an efficiency measure for all credit programs. For example, the first measure is the percentage of counties provided telemedicine services under the DLT grant or loan program. This measure directly ties to the long-term measure: percentage of medically underserved counties receiving telemedicine facilities under the DLT grant or loan program. The second measure is the percentage of rural counties that receive distance learning facilities through DLT funds. This measure directly ties to the long-term measure: percentage of rural school districts offering distance learning classes.

Evidence: Percent of rural counties that receive telemedicine and distance learning, number of rural healthcare providers receiving telemedicine facilities, number of rural educational providers receiving distance learning facilities, and the percentage of public K-12 schools receiving distance learning facilities.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: Baselines and ambitious targets through 2010 have been developed. The targets are ambitious because the number of counties receiving distance learning and telemedicine facilities is determined by the size and scope of the grant and loan applications received. Some projects that are small in scale may score higher than larger projects with more facilities. However, the program goal is to provide distance learning and telemedicine as widely as possible, within the requirements of the program.

Evidence: Percentage of rural counties received telemedicine facilities target is 31.21% by 2010, a 272% increase from 2005; Percentage of public schools receiving distance learning facilities increases to 11% by 2010, a 511% increase over 2006.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Grantees must be the owners/operators of the system they are building. Therefore, they are committed to providing service to educational or health care facilities. The program makes grants available for the end-user computer equipment to be placed in schools or medical facilities; as such, when the equipment is deployed in a rural county, the performance goal is supported.

Evidence: Standard Form 269A; 7 CFR 1703 Subparts D and E.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: USDA's Rural Development does not have a policy or process to have independent evaluations of programs on a regular or as needed basis. However, Management Control Reviews are performed on a periodic basis, not to exceed 5 years. These reviews look at how RUS is implementing the DLT program and provides recommendations for improvement. These reviews do not focus on program performance.

Evidence: Management Control Reviews of the grant program were completed in 1999 and in 2004. The 1999 report concluded that the program was well run, but noted some areas for improvement (1999 MCR, DLT grants). The 2004 report noted some areas for improvement but concluded that in general the program was well run (2004 MCR, DLT Grants). The grant program has been categorized as "low-risk" by the Management Control Group.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: USDA's Rural Development does not have the ability to track and report all direct and indirect costs at the program level. Due to this, there is no way of knowing the total costs associated with operating the DLT program. Funding requests do not detail why the requested funding level is reasonable and adequate and why the requested performance resource mix is appropriate. However, the Rural Development budget-planning process ensures that the direction of the agency financing activities meet annual and long-term performance goals. Funding requests are based on estimated demand based on pending applications and staff experience. Performance measures from both the strategic and annual performance plans are monitored to determine the maximum benefit from appropriated funding toward meeting pre-established goals. The Rural Development budget requests depend on the review and approval of USDA's Budget and Performance Integration Board. This provides a secondary executive review to ensure alignment of funding and performance with USDA's strategic goals. Although the funds for the DLT Loan Program have not been fully obligated over the last few years, for economic and technical reasons, there has been an increase in the number of loan applications over the last year, and interest in the program remains high. Demand for the Loan program is impacted by industry factors such as: (1) the inability of most public educational institutions to incur debt; and (2) issues related to reimbursement for telemedicine services from public and private insurers. As noted in the report, "Private Payer Reimbursement for Telemedicine Services in the U.S.," from researchers at Michigan State University, "In order for telemedicine programs across the country to become more stable and economically viable, it is essential that healthcare organizations offering telemedicine services begin to start billing private payers, especially since many states cannot rely on Medicaid for reimbursement." The program staff sees evidence that interest in the loan program is increasing and has recently initiated a focus on promoting use of the loan program as a source of funds for automating medical records activities.

Evidence: USDA's Congressional Justification materials, Rural Development's Strategic Plan, GPRA documents, "Private Payer Reimbursement for Telemedicine Services in the U.S," Pamela Whitten, Ph.D. and Laurie Buis, M.S.I., Michigan State University.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The DLT program has developed annual and long-term performance measures, baselines and targets. Performance measures are reviewed on an annual basis to ensure current applicability to program delivery.

Evidence: A review of long-term performance measures, annual performance measures, and outcomes is required as part of the Rural Development budgeting process.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: All borrowers are required to annually file financial and performance data. Information is used to process and monitor loans allowing for effective and consistent management of the loan portfolio. In addition, a grant project performance activity report is required on an annual basis until the project is complete and the funds are expended. Performance information provides assurance that the project is on track and funding is being utilized in a proper manner. If there are any deviations from the proposed project, a corrective action (site visit or expedited audit) would be initiated.

Evidence: 7 CFR Ch. XVII, Subpart D Distance Learning and Telemedicine Loan and Grant Program General; Form 269A; Grant Agreement

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: The Federal manager directly responsible for the DLT program is at the SES level with specific performance measures that ensure accountability. Loan contracts require borrowers to use loan funds for specific purposes. Borrowers are required to provide annual financial and performance data which is reviewed by program staff. Grants are monitored to assure that the projects are completed in accordance with the scope of work. The Grantee submits to RUS a 'Request for Advance or Reimbursement' bearing the original signature of the authorized agent of the Grantee. Advances of funds are only for approved purposes of the grant. Invoices are required to ensure that program dollars are spent only for intended purposes. Each project is audited by RUS field accountants after the project has been completed.

Evidence: Program Manager Performance Elements and Evaluations; 7 CFR 1703, Subparts D and E; Distance Learning and Telemedicine Application Guide; SF 270

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: Funds are not always obligated in a timely manner. Three of the past five years, this program has carried forward large unobligated balances. In addition, this program routinely does not obligate significant portions of appropriated loans. However, efforts to decrease appropriations for loans, so that the funding is more in line with expected demand, have been made. These efforts have been met with resistance. In addition, Rural Development is improving their application review and approval process which should reduce the amount of grant funding carried over in the future.

Evidence: 7 CFR 1703, Subparts D and E; Distance Learning and Telemedicine Application Guide; SF 270

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: This program does not have baselines and targets developed for its efficiency measure. Baselines and targets will not be developed until USDA's Rural Development has the ability to track and report all direct and indirect costs at the program level. However, program staff works to improve the program when deficiencies are found. In 2006, a major streamlining effort was undertaken to improve application processing. Rural Development required applications to be complete when submitted with regard to applicant eligibility, project eligibility, and scoring. This made the process more equitable because every application was now treated the same. In the past, staff would often give applicants who did not provide a complete application additional months to arrange their matching funds, and as a consequence, increased the processing time. Staff provided extensive additional guidance in the Application Guide. This allowed scoring to begin earlier. Rural Development also changed the internal review process so that several pieces could be accomplished in parallel by teams instead of by one reviewer serially. The previous record for processing from deadline to submission of the scores to the Administrator was 105 days and was typically about 150. In FY2006, it was accomplished in 81 days. Beginning in FY 2004, electronic application filing was initiated through the Grants.Gov process; while the process was not heavily utilized in its initial year, it is anticipated that electronic filing will greatly improve program efficiencies. In addition, revisions were made to the application guide and a new "tool kit" was developed to better assist applicants in the application process and in understanding the program requirements. All information is posted on the RUS web site.

Evidence: DLT Application Guide and Tool Kit; www.usda.gov/rus/telecom/dlt/dlt.htm; The 1999 Management Control Review found a lack of standardization in directions for scoring worksheets and documentation. New scoring worksheets and directions were established.

Does the program collaborate and coordinate effectively with related programs?

Explanation: RUS is a member on the Telemedicine Working Group coordinated by the HHS Office for the Advancement of Telehealth. RUS works with the Appalachian Regional Commission (ARC) since ARC can provide matching dollars to RUS applicants. Information about ARC is included in the DLT grant application guide. RUS also collaborates with other USDA programs that are similar including the RUS' Telecommunication Program and the Rural Housing Services' Community Facilities Program. This collaboration is done to facilitate projects that may be common to the programs. Rural Development has conducted an information-sharing meeting with the Federal Communications Commission to learn about the newly announced FCC Telehealth program and to identify ways that the two agencies can collaborate on their programs.

Evidence: Staff Instruction 1703-2 for post review responsibilities.

Does the program use strong financial management practices?

Explanation: USDA's Rural Development has strong financial management practices. The program has procedures in place to ensure that loan and grant approvals are given to eligible entities and loan advances are made to participants who comply with contract terms and only for approved purposes. The program also has procedures in place to comply with FCRA, DCIA and OMB Circular A-129. In addition, Rural Development has received clean audit opinions since 2001. Projects must demonstrate sustainability, i.e., the ability to operate on their own after the grant or loan has been made. Each project must demonstrate its financial capacity to prove feasible. In addition, each project is audited by RUS field accountants after the project has been completed.

Evidence: Rural Development's annual financial statement audits; 7 CFR 1703 Subparts D and E; Staff Instruction 1703-2; post grants audits.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: USDA's Rural Development has a process in place to perform Management Control Reviews at least once every five years. The Management Control Review process is used to analyze efficiency, identify management deficiencies and recommend improvements. This program has been reviewed in 1999 and 2004 and has implemented changes to address identified deficiencies.

Evidence: For example, one review noted in some instances that projects had not been visited by field staff in a timely manner. The program issued a staff instruction to clearly layout responsibilities for the field staff in the post review process.

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: An extensive, rigorous competitive review process is undertaken, including qualified reviewers, strict guidelines and safeguards to avoid conflict of interest. In providing financial assistance, RUS gives priority to rural areas that have the greatest need for distance learning and telemedicine services. In general, the need is greatest in areas that are economically challenged, costly to serve, and experiencing outward migration. Points are given for comparative rurality, the need for and benefits from the service, economic need, ability to leverage, innovativeness, cost effectiveness, and participation of EZ/EC and champion communities. Revisions have been made to the application guide and a new "tool kit" was developed to better assist applicants in the application process and in understanding the program requirements. All information is posted on the RUS web site. In 2006, a major streamlining effort was undertaken to improve application processing. Rural Development required applications to be complete when submitted with regard to applicant eligibility, project eligibility, and scoring. This made the process more equitable because every application was now treated the same. In the past, staff would often give applicants who did not provide a complete application additional months to arrange their matching funds, and as a consequence, increased the processing time. Staff provided extensive additional guidance in the Application Guide. This allowed scoring to begin earlier. In all cases, release of funds can be and is deferred pending submission and approval of an acceptable proposal of work against which performance is assessed. Rural Development works with new and first-time recipients to ensure they understand Federal Financial Assistance regulations and policies and to ensure the proposed project is sound, relevant and meritorious. Rural Development conducts outreach to encourage new grantees through presentations at professional societies and other stakeholder meetings, and through frequent grantsmanship workshops.

Evidence: DLT Application Guide and Tool Kit; www.usda.gov/rus/telecom/dlt/dlt.htm; 7 CFR 1703, Subparts D and E.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: RUS General Field Representatives are required to meet with grantees and visit the project from time to time to ensure proper operation. In addition, each project is audited by USDA accountants after the project is complete.

Evidence: Program Manager Performance Elements and Evaluations; 7 CFR 1703, Subparts D and E; Distance Learning and Telemedicine Application Guide; SF 270.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: Though a grant project performance activity report is required on an annual basis until the project is complete and the funds are expended, only limited data is available to the general public in a transparent and meaningful manner. The program staff is currently conducting a status review of all DLT grants and loans from 2001 to 2005, and will provide aggregate performance data on the public website once the review is complete.

Evidence: 7 CFR Ch. XVII, Subpart D Distance Learning and Telemedicine Loan and Grant Program General; Form 269A; Grant Agreement

Is the program managed on an ongoing basis to assure credit quality remains sound, collections and disbursements are timely, and reporting requirements are fulfilled?

Explanation: USDA assures credit quality by reviewing all applicants using standardized credit risk models and required financial analysis. Borrower's performance is monitored on an ongoing basis. Annually borrowers submit financial and statistical data and an audited financial statement. Procedures are established to monitor borrower compliance. Financial information is reviewed and each borrower's risk is established by a pre-established set of financial criteria. Borrowers with high or imminent risk are reviewed for the cause and agreed upon corrective action is pursued. The Agency's accounting staff review borrower's audited financial statements and monitor borrower compliance. Deficiencies noted in the audit and downward trends in financial performance are reported to the program for correction. The accounting staff independently tracks the program's progress in obtaining corrective action. Monthly the accounting staff notifies the program staff of any late payment collections, at which time the program takes immediate action to remedy.

Evidence: 1) 7 CFR 1703 Subparts D and E; 2) Staff Instruction 1703-2; 3) post grants audits; 4) Borrowers' audited financial statements: 5) Accounting and Program tracking reports, which provide information on deficiencies noted in financial audits, downward trends in financial performance and progress in obtaining corrective action; 6) Accounting staff's audit review checklists; and 7) Monthly Delinquency Report, which provides information on any late payment collections.

Do the program's credit models adequately provide reliable, consistent, accurate and transparent estimates of costs and the risk to the Government?

Explanation: Rural Development has changed the credit models used to calculate subsidy rates. The OIG has audited these new models. Loan applications are reviewed and evaluated using the following criteria: Cash flow from operations/total debt service; Times Interest Earned ratio; Total Equity/Total Assets ratio; EBITDA/Total Assets ratio; Current ratio. A business review is also conducted which examines the applicant's operating environment, operating efficiency, system quality and technology. Applicants are required to include detailed cost and engineering data which ensure that loan applications receive a thorough review by engineering and financial experts on the program staff. An Engineering Cost memo is prepared by Rural Development program staff and used to monitor progress as the project progresses. Most variation experienced by the Distance Learning/Telemedicine program's subsidy rates is due to interest rate fluctuations that are not within the program's control. Subsidy rates have been fairly consistent over the past five years.

Evidence: DLT loan subsidy rates range from 0% to 1.5% over five years.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Data reflects progress towards meeting long-term goals. This program has met or exceeded annual measures that reflect progress towards meeting long-term goals. However, since only one year of data for annual measures that compares with a target are available, at this time a small extent is all that is supported.

Evidence: Rural Development is making progress in reaching medically underserved counties. In 2005, 8% of the medically underserved counties in the U.S. received funds under the DLT program. In 2006, that number rose to 20%. The percentage of rural school districts offering distance learning classes continues to grow. As the percentage of rural counties receiving distance learning facilities funding through this program increases, the percentage of rural school districts offering distance learning classes should increase. So the increase from 13.4% to 15.67% in rural counties receiving distance learning facilities through this program reflects progress towards the long term goal of providing distance learning classes in 60% of rural school districts by 2010.

Does the program (including program partners) achieve its annual performance goals?

Explanation: This program has met or exceeded 6 out of 7 annual performance measures. However, since there is only data for one year that compares with a target, a small extent is all that is supported.

Evidence: Annual measures for 2006 include: 25.92% of rural counties received telemedicine facilities (compared to a target of 6.94%); 15.67% of rural counties received distance learning facilities (compared to a target of 10%).

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: Since this program has no baselines or targets for its efficiency measure and received a no to question 3.4, a no is given. However, the program does work to improve its efficiencies. The application guide was revised in response to a Congressional suggestion that the material was to cumbersome. As such, a new "tool kit" was developed and the guide was reformatted to be more user friendly to the applicant. In addition, applications are now accepted electronically though the Grants.Gov initiative. All relevant program information is available on the RUS web site. In FY2006, application processing, review and scoring processes were reviewed and streamlined to improve processing time. The program has achieved efficiencies in processing by leveraging other Rural Development Telecommunications Program resources during peak application processing times. There are only two Full Time personnel, dedicated to the DLT Program, responsible for the ongoing management of the program, including developing policy, updating application material and providing information for outreach and education. As with other competitive grant programs, there is a peak workload period of about 3 to 4 months during which the grant applications must be reviewed and scored. During this peak period, employees from other parts of the Telecommunications Program are designated to support the DLT staff in the application review process. Therefore, existing resources are leveraged to avoid the need for more personnel for the DLT program.

Evidence: DLT Application Guide and Tool Kit; www.usda.gov/rus/telecom/dlt/dlt.htm

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: There are other Federal and state government programs that provide funding for telemedicine and distance learning projects, as well as private for-profit and nonprofit organizations. Few of these other grant or loan programs that directly compare to the DLT Loan and Grant Program in the scope and scale of the projects and the budget authority available. The grant application scoring and evaluation approach taken by Rural Development has been developed over many years of experience to specifically target rural communities with the greatest need for support. In addition, Rural Development is not aware of any other Federal organization which provides the type of financial analysis and technical support to applicants that is available under the DLT Loan Program. As of the GAO report in 1999, the RUS DLT program did not have the highest administrative cost to program cost ratio. However, the DLT program was one of the highest of the ten programs that target technology. In addition, the DLT program had one of the higher FTE totals. Although this information is dated, there have not been any significant program changes since 1999.

Evidence: DLT NOFA dated 4/10/07; The Catalog of Federal Domestic Assistance: 93.211 Telehealth Network Grants, 93.887 Health Care and Other Facilities; 10.226 Secondary and Two-Year Postsecondary Agriculture Education Challenge Grants; FCC Order 06-144, dated 9/24/06. GAO report HEHS-99-133 Telecommunications Technology Federal Funding for Schools and Libraries.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: No regular independent evaluations are conducted, nor does Rural Development have a process in place to perform regular independent program evaluations. However, other aspects of performance including financial management are reviewed. Financial statement audits have indicated that the program is effective and achieving its goals. Management Control Reviews look at how RUS is implementing the DLT program and provides recommendations for improvement.

Evidence: Management Control Reviews of the grant program were completed in 1999 and in 2004. The 1999 report concluded that the program was well run, but noted some areas for improvement (1999 MCR, DLT grants). The 2004 report noted some areas for improvement but concluded that in general the program was well run (2004 MCR, DLT grants). The grant program has been categorized as "low-risk" by the Management Control Group. OIG financial statement audits.