Detailed Information on the
Broadcasting to Latin America Assessment

Providing advertisment and exposure on major networks for all new programming (within available funds), in order to ensure that audience awareness and program quality ratings improve. Eventually, all programming should have advertising and exposure.

Cutting costs by fully implementing organizational efficiencies and reducing unnecessary reduduncies.

Designing and implementing additional long-term performance measures that align with the BBG mission and vision.

Measure: Audience Reach for VOA Creole broadcasting in Haiti. (%)

Explanation:A key indicator of whether or not BBG broadcasts are reaching their intended audience is the number of people who choose to listen to or view BBG programs. Audience Reach, also called weekly audience, is a measure of the regular listening/viewing audience. Regular audience is defined as all adults in a given population listening/viewing at least once a week, as determined by an audience survey that has an adequately designed sample. The number is a reflection of several annual common measures, such as program quality and the audience's awareness of the program, used for all BBG language services broadcasting in Arabic. Annual measures of audience size, as well as annual performance goals for each individual service, factor into the overall Audience Reach for broadcasting in Creole in Haiti.

Measure: Audience reach for VOA Spanish broadcasting (in millions of audience listeners/viewers)

Explanation:A key indicator of whether or not BBG broadcasts are reaching their intended audience is the number of people who choose to listen to or view BBG programs. Audience reach, also called weekly audience, is a measure of the regular listening/viewing audience Regular audience is defined as all adults in a given population listening/viewing at least once a week, as determined by an audience survey that has an adequately designed sample. The number is a reflection of several annual common measures, such as program quality and the audience's awareness of the program, used for all BBG language services. Annual measures of audience size, as well as annual performance goals for each individual service, factor into the overall audience reach for VOA broadcasting in Spanish. Note: The total audience reach has been calculated since 2006, while prior to this time only a sample of individual countries in the region was available.

Measure: Audience Awareness in Haiti, for VOA broadcasting in Creole. (%) Audience Awareness is the percent of the sampled population, prompted or unprompted, that acknowledges BBG station names in an audience survey that has an adequately designed sample. Note: This measure can fluctuate from year to year based on whether programs are transmitted on preferred media, are placed with popular affiliates, or have sufficient advertising. If annual declines are noted, future targets are adjusted.

Explanation:The statistic is expressed as a percent of the sampled population that knows or recognizes BBG stations, which can then translate to the percent of the entire population of the country or region of which the sample was representative. This measure of the extent to which a target population recognizes BBG is influenced by efforts made by each language service such as ensuring that programs are transmitted on preferred media, are placed with popular affiliates, and have sufficient advertising. Annual measures of the percent of a target audience that is aware of BBG programming are tracked by each individual language service, and directly support the long-term measures of growing the size of the audience and increasing the overall perceived reliability of BBG programming. The data shown here is specific to Haiti.

Measure: Program Quality Score for Radio Marti. (Range 1-4; 4 being highest) Note: Within the range, small variations are common, but maintaining the rating or progressing towards the next rating is significantly ambitious, especially as the rating nears the perfect score of 4.

Explanation:The Program Quality Score is an outcome measure based in part on how the target population perceives the quality of the specific BBG language services. This measure is an industry standard used to determine the target audience's assessment of the quality of media. A monitoring panel of regular listeners from the target population rates the program's accuracy, reliability, authoritativeness, objectivity, comprehensiveness, and other variables reflecting distinct statutory, policy, and mission mandates for the different stations. Another component of the Program Quality Score is the evaluation of IBB delivery systems (engineering and transmission), marketing and program placement efforts, and other broadcasting support functions. Program content and presentation are evaluated by both internal and external reviewers to determine the overall Program Quality Score. In total, the Program Quality Score is a comprehensive, weighted score, which ranges from 1-4, with 1.0-1.3 = poor; 1.4-1.6 = poor to fair; 1.7-2.3 = fair; 2.4-2.6 = fair to good; 2.7-3.3 = good; 3.4-3.6 = good to excellent; 3.7-4.0 = excellent. Annual measures of the perceived quality of BBG programming are tracked by each language service, and directly support the long-term measures of growing the size of the audience and increasing the overall perceived reliability of the programming. The data shown here is specific to Radio Marti.

Measure: Cost per audience member for VOA broadcasting in Spanish to target populations in Latin America.

Explanation:This efficiency measure tracks the cost per audience member of each language service. Cost per audience member is the total cost of programming and delivery divided by the unduplicated audience for all media, and is calculated for each language service. The ratio (measure) must be evaluated in context of the changes in either the cost or audience size. For example, a short-term capital cost increase to improve transmission may not increase audience size until after a lengthy installation. In some cases, audience size may be kept artificially low by restrictions imposed by the host government or increased competition, not necessarily poor programming or inefficient management. Costs per audience member may fluctuate annually and also differ compared to other language services based on a combination of factors in each location, including: preferred media in the region, hours of programming, competition from other broadcasters, jamming, and government restrictions.

Is the program purpose clear?

Explanation: The program purpose is to promote and sustain freedom and democracy by broadcasting accurate and objective news and information about the United States and the world to audiences overseas (in this case, audiences in Latin America). The BBG mission embraces the longstanding dual components of U.S. international broadcasting: (a) representing America's culture and official policies and (b) serving as a surrogate for indigenous free press.

Evidence: ?? BBG Strategic Plan (2002-2007) ?? U.S. International Broadcasting Act of 1994 ?? Foreign Affairs Reform and Restructuring Act of 1998 ?? Voice of America (VOA) Charter ?? Radio Broadcasting to Cuba Act ?? Television Broadcasting to Cuba Act"

Does the program address a specific and existing problem, interest, or need?

Explanation: The program addresses the need for accurate news and information about the United States and events in Latin America in languages spoken in those countries (Spanish to all countries, except Haiti where broadcasts are in Creole). The dearth of free, unbiased media and the existence of anti-American reporting in Latin America create a need for timely and accurate reporting of U.S. policies and actions as well as events in the region and the world at large. To address this need, in 2005 VOA broadcasts 35 hours per week in Spanish to Latin America and 9.5 hours per week in Creole to Haiti. The Office of Cuba Broadcasting broadcasts 162 hours per week of Radio Marti in Spanish to Cuba and 168 hours per week of TV Marti in Spanish to Cuba. Target countries for VOA in Latin America include: Colombia, Haiti, Venezuela, Ecuador, Bolivia, and Peru.

Evidence: ?? The 2005 Freedom House Press rating for Cuba, Venezuela, Columbia and Haiti is "Not Free"; ?? The rating for Guatemala, Paraguay, Honduras, Panama, Nicaragua, Mexico, El Salvador, Ecuador, Peru, Brazil, Dominican Republic, and Bolivia is "Partly Free." ?? Source: (www.freedomhouse.org/research/pressurvey/regionaltables2005.pdf) ?? BBG FY 2006 Budget Justification to Congress

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The Broadcasting Board of Governors (BBG) is the only entity responsible for all U.S. government and government-sponsored, non-military international broadcasting. As such, its broadcasts are not duplicative of other Federal, state, or local contributions to the problems being addressed in Latin America. Although other media outlets--including CNN, the BBC, and local indigenous organizations--broadcast to this region, only U.S. international broadcasting (Voice of America and the Office of Cuba Broadcasting) has immediate and guaranteed access to accurate information about U.S. interests and policies and can be relied upon to provide a distinctly American perspective on events. The mission of BBG broadcasting ("to promote and sustain freedom and democracy by broadcasting accurate and objective news and information about the United States and the world to audiences overseas") distinguishes it from commercial media organizations.

Evidence: ?? The U.S. International Broadcasting Act of 1994 and the Foreign Affairs Reform and Restructuring Act of 1998 state that U.S. international broadcasting should not duplicate the activities of private United States broadcasters or the activities of government-supported broadcasting entities of other democratic nations.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: There is no conclusive evidence that another approach would be more efficient or effective at achieving the program's goals.

Evidence: In the GAO report "New Strategic Approach Focuses on Reaching Large Audiences but Lacks Measurable Program Objectives" (GAO-03-772) the Board identified the internal challenge of consolidating and rationalizing the overall enterprise. "The diversity of the BBG - organizations with different missions, different frameworks, and different constituencies - makes it hard to bring all the separate parts together in a more effective whole??" In the same report the Board also identified a solution to this challenge, a "single system" approach to broadcasting. In follow-up testimony to Congress (GAO-04-711T) the GAO found that "The Broadcasting Board of Governors has responded to a disparate organizational structure by developing a new strategic approach to broadcasting??" and "To streamline its operations, the Board has used its annual language service review process to address such issues as how resources should be allocated among language services on the basis of their priority and impact, what degree of overlap should exist among services, and whether services should be eliminated because they have fulfilled their broadcast mission." In addition, the Broadcasting Board of Governors has identified and acted on additional opportunities to increase efficiencies. For example, VOA TV and the former WORLDNET merged in 2004 and created a focused, multimedia organization that will reach larger audiences with enhanced programming in the most effective media.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The answer to this question has been based on BBG's targeting efforts in four areas: 1) choice of delivery medium (AM, FM, shortwave, or TV); 2) efforts to overcome jamming; 3) choice of language; and 4) program content. 1) Choice of delivery medium: As part of the research conducted for Program Review, InterMedia (an independent research organization under contract to the BBG) conducts national surveys on the type of media available to and preferred by target audiences. Based on the results of the survey and input from the IBB Office of Engineering about technical feasibility, the BBG selects the media mix that will be most effective at reaching the target audience. ?? In Cuba, OCB offers 24 hour television and radio programming, which is available on satellite television, local UHF, shortwave, and AM. This meets the media preferences and capabilities of listeners and viewers. ?? In Haiti, VOA broadcasts 9.5 hours of radio per week (1 hr 7 x week and an additional 30 minutes 5 x week.) via SW, satellite, Internet and FM affiliate stations. 60% of the population gets their news from radio and 50% through TV. 97% of VOA listeners tune in through FM radio and 2% through shortwave. ?? For Latin America, VOA broadcasts 34 hrs 14 minutes of radio per week via SW, satellite, Internet and affiliate stations, and 1 hour 25 minutes of television per week via satellite, Internet and affiliates. Studies in Ecuador and Venezuela show that 97% and 96% use television weekly, 83% and 85% use radio weekly, and 2% use shortwave on a daily basis in each country. 2) Jamming issues: Jamming is an issue in Cuba, where OCB uses a multi-pronged approach to deliver its programming, including land-based transmitting stations, an Aerostat balloon, the Hispasat direct to home satellite system, and an airborne broadcasting platform. 3) Choice of language: BBG programming is broadcast in the local languages of the target areas. VOA broadcasts in Spanish to Latin American and Cuba, where it is the predominant and official language. VOA broadcasts in Creole to Haiti, where Creole and French are both official languages of Haiti. Few Haitians speak French (10% - 20%) and essentially all speak Creole. 4) Program Content: As part of the research conducted for Program Review, control panels and focus groups are used to determine the information needs of target populations and obtain feedback on several program attributes, such as the relevance and objectivity of content. Internal content analysis and production analysis by the Office of Performance Review also inform program content decisions.

Evidence: ?? InterMedia Research Presentation VOA Spanish to Latin America Program Review; D3 Systems Research Presentation VOA Creole to Haiti; InterMedia Research documents for TV Marti and Radio Marti ?? State Department Background Notes for countries in Latin America, ?? State Department Background Note for Haiti ?? U.S. International Broadcasting Act of 1994 ?? Foreign Affairs Reform and Restructuring Act of 1998 ?? Voice of America Charter ?? Radio Broadcasting to Cuba Act ?? Television Broadcasting to Cuba Act ?? Program Review Scoring Procedures ?? Language Service Review process

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The BBG has three long-term measures for this program: 1) audience reach in target populations across Latin America; 2) audience reach in Cuba; and 3) audience reach in Haiti. To reflect the diversity of languages and baseline audience reach levels, separate long-term targets were set for Spanish broadcasting to the Latin American region, Spanish broadcasting to Cuba, and Creole broadcasting to Haiti. Audience reach, also called weekly audience, is an indicator based upon measurement of the regular listening/viewing audience. Regular audience has over the years been consistently defined as all as all adults listening/viewing at least once a week, as determined by an audience survey that has an adequately designed sample. The BBG continues to explore additional or improved long-term measures or goals that address the quality and relevance of broadcasting.

Evidence: ?? BBG Strategic Plan (2002-2007) ?? U.S. International Broadcasting to Latin America FY 2007 Performance Plan ?? Creole FY 2007 Annual Performance Plan ?? Cuba FY 2007 Annual Performance Plan ?? Spanish FY 2007 Annual Performance Plan

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The long-term targets for increasing audience reach are sufficiently ambitious given the large population of the Latin American region and the closed nature of many of its societies. The related annual goals and action steps will track the progress toward achieving these long-term goals.

Evidence: The long-term goal for Spanish broadcasting to Latin America is to increase cumulative audience reach from 5% in 2004 to 7% in 2009. (The FY 2004 baseline is expressed as a range of values for individual countries in the region. Audience reach in FY 2004 ranged from 0.2% to 9.0% of the population.) The long-term goal for broadcasting to Cuba is to increase audience reach from 2% in 2004 to 10% in 2009. The Cuban government aggressively blocks broadcasts by jamming the signals carrying programs from Voice of America and the Office of Cuba Broadcasting. The on-going and planned anti-jamming efforts of the BBG were considered when setting the long-term goal. The long-term goal for Creole broadcasting to Haiti is to increase audience reach to 55% of the population by 2009. The FY 2004 baseline is 45%. The baseline for the common measures is the FY 2004 level. ?? U.S. International Broadcasting to Latin America FY 2007 Performance Plan ?? Creole FY 2007 Annual Performance Plan ?? Cuba FY 2007 Annual Performance Plan ?? Spanish FY 2007 Annual Performance Plan ?? Language Service Review ?? Program Review

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: There are 4 annual performance measures for each program in Latin America, Haiti, and Cuba: 1) percent of audience awareness; 2) program quality; 3) signal strength; and 4) cost per listener. The last common measure - cost per listener - is an efficiency measure. The program quality score aggregates both internal and external evaluations of the program's content and presentation. The content evaluation rates the program's accuracy, reliability, authoritativeness, objectivity, comprehensiveness and other variables reflecting distinct statutory, policy, and mission mandates for the different stations. The presentation evaluation involves separate sub-criteria for each production unit unique to its media and the program. Ratings of the program's content and presentation are aggregated into an overall program quality rating, which ranges from 1-4. Each language service or area has developed individual annual action steps with milestones tailored to the nature of the broadcasting and target population. These contribute to achieving the annual and long-term goals.

Evidence: ?? U.S. International Broadcasting to Latin America FY 2007 Performance Plan ?? Creole FY 2007 Annual Performance Plan ?? Cuba FY 2007 Annual Performance Plan ?? Spanish FY 2007 Annual Performance Plan ?? BBG Strategic Plan (2002-2007) ?? Language Service Review ?? Program Review (Source: Performance Review Scoring Procedures)

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The annual targets are sufficiently ambitious given the large population of the Latin American region and the closed nature of many of its societies. Each program's action steps will track the progress toward achieving these annual goals.

Evidence: The baseline for the common measures is the FY 2004 level. Annual targets are set through FY 2007. See the Measures tab of this report for examples of this program's annual goals. ?? U.S. International Broadcasting to Latin America FY 2007 Performance Plan (see tables of measures and targets) ?? Language Service Review ?? Program Review (Source: Performance Review Scoring Procedures)

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: VOA's Spanish service, VOA's Creole service, and the Office of Cuba Broadcasting contributed to establishing the BBG Strategic Plan (2002-2007) and the development of the U.S. International Broadcasting to Latin America FY 2007 Performance Plan. Each language service also developed an individual annual plan that includes action steps that support the overall BBG long-term goals of the program.

Evidence: ?? Language Service Review ?? Program Review ?? BBG Strategic Plan (2002-2007) ?? U.S. International Broadcasting to Latin America FY 2007 Performance Plan ?? Creole FY 2007 Annual Performance Plan ?? Cuba FY 2007 Annual Performance Plan ?? Spanish FY 2007 Annual Performance Plan

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Internal review: The International Broadcasting Board's Office of Program Review conducts annual Program Reviews and an annual Language Service Review. This Office operates independently from all BBG language services. Annual Program Reviews include an assessment of each language service and program, with regularly scheduled follow-up meetings. Within ten days of the review the VOA Director or program director, the Division Director, the Service Chief, and appropriate Performance review personnel reconvene to consider the results of the review, and to determine the issues that need to be addressed in an Action Plan. The Action Plan includes items for all IBB and VOA elements, not just the broadcast service; support elements are also tasked with action items. Three months after the Program review, the group that assembled the Action Plan convenes again to check the progress on recommendations generated at the Action Plan meeting. The reviews assess program content, IBB delivery systems (engineering and transmission), marketing and program placement efforts, Internet development, and production content and presentation. BBG's annual Language Service Review looks at BBG's overall broadcasting portfolio and to assess two basic issues: (1) where should BBG broadcast and (2) how well is BBG broadcasting. Through Language Service Review the importance of reaching each of the target audiences is evaluated and the highest priority audiences among all of those targeted are identified. Resource allocations or reallocations to reach specific target audiences are made based on the results of this evaluation. External review: BBG contracts with InterMedia, an independent firm that conducts annual surveys for each language service and entity, evaluating a variety of factors: market profile, media access and use, and international broadcasting impact and trends. Surveys include questions about ownership of radios and TVs (UHF and cable/satellite systems) and "yesterday use" of radio, TV and internet, "Yesterday use" of radio is further broken down by format: FM, AM, and short wave. Activities of Internet users are profiled (e.g. send email, listen to music, find latest news). Government review: Between September 2000 and February 2004 the GAO conducted three evaluations of the BBG. While none of the reports focused specifically on broadcasting to Latin America, the findings, recommendations and resulting actions by the BBG impacted all broadcast services, including those targeting Latin America. Follow-up testimony was provided in 2004. In 2002, at the request of the IBB, the OIG reviewed the effectiveness and implementation of program changes at OCB.

Evidence: ?? Language Service Review ?? Program Review (Source: Performance Review Scoring Procedures and Analysis for Creole, Spanish and OCB program reviews) ?? InterMedia Research Presentation VOA Spanish to Latin America Program Review; D3 Systems Research Presentation VOA Creole to Haiti; InterMedia Research documents for TV Marti and Radio Marti ?? GAO Report: "Strategic Planning and Performance Management System Could be Improved" (GAO/NSIAD-00-222) ?? GAO Report: "New Strategic Approach Focuses on Reaching Large Audiences but Lacks Measurable Program Objectives" (GAO-03-772) ?? GAO Report: "Enhanced Measure of Local Media Conditions Would Facilitate Decisions to Terminate Language Services" (GAO-04-374) ?? GAO Testimony: "Challenges Facing the Broadcasting Board of Governors" (GAO-04-711T) ?? OIG Report: Review of the Effectiveness and Implementation of Office of Cuba Broadcasting's New Program Initiatives (IBO-A-03-01)

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: BBG has excelled at clarifying its mission, establishing implementation strategies and performance objectives that support the mission, as well as establishing performance indicators. Actions taken in 2006 and 2007 have enhanced the linkages between these strategic tools and increasingly shown how they are supported and implemented by BBG's budget each year. In its FY 2006 Budget request the BBG explicitly linked all costs to six annual performance objectives that support BBG's mission, and will do so again in its FY 2007 budget request. In addition, each objective will link annual performance goals to the requested resources with performance measures that will be monitored against the targets set for FY 2007. In FY 2007, the BBG will show how the performance objectives for the various broadcasting entities (including VOA and OCB) and support elements support the overall BBG strategic goal and annual performance goals. In FY 2007, the BBG will again include the agency's annual performance plan and performance measures in budget requests.

Evidence: ?? FY 2006 Congressional Justification ?? FY 2006 and FY 2007 Budget Requests to OMB

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: BBG has developed performance plans for Broadcasting to Latin America in FY 2005. The performance plan is based on the BBG Strategic Plan and will tie individual program goals to the performance goals of the agency. BBG has continued to improve its performance goals and indicators: in 2004 BBG added the agency-wide performance indicator of audience awareness of each program, including programs in Latin America. Improving performance in terms of audience awareness will strengthen BBG's ability to reach the audience and gauge the effectiveness of marketing, branding and affiliations. With regard to relevant GAO reports, between September 2000 and February 2004 the GAO conducted three evaluations of the BBG. While none of the reports focused specifically on broadcasting to Latin America, the findings, recommendations and resulting actions by the BBG impacted all broadcast services, including those targeting Latin America. Follow-up testimony provided in 2004 shows that the Board has taken meaningful steps to address all identified strategic planning deficiencies. With regard to relevant IG reports, in its review of the effectiveness and implementation of program changes at OCB, the OIG recommended that OCB improve its strategic planning by establishing a plan to conduct independent audience research regularly. The BBG agreed with this recommendation and has significantly improved its research methods.

Evidence: ?? BBG Strategic Plan (2002-2007) ?? Broadcasting to Latin America Annual Performance Plan ?? GAO Report: "Strategic Planning and Performance Management System Could be Improved" (GAO/NSIAD-00-222) ?? GAO Report: "New Strategic Approach Focuses on Reaching Large Audiences but Lacks Measurable Program Objectives" (GAO-03-772) ?? GAO Report: "Enhanced Measure of Local Media Conditions Would Facilitate Decisions to Terminate Language Services" (GAO-04-374) ?? GAO Testimony: "Challenges Facing the Broadcasting Board of Governors" (GAO-04-711T) ?? OIG Report: Review of the Effectiveness and Implementation of Office of Cuba Broadcasting's New Program Initiatives (IBO-A-03-01)

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The BBG analyzes the performance data gathered through its annual Language Service Reviews and Program Reviews, including polling date gathered from Intermedia (from statistically valid surveys and focus groups) and uses the results to adjust resources and set strategic priorities. Program Reviews also include content and production analysis that assist managers in improving their specific programming in such areas as music bridges, lively pace, interactive call-in shows, or younger voices to attract a larger audience. The BBG also conducts follow-up meetings to ensure that programs have successfully made adjustments and changes that were recommended during the Language Service Reviews and Program Reviews.

Evidence: ?? Language Service Review ?? Program Review ?? VOA Latin American Division individual language service performance plans and Office of Cuba Broadcasting performance plan ?? BBG Performance- based Employee Appraisals

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Senior managers, including SES and GS 13-15, are under a performance based appraisal system that requires critical elements and ratings that are tied to the objectives of the BBG with measurable goals and targets. Through the ongoing appraisal reviews and final annual rating, managers are held accountable for the performance results. The annual Language Service Review holds managers and partners directly accountable for results with the same common measures of audience reach, audience awareness, program quality and cost per listener considered among other factors. In addition, the CFO's mid-year, 3rd quarter and year-end reviews of budget and performance results hold managers accountable for their annual resource level against expected expenditures, schedules and accomplishments. Contractors are accountable to the same program standards. The Program Review process also measures the quality of their work. Many contractors work on a "pay-for-product" basis, with payment dependent upon the timely receipt of quality products. For example, many "stringers" receive payment for each quality news story they deliver.

Evidence: ?? Performance based Employee Standards/Appraisals; ?? Language Service Review

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: All requisitions must include verification that funds are available, and the funds are obligated when a contract or agreement is signed or a purchase order is done. The mid-year, 3rd quarter and year-end review provides an opportunity for managers to verify that unliquidated obligations are valid and that once an obligation has been fully expended, remaining balances are deobligated. Financial transactions are coded to both an organization and to a function or project. Given that BBG has a high percentage of fixed costs (salaries, transmission and infrastructure expenses, rent, etc.), funds must be obligated in a timely manner in order to continue operations. The BBG requires an annual financial plan and monthly status reports from the IBB and the grantees. The financial plans must detail anticipated costs for specific line items by each month of the fiscal year. BBG conducts a mid-year, 3rd quarter, and year-end review of each entity's financial plans and budgets to monitor funding and expenditures. BBG also requires a monthly status report of expenditures, disbursements and obligations, as well of the submission of the SF-169 Financial Status Report, and SF-172 Report of Federal Cash Transactions from the grantees. The reviews and the allotment control procedures enable the Agency to track obligations to ensure that funds are used for the intended purpose by the appropriate Agency office.

Evidence: ?? SF-169 Financial Status Report; ?? SF-172 Report of Federal Cash Transactions; ?? Annual financial plans from Voice of America and the Office of Cuba Broadcasting

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The annual Language Service Review process includes reviewing each program's progress on one efficiency measure: cost per listener. The Program Review process also assesses the efficiency and productivity of each language service. The CFO's reviews of budget and performance results hold managers accountable for their annual resource level against expected accomplishments. Enhancements are tracked separately with specific performance targets based on the increased resources.

Evidence: ?? VOA Latin American Division individual language service performance plans and Office of Cuba Broadcasting performance plan ?? Language Service Review ?? Program Review

Does the program collaborate and coordinate effectively with related programs?

Explanation: In terms of broadcasting, BBG is solely responsible for all non-military, U.S. international broadcasting. It is the only entity with the express objective of promoting freedom and democracy through broadcasting of objective news and information about the U.S. and the world to audiences in Latin America. There are no related programs that share similar broadcasting goals and objectives with which the BBG can collaborate. In terms of international communications, the BBG participates in weekly strategic communication coordination and support meetings through the State Department's International Information Program Interagency Strategic Communication Fusion Team. Members of the Fusion Team share information about their respective plans and activities in order to leverage each other's communication with international publics. The team coordinates and de-conflicts the production and the dissemination of information products.

Evidence: ?? U.S. International Broadcasting Act of 1994 ?? Foreign Affairs Reform and Restructuring Act of 1998 ?? Voice of America Charter ?? Fusion Team weekly agendas

Does the program use strong financial management practices?

Explanation: The BBG is cross-serviced by the State Department to provide financial services using the Broadcasting Financial Management System (BFMS) system. Internal controls are in place to provide for the separation of duties, minimize erroneous payments, ensure recording of receipts, and the accurate reporting to external entities. The BBG Administrative Officers obligate financial transactions accurately and reconcile their allotments on a monthly basis. In the CFO Office, the Financial Operations, records and makes disbursements, reconciles the Fund Balances with Treasury, records and makes deposits, and prepares the monthly, quarterly and annual statement reports for the agency. FY2004 was the first year the BBG prepared the Performance and Accountability Report (PAR) and conducted a financial statement audit. The PAR was submitted on time and the BBG received an unqualified opinion on its Balance Sheet.

Evidence: ?? BFMS System. ?? An independent audit conducted in October 2004 by the OIG offered an unqualified opinion of the BBG financial statements. ?? The State Department has also received clean audit opinions.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The Office of Program Review conducts an independent analysis of program content and presentation, which includes the management of the program. The Action Plans resulting from the Program Reviews lay out specific and meaningful steps to correct deficiencies when they are identified. The Office of Program Review regularly follows up with managers to track progress. The senior managers' performance based appraisal system requires critical elements and ratings based on the mission, and are tied to the objectives of the BBG. Specific goals with measurable targets are also set for each element within the appraisal. Neither the GAO nor the OIG has made specific recommendations that address management deficiencies in Latin American broadcasting programs.

Evidence: ?? Program Reviews ?? BBG Manual of Operations and Performance Based Appraisal System, Section 490

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: For VOA Creole to Haiti, the audience reach has increased from 40.5% weekly audience in FY 2002 to 45% estimated in FY 2005. For Radio and TV broadcasting to Cuba, the audience reach has remained constant from 2004 to estimated levels in 2005. However, recent gains in program quality scores for broadcasting to Cuba support the expected progress towards the long-term goals set beyond FY 2005. VOA Spanish also shows some gains toward the audience reach long-term goal: in 2002 the radio audience size ranged from 0.2%-5.4% across the region and in 2005 the radio audience size ranged from 0.2%-9%. In 2003, VOA Spanish's television audience ranged from 1.1%-2.8% and in 2005 the television audience size ranged from 0.7%-3.7%.

Evidence: ?? U.S. International Broadcasting to Latin America FY 2007 Performance Plan

Does the program (including program partners) achieve its annual performance goals?

Explanation: There are 4 annual measures for each program in Latin America, Haiti, and Cuba: 1) percent of audience awareness; 2) program quality, 3) signal strength, and 4) cost per listener (addressed in question4.3). For percent of audience awareness, data are available from 2002-2005 for broadcasting to Latin America and for broadcasting to Haiti and both show a one-time decrease in audience awareness in 2002 or 2003, and steady levels from 2004 to 2005. For broadcasting to Cuba, data are available from 2004 and 2005 and show steady levels of audience awareness. For program quality, ratings for broadcasting to Latin America have dipped since 2002, but show steady increases from 2003-2005. For broadcasting to Cuba, radio and television programs show increased quality ratings. For broadcasting to Haiti, program quality has decreased from 2004 to 2005, but the 2005 level is consistent with 2002 and 2003 levels. For signal strength, all programs in Latin America and Haiti have increased signal strength, and programs in Cuba have remained constant or declined slightly (due to local jamming efforts).

Evidence: ?? U.S. International Broadcasting to Latin America FY 2007 Performance Plan

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The BBG has a common measure for all language services, cost-per-listener, that tracks the cost effectiveness of broadcasting. Since the weekly audience listening rate is not always available for those areas closed to surveys, not all years have the cost per listener rate. Although there are increases in the cost per listener from FY 2004 to the expected FY 2005 values in broadcasting to Latin America, Haiti, and Cuba, the agency anticipates lower cost per listener in future years due to anticipated increased efficiencies and greater audience reach. (This is based on the most recent Program Reviews and Language Service Reviews.) In the past year the VOA Spanish service and OCB have increased efficiency in radio and TV production by holding joint editorial meetings, providing joint coverage of major events, and by sharing technical production and internal feed capabilities.

Evidence: ?? VOA Latin American Division individual language service performance plans and Office of Cuba Broadcasting performance plan ?? Language Service Review ?? Program Review

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: BBG is solely responsible for all non-military, U.S. international broadcasting.

Evidence: ?? U.S. International Broadcasting Act of 1994 ?? Foreign Affairs Reform and Restructuring Act of 1998 ?? Voice of America Charter ?? Radio Broadcasting to Cuba Act ?? Television Broadcasting to Cuba Act

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Yearly independent research indicates that all broadcasting services in Cuba, Haiti, and Latin America reach target audiences with appropriate and effective material. Also, annual Program Reviews and Language Service Reviews indicate that the Latin American Division and OCB are effective programs and are achieving results.

Evidence: ?? InterMedia research including surveys, monitoring panels, and listener panels ?? Program Review schedules, results and action plans ?? Language Service Review analysis and results