Detailed Information on the
Broadcasting to Near East Asia and South Asia Assessment

Designing and implementing additional performance measures that capture the quality and credibility of the radio and television programs.

Revising the current agency strategic plan to build on a market-based approach to broadcasting.

Designing and implementing additional long-term performance measures that align with the BBG mission and vision.

Ensuring that funding for broadcasting to Near East Asia and South Asia is explicitly linked to the accomplishment of relevant performance goals.

Measure: Audience Reach in Afghanistan. (%) Note: The program has established a long-term goal for audience size that maintains a very high level of Audience Reach overall, even though the program anticipates that competition for audience from other broadcasters and preferred media choices will increase substantially over the next few years.

Explanation:A key indicator of whether or not BBG broadcasts are reaching their intended audience is the number of people who choose to listen to or view BBG programs. Audience Reach, also called weekly audience, is a measure of the regular listening/viewing audience. Regular audience is defined as all adults in a given population listening/viewing at least once a week, as determined by an audience survey that has an adequately designed sample. The number is a reflection of several annual common measures, such as program quality and the audience's awareness of the program, used for all BBG language services broadcasting in Arabic. Annual measures of audience size, as well as annual performance goals for each individual service, factor into the overall Audience Reach for broadcasting in Afghanistan.

Measure: Audience Reach in Iran. (%) Note: increased anti-Americanism, more media choices in Iran, and government restriction against international broadcasting influenced the slight decrease in BBG audiences in 2007; therefore targets for future years were adjusted, since these factors will likely continue.

Explanation:A key indicator of whether or not BBG broadcasts are reaching their intended audience is the number of people who choose to listen to or view BBG programs. Audience Reach, also called weekly audience, is a measure of the regular listening/viewing audience. Regular audience is defined as all adults in a given population listening/viewing at least once a week, as determined by an audience survey that has an adequately designed sample. The number is a reflection of several annual common measures, such as program quality and the audience's awareness of the program, used for all BBG language services broadcasting in Arabic. Annual measures of audience size, as well as annual performance goals for each individual service, factor into the overall Audience Reach for broadcasting in Iran.

Measure: Audience Reach in Bangladesh. (%)

Explanation:A key indicator of whether or not BBG broadcasts are reaching their intended audience is the number of people who choose to listen to or view BBG programs. Audience Reach, also called weekly audience, is a measure of the regular listening/viewing audience. Regular audience is defined as all adults in a given population listening/viewing at least once a week, as determined by an audience survey that has an adequately designed sample. The number is a reflection of several annual common measures, such as program quality and the audience's awareness of the program, used for all BBG language services broadcasting in Arabic. Annual measures of audience size, as well as annual performance goals for each individual service, factor into the overall Audience Reach for broadcasting in Bangladesh.

Measure: Audience Awareness in Pakistan. (%) Audience Awareness is the percent of the sampled population, prompted or unprompted, that acknowledges BBG station names in an audience survey that has an adequately designed sample. Note: This measure can fluctuate from year to year based on whether programs are transmitted on preferred media, are placed with popular affiliates, or have sufficient advertising. If annual declines are noted, future targets are adjusted.

Explanation:Audience Awareness is the percent of the sampled population, prompted or unprompted, that acknowledges BBG station names in an audience survey that has an adequately designed sample. The statistic is expressed as a percent of the sampled population that knows or recognizes BBG stations, which can then translate to the percent of the entire population of the country or region of which the sample was representative. Annual measures of the percent of a target audience that is aware of BBG programming are tracked by each individual language service, and directly support the long-term measures of growing the size of the audience and increasing the overall perceived reliability of BBG programming. The data shown here is specific to Pakistan.

Measure: Audience Awareness in India. (%) Audience Awareness is the percent of the sampled population, prompted or unprompted, that acknowledges BBG station names in an audience survey that has an adequately designed sample. Note: This measure can fluctuate from year to year based on whether programs are transmitted on preferred media, are placed with popular affiliates, or have sufficient advertising. If annual declines are noted, future targets are adjusted.

Explanation:Audience Awareness is the percent of the sampled population, prompted or unprompted, that acknowledges BBG station names in an audience survey that has an adequately designed sample. The statistic is expressed as a percent of the sampled population that knows or recognizes BBG stations, which can then translate to the percent of the entire population of the country or region of which the sample was representative. Annual measures of the percent of a target audience that is aware of BBG programming are tracked by each individual language service, and directly support the long-term measures of growing the size of the audience and increasing the overall perceived reliability of BBG programming. The data shown here is specific to India.

Measure: Program quality score for the Afghan Radio Network. (Range 1-4; 4 being highest) Note: Within the range, small variations are common, but maintaining the rating or progressing towards the next rating is significantly ambitious, especially as the rating nears the perfect score of 4.

Explanation:The program quality score is an outcome measure based in part on how the target population perceives the quality of the specific BBG language services. This measure is an industry standard used to determine the target audience's assessment of the quality of media. A monitoring panel of regular listeners from the target population rates the program's accuracy, reliability, authoritativeness, objectivity, comprehensiveness, and other variables reflecting distinct statutory, policy, and mission mandates for the different stations. Another component of the program quality score is the evaluation of IBB delivery systems (engineering and transmission), marketing and program placement efforts, and other broadcasting support functions. Program content and presentation are evaluated by both internal and external reviewers to determine the overall Program Quality Score. In total, the Program Quality Score is a comprehensive, weighted score, which ranges from 1-4, with 1.0-1.3 = poor; 1.4-1.6 = poor to fair; 1.7-2.3 = fair; 2.4-2.6 = fair to good; 2.7-3.3 = good; 3.4-3.6 = good to excellent; 3.7-4.0 = excellent. Annual measures of the perceived quality of BBG programming are tracked by each language service, and directly support the long-term measures of growing the size of the audience throughout Near East Asia and South Asia, and increasing the overall perceived reliability of the programming. The data shown here is specific to the Afghan Radio Network.

Measure: Cost-per-audience member for broadcasting to Pakistan. Note: in 2005 television programming was added, leading to a large jump in program costs.

Explanation:This efficiency measure tracks the cost per audience member of each language service. Cost per audience member is the total cost of programming and delivery divided by the unduplicated audience for all media, and is calculated for each language service. The ratio (measure) must be evaluated in context of the changes in either the cost or audience size. For example, a short-term capital cost increase to improve transmission may not increase audience size until after a lengthy installation. In some cases, audience size may be kept artificially low by restrictions imposed by the host government or increased competition, not necessarily poor programming or inefficient management. Costs per audience member may fluctuate annually and also differ compared to other language services based on a combination of factors in each location, including: preferred media in the region, hours of programming, competition from other broadcasters, jamming, and government restrictions. The data shown here is specific to Pakistan.

Is the program purpose clear?

Explanation: The program purpose is to promote and sustain freedom and democracy by broadcasting accurate and objective news and information about America and the world to audiences overseas (in this case, audiences in Near East and South Asia).

Evidence: U.S. International Broadcasting Act of 1994; Foreign Affairs Reform and Restructuring Act of 1998; Voice of America Charter

Does the program address a specific and existing problem, interest or need?

Explanation: The program is meant to address the need for accurate news and information about the United States and events in Near East and South Asia in languages spoken in those countries.

Evidence: The dearth of free, unbiased media and the existence of anti-American reporting in these regions creates a need for timely and accurate reporting of U.S. policies and actions as well as events in the regions and the world at large. Freedom House Press Freedom Ratings for Near East and South Asia countries is overwhelmingly "Not Free".

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The Broadcasting Board of Governors (BBG) is the independent Federal entity responsible for all U.S. government and government-sponsored, non-military international broadcasting. As such, its broadcasts are not duplicative of other Federal contributions to the problem being addressed in Near East and South Asia. In those countries that have broadcasting by VOA, RFE/RL, and the Middle East Television Network the BBG ensures the missions are distinct, complementary and not duplicative. For example, in FY 2004 the BBG is phasing out RFE/RL's Radio Free Iraq broadcasting. In 2004, BBG merged all of its Arabic radio and television broadcasting under a new grantee, the Middle East Television Network (MTN). VOA and RFE/RL contribute their individual programming to Radio Farda (Persian) to create one streamlined broadcast.

Evidence: The U.S. International Broadcasting Act of 1994 and the Foreign Affairs Reform and Restructuring Act of 1998 state that U.S. international broadcasting should not duplicate the activities of private United States broadcasters or the activities of government-supported broadcasting entities of other democratic nations. The FY 2004 Program Plan provides details of reductions and streamlining of dual broadcasting.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: There is no conclusive evidence that another approach would be more efficient/effective at achieving the program's goals. To further achieve its goals, in 2004 VOA announced a new structure which merges VOA and the former WORLDNET TV to create a focused, multimedia organization that will reach larger audiences with enhanced programming in the most effective media. BBG's Arabic broadcasting was merged under the new grantee, MTN, to combine staff resources and ensure that U.S. Arabic broadcasting is coordinated.

Evidence: U.S. International Broadcasting Act of 1994; Foreign Affairs Reform and Restructuring Act of 1998; Voice of America Charter; VOA and WORLDNET merger.

Is the program effectively targeted, so that resources will reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: The BBG is the sole Federal entity that broadcasts to Near East and South Asia for the purpose of delivering accurate and objective news and information about the United States and the world. Annual Language Service and Program reviews continually refine the target audience and broadcast in the preferred medium. The lack of unbiased news and information on television broadcasts in the Near East resulted in BBG's creation of Alhurra in February 2004 under the Middle East Television Network (MTN). Alhurra is a 24-hour Arabic satellite television network broadcasting news and information to the region. Radio Sawa (Arabic) and Radio Farda (Persian) specifically target the 15-30 age group. Programming is based on continuous research into the listening preferences of this age group, with news, information and music combined to draw audiences and communicate news about the region and world.

Evidence: U.S. International Broadcasting Act of 1994; Foreign Affairs Reform and Restructuring Act of 1998; Voice of America Charter; Near East and South Asia Program Reviews and annual Language Service Reviews

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: BBG's Near East and South Asia performance plans include a long-term outcome measure that ties annual language service activities to the BBG's overall strategic goals. The primary long-term measure is to increase audience reach. Given that measuring the program's impact on sustaining freedom and democracy in the region is virtually impossible, audience reach is the most important and quantifiable measure of the program's success. This measure is a culmination of a number of annual common measures used across all of the VOA and RFE/RL language services, as well as individual annual action steps for each broadcasting area. The impact of annual activities should result in increased audience awareness across the program and an overall growth in audience size. BBG continues to explore additional or improved long-term measures or goals that address the quality and relevance of broadcasting. BBG is also drafting a performance plan for the brand new Middle East Television Network, and will incorporate the Radio Sawa performance plan.

Evidence: BBG Strategic Plan. Near East and South Asia individual performance plans.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: BBG has set a long term target for the primary long-term performance measure of increasing audience reach for each of its language services in this region. The baseline for each is the FY 2002 level, and the targets are aggressive and ambitious. For example, the BBG launched a new Arabic satellite television service in February 2004, Alhurra, and have set the very ambitious goal of an audience reach of 21 million viewers in five years. The five-year target for audience reach for Radio Sawa (Arabic) is to increase from 3.8 million listeners per week to 21 million. The other language services have similarly ambitious targets. The related annual goals and action steps will track the progress toward achieving the long-term goal for each language service.

Evidence: Near East and South Asia performance plans; Language Service Review; Program Review

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The BBG's Near East and South Asia broadcasts have annual performance measures that support the long-term goal of increasing audience reach. There are 5 common measures for the program: 1) percent of weekly audience, 2) percent of audience awarenss, 3) program quality rating, 4) signal strength rating, and 5) cost per listener. The last common measure - cost per listener - is an efficiency measure. Each language service or area also has, or is developing, individual annual action steps with milestones tailored to the nature of the broadcasting and target population. These contribute to achieving the annual and long-term goals.

Evidence: Near East and South Asia performance plans; Language Service Review; Program Review

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The common measures have separate baselines and annual targets for each broadcasting area, and in many cases for VOA and the grantees separately. BBG established these separate targets so that language service managers could easily track performance for each country or region they broadcast to, helping them identify areas that need improvement. The baseline for the common measures is the FY 2002 level.

Evidence: Program individual performance plans; Language Service Review; Program Review

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: As a grantee of BBG, RFE/RL contributed to establishing the new Strategic Plan and committed to the FY 2006 performance plans for Near East and South Asia, including the annual action steps for its broadcasting that support the overall BBG long-term goals in the program. Additionally, BBG is working with its new grantee, MTN, to create an MTN performance plan and integrate the Radio Sawa performance plan as part of the merger of Arabic broadcasting entities.

Evidence: Near East and South Asia performance plans; Annual Language Service Reviews; Program Reviews; BBG Annual Performance Plans and Reports.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: BBG's annual Language Service Reviews and Program Reviews conducted by the BBG's Office of Program Review (OPR) serve this purpose. OPR operates independently from all BBG language services. Annual reviews of VOA language services include an initial assessment with regularly scheduled follow-up meetings to check the progress on recommendations generated by the review. The reviews assess radio content, engineering and transmission, marketing efforts, internet development, and production value. BBG contracts with InterMedia, an independent firm that conducts annual surveys for each language service and program,including VOA, RFE/RL and the Middle East Television Network. InterMedia evaluates a variety of factors: market profile, media access and use, and international broadcasting impact and trends. InterMedia also profiles audience listening habits.

Evidence: The annual Language Service Review assesses two basic issues: (1) where should BBG broadcast and (2) how well is BBG broadcasting. The Program Reviews are annaul uality-control mechanisms based on field research and external analyses of program content and presentation. Program Review Schedules.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: In FY 2006, the BBG plans to include annual performance plans with performance measures for Near East and South Asia in the budget request. As in FY 2005, all budget enhancement requests for the Agency will be performance based with measures, goals and annual targets. The BBG intends to build upon this progress with fully integrated performance-based budgets in the future.

Evidence: FY 2005 Budget Request; language Service Review; Program Review; FY 2006 BBG Budget Request

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: BBG continues to use the individual language service or broadcast area performance plan template, including broadcasting to Near East and South Asia. BBG is working with the newly created Middle East Television Network to develop a performance plan for Alhurra and incorporate existing plans for Radio Sawa. A newly reported common measure of Audience Awareness will further strengthen the goal of reaching the audience and gauging the effectiveness of marketing, branding and affiliations. The performance plans are based on the BBG strategic plan, and they attempt to tie individual program goals to the strategic goals of the agency. BBG also continues to review and revise the current agency strategic plan to build on the innovative new approach to international broadcasting, "Marrying the Mission to the Market."

Evidence: BBG Strategic Plan, Near East and South Asia Performance Plans

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: Using performance data gathered through its annual Language Service Reviews and Program Reviews, the BBG examines the results to adjust resources and set strategic priorities. Program Reviews also include content and production analysis to assist managers in improving performance. These same data are included in the Performance Plans.

Evidence: Language Service Review, Program Review, Near East and South Asia Performance Plans.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Senior managers' "Line of Sight" performance appraisal system requires critical elements and ratings that are tied to the objectives of the BBG. The annual Language Service Review holds managers and partners directly accountable for results by the corresponding resource allocations, including the grantees broadcasting to Near East and South Asia. Contractors are accountable to the same program standards. The Program Review process also measures the quality of their work. The grantees must report the same common measures. Many contractors work on a "pay-for-product" basis, with payment dependent upon the timely receipt of quality products. For example, many "stringers" receive payment for each quality news story they deliver.

Evidence: "Line of Sight" Performance Standards/Appraisals; Manual of Operations and Administration, Sec. 490 - Performance Appraisal System; Language Service Review

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: All requisitions must include verification that funds are available, and the funds are obligated when a contract or agreement is signed or a purchase order is done. The mid-year and 3rd quarter review provides an opportunity for managers to verify that unliquidated obligations are valid and that once an obligation has been fully expended, remaining balances are deobligated. Financial transactions are coded to both an organization and to a function or project. Given that BBG has a high percentage of fixed costs (salaries, transmission and infrastructure expenses, rent, etc.), funds must be obligated in a timely manner in order to continue operations. The BBG requires an annual financial plan and monthly status reports from its grantees. The financial plans must detail anticipated costs for specific line items by each month of the fiscal year. BBG conducts a mid-year, 3rd quarter, and year-end review of each grantee's financial plans and budgets to monitor funding and expenditures. BBG also requires a monthly status report of expenditures, disbursements and obligations as well as the submission of the SF-169, Financial Status Report and SF-172, Report of Federal Cash Transactions.

Evidence: The reviews and the allotment control procedures enable the Agency to track obligations to ensure that funds are used for the intended purpose by the appropriate Agency office. SF-169, Financial Status Report and SF-172, Report of Federal Cash Transactions. Montly status reports from RFE/RL and MTN on expenditures, disbursements, obligations. Emergency Response Fund monthly reports.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The annual Language Service Review process includes an efficiency measure -- cost per listener. The Program Review process also assesses the efficiency and productivy of each language service. The merger of VOA and WORLDNET TV will improve efficiency through utilization of multimedia resources to effectively reach the audience in the preferred media. BBG recently reorganized its various IT components into one structure to avoid duplication of effort and centralized efficiencies.

Evidence: East Asia and Eurasia Performance Plans; Language Service Reviews; Program Review ; VOA and WORLDNET merger; IT restructuring

Does the program collaborate and coordinate effectively with related programs?

Explanation: BBG is solely responsible for all non-military, U.S. international broadcasting. It is the only entity with the express objective of promoting freedom and democracy through broadcasting objective news and information about the U.S. and the world to audiences in Near East and South Asia. There are no related programs that share similar goals and objectives with which the BBG can collaborate. Within the BBG, VOA, RFE/RL and the Middle East Television Network collaborate on program content and the BBG has worked to reduce the duplication.

Evidence: U.S. International Broadcasting Act of 1994; Foreign Affairs Reform and Restructuring Act of 1998; Voice of America Charter

Does the program use strong financial management practices?

Explanation: The BBG is cross-serviced by the State Department to provide financial services using the BFMS system. Internal controls are in place to minimize erroneous payments. The BBG Administrative Officers obligate financial transactions, and the State Department processes the payments, thus ensuring checks and balances on BBG accounts.

Evidence: BFMS System. The State Department has received clean audit opinions.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The Office of Program Review conducts an independent analysis of program content and presentation, which includes the management of the program. The Action Plans resulting from the Program Reviews lay out specific and meaningful steps to correct deficiencies when they are identified. The Office of Program Review regularly follows up with managers to track progress. The grantees follow a similar process and report the common measures to the BBG. The senior managers' "Line of Sight" performance appraisal system requires critical elements and ratings based on the mission and is tied to the objectives of the BBG.

Evidence: Program Reviews; BBG Manual of Operations and Administration "Line of Sight" Performance Appraisal System, Section 490

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The BBG has achieved strong results and is on track to reach it's long term goal for each language service. In 2004 a few language services experienced a slight decrease in the number of listeners per week, but demonstrated strong increases in its priority language services. Broadcasting to Afghanistan, Iran, and the Caucusus all experienced increases in audience reach. The biggest achievements were in Arabic broadcasting, both radio and television. Radio Sawa (Arabic) far outpaced its annual audience targets, increasing from 3.1 million weekly listeners in 2002 to 10.7 million in 2003, and 14.3 million in 2004. Radio sawa is on track to reach 21 million listeners by 2008. In February 2004 the BBG launched an Arabic satellite television network, Alhurra, which reached a weekly audience of 8.5 million viewers in 2004.

Evidence: Language Service Review, Progam Review, Near East and South Asia Performanc Plans

Does the program (including program partners) achieve its annual performance goals?

Explanation: In 2003, BBG's Arabic broadcaster Radio Sawa far exceeded its weekly audience target of 5.1 million listeners, with a weekly listening rate of 10.5 million people. Broadcasting to Iran also made large gains in 2003, reaching 13% of the population (doubling the original target of 6.5%). BBG's new pan-Arabic satellite television network, Alhurra, has shown immediate results, and is estimated to garner a weekly audience of 8.5 million viewers by the end of FY 2004. The BBG has had less success in its smaller language services broadcasting to Bangladesh and India, with its estimated targets for 2004 decreasing. Several language services also suffered a decline in audience awareness in 2003, but BBG aims to increase awareness in 2004 and 2005. Generally, since 2002 the Near East and South Asia language services have been improving in the most important measure, weekly audience reach.

Evidence: Near East and South Asia Performance Plans, Program Reviews, Language Service Reviews.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The BBG has a common measure for all language services, cost-per-listener, that tracks the cost effectiveness of broadcasting. Since the weekly audience listening rate is not available for all language services in the Near East & South Asia, BBG cannot track the cost-per-listener of all language services. Those with data indicate that the cost-per-listener is fluctuating due to varying listening rates from year to year. However, most language services with cost-per-listener data indicate the costs decreased in FY 2003 from the previous year.

Evidence: Near East and South Asia Performance Plans, Program Reviews, Language Service Reviews.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: BBG is solely responsible for all non-military, U.S. international broadcasting. It is the only entity with the express objective of promoting freedom and democracy through broadcasting objective news and information about the U.S. and the world to audiences in Near East and South Asia. There are no related programs that share similar goals and objectives with which to compare BBG's performance.

Evidence:  

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Annual Program Reviews and Language Service Reviews indicate that BBG's broadcasting to Near East and South Asia is improving, with audience reach to Arabic-speaking populations increasing dramatically in the past two years. InterMedia and AC Nielsen have also conducted independent surveys that indicate BBG's Radio Sawa is one of the most popular international broadcasters in the region. The weekly audience reach in 2003, which measures the percent of target audience tuning in, ranges from a low of 1.1% in Bangladesh to a high of 49% in Afghanistan, indicating a wide range of impact on the target populations.

Evidence: Program Review schedules, results and action plans; Language Service Review analysis and results.