Detailed Information on the
Engineering and Technical Services for US International Broadcasting Assessment

Achieving a permanent reduction in total engineering and technical costs per audience member from the 2004 baseline of $0.98 to $0.55 in 2013.

Ensure that the satellite effectiveness index is maintained or increased. This index provides a quantitative measure of the ability of the satellite network to access the population of TV households, which enables the BBG to reach the target audience at the time and in the media they may prefer. It accounts for TV population, total satellite network capacity, signal strength, and prime-time flexibility. This index considers five criteria: 1) Coverage of the satellite in channel-hours; 2) Coverage to small (3 meters or less) antennas; 3) Time-zone flexibility to ensure prime-time coverage; 4) TV households reached; and 5) Ability to feed other satellites as part of the network.

Revising the current agency strategic plan to build on a market-based approach to broadcasting.

Measure: Cost-per-audience member for all broadcasting. ($) Note: The long-term outcome goal of BBG Engineering and Technical Services is to increase the efficiency of its program by utilizing an effective mix of popular media to reach the largest population in targeted audiences in the most cost-effective manner. This goal will be measured using the efficiency indicator "BBG Engineering and Technical Services Program Cost per Audience Member."

Explanation:This long-term outcome and efficiency measure is the total annual cost for radio and television transmission and Internet delivery for the BBG divided by the total BBG audience. These costs include the construction, operation and maintenance of transmitting stations and satellite networks, as well as the external delivery of Internet broadcasts. The indicator does not include internal IT costs, such as computing services or web content development. This indicator also excludes consumable expenses (power and parts). Consumable expenses are associated with specific language services and so are reflected in other BBG PART evaluations as efficiency measures for the language services. The ratio (measure) must be evaluated in context of the changes in either the cost or audience size. For example, a short-term capital cost increase to improve transmission may not increase audience size until after a lengthy installation. In some cases, audience size may be kept artificially low by restrictions imposed by the host government or increased competition, not necessarily ineffective program delivery or inefficient management.

Measure: Satellite Effectiveness Index (TV) Note: The BBG's continued commitment to convert the satellite distribution network to more efficient and effective digital capability, and the resulting costs, contributed to the steady rise in this index through 2007 for TV audiences. However, the index is expected to be maintained or decline in future years as some program distribution is shifted from satellites to more cost effective fiber optic circuits, thus reducing the number of TV households reached by satellite; less effective satellite services are dropped, thus reducing overall network capacity; and the limit of additional capacity that the BBG can acquire is reached.

Explanation:This index provides a quantitative measure of the ability of BBG's satellite network to reach the desired population of TV households around the world. The measure accounts for TV population, total satellite network capacity, signal strength, and primetime flexibility. The index considers five criteria: 1) Coverage of the satellite in channel hours; 2) Coverage to small (three meters or less) satellite dish antennas; 3) Time-zone flexibility to ensure prime-time coverage; 4) TV households reached; 5) Ability to feed other satellites as part of the network. The index was developed by the BBG's Office of Engineering in cooperation with the Broadcasting Satellite Users' Board. The index combines a weighted measure of each criterion in a mathematical formula for each satellite that BBG uses. Scores for this outcome measure reflect the contribution of all the satellites in the BBG network. Each of the five criteria described above are characteristics of an effective satellite network that will contribute to reaching BBG's global TV audiences. Because of the complex interaction among the criteria, targets will vary for this dimensionless index. For example, decreases in the Satellite Effectiveness Index are expected to occur when the BBG switches to other delivery modes (such as fiber optic links) in targeted areas, which are reliable and cost effective for specific requirements but lack the broad regional coverage and flexibility that satellite links provide. Therefore, the Satellite Effectiveness Index, which assess only the satellite dimension of our network performance, will go down. The goal is to maintain scores or reflect fluctuations caused by planned changes in programming, such as overall satellite usage, but to prevent declines in scores caused by a drop in audience or limitations of the satellite network capacity.

Measure: Transmission Network Consumable Expense. ($ millions) Note: This measure of annual consumable expense (power and parts) transmission costs can fluctuate from year to year based on exchange rates, the volatile costs of electricity, jamming by host governments (which incurs additional costs to combat), or the transition to preferred media on delivery systems outside of the transmission network (which will reduce costs). If annual changes are noted or projected, future targets are adjusted. The goal for this output measure is to monitor the balance between costs and hours of programming on shortwave and medium wave. The variable costs of power and replacement parts have been rising rapidly, plus the increase in new surge broadcasting starting in 2008 will cause the overall expense to increase in the next few years.

Explanation:This measure is equal to the total annual cost of power and parts to operate the shortwave and medium wave transmitters in the BBG network around the world and the costs of leasing transmission time on other shortwave and medium wave stations to augment the capabilities of the BBG owned network. These costs generally vary as the number of hours of programming varies, i.e., costs increase as more programming is added and costs decrease as less programming is placed on the network. The fixed costs of operating a BBG-owned station (personnel, facilities maintenance, building and grounds upkeep, land leases, etc.) are not included because such costs vary little with the number of hours of programming transmitted. The variable costs of running a transmitting station, particularly the costs of electricity and fuel to run generators, are climbing dramatically worldwide and unfavorable exchange rates magnify these increases. At the same time, the total hours of programming that BBG transmits by shortwave and medium wave are declining slowly because of the shift to more audience preferred media (such as FM radio, television, and Internet services), although the addition of urgent surge broadcasting requirements in crisis areas has moderated this decline.

Measure: Annual Transmitter Hours. (# of hours)

Explanation:This measure gives an indication of the scope of the global transmission network by tallying the total numbers of hours that transmitters are in use for broadcasting programs. A typical program is often carried on a number of transmitters simultaneously to provide reliable coverage over a broad geographical area and to satisfy different media preferences of the audience. The BBG network transmitters support a number of different technical media: shortwave radio, medium wave (or AM) radio, FM radio, digital radio, television, and satellite transmissions. In addition, the network supports television programs for the State Department's American Embassy Television Network and some radio broadcasts for countries that host BBG transmitting facilities.

Is the program purpose clear?

Explanation: The purpose of this program is to implement, operate and maintain a global communication system to deliver the multi-media programs of the Broadcasting Board of Governors (BBG) broadcasting entities to audiences around the world, in the most efficient and effective manner possible. BBG Engineering and Technical Services is responsible for the transmission of shortwave, medium wave, and FM radio, TV programming, and web-delivery of internet programming to audiences nearly any place in the world.

Evidence: ?? BBG Strategic Plan (2002-2007) ?? U.S. International Broadcasting Act of 1994 ?? Foreign Affairs Reform and Restructuring Act of 1998

Does the program address a specific and existing problem, interest, or need?

Explanation: In order to carry out its mission of broadcasting accurate and objective news and information about the United States and the world to audiences overseas the BBG broadcasting entities--including Voice of America (VOA), Radio and TV Marti, and Radio Free Asia (RFA), among others--must have the technical capability available to broadcast news and information via television, radio, and internet. The Office of Engineering and Technical Services provided nearly 1,000,000 broadcast hours to all the various BBG broadcasting entities in 2004. To remain competitive in a developing media market, the BBG also needs resources for new technology and application development, for anti-jamming initiatives for example, which this program also provides. BBG Engineering and Technical Services also serves the media transmission needs of other international programs within the U.S. government, for example by transmiting worldwide the television signals of the American Embassy Television Network for the Department of State, and performing the maintenance of communication (TVRO) facilities for State. BBG Engineering and Technical Services also occasionally works with other federal agencies including the United State Agency for International Development (USAID), the Department of Defense, the Department of Commerce, and the Federal Communications Commission.

Evidence: ?? U.S. International Broadcasting Act of 1994; ?? Foreign Affairs Reform and Restructuring Act of 1998 ?? MOU between BBG and State Department (drafted 2004, updated in 2006)

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: BBG Engineering and Technical Services include the technical offices of the International Broadcasting Bureau (IBB Engineering), Radio Free Asia (RFA), and Radio Free Europe/Radio Liberty (RFE/RL). The technical offices in the IBB, RFA, and RFE/RL coordinate efforts to ensure that BBG program delivery needs are met without duplication and that new technologies of potential use to BBG are explored, developed, and fielded. Functions are centralized when economies can be gained from doing so, and are distributed as required by the needs of the broadcasting entities. IBB Engineering has primary responsibility for the transmission of BBG radio and television programs. Responsibility for Internet delivery is shared among IBB Engineering, RFA, and RFE/RL. The Foreign Affairs Reform and Restructuring Act of 1998 mandates that IBB Engineering alone is responsible for transmitting the programming of the broadcast entities via radio and television. As far as duplication of private efforts, some aspects of IBB operations are inherently governmental; for example, those that require government-to-government agreements or the operation of long-held government assets. IBB Engineering contracts with private sector vendors (satellite transmission system, for example) and external radio operators for the operations and maintenance of IBB assets in locations such as Djibouti. There is no compelling evidence that the private sector could perform this function any more efficiently than the IBB. Some transmissions are rebroadcast by affiliates in the local markets. IBB Engineering provides technical support to the Office of Marketing and Program Placement, which arranges and organizes affiliate support.

Evidence: ?? The U.S. International Broadcasting Act of 1994 and the Foreign Affairs Reform and Restructuring Act of 1998 state that U.S. international broadcasting should not duplicate the activities of private United States broadcasters or the activities of government-supported broadcasting entities of other democratic nations. ?? U.S. International Broadcasting Act of 1994; ?? Foreign Affairs Reform and Restructuring Act of 1998 ?? Operations and Maintenance Contract with Radio and Television Diffusion de Djibouti ?? The BBG Engineering and Technical Services meets and exceeds industry standards such as "on air availability": 99.8% on air availability for a global transmission operation with thousands of daily transmission switches and a complexity of simultaneous radio and television transmissions likely exceed any transmission systems found in the private sector.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: There is no evidence that a different approach or mechanism would be more effective or efficient at meeting the programs goals. BBG Engineering and Technical Services includes the transmission of radio and TV programming and web-delivery of internet programming to audiences nearly any place in the world. There are several benefits to having an in-house capability to provide engineering services: continuity of operations; the ability to easily direct and re-direct engineering and technical resources; participation in agency-wide and language-specific strategic planning, research, and problem-solving; the ability to negotiatiate with other agencies and host governments for technical agreements; and the flexibility to adjust engineering specifics (such as antenna direction) to reach target audiences most effectively. As stated above, functions are centralized when economies of scale can be gained from doing so, and are distributed to either separate broadcasting entities or to the private sector as appropriate.

Evidence: ?? U.S. International Broadcasting Act of 1994; ?? Foreign Affairs Reform and Restructuring Act of 1998

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: Engineering and Technical Services reaches two groups of intended beneficiaries. The first is the assortment of internal broadcasting organizations within the BBG, in terms of delivering the signal; the second is composed of worldwide listeners, in terms of actually reaching the desired audience. BBG broadcasting: Through the annual Capital and Information Technology Planning Process the Planning and Coordination teams within the Engineering office evaluate the capacity of existing capital assets to bridge the performance gap between current and planned program results and identify the technical requirements necessary to close the gaps in achieving annual and long-term performance goals. The relative need and impact of each new investment is weighed by the BBG Executive Review Committee Staff Group, which recommends specific proposals to the Board. This evaluation feeds into annual resource allocation decisions in terms of capital investment budget enhancement requests to OMB. Throughout this process, Engineering and Technical Services makes sure that they are meeting the agency's highest priority capital projects, i.e. the intended internal beneficiaries. Also, by participating in the Program Review process, which assessed the effectiveness of each language program, the Engineering department assures that its resources are reaching target audiences, and if not, the work plan that is established as a follow up to the program review can be designed to improve targeting. Listeners: BBG Engineering and Technical Services also participates in the annual Program Review and Language Service Review processes to inform decisions about the infrastructure and technical support services necessary to carryout the broadcasting mission. Program Review conducts audience and market research that determines whether or not the target audiences are being reached and whether unintended audiences are being reached. Language Service Review evaluates the importance of reaching each of the target audiences. The BBG Board uses the results of these reviews to adjust the transmission of radio and TV programming, in terms of devoting more resources, different technologies, or making mechanical adjustments to existing transmitters, and in terms of web-delivery of internet programming to target and reach specific target audiences. Selection of appropriate media (e.g. FM, medium wave or shortwave radio, TV, internet) is one of several components that contribute to audience reach. BBG Engineering and Technical Services works in conjunction with the broadcast entities, IBB research, independent market researchers, and the Office of Marketing and Program Placement to determine the appropriate technology to be used to reach a given target audience. In cases in which the appropriate technology is available, but the US government's transmission signal is degraded, either by jamming or other means, then BBG Engineering and Technical Services employs appropriate anti-jamming techniques to improve reception.

Evidence: ?? U.S. International Broadcasting Act of 1994 ?? Foreign Affairs Reform and Restructuring Act of 1998 ?? Capital and Information Technology Planning Process ?? Program Review ?? Language Service Review ?? Excerpts from InterMedia research report on Persian language broadcasting ?? VOA Persian Action Plan Follow Up Notes -- June 17, 2003 show that an action was recommended for Engineering and Technical Services to investigate the costs of placing the Perisan weekly TV/radio simulcast on a more effective satellite for reaching Iran, which was done and implemented. ?? Action plans/correspondence from Program Reviews demonstrating IBB Engineering and Technical Service's responsiveness to issues raised by BBG language services ?? BBG is also engaged in a shift from transmitting via short-wave radio to FM radio, television, and internet broadcasting, in an effort to reach target audiences more effectively and minimize the unintended beneficiaries.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The BBG has one long-term outcome measure for Engineering and Technical Services: "Program cost per audience member." The goal is to maintain an efficiency level that is 15% lower than the baseline level in 2004 by 2009, and BBG will do so by reaching the largest targeted audiences in the most cost-effective manner. The measure is derived by dividing the total engineering and technical services program costs by the total BBG audience. The total BBG audience measures the actual number of listeners and viewers that watch or listen to BBG programming, and is therefore an outcome measure. Program costs include the costs of construction, operation and maintenance of transmitting stations and satellite networks, as well as the external delivery of Internet broadcasts. An efficiency goal is acceptable for this program since efficiency (in terms of cost) and effectiveness (in terms of audience size) is a part of this program's purpose.

Evidence: In FY 2004 the actual program cost per Audience Member was $0.98, meaning that the goal for 2009 and beyond is no more than $0.83 per listener. ?? BBG Strategic Plan (2002-2007) ?? BBG Engineering and Technical Services FY 2007 Performance Plan

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The long-term target for increasing the efficiency of program cost per audience member is sufficiently ambitious since it entails successful implementation of planned efficiencies, and will be subject to variable consumable costs such as exchange rates and power costs. High or unexpected increases in these variable costs, or a transition to more expensive technologies such as television, may impact the overall program cost in the future and will be monitored through the annual measure of Transmission Network Consumable Expense. Any decrease in audience size will have to be met with increased efficiencies over the years in order to maintain a cost of less than $0.83 per listener.

Evidence: In FY 2004 the actual program cost per Audience Member was $0.98, meaning that the goal for 2009 and beyond is no more than $0.83 per listener. Initial estimates for FY 2005 and FY 2006 are $0.78 and $0.71, respectively. This significant increase in efficiency in 2005-2006 reflects an initial surge in audience size due to new programming in the Arab world. The uncertainty of continued audience increases in light of the developing media market combined with variable costs that are unpredictable make meeting the target sufficiently ambitious for the BBG. ?? BBG Strategic Plan (2002-2007) ?? BBG Engineering and Technical Service FY 2007 Performance Plan

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: There are four annual performance measures that support the long-term goal of increasing the efficiency of the program: 1) Radio Signal Strength Index; 2) Satellite Effectiveness Index; 3) Annual Transmitter Hours; and 4) Transmission Network Consumable Expense. The first three are components of broadcast reach, while the fourth goal is a subset of the overall Program Cost. The first two measures are outcome-oriented since they reflect the effectiveness of the program in terms of producing audible and visable signals anywhere in the world. The second two are output measures and are used to monitor the overall size and cost of the program's operations, which can be used to guage overall efficiencies and for strategic planning purposes. Radio Signal Strength rates whether programs are capable of being heard by target audiences on AM or MW frequencies, and are rated by internal and external monitors on a scale from 1 (indicating no reception) to 5 (indicating excellent reception). Satellite Effectiveness is a composite rating of 1) coverage in channel hours; 2) coverage to small satellite dish antennas; 3) time-zone flexibility to ensure prime-time coverage; 4) TV households reached; 5) ability to feed other satellites as part of the network. Annual transmitter hours is a tally of the total numbers of hours that transmitters are in use for broadcasting programs. Transmission Network Consumable Expense is equal to the total annual cost of power and parts to operate the SW and MW transmitters in the BBG network around the world and the costs of leasing transmission time on other SW and MW stations. By tracking the goals and measures since FY 2002 and setting targets for each performance measure through FY 2007, they indicate progress towards achieving the long-term goal.

Evidence: ?? BBG Strategic Plan (2002-2007) ?? BBG Engineering and Technical Service FY 2007 Performance Plan

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The annual targets for BBG's annual measures are sufficiently ambitious given the fluctuating nature of exchange rates, fuel costs, weather issues, etc.

Evidence: The Radio Signal Strength Index measures only cross-border shortwave and medium wave signals that can be easily impacted by weather and sunspot activity, so maintaining an Index between 2.5 and 3.5 as a summary measure for all broadcasting worldwide is considered ambitious. Since the baseline of FY 2002, the Satellite Effectiveness Index has shown steady increased from 5.80 to 7.90 in 2004, and expected value of 9.0 in FY 2005 that will be maintained in 2006. The annual transmitting hours are projected to increase by 4% in 2005 and by 17% in 2006 (from the previous year), while the transmission network consumable expense is projected to decrease by 6% in 2005 and 3% in 2006 (from the previous year). ?? BBG Engineering and Technical Service FY 2007 Performance Plan

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: The IBB's Office of Engineering and Technical Services and the technical offices within two of the BBG's grantee organizations, Radio Free Asia and Radio Free Europe/Radio Liberty, contributed to establishing the Strategic Plan and in the development of the annual performance plan, which will include the annual and long-term goals of the program. The headquarters offices represent the transmitting stations. Senior managers within the BBG Engineering and Technical Services Programs commit and work towards the goals through the BBG performance based appraisal system, which is tied to program objectives and goals. Contracts for the program are managed by contract officers who monitor compliance with performance standard clauses that impact the goals of the program, such as schedule, cost and technical requirements. The Engineering and Technical Services participation in all language service Program Reviews and the annual Language Service Reviews also enforce commitment to the annual and long-term goals in support of the broadcasting mission of the BBG. Government partners also contribute to the mission of this program: State helps to facilitate international relationships on behalf of the BBG and provides administrative functions overseas such as invoice payment processing, contract awarding, personnel support, and fund certification through the International Cooperatiive Administrative Support Services (ICASS) memorandum of agreement. The Secretary of State is also a legally mandated member of the Board, and has the same role and responsibility as other board members in commiting to and working toward the annual and long-term goals of the program.

Evidence: ?? BBG Strategic Plan (2002-2007) ?? Language Service Review ?? Program Review ?? Performance Based Employee Standards/Appraisals

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Internal: BBG Engineering and Technical Services are included in the annual Language Service Reviews and Program Reviews conducted by the IBB's Office of Program Review, and RFA's and RFE/RL's program review offices. The offices operate independently from all BBG language services. Annual reviews include an initial assessment with regularly scheduled follow-up meetings to check the progress on recommendations generated by the review. The reviews assess engineering and transmission and internet development, in addition to radio content, marketing efforts, and production value. External: Three methods are used to measure signal quality of shortwave and cross border medium wave transmissions: 1) internationally distributed automated monitoring systems (computers) that assess the signal strength inside (where feasible) and surrounding the targeted transmission area; 2) trained staff monitors that tune to and evaluate the quality of signals intended for a targeted area; 3) annual or more frequent listener surveys for each language program is conducted by independent researchers such as Intermedia and AC Nielson. Government: In FY 2003 and FY 2004 the Inspector General conducted reviews of BBG technical operations and information security (annual audit) and use of the internet and other new technologies (Report Numbers: IBO-A-03-01, IT-A-03-14, IT-A-04-07).. The Inspector General conducted inspections at 7 of the 15 IBB transmitting stations in FY 2003 and FY 2004. The schedule of inspections is determined by the OIG (Report Numbers: IBO-I-04-04, IBO-I-04-02, IBO-I-04-13, IBO-I-04-10, IBO-I-04-06, IBO-I-04-05, and IBO-I-05-05). None of the GAO's reports specifically address the work of BBG Engineering and Technical Services.

Evidence: ?? Language Service Review ?? Program Review ?? Excerpts from InterMedia research report on Persian language broadcasting (Selection of the independent researchers is done through a federal competitive contract process.) ?? OIG: Review of the Information Security Program at the Broadcasting Board of Governors (IT-A-03-14) ?? OIG: Review of the Broadcasting Board of Governors' Use of the Internet and Related New Technologies (IBO-A-04-03) ?? OIG: Inspection of the International Broadcasting Bureau's Kuwait Transmitting Station (IBO-I-04-04) ?? OIG: Inspection of the International Broadcasting Bureau's Transmitting Station in Sri Lanka (IBO-I-04-02) ?? OIG: Review of Information Security Program at Broadcasting Board of Governors (IT-A-04-07) ?? OIG: Inspection of the International Broadcasting Bureau's Djibouti Transmitting Station (IBO-I-04-13) ?? OIG: Inspection of the International Broadcasting Bureau's Germany Transmitting Station (IBO-I-04-10) ?? OIG: Inspection of Broadcasting Board of Governor's Operations in Thailand (IBO-I-04-06) ?? OIG: Inspection of the International Broadcasting Bureau's Transmitting Station in Greece (IBO-I-04-05) ?? OIG: Inspection of the International Broadcasting Bureau's Delano, California Transmitting Station (IBO-I-05-05)

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: BBG has excelled at clarifying its mission, establishing implementation strategies and performance objectives that support the mission, as well as establishing performance indicators. Actions taken in 2006 and 2007 have enhanced the linkages between these strategic tools and increasingly shown how they are supported and implemented by BBG's budget each year. In its FY 2006 Budget request the BBG explicitly linked all costs to six annual performance objectives that support BBG ' s mission, and will do so again in its FY 2007 budget request. In addition, each objective will link annual performance goals to the requested resources with performance measures will be monitored against the targets set for FY 2007. In FY 2007, the BBG will show how the performance objectives for the various broadcasting entities (including VOA and OCB) and support elements support the overall BBG strategic goal and annual performance goals. In FY 2007, the will again include the agency's annual performance plan and performance measures in budget requests. The Capital and Information Technology Planning Process is used in the selection of capital enhancements proposals. The Process evaluates the capacity of existing capital assets to bridge the performance gap between current and planned program results, and identifies those projects necessary to close the gaps in achieving annual and long-term performance goals.

Evidence: ?? FY 2006 Congressional Justification ?? Language Service Review ?? Program Review ?? FY 2006 and FY 2007 Budget Requests to OMB ?? Capital and Information Technology Planning Process

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The BBG has created a performance plan that reflects the functions of the BBG Engineering and Technical Services. The performance plan is based on the BBG Strategic Plan, and it ties individual program goals to the strategic goals of the agency. As part of a continual effort to increase efficiencies and improve management, Engineering and Technical Services recently reorganized its IT department after consulting with contractors on the best practices in the IT industry. The BBG Internet Advisory Team was established to monitor internet development across all the entities, establish standards and common practices and develop internet measures as web broadcasting continues to become an important way to reach the audience in certain markets. The Capital and Information Technology Planning Process was initiated in 2004 to identify agency-wide capital enhancements. The Process evaluates the capacity of existing capital assets to bridge the performance gap between current and planned program results, and identifies those projects necessary to close the gaps in achieving annual and long-term performance goals. In the past, capital enhancement requests were considered in the same manner as the non-capital enhancement requests. It was not required that a gap analysis be conducted and there were no capital planning teams or an executive committee ensuring that the development of the capital enhancements were coordinated across offices and entities and linked to performance goals and planned results. Last, in 2002 the BBG adopted a new strategy to "Marry the Mission to the Market." This new strategy permeated the entire agency, and Engineering has been responsible for evaluating the best media to reach target audiences, based on available technology, existing infrastructure, speed of implementation, and cost.

Evidence: ?? BBG Strategic Plan (2002-2007) ?? BBG Engineering and Technical Services FY 2007 Performance Plan ?? FY 2006 and FY 2007 Budget Requests to OMB ?? Capital and Information Technology Planning Process

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The BBG analyzes the performance data related to Engineering and Technical Services through the annual Language Service Review and Program Reviews. Annual Program Reviews include an initial assessment of each program's signal quality, transmission and internet development, in addition to content, marketing efforts, and production value, with regularly scheduled follow-up meetings to check the progress on recommendations generated by the review. The Board uses this information to adjust resources and set strategic priorities. The Program review data is also used in the development of the annual performance plans for engineering and to check whether existing performance targets are being met. The annual plans include action steps that the management of BBG Engineering and Technical Services has identified as necessary for the accomplishment of the annual and long-term goals. Management also uses this performance information to inform their evaluations of and performance plans for individual employees. The Capital and Information Technology Planning Process is used to evaluate the capacity of existing capital assets to bridge the performance gap between current and planned program results, and identifies those technical requirements necessary to close the gaps in achieving annual and long-term performance goals. (See response to question 2.8) Additional information on independent evaluations of program effectiveness (including signal monitoring) is available in question 2.6.

Evidence: ?? Capital and Information Technology Planning Process ?? Language Service Review ?? Program Review ?? BBG FY 2007 Engineering and Technical Services Performance Plan

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: The IBB Office of Engineering conducts monthly reviews of all project activities including contracts to ensure that projects are meeting mission objectives. Managers are responsible for both the execution of budget line items and the management of contracts, which are written to assure adherence to schedule, cost, and quality (performance). Senior managers, including SES and GS 13-15, are under the BBG performance based appraisal system that requires critical elements and ratings that are tied to the objectives of the BBG with measurable goals and targets. Through the ongoing appraisal reviews and final annual rating, mangers are held accountable for the performance results. The annual Language Service Review holds managers and partners directly accountable for results. The impact of engineering and technical services in achieving individual language service results are reviewed and evaluated throughout the process. The CFO's mid-year reviews of budget and performance results hold managers (and corresponding contracts) accountable for their annual resource level against expected expenditures, schedules and accomplishments.

Evidence: ?? Project Status Review ?? BBG Performance- based Employee Appraisals ?? Language Service Review

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Financial plans for each element within the BBG are developed at the budget object class level. These plans are carefully scrutinized througout the year as requisitions are made, and the Office of the Chief Financial Officer (CFO) conducts formal reviews with program managers at mid-year, the third quarter, and year-end, with detailed justifications required for any deviations from the financial plans greater than 15%. All requisitions must include verification that funds are available, and the funds are obligated when a contract or agreement is signed or a purchase order is done. The mid-year, 3rd quarter and year-end review provide an opportunity for managers to verify that unliquidated obligations are valid and that once an obligation has been fully expended, remaining balances are deobligated. Financial transactions are coded to both an organization and to a function or project, and on a routine basis, spending line reports based on these codes are run and reviewed by the Budget Office to ensure that funds are being obligated and expended according to the financial plan. Given that BBG has a high percentage of fixed costs (salaries, transmission and infrastructure expenses, rent, etc.), funds must be obligated in a timely manner in order to continue operations.

Evidence: ?? SF-169 Financial Status Report ?? SF-172 Report of Federal Cash Transactions ?? Monthly status reports on the IBB, RFA and RFE/RL on expenditures, disbursements, and obligations ?? Quarterly review reports

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The BBG uses the measure of BBG Engineering and Technical Services Program Cost per Audience Member to evaluate the efficiency of the program. The efficiency indicator (program cost per audience member) is used to track improvements in efficiency over time. Engineering is expected to maintain or improve the achievement of all performance indicators at a relatively constant resource level, thus the annual budget and capital/IT planning procedures assure that effeciencies and effectiveness are measured and attained. Requests for enhanced resources must be tied to proportionally higher results than a standard increase in operating costs. IBB Engineering also achieves efficiencies by using incentives in its contracts, for example using competitive sourcing to assure cost effectiveness. These projects are managed by Engineering to assure compliance with the cost, schedule and performance dictated in the contract. Contracts include "negative incentives" in accordance with the Federal Acquisitions Regulations to discourage changes from the time or performance indicated in the scope of contract.

Evidence: ?? BBG Engineering and Technical Services FY 2007 Performance Plan ?? Language Service Review process ?? Program Review process ?? International Broadcasting Bureau IT restructuring ?? Federal Acquisitions Regulations 52.211-11, 52.211-12, 11.503

Does the program collaborate and coordinate effectively with related programs?

Explanation: Within the BBG, the IBB Office of Engineering and Technical Services and the technical functions of the IBB, RFA, and RFE/RL coordinate efforts to ensure that BBG program delivery needs are met and that new technologies of potential use to BBG are explored, developed, and fielded. Functions are centralized when economies of scale can be gained from doing so, and are distributed as required by the needs of the broadcasting entities. IBB Engineering has primary responsibility for the transmission of BBG radio and television programs. Responsibility for Internet delivery is shared among IBB Engineering, RFA, and RFE/RL. IBB engineering also transmits worldwide the television signals of the American Embassy Television Network for the Department of State, performs the maintenance and installation of communication (TVRO) facilities for State. This collaboration has had numerous benefits, for example: the State Department has helped in China to gain placement for VOA radio, TV, and text; the two agencies have collaborated on regulatory issues dealing with international broadcasting such as jamming, licensing, and communicating contractual and technical information with host countries. This has enabled BBG to operate more efficiently in host countries while ensuring that the best interests of the United States are served during negotiations.

Evidence: ?? U.S. International Broadcasting Act of 1994 ?? Foreign Affairs Reform and Restructuring Act of 1998 ?? State Department Memorandum of Understanding

Does the program use strong financial management practices?

Explanation: The BBG is cross-serviced by the State Department to provide financial services using the Broadcasting Financial Management System (BFMS) system. Internal controls are in place to provide for the separation of duties, minimize erroneous payments, ensure recording of receipts, and the accurate reporting to external entities. The BBG Administrative Officers obligate financial transactions accurately and reconcile their allotments on a monthly basis. Processes on the program level and in the CFO's office also exist to verify that daily (or less frequent) expenditures are made according to the Engineering and Technical Services' annual financial plan. Financial Operations (within the CFO office) records and makes disbursements, reconciles the Fund Balances with Treasury, records and makes deposits, and prepares the monthly, quarterly and annual statement reports for the agency. FY2004 was the first year the BBG prepared the Performance and Accountability Report (PAR) and conducted a financial statement audit. The PAR was submitted on time and the BBG received an unqualified opinion on its Balance Sheet.

Evidence: ?? BFMS System ?? An independent audit conducted in October 2004 by the OIG offered an unqualified opinion of the BBG financial statements ?? The State Department has also received clean audit opinions

Has the program taken meaningful steps to address its management deficiencies?

Explanation: First, the Office of Program Review conducts an independent annual analysis of engineering and transmission and internet development for each language service, in addition to radio content, which includes the management of the engineering program if appropriate. The Action Plans resulting from the Program Reviews lay out specific and meaningful steps to correct deficiencies when they are identified. The Office of Program Review regularly follows up with managers to track progress. While it is too tedious to mention specific steps that have been taken as a result of this review, the process ensures that steps are taken to address deficiencies. Second, BBG recently adopted a performance based appraisal system for senior managers that are tied to the objectives of the BBG. Specific goals with measurable targets are also set for each element within the appraisal. Third, as part of a continual effort to increase efficiencies and improve management, Engineering and Technical Services recently reorganized its IT department after consulting with contractors on the best practices in the IT industry. Fourth, between September 2000 and February 2004 the GAO conducted three evaluations of the BBG. While none of the reports focused specifically on BBG engineering and technical services, the findings, recommendations and resulting actions by the BBG impacted the entire agency, including this program. Follow-up testimony provided in 2004 shows that the Board has taken meaningful steps to address its management deficiencies.

Evidence: ?? Program Reviews ?? BBG Performance Based Appraisal System ?? GAO Report: "Strategic Planning and Performance Management System Could be Improved" (GAO/NSIAD-00-222) ?? GAO Report: "New Strategic Approach Focuses on Reaching Large Audiences but Lacks Measurable Program Objectives" (GAO-03-772) ?? GAO Testimony: "Challenges Facing the Broadcasting Board of Governors" (GAO-04-711T) ?? Documents related to the IT Reorganization N.B. BBG is not part of the President's Management Agenda, but has adopted elements from the Agenda, some of which address management deficiencies. For example, this program uses competitive sourcing for the award of new contracts to assess cost effectiveness.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: As explained in question 2.1, in FY 2004 the actual program cost per Audience Member was $0.98, meaning that the goal for 2009 and beyond is no more than $0.83 per listener. The estimate for FY 2005 and the target for FY 2006 are $0.78 and $0.71 respectively, showing that BBG is on track to achieve this goal.

Evidence: ?? BBG Engineering and Technical Service FY 2007 Performance Plan

Does the program (including program partners) achieve its annual performance goals?

Explanation: Since 2002, the Satellite Effectiveness Index has shown steady increases from 5.80 to 7.10 in 2003, 7.90 in 2004, and expected value of 9.0 in FY 2005. The goal for the Radio Signal Strength Index is to maintain an Index between 2.5 and 3.5, which has been met every year since 2002. (Actual values are 2.8-3.1) Annual transmitter hours have steadily increased from 705,000 hours in 2002 to 879,000 hours in 2003, 967,000 hours in 2004 and an expected 1,001,000 hours in 2005.

Evidence: ?? BBG Engineering and Technical Service FY 2007 Performance Plan

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The BBG uses the measure of BBG Engineering and Technical Services Program Cost per Audience Member to evaluate the efficiency of the program. Data are not available for 2003 and before, but the value for 2004, the estimate for 2005, and the goal for 2006 show increased efficiency each year: $0.98 in 2004, $0.78 in 2005, and $0.71 in 2006.

Evidence: ?? BBG Engineering and Technical Service FY 2007 Performance Plan

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: There are no related programs that share similar goals and objectives with which the performance of this program can be compared. As stated in Question 1.8, the BBG Engineering and Technical Services meets and exceeds industry standards such as "on air availability"; 99.8% on air availability for a global transmission operation with thousands of daily transmission switches and a complexity of simultaneous radio and television transmissions likely exceed any transmission systems found in the private sector. There is no compelling evidence that the private sector could perform the engineering and technical services function more efficiently than the BBG, and there is no single private organization having the vast capabilities required with which to perform such a comparison.

Evidence: ?? U.S. International Broadcasting Act of 1994 ?? Foreign Affairs Reform and Restructuring Act of 1998

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Annual Program Reviews and Language Service Reviews indicate that Engineering and Technical Services is effective in supporting the broadcasting entities and is achieving results. In FY 2003 and FY 2004 the Inspector General conducted reviews of BBG technical operations and information security (annual audit) and use of the internet and other new technologies (Report Numbers: IBO-A-03-01, IT-A-03-14, IT-A-04-07).. The Inspector General conducted inspections at 7 of the 15 IBB transmitting stations in FY 2003 and FY 2004. The evaluations indicated that the program is effective and achieving results or making significant progress. None of the GAO's reports specifically address the work of BBG Engineering and Technical Services.

Evidence: ?? Program Review schedules, results and action plans ?? Language Service Review analysis and results ?? InterMedia research including surveys, monitoring panels, and listener panels ?? OIG: Review of the Information Security Program at the Broadcasting Board of Governors (IT-A-03-14) ?? OIG: Review of the Broadcasting Board of Governors' Use of the Internet and Related New Technologies (IBO-A-04-03) ?? OIG: Inspection of the International Broadcasting Bureau's Kuwait Transmitting Station (IBO-I-04-04) ?? OIG: Inspection of the International Broadcasting Bureau's Transmitting Station in Sri Lanka (IBO-I-04-02) ?? OIG: Review of Information Security Program at Broadcasting Board of Governors (IT-A-04-07) ?? OIG: Inspection of the International Broadcasting Bureau's Djibouti Transmitting Station (IBO-I-04-13) ?? OIG: Inspection of the International Broadcasting Bureau's Germany Transmitting Station (IBO-I-04-10) ?? OIG: Inspection of Broadcasting Board of Governor's Operations in Thailand (IBO-I-04-06) ?? OIG: Inspection of the International Broadcasting Bureau's Transmitting Station in Greece (IBO-I-04-05) ?? OIG: Inspection of the International Broadcasting Bureau's Delano, California Transmitting Station (IBO-I-05-05)