Detailed Information on the
General Services Administration - USA Services Assessment

Develop a government-wide contact center manager's forum to share information and best practices.

Initiate high quality, comprehensive, independent evaluations of USA Services on a regular basis to assess whether the program is effective and achieving results and what areas might be improved.

Complete the first comprehensive, independent evaluation of USA Services during FY 2007.

As recommended by the 2007 independent evaluation, initiate an Innovations Council to evaluate and prioritize needed technical support tasks.

Measure: Citizen Touchpoints

Explanation:USA Services meets its goal of making the government more citizen-centered by providing direct services to citizens through easily accessible channels that deliver accurate, consistent and timely government information. A concrete measure of how effectively USA Services is accomplishing its goal is the number of times that citizens use the services provided by USA Services. The fact that millions of citizens use and re-use these widely available services proves that they are successful in using USA Services to obtain the government information they seek. This outcome measure combines tallies of direct and assisted website visits, telephone contacts, e-mails processed, publications distributed, subscriber e-mailings, and telephone and e-mail activity conducted under USA Services Task Orders to show how many citizens are being served. The number of citizen contacts or "Citizen Touchpoints" provides a valid measure of citizen awareness, usage, and satisfaction with USA Services information channels.

Measure: Governmentwide Website ACSI Satisfaction Benchmark

Explanation:USA Services goals include making all of government more citizen-centered so that regardless of source, citizens can expect easy access to accurate, consistent and timely information. A primary information channel supported by all agencies is the web. USA Services provides guidance, training, and support to web managers regarding web best practices across all levels of government. Currently over 90 Federal websites use the nationally recognized and accepted American Customer Satisfaction Index (ACSI) to measure how citizens rate their websites. ACSI's methodology is the most respected, credible and well-known nationwide measure of website satisfaction and is used by corporate America to measure customer satisfaction. ACSI publishes a quarterly index to track the government's aggregate satisfaction score. USA Services success in helping to improve citizen web interactions across the government should result in raising the government-wide aggregate. It should be noted that changes in individual ACSI scores are generally modest unless a major redesign of a website occurs. As an compilation of over 90 website scores, improvements are incremental. USA Services' long-term goal is to match or exceed the aggregate benchmark set by private sector e-business websites which currently stands at 75.9 (3.9 points better than the FY05 government aggregate). Steady improvement to eliminate this gap is a valid measure of the success of USA Services efforts to improve Federal websites.

Measure: FirstContact and Web Solutions Task Orders

Explanation:USA Services has a responsibility to make all of government more citizen-centered. To support other agency efforts to improve their approach to citizens, USA Services provides two contracting vehicles for use by agencies government-wide to obtain USA Services validated 1) contact center solutions and 2) web hosting and professional services support. Agencies using these contract vehicles will obtain performance-based services that meet or exceed the contact center and website standards and best practices that have been developed by USA Services. This output measure shows the numbers of task orders issued under these supporting contracts and thereby the impact of USA Services efforts to make government more citizen-centered.

Measure: Cost Per Citizen Touchpoint

Explanation:This measure matches effective use of resources to more successfully achieve the USA Services mission at reduced unit costs. It is calculated by taking the total budget for the operations included in USA Services and dividing those funding resources by the number of citizen touchpoints. The target is to reduce the cost per contact over time by increasing the magnitude of contacts faster than budgets.

Is the program purpose clear?

Explanation: USA Services' mission is to help make the government more citizen-centric by providing citizens with easy access to accurate, consistent, and timely government information. To achieve this mission, it provides citizens direct information about and from all levels of their government through an array of integrated information channels including FirstGov.gov and other websites, government-wide web-based search, telephone and e-mail inquiry response from the National Contact Center, and print materials distributed from Pueblo, Colorado. USA Services also is responsible for helping agencies government-wide improve their interactions with citizens. It accomplishes this goal by providing agencies with easy and cost-effective access to contact center services through the FirstContact contract and to web hosting and professional services through the Citizen Services Infrastructure contract. USA Services also provides leadership to web and contact center managers across all levels of government by organizing and managing interagency steering committees and intergovernmental collaboration that develop and promulgate best practices.

Evidence: "USA Services e-Gov Initiative: http://www.whitehouse.gov/omb/egov/c-1-5-usas.html Improving Public Access to and Dissemination of Government Information and Using the Federal Enterprise Architecture Data Reference Model: http://www.whitehouse.gov/omb/memoranda/fy2006/m06-02.pdf 2005 Annual GSA Performance and Accountability Report: http://www.gsa.gov/gsa/cm_attachments/GSA_DOCUMENT/GSAFullPAR_111505_Final_R2F-aAB_0Z5RDZ-i34K-pR.pdf 2007 GSA Congressional Justification: http://www.gsa.gov/gsa/cm_attachments/GSA_DOCUMENT/2007CJ_R2E-s65_0Z5RDZ-i34K-pR.pdf 2006 GSA Congressional Justification: http://www.gsa.gov/gsa/cm_attachments/GSA_DOCUMENT/2006CJ_R2E-s65_0Z5RDZ-i34K-pR.pdf" http://www.gsa.gov/intergov

Does the program address a specific and existing problem, interest, or need?

Explanation: Citizens have a vital need to learn about, access, and use the services and benefits that are available to them from the Federal government. The unique benefit of USA Services is that it offers citizens multiple channels that provide easy access to consistent and authoritative government information. USA Services also works to improve the quality of all Federal agency channels available to citizens government-wide. There is much evidence that citizens need an improved experience when dealing with the Federal government. The Pew Internet and American Life Project reported in its 2004 study, How Americans Get in Touch With Government that 59% of people who contact the government by phone encountered a problem and 46% of those using the web encountered a problem. They report that it takes too long on the phone and they can't find the right information on the web. USA Services research of citizen expectations of government information services reveals that: 1) citizens expect that the information they need will be accessed through a combination of channels and be consistent, no matter how they contact the government; 2) the manner in which citizens contact the government is dependent upon the reason for and the nature of the contact; 3) citizens expect improvements in the channels they use the most; and 4) the government is not effectively communicating the availability of existing government services and contact channels. During a series of nationwide focus group sessions in the summer of 2005, USA Services found that citizens generally lack awareness of the currently available government channels and services. The need for multiple channels is demonstrated in a 2004 report by the Department of Commerce's National Telecommunications and Information Administration. Their research presented in A Nation Online: Entering the Broadband Age found that many Americans??41.3 percent of the total U.S. population??still do not use the Internet from any location, regardless of availability or affordability. USA Services not only provides direct information to citizens through a variety of channels but also provides leadership to improve citizen customer service government-wide.

Evidence: "Report to Congress on the Benefits of the President's E-Gov Initiatives (1/6/2006): http://www.whitehouse.gov/omb/inforeg/e-gov/e-gov_benefits_report_2006.pdf 2005 Annual GSA Performance and Accountability Report: http://www.gsa.gov/gsa/cm_attachments/GSA_DOCUMENT/GSAFullPAR_111505_Final_R2F-aAB_0Z5RDZ-i34K-pR.pdf 2005 Appendix II: Summary Chart of Performance Goals and Measures: http://www.gsa.gov/gsa/cm_attachments/GSA_DOCUMENT/2005%20PAR%20appendix%20II%20completed_R2F-aAB_0Z5RDZ-i34K-pR.pdf ACSI Overall Federal Government Scores with Historical Scores of Agencies Measured 1999-2005: http://www.theacsi.org/government/govt-all-05.html Q1 2006 ACSI E-Government Satisfaction Index: http://www.foreseeresults.com/WhitePapers/ACSI_EGov_Commentary_Mar06.pdf USA Services Program website: http://www.usaservices.gov/ Webcontent.gov: http://www.firstgov.gov/webcontent/index.shtml"

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: USA Services delivers a comprehensive scope of services; delivering consistent official government information directly to citizens through integrated easily accessed multiple channels, and also leading Federal agencies to improve their customer service to citizens. This scope is unique and exceeds the more limited citizen service offerings in the private and public sector. For example, in the web space, there are several commercial organizations that provide government search services (Google, Yahoo, etc.) and portal access to government agencies (AOL). However, these firms do not match the scope of USA Services. They do not provide official, authoritative Federal information; they permit advertising which skews results based on sponsorship; and they are less current than FirstGov.gov. No private sector organization provides official government information through the broad array of integrated channels that USA Services does. None provide toll-free telephone information about government programs; none accept and respond to citizen e-mail inquiries about government programs; none handle misdirected e-mails and phone calls received by government agencies; and none distribute government publications. USA Services is alone in its leadership role to improve citizen services government-wide and in its collaboration with federal, state and local partners to encourage best practices at all levels of government.

Evidence: "Improving Public Access to and Dissemination of Government Information and Using the Federal Enterprise Architecture Data Reference Model: http://www.whitehouse.gov/omb/memoranda/fy2006/m06-02.pdf 2007 GSA Congressional Justification: http://www.gsa.gov/gsa/cm_attachments/GSA_DOCUMENT/2007CJ_R2E-s65_0Z5RDZ-i34K-pR.pdf 2006 GSA Congressional Justification: http://www.gsa.gov/gsa/cm_attachments/GSA_DOCUMENT/2006CJ_R2E-s65_0Z5RDZ-i34K-pR.pdf How the President's Management Agenda (PMA) has helped GSA - http://www.whitehouse.gov/results/agenda/report8-04/general_services_administration.pdf"

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The Office of Citizen Services (OCS), which houses the USA Services program, has management and budgetary authority over all of GSA's citizen-facing activities, specifically the National Contact Center, FirstGov.gov and other informational websites, and the Pueblo program that helps citizens order all agencies' publications. OCS also manages the activities through which USA Services provides contracting and collaborative support to other Federal agencies to help them provide high quality responses to citizen inquiries. Therefore, USA Services has all the authorities and resources needed to accomplish its mission and meet its performance goals. There is no evidence or reason to believe that other organizational approaches would produce the efficiencies or achieve the results currently realized by USA Services.

Evidence: "S.3259 A bill to authorize the permanent establishment of a system of Federal Information Centers. Public Law 95-41: http://thomas.loc.gov/cgi-bin/bdquery/z?d095:SN03259:@@@R|TOM:/bss/d095query.html| The E-Government Act of 2002 (Public Law 107-347): http://frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname=107_cong_public_laws&docid=f:publ347.107.pdf Executive Order 11566, Consumer product information, Signed: October 26, 1970 http://www.archives.gov/federal-register/codification/executive-order/11566.html TITLE 40--PUBLIC BUILDINGS, PROPERTY, AND WORKS CHAPTER 16--GENERAL SERVICES ADMINISTRATION Sec. 761. Consumer Information Center Fund, General Services Administration - Pub. L. 98-63, title I, Sec. 101, July 30, 1983, 97 Stat. 321.: http://frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname=browse_usc&docid=Cite:+40USC761 TITLE 40--PUBLIC BUILDINGS, PROPERTY, AND WORKS CHAPTER 16--GENERAL SERVICES ADMINISTRATION Sec. 761a. Consumer Information Center Fund; acceptance and expenditure of gifts - Pub. L. 105-65, title III, Oct. 27, 1997, 111 Stat. 1377.: http://frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname=browse_usc&docid=Cite:+40USC761a Reorganization of CIC to include FIC - GSA Order ADM 5440.535 Date: 01/28/2000 OCSC Creation - GSA Order ADM 5440.564 - 06/17/2002 OCSC Reorganization - GSA Order ADM 5440.593 - 02/14/2006"

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: One of the primary goals in establishing USA Services was to provide all citizens with easy access to government information regardless of how they want to access that information. Demographics and socio-economic realities may predispose one group of citizens to favor one information channel over another and USA Services is designed to allow citizens to use whatever channel works best for them. Staff and funding resources are allocated to the different information channels based on the levels of citizen usage of the different channels and the cost of operating the channels. While the cost to reach individual citizens through the web is relatively low, the audience is immense and growing and demands a significant allocation of resources. The costs per touchpoint for e-mail, telephone and print distribution are significantly higher than web access, but those channels have lower usage. Since the cost to reach citizens via the web is lower, USA Services seeks to guide as many citizens as possible to the web while maintaining efficient services via other channels. Reaching all citizens is the target of USA Services but special emphasis is also placed on maintaining channels that serve those who cannot afford internet access. The high levels of customer satisfaction reported for the different channels indicates that the program is successful in distributing its resources fairly to the different groups of citizens benefitting from USA Services.

Evidence: "USA Services e-Gov Initiative: http://www.whitehouse.gov/omb/egov/c-1-5-usas.html FY 2005 Report to Congress on the Implementation of the E-Government Act of 2002 (lots about FirstGov.gov): http://www.whitehouse.gov/omb/inforeg/reports/2005_e-gov_report.pdf Implementing the President's Management Agenda for E-Government: http://www.whitehouse.gov/omb/egov/2003egov_strat.pdf 2005 Annual GSA Performance and Accountability Report: http://www.gsa.gov/gsa/cm_attachments/GSA_DOCUMENT/GSAFullPAR_111505_Final_R2F-aAB_0Z5RDZ-i34K-pR.pdf 2005 Appendix II: Summary Chart of Performance Goals and Measures: http://www.gsa.gov/gsa/cm_attachments/GSA_DOCUMENT/2005%20PAR%20appendix%20II%20completed_R2F-aAB_0Z5RDZ-i34K-pR.pdf USA Services Program website: http://www.usaservices.gov/ FirstGov.gov website: http://www.FirstGov.gov/ FirstGov en espanol: http://www.espanol.gov/ Pueblo Publication website: http://www.pueblo.gsa.gov/ Consumer action website: http://www.consumeraction.gov/ Kids website: http://www.kids.gov/ Webcontent.gov: http://www.firstgov.gov/webcontent/index.shtml"

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The long term goal of USA Services is to provide citizens with easy access to accurate, timely and consistent information about the government. Whether the citizen uses a channel offered directly by USA Services, or one offered by another government agency that provides excellent service because it adheres to USA Services standards of quality, the result is the same: a correct, timely answer to the citizen's question. To monitor the successful achievement of this goal, USA Services uses two outcome measures that provide a comprehensive, continuing overview of program performance. The first tracks citizen "touchpoints" or the volume of usage by the public of all USA Services channels, including web, telephone, e-mail, and publications. The second measure is the level of citizen satisfaction with all agency websites government-wide. The number of citizen contacts or "citizen touchpoints" provides a valid measure of citizen awareness, usage, and satisfaction with USA Services information channels. The fact that millions of citizens choose to use and re-use these easily accessible services demonstrates that they are successfully using USA Services to obtain the government information they seek. USA Services also provides guidance, training, and support to other agencies to improve their responses to citizens. The primary citizin information channel supported by all agencies is the web. USA Services provides leadership in improving the Federal web presence through its Web Manager University and through the website webcontent.gov, a consolidation of web best practices. GSA uses the independent American Customer Satisfaction Index to measure the impact of USA Services leadership on improving agency websites government-wide. This nationally known and accepted survey methodology was developed by the University of Michigan Business School and is widely used in both the public and private sectors. In conducting the survey, users of websites are asked to rate elements such as content, look-and- feel, and navigation. Currently, the ACSI survey is used by over 90 different Federal websites and many websites operated by other governments and the private sector. USA Services tracks the government-wide aggregate score which should trend upwards if the leadership activities of USA Services are having a positive outcome. ACSI publishes an aggregate score for private sector websites and the long term goal customer satisfaction goal of USA Services is for the government aggregate score to meet or exceed the private sector score.

Evidence: "2005 Annual GSA Performance and Accountability Report: http://www.gsa.gov/gsa/cm_attachments/GSA_DOCUMENT/GSAFullPAR_111505_Final_R2F-aAB_0Z5RDZ-i34K-pR.pdf 2005 Appendix II: Summary Chart of Performance Goals and Measures: http://www.gsa.gov/gsa/cm_attachments/GSA_DOCUMENT/2005%20PAR%20appendix%20II%20completed_R2F-aAB_0Z5RDZ-i34K-pR.pdf ACSI Overall Federal Government Scores with Historical Scores of Agencies Measured 1999-2005: http://www.theacsi.org/government/govt-all-05.html Q1 2006 ACSI E-Government Satisfaction Index: http://www.foreseeresults.com/WhitePapers/ACSI_EGov_Commentary_Mar06.pdf USA Services Program website: http://www.usaservices.gov/"

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: USA Services is a steady state, mature program effort that has been in full-scale operation for several years and serves citizens and customer agencies every day. Its long-term goal is to give citizens easy access to accurate, timely, and consistent government information whether through a USA Services channel or a partner agency channel. USA Services has established a comprehensive set of measures with set baselines and targets to measure progress in meeting this goal. The targets and timeframes set in these measures are ambitious. However, after the initial burst of publicity and success that brings large audiences to the service, it becomes increasingly challenging to reach groups of citizens who may remain unaware of the services. USA Services has worked very hard to increase usage by single digits in FY 2005 and in the currently unfolding FY 2006. Unexpected events or emergencies may also drive increased usage of USA Services channels but can not be anticipated when projecting targets. This was demonstrated on numerous occasions such as providing support in the Hurricane Katrina disaster, during the weeks following the Southeast Asian Tsunami disaster, and most recently when USA Services supported the Department of Veterans Affairs in the response to veterans after the theft of personal data on veterans and active military personnel. It is more difficult to generate large annual increases in citizen touchpoints unless unanticipated events intervene.

Evidence: "2005 Annual GSA Performance and Accountability Report: http://www.gsa.gov/gsa/cm_attachments/GSA_DOCUMENT/GSAFullPAR_111505_Final_R2F-aAB_0Z5RDZ-i34K-pR.pdf 2005 Appendix II: Summary Chart of Performance Goals and Measures: http://www.gsa.gov/gsa/cm_attachments/GSA_DOCUMENT/2005%20PAR%20appendix%20II%20completed_R2F-aAB_0Z5RDZ-i34K-pR.pdf ACSI Overall Federal Government Scores with Historical Scores of Agencies Measured 1999-2005: http://www.theacsi.org/government/govt-all-05.html Q1 2006 ACSI E-Government Satisfaction Index: http://www.foreseeresults.com/WhitePapers/ACSI_EGov_Commentary_Mar06.pdf USA Services Program website: http://www.usaservices.gov/"

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: USA Services has a comprehensive set of annual outcome, output, and efficiency measures that provide a clear picture of progress towards meeting long-term goals. These annual measures have established baselines and specific targets for program performance through 2008 and beyond. By achieving these targets, USA Services demonstrates progress toward its goal of providing all citizens with easy access to accurate, timely, and consistent government information. USA Services receives and reports performance information on a daily, monthly and quarterly basis. Depending upon the frequency with which this information becomes available, reports are produced on a series of performance measures to demonstrate progress to Program Managers and to the GSA Administrator. As part of this PART process, USA Services has revisited and revised its performance measures to bring them more in-line with the long-term outcome goals which are to reach the most citizens with our services, effectively and efficiently satisfy their need for government information, and provide a level of high quality service that leaves them wanting to return. USA Services support for agencies is directed at assisting their efforts in reaching and providing great service to their constituent audiences. The annual measures demonstrate progress toward the long-term goal of providing citizens with eacy access to accurate, timely, and consitent information about the government.

Evidence: On a monthly and quarterly basis, USAS currently reports to management on the following measures: Public Contacts, FirstGov.gov Page Views, USA Services Working Agreements, FirstContacts Task Orders, Web Self-Help for Citizens, FirstGov Search ACSI Survey, FirstGov Website Uptime, Citizen Visits - GovBenefits, Citizen Visits - Forms.gov, Citizen Visits - Disability.gov, Citizen Visits - FirstGov, Number of Search Queries - Affiliates, Number of Search Queries - FirstGov. With OMB acceptance of the revised long-term measures, USAS will report to monthly, quarterly and/or annually to management citizen touchpoints, citizen satisfaction, effectiveness of public service outreach, and cost per citizen touchpoint. GSA Administrator's quarterly performance briefing (02-22-06 OCSC Performance Review.ppt) 2005 Annual GSA Performance and Accountability Report: (see copy under Question 1-1) http://www.gsa.gov/gsa/cm_attachments/GSA_DOCUMENT/GSAFullPAR_111505_Final_R2F-aAB_0Z5RDZ-i34K-pR.pdf 2007 GSA Congressional Justification: (see copy under Question 1-1) http://www.gsa.gov/gsa/cm_attachments/GSA_DOCUMENT/2007CJ_R2E-s65_0Z5RDZ-i34K-pR.pdf 2004 GSA Performance Plan: http://www.gsa.gov/gsa/cm_attachments/GSA_DOCUMENT/2004PerfPlan_R2F-aAO_0Z5RDZ-i34K-pR.pdf 2002 GSA Strategic Plan: http://www.gsa.gov/gsa/cm_attachments/GSA_DOCUMENT/GSAStrategicPlan_R2E-s43_0Z5RDZ-i34K-pR.doc

Does the program have baselines and ambitious targets for its annual measures?

Explanation: As noted in question 2.3, there is baseline data for each of the measures that USA Services reports on regularly to management, and each measure has ambitious targets set for the performance periods. This baseline data shows that USA Services has been successful in improving performance across a variety of measures and provides a firm foundation for setting ambitious targets going forward. Targets and timeframes for USA Services measures are set based upon a realistic analysis of what USA Services has previously been able to accomplish and the resources available going forward. With respect to the outcome measure of the government-wide American Customer Satisfaction Index, experts at ForeSee Results, the organization that conducts the ACSI web surveys, have stated that the mid-70's is an excellent score for broad-based websites and that ACSI score increases are likely to be gradual. USA Services continually monitors channel performance and works to improve the citizen experience so they will return and recommend us to others. Additionally, USA Services output measure of the volume of task orders awarded under its two contract vehicles (First Contact and the Citizen Services Infrastructure) and its efficiency measure to reduce cost per touchpoint, directly support he program's goals. Improvements in all of the measures only come as the result of intensive, sustained efforts to reach and impress citizens and agency partners with our ability to meet their needs.

Evidence: On a monthly and quarterly basis, USAS currently reports to management on the following measures: Public Contacts, FirstGov.gov Page Views, USA Services Working Agreements, FirstContacts Task Orders, Web Self-Help for Citizens, FirstGov Search ACSI Survey, FirstGov Website Uptime, Citizen Visits - GovBenefits, Citizen Visits - Forms.gov, Citizen Visits - Disability.gov, Citizen Visits - FirstGov, Number of Search Queries - Affiliates, Number of Search Queries - FirstGov. With OMB acceptance of the revised long-term measures, USAS will report to monthly, quarterly and/or annually to management citizen touchpoints, citizen satisfaction, effectiveness of public service outreach, and cost per citizen touchpoint. GSA Administrator's quarterly performance briefing (02-22-06 OCSC Performance Review.ppt) (see copy under Question 2.3) 2005 Annual GSA Performance and Accountability Report: (see copy under Question 1-1) http://www.gsa.gov/gsa/cm_attachments/GSA_DOCUMENT/GSAFullPAR_111505_Final_R2F-aAB_0Z5RDZ-i34K-pR.pdf 2007 GSA Congressional Justification: (see copy under Question 1-1) http://www.gsa.gov/gsa/cm_attachments/GSA_DOCUMENT/2007CJ_R2E-s65_0Z5RDZ-i34K-pR.pdf 2004 GSA Performance Plan: (see copy under Question 2-3) http://www.gsa.gov/gsa/cm_attachments/GSA_DOCUMENT/2004PerfPlan_R2F-aAO_0Z5RDZ-i34K-pR.pdf 2002 GSA Strategic Plan: (see copy under Question 2-3) http://www.gsa.gov/gsa/cm_attachments/GSA_DOCUMENT/GSAStrategicPlan_R2E-s43_0Z5RDZ-i34K-pR.doc

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: USA Services has two types of partner groups. The first is made up of the contractors who directly support the daily operations of the FirstGov.gov website and the National Contact Center (NCC). These contractors are fully acquainted with our program objectives and are committed to providing services which directly contribute to the program goals. The NCC contractor regularly reports and we monitor their performance in such areas as average speed of answer, hold time, call handle time, abandoned call rates, service levels, and total time to handle e-mails (see RFQ and SOW sections in evidence). Additionally, we work together with the contractor to review the quality of their performance in handling calls and e-mails and to insure that they completely understand the levels of service we wish to provide the public. The second type of partners are the Federal, state, and local agencies that participate in numerous collaborative efforts, including two in which USA Services has a lead role: 1) the Web Managers Advisory Group, (www.webcontent.gov) 40 senior government web managers who collaborate on creation of "Recommended Policies and Guidelines for Federal Public Websites". Their recommendations were used by OMB to issue "Policies for Federal Agency Public Websites" (copies of both are provided in the Evidence section), and 2) the Citizen Service Level Interagency Committee (CSLIC) with over 80 members which produced "Proposed Performance Measures, Practices and Approaches for Government-wide Citizen Contact Activities" containing guidelines for handling telephone calls, e-mails and other contact center related issues (copy is provided in the Evidence section). Also, USA Services operates the Web Manager University which conducted its first series classes this spring to sold-out audiences of Federal web managers who carry this knowledge back and apply it on agency web portals. All partners apply what is shared in these groups to their agency websites and contact centers. Partner commitments of staff time and contribution of resources demonstrate their support for achieving the USA Services goal of creating a more citizen-centric government.

Evidence: NCC-RFQ-Metrics(Sec-19).doc SectionC-SOW-Objectives(C.1.3).doc SectionJ-SOW-Metrics(J.3.13).doc OMB Policies for Federal Agency Public Websites http://www.whitehouse.gov/omb/memoranda/fy2005/m05-04.pdf Web Manager University Training Program: http://www.firstgov.gov/webcontent/resources/training/university.shtml Web Managers Advisory Council: http://www.firstgov.gov/webcontent/about/council.shtml Proposed Performance Measures, Practices and Approaches for Government-wide Citizen Contact Activities: http://usaservices.gov/word/performreport.doc CSLIC Committee Members: http://usaservices.gov/committee.htm Recommended Policies and Guidelines for Federal Public Websites, Final Report of the Interagency Committee on Government Information, Submitted to the Office of Management and Budget, June 9, 2004: http://www.firstgov.gov/webcontent/about/documents/icgi_report.html Federal Citizen Information Center Services to Agencies: http://www.pueblo.gsa.gov/svcagncy.htm Federal Citizen Information Center Publication Partners: http://www.pueblo.gsa.gov/sponsor.htm Intergovernmental Advisory Board: http://www.gsa.gov/intergov USA Services Partner Agency Testimonials: Department of Health and Human Services - Program Support Center Department of Health and Human Services - Human Trafficking Department of State's Office of Overseas Citizens Services Department of the Interior's Fish and Wildlife Service Federal Emergency Management Agency GovBenefits

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: USA Services is a relatively new Program and it has never had a high quality, comprehensive, independent evaluation of the total program. However, individual USA Services components have been evaluated by independent reviewers. In August 2005, the Office of Management and Budget's, Office of Electronic Government and Information Technology conducted a Gap Analysis of Electronic Government Initiatives to determine their conformance to legislative, regulatory, and management polices. USA Services was included in this analysis and, except for a few minor discrepancies, was found to be performing at a high level. The program is currently working under OMB's guidance to meet requirements made in a corrective action plan. In February 2006, a GAO audit, Federal Contact Centers: Mechanism for Sharing Metrics and Oversight Practices along with Improved Data Needed, looked at the USA Services program and the National Contact Center (NCC) along with 4 other agency contact centers and found the NCC to meet all of their criteria for information quality. Currently, USA Services is developing a statement of work for a contract to obtain the first high quality, comprehensive evaluation of the program. This evaluation will be available during the second quarter of FY 2007. Additionally, in the future, USA Services will regularly (every two years) have a program evaluation done to insure that the program remains on track and to support program improvements.

Evidence: Gap Analysis of Electronic Government Initiatives, Office of Management and Budget's, Office of Electronic Government and Information Technology, August 2005 (USA Services Gap Assessment Aug 31, 2005.doc) Government Accountability Office (GAO) Audit, Federal Contact Centers: Mechanism for Sharing Metrics and Oversight Practices along with Improved Data Needed, February 2006 (GAO-FedContactCtrs-Feb06.pdf) ACSI Overall Federal Government Scores with Historical Scores of Agencies Measured 1999-2005: http://www.theacsi.org/government/govt-all-05.html Q1 2006 ACSI E-Government Satisfaction Index: http://www.foreseeresults.com/WhitePapers/ACSI_EGov_Commentary_Mar06.pdf FirstGov.gov Awards http://firstgov.gov/About/Awards.shtml

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: Not yet. Currently, the resources required to support both direct and indirect costs for several USA Services components are shown under the separate accounts funding those components. GSA's FY 2008 Budget Request perpetuated the separation of performance and funding discussions by funding source. It is expected that the FY 2008 Congressional Justification for this program will present a single picture of the program's performance goals and funding from all sources associated with achieving these goals.

Evidence: "2007 GSA Congressional Justification: http://www.gsa.gov/gsa/cm_attachments/GSA_DOCUMENT/2007CJ_R2E-s65_0Z5RDZ-i34K-pR.pdf 2006 GSA Congressional Justification: http://www.gsa.gov/gsa/cm_attachments/GSA_DOCUMENT/2006CJ_R2E-s65_0Z5RDZ-i34K-pR.pdf"

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: USA Services has developed and refined its long-term goals and measures to more accurately reflect the overall purpose of the program. Management tracks both high and intermediate level strategies through use of GSA Program Management Planning (PMP) and Performance Measure Tracking (PMT) systems with regular monthly and quarterly reviews to insure that USA Services is on course. The GSA Program Management Process is a strategic planning mechanism that provides a systematic approach to determining customer requirements, setting performance goals based upon those requirements, developing sound action plans based on those requirements, allocating resources in a manner that integrates budget and performance goals, and executing action plans to achieve the desired results. There is currently no indication that there is a deficiency in USA Services strategic planning.

Evidence: "2005 Annual GSA Performance and Accountability Report: http://www.gsa.gov/gsa/cm_attachments/GSA_DOCUMENT/GSAFullPAR_111505_Final_R2F-aAB_0Z5RDZ-i34K-pR.pdf 2005 Appendix II: Summary Chart of Performance Goals and Measures: http://www.gsa.gov/gsa/cm_attachments/GSA_DOCUMENT/2005%20PAR%20appendix%20II%20completed_R2F-aAB_0Z5RDZ-i34K-pR.pdf"

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: USA Services collects performance data on all of its direct service contractor operated channels and other activities on a daily and weekly basis. Data collected include such statistics as calls answered, average speed of answer, service levels, abandonment rates, accesses to the FAQ system, most popular searches, etc. Each month, the data is rolled up and reported to all program managers and is recorded and tracked in GSA's Performance Measure Tracking system which is available to all GSA Senior managers. On a quarterly basis, program managers brief the GSA Administrator or a designee on USA Services performance, progress against established targets, and any significant issues affecting program operations. During these reviews, decisions are made to revise program efforts and shift resources. Also, USA Services receives daily reports from the NCC contractor on performance for Tier 1 partners who we support through the National Contact Center. We monitor partner website satisfaction levels through the quarterly American Customer Satisfaction Index program and the Government benchmark it produces. We have just awarded a contract to conduct a more comprehensive Baseline Assessment of all contact center activity government-wide. All of these tools help measure progress towards improved citizen service and to compare performance across government. USA Services also maintains close contact with web managers, contact center agency partners, and state, local, and international officials through regular meetings, teleconferences, and ListServ interaction to determine their observations of citizen needs and to help improve program performance.

Evidence: The data used comes from a variety of reliable information sources including industry standard WebTrends analytic reports for the websites, Contractor reports on telephone, e-mail, and publication distribution activity. These reports include a large amount of internal detail that is studied by operations staff to determine how systems are working and what changes should be made. The following samples for June 2006 are included on the Evidence CD: FirstGov All WebTrends Jun 2006.xls GSA_Email_Activity_Report_Jun 2006.xls GSA_Email_Performance_Report_Jun 2006.xls GSA_NCC_Call_Summary_June2006.xls GSA-FAQ Hits Jun 2006.xls Pueblo Quarterly Distribution Summary 3rdQtr FY06.xls Pueblo Weekly Inventory AE Seq.pdf Pueblo Weekly Inventory Item # Seq.pdf Pueblo Weekly Replenishment Item # Seq.pdf Pueblo Weekly Publications Alpha Seq.pdf Pueblo Weekly Publications Item # Seq.pdf Pueblo Weekly Replenishment AE Seq.pdf

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: USA Services program managers are evaluated under GSA's new APPAS rating system which links pay to program performance. As one of five Presidential e-Gov initiatives managed by GSA, USA Services performance is closely monitored by GSA. On a quarterly basis, USA Services managers brief the GSA Administrator or a designee on performance progress measured against established targets, budgetary goals, program schedules, and any significant issues affecting program operations. USA Services also provides regular milestone reports to OMB's Office for Electronic Government and Information Technologies and meets regularly with the OMB Manager for the Government-to-Citizen Portfolio. USA Services has performance-based contracts with vendors providing search, hosting, and contact center services. All task orders issued to support partner agencies under FirstContact are performance-based. We monitor contractor performance against industry accepted standards to insure that performance meets or exceeds expectations. Contractor performance in the areas of web hosting, response to telephone and e-mail inquiries, and publication distribution are monitored daily with regular weekly conference calls and quarterly site visits to insure that they meet program performance targets within set schedules and costs. Failure to meet program performance standards can result in withholding of payment until problems are resolved or failure to renew option years in the contract.

Evidence: Included are the APPAS documents for: Martha Dorris, Deputy Associate Administrator for Citizens Services (FY2006 SES Dorris Performance Plan.doc) Teresa Nasif, Director, Federal Citizen Information Center (FY2006 SES Nasif Performance Plan.doc) Stuart Willoughby, Program Manager, USA Services (FY2006 Willoughby Performance Plan.doc) The following sample contact center reports for June 2006 are included on the Evidence CD to illustrate the types of performance data reported and monitored for the NCC contract. GSA_Email_Activity_Report_Jun 2006.xls GSA_Email_Performance_Report_Jun 2006.xls GSA_NCC_Call_Summary_June2006.xls GSA-FAQ Hits Jun 2006.xls The following sample website report for June 2006 is included on the Evidence CD to illustrate the types on performance information reported and monitored for the FirstGov.gov website. FirstGov All WebTrends Jun 2006.xls

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: GSA's annual independent audits have consistently verified that procedures are in place to assure that all program funds are obligated in a timely manner and spent for the intended purposes. USA Services obligates funds on a quarterly schedule and tracks expenses on a monthly basis. GSA's accounting systems and procedures require that obligations be established and identified according to intended purpose prior to processing payments for the corresponding goods and services. This ensures that GSA's payments for this program correspond to their intended purposes. Unobligated balances for the FCIC portion of USA Services are returned to the FCIC Fund for use in future years with Congressional approval. USA Service's principle partners are the contact center contractors used by federal agencies under FirstContact task orders. Other major contractor partners provide Firstgov.gov infrastructure services such as search and hosting. Depending upon the size of the contract and the wishes of the agency customers, some agencies are responsible for making payments directly to the contractors while others are billed by USA Services to reimburse the contractor. On the latter, USA Services receives the invoice, works directly with the agency to certify its validity and when appropriate authorizes payment from monies obligated by the customer agency. Customer agencies are regularly apprised of obligation balances and additional funds are requested to insure that contractor invoices can be covered.

Evidence: Financial Management 5 Year Plan 2003 through 2007: http://www.gsa.gov/gsa/cm_attachments/GSA_DOCUMENT/GSACFO%205%20yr%20Plan%20FY%202003-2007_R2E-s46_0Z5RDZ-i34K-pR.doc

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: USA Services has established the cost per citizen touchpoint as its primary measure of efficiency and cost-effectiveness and has established a baseline and annual targets for this measure. This measure is derived by dividing the total Program budget dollars by the number of annual citizen touchpoints to show a cost per touchpoint. Program management monitors actual efficiency results each quarter and continually looks for opportunities to encourage citizens to shift their search for information from less cost-effective information channels (e.g., telephone and e-mail) to more cost-effective channels (e.g., websites and web-based frequently asked questions). Examples of actions taken to shift citizens to more cost-effective channels include modification of website design and navigation to highlight the availability of the continually updated Frequently Asked Questions (FAQs) posted on the Internet so that citizens can find answers to current questions without having to resort to e-mails and phone calls. USA Services has added the FAQ database to the FirstGov search so that search terms from the public direct people to actual answers in the FAQ database rather than just listings of results. Also, USA Services conducts annual Public Service Advertising campaigns which primarily focus on the information available via the web at FirstGov.gov and making citizens more aware of the ease of obtaining information and answers to questions over the Internet.

Evidence: "USA Services FirstContact contract: http://usaservices.gov/firstcontact.htm FirstContact Frequently Asked Questions: http://usaservices.gov/howdoifc.htm"

Does the program collaborate and coordinate effectively with related programs?

Explanation: USA Services plays a leadership role in coordinating with a variety of organizations not only to share ideas, best practices, and lessons learned but to help establish benchmarks for development of high performing web and contact center based solutions. Two organizations in which USA Services has a lead role are: 1) the Web Managers Advisory Group, (www.webcontent.gov) 40 senior government web managers who collaborate on creation of Government-wide standards for websites, 2) the Citizen Service Level Interagency Committee (CSLIC) with over 80 members which establishes recommendations for handling telephone calls, e-mails and other contact center related issues. USA Services works closely with state, local and international governments to share U.S. best practices. USA Services participates in meetings with a variety of intergovernmental and industry committees, forums and events such as the Internet Managers Consortium, Intergovernmental Advisory Board, the International Council for Information Technology in Government Administration, the Intergovernmental Teleconference Group, etc.. Additionally, we regularly communicate and work closely with over 100 Federal program offices and all key information suppliers at agencies to ensure that they are aware of the USA Services program and the services and solutions it offers.

Evidence: "USAS's FirstGov.gov Website Program took the lead in working with the Web Managers Advisory Group to develop and publish guidelines which were endorsed by OMB for Governmentwide application on websites. USAS has formed the Citizen Service Level Interest Committee (CSLIC) to develop the same type of guidelines for contact centers. Proposed Performance Measures, Practices and Approaches For Government-wide Citizen Contact Activities: http://usaservices.gov/word/performreport.doc CSLIC Committee Members: http://usaservices.gov/committee.htm Web Managers Advisory Council: http://www.firstgov.gov/webcontent/about/council.shtml Recommended Policies and Guidelines for Federal Public Websites, Final Report of the Interagency Committee on Government Information, Submitted to the Office of Management and Budget, June 9, 2004: http://www.firstgov.gov/webcontent/about/documents/icgi_report.html"

Does the program use strong financial management practices?

Explanation: The USA Services program is audited as part of the independent "GSA Consolidated Annual Financial Statements" audit process. There have been no material weaknesses identified for the Federal Citizen Information Center Fund or for GSA's Operating Expenses since the annual audit process commenced 14 years ago. USA Services financial practices are closely monitored by a dedicated OCSC Financial Director who is also the Controller of the Federal Citizen Information Center Fund. All budgeting and expenditures are continuously reviewed.

Evidence: "USA Services participates each year in GSA's structured budget development process which starts with establihing a strategic view of USAS's processes, measures and targets for up-coming fiscal years, and ends with determining initiatives and their costs for the upcoming budget cycle. 2007 GSA Congressional Justification: http://www.gsa.gov/gsa/cm_attachments/GSA_DOCUMENT/2007CJ_R2E-s65_0Z5RDZ-i34K-pR.pdf"

Has the program taken meaningful steps to address its management deficiencies?

Explanation: Prior to the creation of the Office of Citizen Services (OCS), the components of USA Services were managed within different parts of GSA without any coordination or unity of purpose. FirstGov and Intergovernmental Solutions were in the Office of Governmentwide Policy, the Federal Information Center was in the Federal Telecommunications Service, and the Federal Citizen Information Center was housed in the Office of Communications. GSA established OCS in 2002 to coordinate the management of its citizen-facing programs and transferred all of those activities to this office as the USA Services program. OCS now has management and budgetary control over all of the units that support the current USA Services vision. USA Services also has procedures in place to monitor its performance on an on-going basis and make program improvements where needed. For example, USA Services conducts ongoing usability testing of FirstGov.gov and other its websites to improve ease of use and navigability. As a result of citizen feedback, a major redesign is underway and will be completed in FY 2007. Similarly, discontent with the previous search capability led to the January 2006 launch of a superior search engine in partnership with MSN and Vivisimo. Research indicated that web users wanted information pushed to them on topics of interest and so USA Services launched GovDelivery which sends alerts directly to subscribers when information is updated. USA Services has established and continues to improve the frequently asked questions (FAQs) system that allows citizens to access the same knowledgebase that NCC telephone agents use. We have integrated the online FAQs with the new FirstGov search service so that, if there are answers related to the search terms used by the public, those answers are provided along with other search results. We have also improved the monitoring of performance by revising our usage of service metrics to reflect website "visits" instead of the less-accurate "page views" that was previously used. USA Services continues to look for ways to eliminate program and process inefficiencies to improve service to citizens and to our government partners.

Evidence: "OCSC Creation - GSA Order ADM 5440.564 - 06/17/2002: http://insite.gsa.gov/Insite/gsa/ep/portletView.do?pageTypeId=8683&channelId=-8790&redirectUrl=http%3A%2F%2Finternotes.gsa.gov%2FInsite%2Fgsad.nsf%2FInsiteDirectivesDisplay%2F48483D0BE30C03C385256BDC006D26DD%3FOpenDocument (Not accessible outside GSA, copies available on request) OCSC Reorganization - GSA Order ADM 5440.593 - 02/14/2006: http://insite.gsa.gov/Insite/gsa/ep/portletView.do?pageTypeId=8683&channelId=-8790&redirectUrl=http%3A%2F%2Finternotes.gsa.gov%2FInsite%2Fgsad.nsf%2FInsiteDirectivesDisplay%2F84FF220D903B5BBC8525711C004AC009%3FOpenDocument (Not accessible outside GSA, copies available on request)"

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: USA Services has shown consistent improvement in meeting its two long-term outcome goals; i.e., expanding citizen use of its information channels and improving citizen satisfaction with all Federal websites. During the baseline year of FY 2004, USA Services achieved over 101 million citizen touchpoints. The citizen touchpoints climbed to 122.6 million in 2005 and nearly 133 million in 2006, an increase of 31% in two years above the targets for both years. During that same time period, the Government-wide ACSI customer satisfaction scores climbed from 70.3 during benchmark year FY 2004 to 73.7 in FY 2006. This increase of 3.4 points over two years is large for a measure for which an annual increase of 0.5 points is considered ambitious.

Evidence: See the PART performance measures section for information on how well USAS has done in achieving its goals.

Does the program (including program partners) achieve its annual performance goals?

Explanation: USA Services has consistently achieved all of its annual performance goals. The increase in citizen touchpoints demonstrates USA Services' success in promoting use of its information channels. The increase in the ACSI scores demonstrates the success of USA Services in providing citizens with easy access to accurate, consistent and timely government information, whether through its websites or through it partner agencies's websites. The annual measures also indicate USA Services' success in providing partner agencies with qualified contact center services. The only measure for which USA Services failed to meet an annual performance target was the efficiency measure of cost per citizen touchpoint where actual results were within nearly 99% of the FY 2006 target.

Evidence: "See the PART performance measures section for information on how well USAS has done in achieving its goals. 2005 Annual GSA Performance and Accountability Report: http://www.gsa.gov/gsa/cm_attachments/GSA_DOCUMENT/GSAFullPAR_111505_Final_R2F-aAB_0Z5RDZ-i34K-pR.pdf 2005 Appendix II: Summary Chart of Performance Goals and Measures: http://www.gsa.gov/gsa/cm_attachments/GSA_DOCUMENT/2005%20PAR%20appendix%20II%20completed_R2F-aAB_0Z5RDZ-i34K-pR.pdf 2007 GSA Congressional Justification: http://www.gsa.gov/gsa/cm_attachments/GSA_DOCUMENT/2007CJ_R2E-s65_0Z5RDZ-i34K-pR.pdf 2006 GSA Congressional Justification: http://www.gsa.gov/gsa/cm_attachments/GSA_DOCUMENT/2006CJ_R2E-s65_0Z5RDZ-i34K-pR.pdf"

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: USA Services has designated cost-per-citizen touchpoint as the principal measure of its efficiency and cost-effectiveness for purposes of reporting program performance. This measure is derived by dividing the total annual program budget dollars by the number of annual citizen touchpoints for all information channels. USA Services successfully increased the number of citizen touchpoints over 31% between FY 2004 and 2006, but increases in spending to enhance the channels resulted in a slightly increased cost per touchpoint from 30.3?? in FY 2004 to 31.5?? in FY 2005 and 2006. However, cost-per-citizen touchpoint is projected to decrease in future years as the volume of citizen usage increases and shifts from high-cost channels, such as telephone and e-mail, to lower cost channels, such as the web and the Frequently Asked Questions online database. Although cost per citizen touchpoint is the only efficiency measure that USA Services reports routinely, it is not the only efficiency or cost effectiveness measure by which the program should be judged. From the time USA Services was launched as an e-Gov initiative in July 2003, it has demonstrated a proven track record for improved efficiencies and cost effectiveness each year. Instead of creating USA Services as a new entity, as originally envisioned, GSA decided to build the program on the existing Federal Citizen Information Center (FCIC) infrastructure. This significantly reduced both the cost and time to required to implement the initial operational stages of this initiative compared to initial estimates. Shortly after USA Services began operation, it was decided to add a new a information channel for e-mail inquiries. By adding this requirement to the existing FCIC National Contact Center contract, the program was able to implement this capability much more quickly and at costs far lower than would have been the case if a new contract had been solicited. USA Services subsequently developed the FirstContact contract vehicle to provide agencies a quicker, cheaper way to respond to citizen calls and emails than establishing their own call centers. Use of this contract vehicle to award task orders to pre-vetted vendors, has saved the government millions of dollars in acquisition costs and has provided a means for agencies to obtain support in emergency situations within a matter of days instead of weeks or months. Another example of USA Services cost-effectiveness is its use of Public Service Advertising (PSA) campaigns to inform citizens of the availability of its information channels. For example, in FY 2005, for a production cost of less than $1.2 million, USA Services received over $16.8 million of free placement on TV and radio broadcasts and in print magazines and newspapers - a fourteen-fold return on the investment. USA Services achieved further savings in FY 2006, when it re-competed contracts supporting the FirstGov search service and the website hosting service at significant savings to the government. Additionally, the custom search services being provided to many affiliate agencies free of charge has enabled those agencies to avoid the significant costs associated with acquisition of their own search services. These examples demonstrate that this program achieves significant improvements in efficiency and cost effectiveness beyond what is reported by the cost per touchpoint measure.

Evidence: USA Services eGov Program Cost Savings Estimating Process, Version 1.2, March 7, 2006, MITRE Corporation. Also see the PART performance measures section for information on how well USAS has done in achieving its goals.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: USA Services delivers a unique combination of services that, taken together, has no counterpart elsewhere in government or in the private sector to which this program can be compared. However, there are many government and private programs that provide some of the individual components of USA Services to the public. Over the years, the performance of USA Services' websites, contact centers, and publication distribution operations have been reviewed and compared to their public and private sector counterparts. In all those comparisons, USA Services activities have met or exceeded the performance of their government or the private-sector counterparts. For example, in recent years, USA Services and its components have either won or been finalists of over 100 awards. USA Services components were finalists in The Stockholm Challenge Award 2006 (FirstGov.gov), and the Government Customer Support Excellence Award, Customer Focus Category (National Contact Center). USA Services won the CIO Leadership Award in May 2006 for its creation of the Citizen Services Cost Calculator, innovative software that enables agencies to pre-determine the cost of contact center services. And it was recently announced that the new FirstGov Search has won the prestigious Pioneer Award from Federal Computer Week which recognizes innovative use of technology in a U.S. Government program

Evidence: "GAO-06-270 Federal Contact Centers: Mechanism for Sharing Metrics and Oversight Practices??: http://www.gao.gov/new.items/d06270.pdf ACSI Overall Federal Government Scores with Historical Scores of Agencies Measured 1999-2005: http://www.theacsi.org/government/govt-all-05.html Q1 2006 ACSI E-Government Satisfaction Index: http://www.foreseeresults.com/WhitePapers/ACSI_EGov_Commentary_Mar06.pdf GAO-B-114829 Comparative Study on the Unit Costs and the Turn-Around Times and the Funding of ...: http://archive.gao.gov/f0102/090678.pdf A list of awards is available upon request, FirstGov.gov Awards are posted at: http://firstgov.gov/About/Awards.shtml"

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: As mentioned previously, there has never been a comprehensive, independent evaluation of the USA Services Program in its entirety. However, individual components of the program have, over the past few years, been reviewed by OMB, GSA, GAO, and the public (through customer satisfaction surveys). All of these reviews have found that the different elements of USA Services are effective and achieve their intended results. In August 2005, OMB's Office of Electronic Government and Information Technology conducted a Gap Analysis of Electronic Government Initiatives to determine their conformance to legislative, regulatory, and management polices. USA Services was included in this analysis and found generally compliant with those policies. In February 2006, a GAO audit looked at USA Services' National Contact Center (NCC) along with 4 other agency contact centers and found the NCC to meet all of their criteria for information quality. American Customer Satisfaction Index (ACSI) surveys on four USA Services websites and on the publication order portion of the program are also conducted on an ongoing basis to evaluate the performance of those components in meeting citizen needs and expectations. These USA Services components consistently receive among the highest scores for their comparable activities. Additionally, over the past few years, USA Services and its components have either won or been finalists of over 100 awards, which demonstrates that the USA Services program is widely considered to be performing at a very high level. Currently, USA Services is developing a Statement of Work for a comprehensive evaluation of the program to be completed during the second quarter of FY 2007. We will continue to conduct these independent evaluations on a regular basis.

Evidence: "GAO-06-270 Federal Contact Centers: Mechanism for Sharing Metrics and Oversight Practices??: http://www.gao.gov/new.items/d06270.pdf Gap Analysis of Electronic Government Initiatives, Office of Management and Budget's, Office of Electronic Government and Information Technology, August 2005. ACSI Overall Federal Government Scores with Historical Scores of Agencies Measured 1999-2005: http://www.theacsi.org/government/govt-all-05.html Q1 2006 ACSI E-Government Satisfaction Index: http://www.foreseeresults.com/WhitePapers/ACSI_EGov_Commentary_Mar06.pdf GAO-B-114829 Comparative Study on the Unit Costs and the Turn-Around Times and the Funding of ...: http://archive.gao.gov/f0102/090678.pdf A list of awards is available upon request, FirstGov.gov Awards are posted at: http://firstgov.gov/About/Awards.shtml USA Services has won the following awards: 1) Top 25 Excellence.gov from the American Council for Technology and Industry Advisory Council (2003), 2) Intergovernmental Solutions Award, People's Choice from the American Council for Technology (2004), and 3) the CIO Leadership Award from the CIO Council for the USA Services Citizen Services Cost Calculator (2006)."