Detailed Information on the
GSA Integrated Technology Services Portfolio Assessment

ITS will establish an independent evaluation capability. An acquisition was completed in Dec 2007 to bring in outside expertise to support Portfolio Management within ITS. Tasks to be completed include: (1) Developing a baseline of current ITS programs and initiatives, including contracts, markets, products and services, customers/clients, (2) Evaluating existing measures and identifying quantifiable value metrics that can provide the ITS Portfolio with the ability to derive greater value from its programs.

ITS will use independent evaluations of the Portfolio effectiveness. A complete Portfolio assessment will be completed with improvements identified.

Measure: External Customer Satisfaction ITS IT Aquisition Center

Explanation:The overall customer satisfaction score is based on the ACSI standards. Aggressive goals are set to achieve a 10% improvement over 5 years with annual targets supporting that improvement. FY06 Customer Satisfaction Survey results: FAS ITS FY 07 Scorecard GSA FY2006 PAR, Appendix I, (page 180) CFI Group Customer Satisfaction Studies for ACSI ACSI Home Page (http://www.theacsi.org)

Measure: Cost avoidance/savings achieved by ITS Portfolio programs

Explanation:The cost avoidance/savings measure was developed to partially gauge if best value is achieved for customer agencies and taxpayers. The actual is calculated by comparing GSA costs to commercial industry (i.e., regional GSA offices provide cost data and it is compared to published statistics by Gardner Group) See: FY05 Network Services Program Review Section 7 for detail. The figures shown for cost savings reflect the ITS Portfolio today, which now includes the SmartBUY program that uses a savings measure along with the Network Services program. In FY05 the Networks only methodology changed, by use of an "Aggregated" instead of "Average" method. The "Aggregated" method compares the FTS2001 prices paid by a customer with the commercial prices offered by their service provider. The "Average" method compares FTS2001 user pricing against the average commercial prices among all service providers.

Measure: IT Acquisition Center Cycle time to process offers

Explanation:IT Aquisition Center Cycle time to process offers is measured to ensure timely delivery to the customer, impacts customer satisfaction, and affects the timeliness of bringing new products and services to the market. Data is from FY06/FY07 ITS Scorecard and trend analysis from 2004 - 2006 Targets were created by using a three year trend linear trend starting in FY04.

Measure: IT Acquisition Center Cycle time to Process Modifications

Explanation:IT Acquisition Center Cycle time to Process Modifications to contracts to ensure prompt delivery to the customer, impacts customer satisfaction, and affects the timeliness of bringing new products and services to the market. Data is from FY06/FY07 ITS Scorecard and trend analysis 2004 - 2007 FY07 targets created by using a three year trend linear trend starting in FY04.

Measure: ITS direct costs for all programs divided by ITS gross margin.

Explanation:This measure reflects the direct costs as a percentage of gross margin (to maintain fiduciary responsibility and financial solvency). A new baseline has been established as a result of the FAS reorganization. Data from ITS FY06 and FY07 Scorecard Long Distance Regional Telecom and Other projects were not part of ITS totals in FY03 through FY05. (Target = FY06 actual - 1.65%) ITS efficiency was not measured as a portfolio prior to 2006

Is the program purpose clear?

Explanation: The purpose of the Office of Integrated Technology Services (ITS) in the Federal Acquisition Service (FAS) of the General Services Administration (GSA) is to develop and manage programs for the acquisition of information technology (IT) and telecommunications products and services and to manage the delivery of network services solutions to Federal agency customers worldwide. ITS also enables state and local governments to purchase IT products, services, and support equipment from Federal Supply Schedule. The mission of ITS is to: "[e]nable government agencies to achieve their missions by providing acquisition programs that efficiently and cost-effectively deliver best value information technology solutions."

Evidence: GSA ADM 5440-591 Change 1 Announcing Changes in the Federal Acquisition Service (10/12/06), Section 211 of the E-Government Act of 2002, Federal Property and Administrative Services Act of 1949.

Does the program address a specific and existing problem, interest, or need?

Explanation: The Federal Government needs to use information technology strategically in order to achieve efficient and effective operations, deliver services to taxpayers, protect the health and safety of Americans, and enhance the national security of the United States. Because many Federal agencies do not have the organic ability or size to negotiate major IT purchases, ITS can achieve savings by negotiating on behalf of multiple agencies and providing: ?? - Voice, data, and video telecommunications services and solutions ?? - Federal agency support at locations around the world ?? - Award and administration of comprehensive IT solution contracts ?? - Planning and start-up of new initiatives to support government wide priorities and emerging technologies - Manage Government-wide Programs (SmartBuy, HSPD-12, E-Authentication)

Evidence: H.R. Report 104-450 Conference Report, Division E - Information Technology Management Reform , in connection with Clinger-Cohen Act of 1996; Statement Assistant Comptroller General Gene Dodaro before Sub-committee on Oversight of Government Management and District of Columbia, Senate Governmental Affairs Committee (7/25/95).

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: Many agencies have the ability to establish government-wide acquisition contacts which serve the same function as those operated in the ITS Portfolio. The GSA reorganization merged the Federal Technology Service (FTS), Federal Supply Service (FSS), and all other GSA IT acquisition programs into one portfolio to (1) leverage the Government's purchasing power, (2) manage programs more efficiently, (3) deliver a suite of IT and telecommunications products, services and solutions to customers and (4) ultimately eliminate the need for agencies to establish their own contracts.

Evidence: Federal Property and Administrative Services Act of 1949. GSA ADM 5440-591 Change 1 Announcing Changes in the Federal Acquisition Service (10/12/06).

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: ITS is free from major design flaws that would prevent it from meeting its defined objectives and performance goals, and there is no strong evidence that another approach or mechanism would be more efficient or effective. FAS consolidates GSA management of Federal Technology Service (FTS), Federal Supply Service (FSS), and all other GSA IT acquisition programs under one roof. This eliminates prior confusion affecting industry and GSA customers. It allows GSA to manage programs more efficiently in a complimentary atmosphere and deliver superior solutions to customers from offerings throughout the ITS portfolio.

Evidence: GSA ADM 5440-591 Change 1 Announcing Changes in the Federal Acquisition Service (10/12/06), FY07 ITS Scorecard. Subsection 109(b) of the Federal Property and Administrative Services Act, requires the GSA Administrator to recover direct and indirect costs of most program activities in the prices paid by customer agencies for goods and services.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: ITS is designed in a way that ensures resources are being used to meet the program's purpose and adequately reach the right beneficiaries. For example, in FY 2006, the ITS SmartBuy initiative's eight agreements saved Federal agency customers $100 million on sales of approximately $500 million in commercial software. In addition, ITS helps its customers purse a more enterprise-level approach to telecommunications services acquisitions consistent with Office of Management and Budget policy on enterprise architecture. This behavioral change mirrors underlying changes in the telecommunications market itself. All activities directly or indirectly serve customer agencies (the intended beneficiaries of the program).

Evidence: FY 2006 GSA Annual Performance and Accountability Report (p.69); FY 2008 Strategy and Action Plan for FAS ITS

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: ITS has two long-term outcome performance measures that support the ITS purpose: (1) External Customer Satisfaction and (2) combined cost avoidance/savings achieved by ITS programs (to provide best value for customer agencies and taxpayers). These measures link to FAS long-term goals based on the Federal Acquisition Regulation, and align with GSA strategic goals.

Evidence: FY 2007 Network Services Strategic Assessment document and Performance Management Tool (PMT) data. ITS Strategic Assessment document dated 3/7/07, FY07 ITS Scorecard.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: ITS has established baselines for its two long-term measures and set ambitious quantified targets for each of them (through 2011). Targets and timeframes for the long-term measures are currently being solidified in strategic planning sessions. The five year targets for improvements in each of the measured areas are: (1) raise the ITS customer satisfaction index rating by 10%, and (2) increase customer cost avoidance by 10%. Refer to measures section for details

Evidence: Performance Management Tool (PMT) data and FAS FY2007 Performance Management Process. The Network Services and IT Schedule 70 Program long-term goals, measures, and targets are updated annually in the GSA PMP Strategic Assessments, Strategy, and Action Plans.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: ITS has annual performance measures that demonstrate progress towards achieving ITS long-term goals: (1) External Customer Satisfaction (2) Combined cost avoidance/savings (identical to the long-term measures identified in the answer to Question 2.1); and (3) Cycle Time to add products and services (to ensure timely delivery to customers, enhance customer satisfaction, and increase the timeliness of bringing new products and services to market).

Evidence: FY 2007 GSA Performance Plan (updated annually in the PMP Strategy and Action Plans). Measures results are posted on the GSA Intranet website (pmt.gsa.gov). GSA FY2006 PAR, Appendix 1.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The ITS scorecard indicates the baselines for each measure and ambitious stretch annual targets that encourage achieving its annual and long-term goals. The annual goals support ambitious year over year improvement to achieve long term goals. The 2007 targets for cost avoidance and customer satisfaction support growth towards the long term goals. The 2007 target for cycle time improvement is based on the data from previous three years. Performance plan/performance budget data targets focus on continued improvement See measures section for detail.

Evidence: ITS Strategic Assessment document dated 3/7/07. Aggregation of current ITS PMT data, FY 2007 ITS Scorecard Data Files

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: ITS partners contribute toward program goals in three ways. First, ITS contractors contribute toward overall program goals by agreeing to and implementing contractual Terms and Conditions that outline specific performance standards by which they are evaluated. Successful achievement of the Terms and Conditions directly relates to the business lines' ability to achieve their annual performance measures and contribute to the Long Term Goals. Second, All MAS acquisition centers including the IT Schedule 70 work towards all the annual and long term Multiple Award (MAS) program goals and share performance measures. Third, other FAS program offices that perform support activities share appropriate performance measures. ITS programs work with customers to ensure they meet their needs. For example the ITS Network Services reviews proposed changes with an Interagency Management Council (IMC) comprised of customers to ensure the changes will address the needs of the customers. Updates are made based on the recommendations of the IMC before the changes are put in place.

Evidence: Individual contract documentation. Section C.4 of the FTS2001 Telecommunications Contract example: Networx Universal Request For Proposal TQC-JTB-05-0001, Section J.1 pp 150-160, FTS 2001 TQC-SS-97-2001 Request For Proposal Section C.2.1.15 page 34 Quarterly network service contract program reviews. The Aquisition Quality Measurement and Improvement Program initiated the use of pre-negotiation panels and post award quality reviews of Multiple Award Schedule contracts dated (3/23/03). Partners' activities are linked to program goals through contract performance measures. In addition, FAS support activities conduct internal customer surveys each year and are graded on the level of support provided to portfolio programs. GAO Report- contract Management: Opportunities to Improve Pricing of GSA Multiple Award Schedules Contracts (2005).

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The ITS programs are subject on a regular basis to independent evaluations which include contract quality reviews and financial audits. In FY 2007, these programs were merged into a new portfolio within FAS, and, to date, an evaluation of sufficient scope has not been conducted. In FY 2008, the entire ITS portfolio will be evaluated to validate or, as needed, modify its business practices. Based on the evaluation, a portfolio improvement plan will be created.

Evidence: Annual PriceWaterhouseCoopers audit to the Inspector General dated 01/22/07 GAO Report Feb 2005 Opportunities to Improve MAS Pricing (see 2.5 evidence) GAO Report July 2005 Improving MAS Schedules Pricing (see 2.5 evidence) CFI Customer Service Report using the American Customer Satisfaction Index (ACSI) through June 2006.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: GSA requires all of its major programs to adhere to its Performance Management Process (PMP) which requires that all external budget requests (OMB Submission and Congressional Justifications) link program funding requests to the accomplishment of its performance measures. GSA has linked its funding requests to performance measures since FY 2004 and will continue this practice going forward. ITS will adjust the measures it links to in the FY 2009 budget process to the measures approved by OMB in this PART review.

Evidence: FY 2008 Congressional Budget Justification http://www.gsa.gov/gsa/cm_attachments/GSA_DOCUMENT/2007CJ_R2E-s65_0Z5RDZ-i34K-pR.pdf

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: GSA's Performance Management Process includes an annual strategic assessment and strategy and action plans. It is an annual collaborative decision-making process that involves senior leaders across the GSA organization. Long-term goals, measures, and targets are developed as part of this process. GSA has linked performance goals to resource requirements for this program in the FY 2008 Congressional budget, and program costs are presented separately. GSA has a Strategic Assessment and Strategic Action Plan (SAP) process to support its budget development and a Performance Measurement Tool (PMT) process for managing results in accordance with its SAPs. FY 2007 strategic efforts are now in process.

Evidence: Guide to the FSS Performance Measurement System, FTS program Scorecard, FY 2008 Congressional Budget Justification, FY 2006 Annual Performance and Accountability Report

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: ITS regularly collects performance data from contractors and customers relating to program goals, and uses it to adjust program priorities and take other appropriate management actions. It has also set ambitious targets based on this data. GSA's senior management meets quarterly to review FAS ITS performance data. Performance data is also used on an on-going basis by program managers in several ways, such as using customer satisfaction data to identify performance strengths and specific customer concerns. In addition, eighty-four percent of the contracting officers in the ITS IT Acquisition Center meet the necessary job qualifications, and all have attended required annual training to maintain their qualifications.

Evidence: FY 2007 ITS Scorecard, monthly performance reports, and annual customer satisfaction survey.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: ITS has identified the managers who are responsible for achieving key program results, and has established clearly defined performance standards and accountability for those managers. They are held accountable through the annual performance review process and ongoing monitoring of major business performance and internal process quality indicators. In addition, ITS contracts contain specific performance standards and contractors are held accountable for meeting those standards. An example of this is an SLA agreement between the General Services Administration (GSA) and the contractor to provide a service at a performance level that meets or exceeds the performance objectives specified in the Request For Proposal (RFP). Networx performance objectives are measurable service attributes that are key indicators of contractor performance.

Evidence: GSA Annual Performance Plan and Appraisal System (APPAS) dated 12/31/05. Quarterly Contractor Program Review. Sample contract. GSA Associate Performance Recognition System (APRS) dated 1/31/05 Networx Universal Request For Proposal TQC-JTB-05-0001, Attachment J.13. Networx Enterprise Request For Proposal TQC-JTB-05-0001, Attachment J.13.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: ITS funds are obligated in a timely manner and spent for the intended purpose. Establishing obligations prior to processing payments is an inherent aspect of GSA's accounting system. ITS reports monthly on actual expenditures compared to planned use. The ITS Portfolio, in cooperation with the FAS Controller's office, utilizes a financial planning process which establishes resource requirements for its operations prior to the execution of the current year's budget and operations. These plans are tracked throughout the year on a monthly basis and compared with actual spending as funds are utilized to execute program operations. Any large variations between planned and actual spending are researched to ensure that program operations are being executed according to plan. GSA and FAS financial practices and processes are also certified through the annual financial audit as documented in the Annual Performance and Accountability Report for GSA.

Evidence: FY 2008 Congressional Budget Justification; GSA FY 2006 Annual Performance and Accountability Report; PriceWaterhouseCoopers audit (dated 01/22/07).

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: GSA completes the annual Fair Act Inventory and identifies positions suitable for outsourcing studies. Contract offer and modification "cycle time," "Direct Dollar costs as a percentage of revenue," and "Direct Dollar cost as a percentage of gross margin" are all measures that encourage the program to become more efficient and be concerned about costs and the Government's return on investment. The ITS efficiency measure is: "Direct costs as a Percentage of Gross Margin." The ITS Schedule 70 participates in the Multiple Award Schedule Express Program created to speed the contract process. The goal of the program is to achieve 30 day contract awards for eligible vendors. This contributes to the overall ITS IT Schedule 70 cycle time measurement.

Evidence: FY 2006 GSA Annual Performance and Accountability Report; Fair Act Inventory, Competitive Sourcing Studies; e-Offer and e-Mod system reports; ITS Scorecard. PMP measures, targets and status updates are posted on the GSA Performance Management Tool system. Multiple Award Schedule Express Program dated 1/16/2007 : www.gsa.gov.masexpress.

Does the program collaborate and coordinate effectively with related programs?

Explanation: ITS plans to strengthen its collaboration with critical partners like Chief Information Officers, Chief Acquisition Officers, and executives leading major business and acquisition programs. ITS also collaborates with state and local government representatives to ensure they understand their authority to use IT Schedule 70. To this end, ITS offers training sessions on how state and local governments may use these contracts. ITS also collaborates with the other programs within the FAS, including AAS, to ensure customer needs for technology solutions are satisfied.

Evidence: FAS ITS Strategic Assessment (3/7/07), FAS ITS briefing (12/11/06)

Does the program use strong financial management practices?

Explanation: The ITS Portfolio is supported by Pegasys, the GSA official financial system of record. The FAS Controller's Office monitors monthly spending and prepares monthly operating expense reports to ensure that the program is operating within annually established budget levels. The Information Technology and General Supply funds that ITS use to pay for its operations are subject to the standards of GSA's annual financial audit and received favorable opinions for FY 2006.

Evidence: GSA FY 2006 Annual Performance and Accountability Report; Price Waterhouse Coopers audit dated 01/22/07; Quarterly Briefings of GSA Administrator; Use of Performance Measurement Tool.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: Managers prepare annual assurance statements, which identify management deficiencies and steps for improvement. GSA has implemented a formalized process for addressing GAO and IG recommendations which ITS implements in its area of operations. GSA is also doing the exploratory research and analysis necessary to provide the products and services customers want when they want them.

Evidence: Quarterly Contract Reviews, Strategic Assessments, and Strategic Action Plans were developed in FY 2006. FYO6 Annual Assurance Statements.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The results for both long term goals have improved over the available data periods. Based on the annual progress noted below in the answer to Question 4.2, ITS is making measurable progress, and is on track, towards achieving both its long-term performance goals (customer satisfaction and cost avoidance/savings).

Evidence: FAS ITS FY 07 Scorecard see measures section

Does the program (including program partners) achieve its annual performance goals?

Explanation: The IT Acquisition Center was historically included as a part of the legacy FSS MAS Program Customer Satisfaction Survey. As part of the FAS reorganization, it is anticipated the IT Acq Center will continue to be surveyed as part of the MAS program, at least for FY07. The MAS Program score improved from from 71.2 in FY 2004 to 73 in FY 2006 (an increase of 2.5%). Cost avoidance accrued by customers of ITS increased from $632 million in FY 2005 to over $720 Million in FY 2006 (an increase of over 13%). Savings are expected to increase to over $730 million in FY 2007 (another 1.4% increase in one year). In addition, ITS has achieved its annual performance goal involving cycle time. The IT Acq Center has reduced the number of days to process offers from 126.4 days in FY 2005 to 117.8 days in FY 2006 (a 7.3% reduction in cycle time in one year). Finally, based on the available goals and measures in the GSA FY2006 Annual Performance and Accountability Report (PAR), all programs covered by the ITS Portfolio met or exceeded those goals for FY2006.

Evidence: FAS ITS FY 07 Scorecard, GSA FY2006 PAR, Appendix I

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: For ITS programs for which there is specific prior year data efficiency improved between FY 2003 and FY 2006 measured by Direct Costs as a percent of Gross Margin. GWACS improved from 34.4% to 26.96% in FY 2006. IT Schedule 70 improved from 12.3% in FY 2003 to 9.7% in FY06.

Evidence: FAS ITS FY06 and FY07 Scorecard

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: Historically, the performance of ITS individual programs have compared very favorably to commercial rates. The latest data presented in the GSA FY2006 PAR shows a 41.5% savings in using the ITS Network Services contracts compared to similar commercial contracts. The IT Acquisition Center was historically included as a part of the legacy FSS MAS Program Customer Satisfaction Survey. As part of the FAS reorganization, it is anticipated the IT Acq Center will continue to be surveyed as part of the MAS program, at least for FY07. When comparing the aggregate American Customer Satisfaction Index (ACSI) score to others in the Information Industry and the Federal Government. For example... FY04 FY05 FY06 ITS Acquisition 71.2 71.4 73 Services (MAS) GWACS 74.4 78.5 78 Information Industry 67.5 65.8 69 All Federal 72.1 71.3 72.3

Evidence: LMI study, "Establishing Baselines for Measuring Acquisition Streamlining FY06 Customer Satisfaction Survey results: FAS ITS FY 07 Scorecard GSA FY2006 PAR, Appendix I, (page 180) CFI Group Customer Satisfaction Studies for ACSI ACSI Home Page (http://www.theacsi.org)

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: The previously referenced studies indicate the program achieves effective results. For example, in the 2004 MAS customer survey, customers were asked about their likelihood of using the MAS program in the future. The 2004 "likelihood to use schedules in the future score" was 90.2, up 0.9 points from 2003. When comparing the ITS IT Aquisition Center and GWACS Customer Satisfaction (ACSI) scores to comparable industry standards, the results are higher and consistently improving.

Evidence: FY 06 Customer Satisfaction Survey Results. GSA FY2006 PAR, Appendix I, (page 180) CFI Group Customer Satisfaction Studies for ACSI http://www.theacsi.org/index.php?option=com_content&task=view&id=18&Itemid=33 ITS FY06 and FY07 Scorecard