E-Government Scorecard Criteria

Item

Red

Agency or Initiative has received a red score in Security, or has any one of the following conditions:

Yellow

Achievement of some core criteria; no red conditions:

Green

Must meet all core criteria; no yellow conditions:

 

Mission Alignment  and Compliance with the E-Government Act of 2002

Alignment and Contribution to Federal, Departmental, and Agency Goals and Priorities and the E-Government Act of 2002

 

1.        Building partnerships

 

Program has established a process for maintaining an ongoing dialogue with interested parties on innovating service and information delivery through IT.

 

·   The program has provided no examples of ongoing dialogues with interested parties, and no specific evidence of an established process for maintaining an ongoing dialogue with interested parties.

 

·  The program has provided specific evidence of ongoing dialogues with other interested parties, but has limited evidence of an established process that will sustain the dialogue over time.

 

· The program has provided specific examples of ongoing dialogues with other interested parties, and has shared plans and/or policies designed to sustain such dialogues in the future.

 

2.        Applying performance measurement to improve customer service and productivity and to comply with legislation

 

Program managers use customer service performance and/or productivity measures as tools to transform agency operations, to align program outcomes with agency and Department goals and objectives, and to achieve compliance with statutory mandates such as the Government Performance Results Act (GPRA) and the Electronic Government Act of 2002.

 

·   The program has provided weak or no evidence of how any of its initiatives will help achieve agency and/or Departmental goals.

·   The program has not provided examples of performance measures designed to promote compliance with GPRA and/or the E-Gov Act.

 

·  The program has provided examples of initiative performance measures, but little evidence of how these measures link to agency and/or Departmental goals.

·  The program has provided limited evidence of how its performance measures promote compliance with GPRA or the E-Gov Act.

 

· The program has provided several examples of initiatives that use performance measures and results to achieve agency and/or Departmental goals.

· The program has provided specific examples of performance measures designed to promote compliance with GPRA and the E-Gov Act.

 

3.        Reducing costs

 

Program has an established methodology for quantifying cost savings and/or cost avoidance, and can quantify actual cost savings for specific initiatives.

 

·   Program has provided no evidence of a formalized methodology for measuring cost savings.

·   Program has provided no examples of how any of its initiatives have reduced costs to the government.

 

·  Program demonstrates an understanding of cost savings analysis, but has provided no evidence of a formalized process or policy to institutionalize the practice.

·  Program has provided some specific examples of how it has calculated cost savings on selected initiatives.

 

· Program has provided evidence of an established methodology for quantifying costs savings and/or cost avoidance.

· Program has provided multiple examples of cost savings analyses it has performed on initiatives that have reduced the cost of performing a government function.

 

4.        Description of how the program supports Paperwork Reduction Act (PRA) and Government Paperwork Elimination Act (GPEA) compliance efforts to reduce paperwork and cost burdens on the public.

·   If relevant, the program does not support efforts to reduce the public paperwork burden.

·   If relevant, the program does not meet GPEA compliance and efforts toward achieving goals are unsatisfactory.

·   The agency does not have a plan in place to ensure that availability of Government information and services is not diminished for those without access to the Internet.

·   The agency has 1 or more unresolved violations of the PRA and/or has filed emergency extensions for 10% or more of its total PRA inventory (Baseline is September 30 of rated year).

·   The agency has not established a plan to ensure that all public-use forms available on the Internet are the currently OMB-approved versions.

·  If relevant, the program has not clearly and fully articulated how it supports efforts to reduce the public paperwork burden.

·  If relevant, the program has not clearly and fully articulated efforts toward achieving GPEA goals in accordance with legislative requirements.

·  The agency has established a plan but has not implemented its use to ensure availability of Government information and services is not diminished for those without access to the Internet.

·   The agency has no unresolved violations and has 1 or more resolved violations of the PRA and/or has filed emergency extensions for 5% or more of its total PRA inventory (Baseline is September 30 of rated year).

·  The agency established a plan to ensure that all public-use forms available on the Internet are the currently OMB-approved versions.

· The relevant program has clearly and fully described how it supports efforts to reduce public paperwork burdens.

· If relevant, the program has clearly and fully articulated its progress and goals for continuous GPEA compliance.

· The agency has established and implemented their plan in to ensure availability of Government information and services is not diminished for those without access to the Internet.

 

·   The agency has had 0 violations of the PRA and/or has filed emergency extensions for 3% or less of its total PRA inventory (Baseline is September 30 of rated year).

·   The agency established and effectively implemented a plan to ensure that all public-use forms available on the Internet are the currently OMB approved versions.

 

Architecture Compliance

Alignment with Federal, Departmental and Agency Enterprise Architecture

 

5.        Transition or sequence planning (Implementation Plans)

 

Agency is successfully planning and communicating the transition of its investments from the “as-is” to a “to-be” architecture.

 

·   Agency provides evidence of transition plans incorporating fewer than 50% of its identified initiatives.

·   Fewer than 50% of Agency transition plans contain an investment Work Breakdown Structure (WBS) that clearly identifies key steps to closing the gap between as-is and to-be.

·   Agency has not submitted an Enterprise Architecture (EA) communication plan that describes the transition strategy.

 

·  Agency provides evidence of transition plans incorporating 50% to 80% of its identified initiatives.

·  50% to 80% of Agency transition plans contain an investment WBS that clearly identifies key steps to closing the gap between as-is and to-be.

·  Agency has submitted, but not yet initiated an EA communication plan that describes the transition strategy.

 

· Agency provides evidence of transition plans incorporating more than 80% of its identified initiatives.

· 80% or more of Agency transition plans contain an investment WBS that clearly identifies key steps to closing the gap between as-is and to-be.

· Agency has submitted and is currently implementing an EA communication plan that describes the transition strategy.

 

6.        Convergence of performance measures with business objectives

 

The Enterprise Architecture (EA) provides detailed performance measures with “line-of-sight” linkage to business objectives.

 

·   Agency provides quantifiable performance metrics for fewer than 50% of its investments.

·   Agency demonstrates “line-of-sight” linkage as described by the Federal Enterprise Architecture (FEA) Performance Reference Model (PRM)  and, if appropriate Exhibit 300 table I.C.2. for fewer than 50% of its investment measures.

·  Agency provides quantifiable performance metrics for 50% to 80% of its investments.

·  Agency demonstrates “line-of-sight” linkage as described by the FEA PRM and, if appropriate Exhibit 300 table I.C.2. for 50% to 80% of its investment measures.

· Agency provides quantifiable performance metrics for more than 80% of its investments.

· Agency demonstrates “line-of-sight” linkage as described by the FEA PRM and, if appropriate Exhibit 300 table I.C.2. for more than 80% of its investment measures.

7.        Integration of security into EA

 

Security measures are defined at each level of the Enterprise Architecture.

 

·   Fewer than 50% of Agency investments identify and address threats, privacy information and information related to confidentiality, integrity and authentication.

·   Fewer than 50% of Agency investments identify and address residual risk related to IT security and privacy.

·    50% to 90% of Agency investments identify and address threats, privacy information and information related to confidentiality, integrity and authentication.

·  50% to 90% of Agency investments identify and address residual risk related to IT security and privacy.

· 90% of Agency investments identify and address threats, privacy information and information related to confidentiality, integrity and authentication.

· 90% or more of Agency investments identify and address residual risk related to IT security and privacy.

8.        Integration between Agency and Department architectures

 

IT investments are described in terms of DOL or Agency functions and interoperability.

 

 

 

 

 

 

·   Fewer than 50% of Agency IT investments identify the specific Agency, Common or Universal functions they support.

·   Fewer than 50% of Agency IT investments identify interoperability requirements.

 

·  Between 50% and 80% of Agency IT investments identify the specific Agency, Common, or Universal functions they support.

·  Between 50% and 80% of Agency IT investments identify interoperability requirements.

 

· 80% or more of Agency IT investments identify the specific Agency, Common, or Universal functions they support.

· More than 80% of Agency IT investments identify interoperability requirements.

 

 

Security Documentation and Testing Compliance

Alignment with Federal and  Departmental IT Security Requirements:  Compliance with  documentation and testing requirements under the Computer Security Act, Privacy Act, FISMA, OMB Security Guidance, DOL security policies; DOL System Development Life-cycle Manual (SDLCM); DOL Computer Security Handbook (CSH); NIST Standards and guidelines.

 

9.        Management of Plan of Actions and Milestones (POA&M)

 

One or none of the following attributes:

·   POA&Ms are submitted on-time

·   Weaknesses are prioritized according to level of risk. 

·   Resources are specified in dollars by funding source and FY.

·   Weakness completion dates are provided.

 

POA&Ms are submitted on-time plus one of the following attributes:

·  Weaknesses are prioritized according to level of risk. 

·  Resources are specified in dollars by funding source and FY.

·  Weakness completion dates are provided.

 

POA&Ms are submitted on-time plus the following attributes:

· Weaknesses are prioritized according to level of risk.   

· Resources are specified in dollars by funding source and FY.

· Weakness completion dates are provided.

· Demonstrated 1% decrease in delayed weaknesses.

10.      Percentage of new employees and contractors that have received security awareness training

Less than 70% of new employees are trained within 60 days of starting date.

 

From 70% to 84% of new employees are trained within 60 days of starting date.

 

From 85% to 100% of new employees are trained within 60 days of starting date.

 

11.      Percentage[1] of Contingency Plans (CP) tested within the past 12 months.

·    For the October review, this means the percentage of systems with at least a notification or a tabletop exercise. [2] 

·    For the April Review, this means the percentage of systems demonstrating increasingly progressive testing (e.g. backup tape or combo exercises.)[3]

Less than 70% of Contingency Plans were tested within the past 12 months.

From 70% to 89% percent of Contingency Plans were tested within the past 12 months.

 

From 90% to 100% of Contingency Plans were tested within the past 12 months.

12.      Percentage of Sensitive Systems that have had technical controls adequately tested in the past 12 months.[4]

Less than 70% of technical controls were tested within the past 12 months.

From 70% to 89% of technical controls were tested within the past 12 months.

From 90% to 100% of technical controls were tested within the past 12 months.

13.      Percentage of Sensitive Systems for which security requirements and cost are monitored by the Program and are on time.[5]

Less than 70% of sensitive systems have security requirements and costs monitored and are on time.

From 70% to 89% of sensitive systems have security requirements and costs monitored and are on time.

From 90% to 100% of sensitive systems have security requirements and costs monitored and are on time. 




 

Project Management

Alignment with Federal, Departmental, and Agency Project Management Requirements

 

14.      Compliance with the Capital Planning Investment Control Process.

 

Clinger-Cohen Act, OMB Circular A-11, OMB Circular A-130, Systems Development Life Cycle (SDLC), Department’s guide to Capital Planning and Investment Control

·   Failure to comply successfully with key requirements.

·   A plan to ensure that all project managers for major IT projects are qualified[6] has not been developed.

·  Successful achievement of some of the key requirements.

·  Ability to demonstrate plans for utilizing qualified project managers for major IT projects and general support systems.

· Successful achievement of all key requirements.

· All major IT projects have qualified project managers.

 

15.      Electronic Capital Planning and Investment Control (eCPIC).

·   Failure to utilize eCPIC as required in the Department’s Capital Planning process (i.e., 300 “Lite” and relevant documentation has not been posted to the resource library (e.g., quarterly reviews, etc.).

·    Limited utilization of eCPIC, (i.e., 300 “Lite” and required resource documentation has not been posted to the resource library or is incomplete (e.g., quarterly reviews, etc.)

· Full utilization of eCPIC (i.e., 300 “Lite” and relevant documentation has been posted to the resource library by the requested date (e.g., quarterly reviews, etc.).

16.      Performance Measures (A-11, A-130).

 

 

·   No performance targets and measures were developed for major project milestones.

·    Performance targets and measures were developed for major project milestones;

·    Developed measures are partially consistent with program goals and objectives.

·    All performance targets and measures to date have not been met.

· Full use of performance targets and measures with major project milestones.

· Developed measures are fully consistent with programs goals and objectives.

· All performance targets and measures to date have been met.

17.      Viability and Risk Analysis.

 

·   Project risk analysis has not been performed; or

·   Risks mitigation strategy or activities have not been developed.

·    Risks and mitigation strategies have been addressed, but the mitigation strategies have not been adequately developed and documented to address cost, schedule, and performance impact.

· Risks and mitigation strategies are adequately documented; and

· The project continues to be viable.

18.      Summarize how any changes to the program’s scope have been assessed for risk, results and mitigation activities.

·   If relevant, no summary provided on changes to the initiative’s scope; and

·   If relevant, risk impact not documented.

·    If relevant, changes to the initiative’s scope have been addressed, but risk assessment and mitigation strategies have not been adequately documented.

· If relevant, changes to the initiative’s scope have been addressed; and

· If relevant, risk assessment, results, and mitigation activities are adequately documented.

19.      Projects are within 10% of cost, schedule and performance.

·   Projects are less than 50% within cost, schedule and performance.

· Projects are between 51 – 89% of cost, schedule and performance.

· Projects are within 90% or more of cost, schedule and performance.

 



[1] Agencies must provide documentation upon testing of the contingency plans to OCIO Security to receive credit for this metric. Refer to the contingency plan testing presentation from the January 2005 Information Technology Security Subcommittee (ITSSC) meeting and the June 17, 2004 memorandum regarding DOL Information Security Requirements for guidance on acceptable contingency plan testing. For definition of tabletop, notification, backup tapes, and combo exercises, see http://labornet.dol.gov/html/computer_security_resources.htm.

[2] Backup tape or combo exercises are acceptable alternatives to tabletop and notification exercises and will address the criteria for both October and April. 

[3] If a notification exercise had been conducted but not a tabletop exercise, then the tabletop exercise will be sufficient.  If a tabletop exercise had been conducted but not a notification exercise, then the notification exercise will be sufficient.  If both notification and tabletop exercises have been performed, then conduct a backup tape or combo exercise.

[4] This is a rolling metric defined by the number of significant systems that have tested all technical controls specified in the system security plan divided by the number of systems for the Agency.  Testing of technical controls is defined as the SSA in conjunction with one or more of the following independent/third party testing programs: (A) OCIO Security TSSM Logical Access Controls Testing (B) OCIO Security SCT&E (C) OIG Audit Testing (D) Independent Security Test and Evaluation

 

[5] Percentage of systems that monitor security deliverables, requirements, and costs in the project schedule (e.g. program management plan, work breakdown structure, Microsoft project schedule).  Present the result as a percentage.  Core deliverables are:  1. System Categorization, 2. Initial Risk Assessment, 3. Complete Risk Assessment, 4. Privacy Impact Assessment, 5. System Security Plan, 6. Plan of Action and Milestones (POA&M), 7. Contingency Plan (CP), 8. Certification Testing (ST&E), 9. Accreditation, 10. Annual CP Testing, 11. Annual Controls Tests, 12. Annual Document Review, 13. Other Security Requirements.  System Categorization is the performance of the Appendix A located in the DOL Guidelines for Identifying Major Information Systems. The initial Risk Assessment consists of a Federal Information Processing Standards (FIPS) 199 assessment and a system description. All requirements are required to be scheduled even for production legacy systems.  For legacy systems, the dates for requirements 1 – 9 will be related to tri-annual re-Accreditation.

[6] For the purposes of the E-Gov scorecard, a qualified project manager (PM) possesses: (a) experience managing IT projects of similar size and scope within 10% of the baseline cost, schedule and performance goals for the project to which currently assigned, (b) a Government Project management certification or a commercial certification such as the Project Management Institute (PMI), and (c) the individual is dedicated to the project or program.