This is the accessible text file for GAO report number GAO-03-584G entitled 'Information Technology: A Framework for Assessing and Improving Enterprise Architecture Management (Version 1.1)' which was released on April 01, 2003. This text file was formatted by the U.S. General Accounting Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. Executive Guide: April 2003: Information Technology: A Framework for Assessing and Improving Enterprise Architecture Management (Version 1.1): GAO-03-584G: Preface: Effective use of enterprise architectures is a recognized hallmark of successful public and private organizations. For over a decade, GAO has promoted the use of architectures, recognizing them as a crucial means to a challenging goal: agency operational structures that are optimally defined, in both business and technological environments. The alternative, as GAO's work has shown, is perpetuation of the kinds of operational environments that saddle most agencies today, in which lack of integration among business operations and supporting information technology (IT) resources leads to inefficiencies and duplication. Why are enterprise architectures so important? Metaphorically, an enterprise architecture is to an organization's operations and systems as a set of blueprints is to a building. That is, building blueprints provide those who own, construct, and maintain the building with a clear and understandable picture of the building's uses, features, functions, and supporting systems, including relevant building standards. Further, the building blueprints capture the relationships among building components and govern the construction process. Enterprise architectures do nothing less, providing to people at all organizational levels an explicit, common, and meaningful structural frame of reference that allows an understanding of (1) what the enterprise does; (2) when, where, how, and why it does it; and (3) what it uses to do it. Through our research of best IT management practices and our evaluations of agency IT management performance, we have identified a set of essential and complementary management disciplines. These include: * IT investment management, * software/system development and acquisition management, * IT services acquisition management, * IT human capital management, * information security management, and: * enterprise architecture management. Using the results of this research and evaluation, we have developed various IT management frameworks and guides. The federal Chief Information Officers (CIO) Council, at times in collaboration with us, has also published such guidance documents. In building on this portfolio of guidance documents, we offer here the first update to our maturity framework for enterprise architecture management.[Footnote 1] Its purpose is to provide federal agencies with a common benchmarking tool for planning and measuring their efforts to improve enterprise architecture management, as well as to provide the Office of Management and Budget with a means for doing the same governmentwide. This update is based on comments received on the initial version. Like the initial version, the update extends A Practical Guide to Federal Enterprise Architecture, Version 1.0, published by the CIO Council, by arranging the core elements in that guide into a matrix of five hierarchical stages and four critical success attributes. Questions and comments about the framework should be directed to me at (202) 512-3439. I can also be reached at hiter@gao.gov. Key contributors to this report were Naba Barkakati, Mark Bird, Barbara Collier, Deborah Davis, Neil Doherty, Tamra Goldstein, and Randolph Tekeley. Randolph C. Hite Director, Information Technology Architecture and Systems Issues: Signed by Randolph C. Hite: Contents: Preface: Section 1. Introduction: What Is an Enterprise Architecture? A Brief History of EA Management Guidance: Section 2. Description of EAMMF Version 1.1: Maturity Stages: Stage 1: Creating EA Awareness: Stage 2: Building the EA Management Foundation: Stage 3: Developing the EA: Stage 4: Completing the EA: Stage 5: Leveraging the EA to Manage Change: Critical Success Attributes: Attribute 1: Demonstrates Commitment: Attribute 2: Provides Capability to Meet Commitment: Attribute 3: Demonstrates Satisfaction of Commitment: Attribute 4: Verifies Satisfaction of Commitment: Core Elements: Stage 1: Creating EA Awareness: Elements for Stage 2: Building the EA Management Foundation: Elements Added for Stage 3: Developing EA Products: Elements Added for Stage 4: Completing the EA Products: Elements Added for Stage 5: Leveraging the EA to Manage Change: Overall View of EAMMF Matrix: Section 3. Uses of EAMMF Version 1.1: Tool for Assessing EA Management Maturity: EA Management Improvement Planning: Appendix. Approach to Developing EAMMF Version 1.1: Figures: Figure 1: Simplified Three-Dimensional View of EAMMF: Figure 2: Transitional View to Two-Dimensional EAMMF Matrix: Figure 3: Two-Dimensional EAMMF Matrix: Figure 4: EAMMF Matrix with Five Stages of Maturity Identified: Figure 5: EAMMF Matrix with Critical Success Attributes Added: Figure 6: Summary of EAMMF Version 1.1: Maturity Stages, Critical Success Attributes, and Core Elements: Tables: Table 1. Criteria for Selecting Automated EA Development and Maintenance Tools: Table 2. Major Categories of Comments and Suggestions: Abbreviations: CIO: chief information officer: C4ISR: Command, Control, Communications, Computers, Intelligence, Surveillance, and Reconnaissance: DOD: Department of Defense: EA: enterprise architecture: EAMMF: Enterprise Architecture Management Maturity Framework: FEAF: Federal Enterprise Architecture Framework: IT: information technology: ITIM: Information Technology Investment Management: OMB: Office of Management and Budget: TEAF: Treasury Enterprise Architecture Framework: This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.Section 1. Introduction: An enterprise architecture (EA) provides a clear and comprehensive picture of the structure of an entity, whether an organization or a functional or mission area. It is an essential tool for effectively and efficiently engineering business processes and for implementing and evolving supporting systems. The concept of an architecture to describe an enterprise first emerged in the mid-1980s, and over the years various frameworks[Footnote 2] for defining the content of EAs have been published. Our work in the early 1990s identified architectures as a critical success factor allowing organizations to effectively apply information technology (IT) to meet mission goals. Since then, we have worked with the Congress, the Office of Management and Budget (OMB), and the federal Chief Information Officers (CIO) Council to recognize the importance of architectures and assist agencies in developing, maintaining, and using them. In our reviews of agency IT management practices and major systems modernization programs, we continue to identify the lack of an architecture as a major management weakness, and we have made numerous recommendations addressing this important area. What Is an Enterprise Architecture? In simple terms, an enterprise can be viewed as any purposeful activity, and an architecture can be characterized as the structure (or structural description) of any activity. Building on this, EAs can be viewed as systematically derived and captured structural descriptions- -in useful models, diagrams, and narrative--of the mode of operation for a given enterprise, which can be (1) a single organization or (2) a functional or mission area that transcends more than one organizational boundary (e.g., financial management, homeland security). The concept of EAs dates back to the mid-1980s. At that time, John Zachman, widely recognized as a leader in the field of enterprise architecture, identified the need to use a logical construction blueprint (i.e., an architecture) for defining and controlling the integration of systems and their components.[Footnote 3] Accordingly, Zachman developed a structure or "framework" for defining and capturing an architecture. In his work, Zachman drew parallels to the field of classical architecture and later to the aircraft manufacturing industry, in which different work products (e.g., architect plans, contractor plans, shop plans, and bills of lading) represent different views of the planned building or aircraft. Similarly, Zachman's framework identified the kinds of work products needed for people to understand and thus build a given system or entity. This framework provides for six windows from which to view the enterprise, which Zachman terms "perspectives" on how a given entity operates: those of (1) the strategic planner, (2) the system user, (3) the system designer, (4) the system developer, (5) the subcontractor, and (6) the system itself. Zachman also proposed six abstractions or models associated with each of these perspectives: these models cover (1) how the entity operates, (2) what the entity uses to operate, (3) where the entity operates, (4) who operates the entity, (5) when entity operations occur, and (6) why the entity operates. Zachman's framework provides a way to identify and describe an entity's existing and planned component parts and the parts' relationships before one begins the costly and time-consuming efforts associated with developing or transforming the entity. Since Zachman introduced his framework, a number of other frameworks have been proposed. In September 1999, the federal CIO Council published the Federal Enterprise Architecture Framework (FEAF), which is intended to provide federal agencies with a common construct for their respective architectures, thereby facilitating the coordination of common business processes, technology insertion, information flows, and system investments among federal agencies. The FEAF describes an approach, including models and definitions, for developing and documenting architecture descriptions for multi-organizational functional segments of the federal government. Similar to the Zachman framework, the FEAF's proposed models describe an entity's business, data necessary to conduct the business, applications to manage the data, and technology to support the applications. More recently, OMB established the Federal Enterprise Architecture Program Management Office to develop a federated enterprise architecture according to a collection of five "reference models": * The Business Reference Model is intended to describe the business operations of the federal government independent of the agencies that perform them, including defining the services provided to state and local governments. * The Performance Reference Model is to provide a common set of general performance outputs and measures for agencies to use to achieve business goals and objectives. * The Data and Information Reference Model is to describe, at an aggregate level, the type of data and information that support program and business line operations, and the relationships among these types. * The Service Component Reference Model is to identify and classify IT service (i.e., application) components that support federal agencies and promote the reuse of components across agencies. * The Technical Reference Model is to describe how technology is supporting the delivery of service components, including relevant standards for implementing the technology. Together, the reference models are intended to facilitate governmentwide improvement through cross-agency analysis and the identification of duplicative investments, gaps, and opportunities for collaboration, interoperability, and integration within and across government agencies. These post-Zachman frameworks differ in their nomenclatures and modeling approach. However, the frameworks consistently provide for defining an enterprise's operations in both (1) logical terms, such as interrelated business processes and business rules, information needs and flows, and work locations and users, and (2) technical terms, such as hardware, software, data, communications, and security attributes and performance standards. The frameworks also provide for defining these perspectives both for the enterprise's current or "as-is" environment and for its target or "to-be" environment, as well as a transition plan for moving from the "as-is" to the "to-be" environment. The importance of developing, implementing, and maintaining an EA is a basic tenet of both organizational transformation and IT management. Managed properly, an EA can clarify and help optimize the interdependencies and relationships among an organization's business operations and the underlying IT infrastructure and applications that support these operations. Employed in concert with other important management controls, such as portfolio-based capital planning and investment control practices, architectures can greatly increase the chances that organizations' operational and IT environments will be configured so as to optimize mission performance. Our experience with federal agencies has shown that investing in IT without defining these investments in the context of an architecture often results in systems that are duplicative, not well integrated, and unnecessarily costly to maintain and interface.[Footnote 4] A Brief History of EA Management Guidance: Since the late 1980s, architecture guidance has emerged within the federal government, beginning with the publication of the National Institute of Standards and Technology guidance in 1989.[Footnote 5] Subsequently, we issued architecture guidance[Footnote 6] and published our research on successful public-and private-sector organizations' IT management practices, which identified the use of architectures as a factor critical to these organizations' success.[Footnote 7] Since that time, other federal entities have issued frameworks for defining the content of EAs, including the Department of Defense,[Footnote 8] Department of the Treasury,[Footnote 9] and the federal CIO Council[Footnote 10] (some of which were described earlier). These frameworks are being used today to varying degrees by many federal agencies. The emergence of federal frameworks and guidance over the last 5 years is largely owing to the Congress's passage of the Clinger-Cohen Act in 1996.[Footnote 11] This act, among other things, requires the CIOs for major departments and agencies to develop, maintain, and facilitate the implementation of architectures as a means of integrating business processes and agency goals with IT. In response to the act, OMB, in collaboration with us, issued guidance on the development and implementation of EAs.[Footnote 12] More recently, OMB issued additional guidance directing that agency investments in IT be based on agency architectures.[Footnote 13] Similarly, the CIO Council, in addition to publishing the FEAF, recently collaborated with us in issuing two additional EA guidance documents. The first addresses enforcement and describes how an organization should go about assessing whether its proposed IT investments are compliant with its EA.[Footnote 14] The second addresses development, maintenance, and implementation, describing in practical terms an end-to-end set of steps for an EA program.[Footnote 15] These steps include how to get started and organized, what kind of management controls are needed, what factors to consider in formulating an EA development approach, how to go about defining the current and target architecture and the plan for sequencing from the current to the target, how to ensure that the architecture is implemented and enforced, and how to systematically refresh and maintain the architecture to ensure its currency and relevance. The need for greater federal agency awareness and use of EAs was also recognized in the E- Government Act of 2002,[Footnote 16] which established the OMB Office of Electronic Government; this office's responsibilities include overseeing the development of EAs within and across federal agencies.[Footnote 17] Section 2. Description of EAMMF Version 1.1: The ability to effectively manage any activity (e.g., architecture development, maintenance, and use) depends upon having meaningful measures of that activity in relation to some standard. Such measurement permits managers to assess progress toward the desired end and to take corrective action to address unacceptable deviations. In February 2002, we issued Version 1.0 of the Enterprise Architecture Management Maturity Framework (EAMMF).[Footnote 18] The framework consists of three basic components: (1) hierarchical stages of management maturity, (2) categories of attributes that are critical to success in managing any endeavor, and (3) elements of EA management that form the core of the CIO Council Practical Guide. These three EAMMF components are interrelated, as depicted in figure 1, and are described in greater detail below. Figure 1: Simplified Three-Dimensional View of EAMMF: [See PDF for image] [End of figure] Elements, or more specifically core elements, are descriptions of a practice or condition that is needed for effective EA management. An example is designating a chief architect. The version of our framework presented here (Version 1.1) specifies 31 core elements, each of which is derived from the CIO Council Practical Guide. Based on the implicit dependencies among these 31 core elements, the EAMMF associates each element to one of five hierarchical management stages, referred to as maturity stages. Each stage reflects the collection of EA management practices and conditions (i.e., core elements) being undertaken by an enterprise at a given maturity level. An example of a stage is building the EA management foundation (Stage 2). The EAMMF also associates each element to one of four types of management attributes, referred to as critical success attributes. Each attribute represents a category or type of management practice and condition (i.e., core element) that is needed to effectively discharge any function. An example of a critical success attribute is demonstrating the institutional commitment to perform the function. Building on figure 1, figure 2 adds the number of core elements, maturity stages, and critical success attributes, and provides a transition to the EAMMF matrix[Footnote 19] presented in figure 3. Figure 2: Transitional View to Two-Dimensional EAMMF Matrix: [See PDF for image] [End of figure] Figure 3: Two-Dimensional EAMMF Matrix: [See PDF for image] [End of figure] The EAMMF is consistent with other maturity frameworks, such as GAO's Information Technology Investment Management (ITIM) framework,[Footnote 20] in that the EAMMF outlines steps toward achieving a stable and mature process for managing the development, maintenance, and implementation of EA. As an organization improves its EA management capabilities, its EA management maturity increases. By establishing the current level of maturity of an organization, managers are able to use the framework to determine steps needed to improve architecture management. Because the EAMMF is derived from the CIO Council Practical Guide, the framework should be viewed as an extension of the Practical Guide and thus used in tandem with it. Accordingly, the EAMMF is not intended to repeat the level of guidance provided in the Practical Guide, but rather to arrange key aspects (i.e., core elements) of the guide into a hierarchical model for use either as an evaluation tool or as a roadmap for EA management improvement. To facilitate this use, we have included references in the descriptions of the core elements indicating the corresponding sections in the Practical Guide. Maturity Stages: The EAMMF is made up of five stages of EA maturity, each of which includes all elements of previous stages. Each of the five stages is described below. To the generic EAMMF structure of figure 3, figure 4 adds the specific names of the five stages. Figure 4: EAMMF Matrix with Five Stages of Maturity Identified (in bold): [See PDF for image] Source: GAO. [End of table] Stage 1: Creating EA Awareness: At Stage 1, either an organization does not have plans to develop and use an architecture, or it has plans that do not demonstrate an awareness of the value of having and using an architecture. While Stage 1 agencies may have initiated some EA activity, these agencies' efforts are ad hoc and unstructured, lack institutional leadership and direction, and do not provide the management foundation necessary for successful EA development as defined in Stage 2. Stage 2: Building the EA Management Foundation: An organization at Stage 2 recognizes that the EA is a corporate asset by vesting accountability for it in an executive body that represents the entire enterprise. At this stage, an organization assigns EA management roles and responsibilities and establishes plans for developing EA products and for measuring program progress and product quality; it also commits the resources necessary for developing an architecture--people, processes, and tools. Specifically, a Stage 2 organization has designated a chief architect and established and staffed a program office responsible for EA development and maintenance. Further, it has established a committee or group that has responsibility for EA governance (i.e., directing, overseeing, and approving architecture development and maintenance). This committee or group is often called a steering committee, and its membership includes both business and IT representatives (i.e., the committee has enterprisewide representation). At Stage 2, the organization either has plans for developing or has started developing at least some EA products, and it has developed an enterprisewide awareness of the value of EA and its intended use in managing its IT investments. The organization has also selected a framework and a methodology that will be the basis for developing the EA products and has selected a tool for automating these activities. Stage 3: Developing the EA: An organization at Stage 3 focuses on developing architecture products according to the selected framework, methodology, tool, and established management plans. Roles and responsibilities assigned in the previous stage are in place, and resources are being applied to develop actual EA products. Here, the scope of the architecture has been defined to encompass the entire enterprise, whether organization-based or function-based. Although the products may not be complete, they are intended to describe the organization in business, performance, information/data, service/application, and technology terms (including security explicitly in each), as provided for in the framework, methodology, tool, and management plans. Further, the products are to describe the current ("as-is") and future ("to-be") states and the plan for transitioning from the current to the future state (the sequencing plan). As the products are developed and evolve, they are subject to configuration management. Further, through the established EA management foundation, the organization is tracking and measuring its progress against plans, identifying and addressing variances, as appropriate, and then reporting on its progress. Stage 4: Completing the EA: An organization at Stage 4 has completed its EA products, meaning that the products have been approved by the EA steering committee (established in Stage 2) or an investment review board, and by the CIO. The completed products collectively describe the enterprise in terms of business, performance, information/data, service/application, and technology for both its current and future operating states, and the products include a transition plan for sequencing from the current to the future state. Further, an independent agent has assessed the quality (i.e., completeness and accuracy) of the EA products. Additionally, evolution of the approved products is governed by a written EA maintenance policy approved by the head of the organization. Stage 5: Leveraging the EA to Manage Change: An organization at Stage 5 has secured senior leadership approval of the EA products and a written institutional policy stating that IT investments must comply with the architecture, unless granted an explicit compliance waiver. Further, decision-makers are using the architecture to identify and address ongoing and proposed IT investments that are conflicting, overlapping, not strategically linked, or redundant. Thus, Stage 5 entities are able to avoid unwarranted overlap across investments and ensure maximum systems interoperability, which in turn ensures the selection and funding of IT investments with manageable risks and returns. Also at Stage 5, the organization tracks and measures EA benefits or return on investment, and adjustments are continuously made to both the EA management process and the EA products. Critical Success Attributes: Associated with the maturity stages described above are characteristics or attributes that are critical to the successful performance of any management function. These critical success attributes are: (1) showing a commitment to perform the function; (2) putting in place the capability (people, processes, and technology) needed to perform the function; (3) demonstrating, via production and results, that the function has been performed; and: (4) verifying, via quantitative and qualitative measurement, that the function was satisfactorily performed. Collectively, these attributes form the basis by which an organization can institutionalize management of any given function or program, like EA management. Each of the EAMMF critical success attributes is described below. Figure 5 presents the four specific critical success attributes, building on the previous figures. Figure 5: EAMMF Matrix with Critical Success Attributes Added (in bold): [See PDF for image] Source: GAO. [End of figure] Attribute 1: Demonstrates Commitment: Because the EA is a corporate asset for systematically managing institutional change, the support and sponsorship of the head of the enterprise are essential to the success of the architecture effort. An approved enterprise policy statement provides such support and sponsorship, promoting institutional "buy-in" and encouraging resource commitment from participating components. Equally important in demonstrating commitment is vesting ownership of the architecture with an executive body that collectively owns the enterprise. Attribute 2: Provides Capability to Meet Commitment: The success of the EA effort depends largely on the organization's capacity to develop, maintain, and implement the EA. Consistent with any large IT project, these capabilities include providing adequate resources (i.e., people, processes, and technology); defining clear roles and responsibilities; and defining and implementing organizational structures and process management controls that promote accountability and effective project execution. Attribute 3: Demonstrates Satisfaction of Commitment: Demonstrating satisfaction of the organization's commitment to develop, maintain, and implement an EA is evidenced by the production of artifacts (e.g., the plans and products). Such artifacts demonstrate "follow through"--actual EA production. Satisfaction of commitment is further demonstrated by senior leadership approval of EA documents and artifacts; this approval communicates institutional endorsement and ownership of the architecture and the change that it is intended to drive. Attribute 4: Verifies Satisfaction of Commitment: This attribute focuses on measuring and disclosing the extent to which efforts to develop, maintain, and implement the EA have fulfilled stated goals or commitments. Measuring such performance allows for tracking progress that has been made toward stated goals, allows appropriate actions to be taken when performance deviates significantly from goals, and creates incentives to influence both institutional and individual behaviors. Core Elements: At the core of the EAMMF are the EA management elements (i.e., practices and conditions) described in the CIO Council Practical Guide. Each of the core elements is briefly described below, along with references to the Practical Guide, where additional explanation and guidance can be found. Stage 1: Creating EA Awareness: At Stage 1, organizations are becoming aware of the value of an EA, but have not yet established the management foundation needed to develop one. Stage 1 has no core elements; by default, an organization that does not satisfy Stage 2 core elements is at Stage 1. Elements for Stage 2: Building the EA Management Foundation [See PDF for image] [End of figure] Source: GAO. At Stage 2, organizations move from basic awareness to building the foundation for effectively developing, maintaining, and implementing an EA. Attribute: Demonstrates commitment: Element: Adequate resources exist. An organization should have the resources (funding, people, tools, and technology) to establish and effectively manage its architecture. This includes identifying and securing adequate funding to support EA activities; hiring and retaining the right people with the proper knowledge, skills, and abilities to plan and execute the EA program; and selecting and acquiring the right tools and technology to support EA activities. Reference: CIO Council Practical Guide, Section 3.1.1: Ensure Agency Head Buy-in and Support; Section 3.1.3: Obtain Support from Senior Executives and Business Units; Section 3.2: Establish Management Structure and Control; Section 6.1.1: Train Personnel: Element: Committee or group representing the enterprise is responsible for directing, overseeing, or approving EA. An organization should assign responsibility for directing, overseeing, and approving the architecture not to just one individual, but to a committee or group with representation from across the enterprise. Establishing this enterprisewide responsibility and accountability is important in demonstrating the organization's commitment to building the management foundation and obtaining buy-in from across the organization. Accordingly, this group should include executive-level representatives from each line of business, and these representatives should have the authority to commit resources and enforce decisions within their respective organizational units. Typically, this group, established by the organization head, serves as a "steering committee" and is responsible for guiding, directing, and approving EA plans and products, including significant changes to either. Reference: CIO Council Practical Guide, Section 3.2.3: Establish an Executive Steering Committee: Attribute: Provides capability to meet commitment: Element: Program office responsible for EA development and maintenance exists. EA development and maintenance should be managed as a formal program. Accordingly, responsibility for EA management should be assigned to an organizational unit and not simply an individual. Typically in the form of a program office, this organizational unit should be devoted to the EA program and responsible for developing a management plan and executing the plan. The plan should include a detailed work breakdown structure, resource estimates (e.g., funding, staffing, and training), performance measures, and management controls for developing and maintaining the architecture. The program office should have qualified staff serving as the core team. Examples of functions performed by the EA program office are risk management, configuration management, quality assurance, and security management. Reference: CIO Council Practical Guide, Section 3.2.5: Establish an EA Program Office: Element: Chief architect exists. An organization should have a chief architect who is responsible and accountable for the EA, and who is supported by the EA program office and overseen by the enterprisewide architecture steering committee. Appointed by the CIO and approved by the organization head, the chief architect is typically an organization executive whose background and qualifications span both the business and technology sides of the organization and who also functions as the EA program manager. The chief architect is responsible for ensuring the integrity of the EA development process, as well as the content of the EA products. The chief architect should be experienced in, among other things, program management, capital planning and investment control, and systems engineering. The chief architect (in collaboration with the CIO, steering committee, and the organization head) is instrumental in obtaining organizational buy-in for EA (including support from the business units), as well as in securing resources to support architecture management functions, such as risk management, configuration management, quality assurance, and security management. Reference: CIO Council Practical Guide, Section 3.2.4: Appoint Chief Architect: Element: EA is being developed using a framework, methodology, and automated tool. To develop the architecture in a consistent and efficient manner, an organization should use an EA framework, methodology, and automated tool. Frameworks provide a defined structure and nomenclature for representing EA information that may come from different parts of the organization. Methodologies, if implemented effectively, define the steps necessary to perform the activities associated with capturing the EA in a coherent, consistent, accountable, and repeatable manner. Automated tools provide an efficient repository for capturing, updating, and disseminating the EA across the organization. Reference: CIO Council Practical Guide, Section 4: Define an Architecture Process and Approach: Framework. A framework provides a formal structure for representing the EA, serving as the basis for the nature and content of the specific products the organization plans to develop, use, and maintain. As such, a framework helps to ensure the consistent representation of information from across the organization. For federal agencies, selecting one of the federal frameworks provides greater interoperability among EAs of various federal organizations. Reference: CIO Council Practical Guide, Section 4.5: Evaluate and Select a Framework: Methodology. A methodology provides a common set of procedures for developing EA products and, if implemented properly, helps to ensure consistency in the procedures used across the organization for developing and maintaining the EA. An organization's methodology or methodologies should govern how the EA products will be developed, maintained, and validated. Methodologies need to be documented, understood, and consistently applied by the EA program team. They should prescribe the standards, steps, tools, techniques, and measures to be used to provide reasonable assurance that expected product quality is attained. Automated tool. An automated tool serves as the repository of architecture artifacts. The choice of tool is based on the organization's needs and the size and complexity of the architecture. EA tools are typically selected based on explicit criteria, including but not limited to those listed in table 1. Reference: CIO Council Practical Guide, Section 4.6: Select an EA Toolset: Table 1. Criteria for Selecting Automated EA Development and Maintenance Tools: Available platforms. Configuration management support. Cost and licensing. Framework support. Integrated and consolidated repository. Interoperability with other tools/repositories. Model size and complexity. Modeling methods and techniques support. Risk management and issue tracking support. Quality assurance support. Traceability to requirements and other enterprise engineering artifacts. Training schedule, cost, and length. Vendor support. Source: CIO Council. [End of table] Attribute: Demonstrates satisfaction of commitment: Element: EA plans call for describing both the "as-is" and the "to-be" environments of the enterprise as well as a sequencing plan for transitioning from the "as-is" to the "to-be.": An organization should have a documented EA program management plan and supporting plans (e.g., configuration management plan and quality assurance plan). Generally, these plans should describe the steps to be taken and tasks to be performed in managing the EA program. They should also provide for development of architectural descriptions of how the organization currently operates (the "as-is" environment), how it intends to operate in the future (the "to-be" environment), and how it will transition from the current "as-is" operating environment to the "to-be" environment. In short, the "as-is" and " to-be" descriptions should be enterprisewide in scope, and they can be developed concurrently. Further, it is expected that the "to-be" descriptions will consume the majority of the EA program's resources. The sequencing plan will generally follow after development of the "as-is" and "to-be" descriptions, and it should include, for example, what system capabilities are to be introduced into the organization, when they are to be introduced (based on their relative value and dependencies), and when legacy systems are to be phased out. The sequencing plan should eventually form the basis for the organization's annual IT capital investment plan, which is a key component of IT investment management. Reference: CIO Council Practical Guide, Section 3.3.2: Develop an EA Program Management Plan: Element: EA plans call for describing both the "as-is" and the "to-be" environments in terms of business, performance, information/data, application/service, and technology. The organization's documented EA management plans should also provide for defining and normalizing[Footnote 21] the current and future architectures in terms relevant to stakeholders from varying organization levels and disciplines. These terms are the organization's business operations, performance measures, information and data needs and definitions, application and service delivery means, and technology profiles and standards. Moreover, these terms or enterprise perspectives should be consistent and aligned with each other. (See Section 1 for more information on these terms of reference.): Reference: CIO Council Practical Guide, Section 3.3.2: Develop an EA Program Management Plan: Element: EA plans call for business, performance, information/data, application/service, and technology descriptions to address security. An organization's EA program management plans should define how it will address security as a distinct area of operational and technology emphasis within the context of each of the terms of reference: business, performance, information/data, application/service, and technology. Reference: CIO Council Practical Guide, Section 3.3.2: Develop an EA Program Management Plan: Attribute: Verifies satisfaction of commitment: Element: EA plans call for developing metrics for measuring EA progress, quality, compliance, and return on investment. An organization's EA management plans should provide for developing metrics and should describe how these will be used to measure (1) progress toward EA goals, (2) the quality of architecture products and management processes, (3) compliance with the architecture, and (4) EA return on investment. Elements Added for Stage 3: Developing EA Products [See PDF for image] Source: GAO. [End of figure] At Stage 3, organizations move from building the EA management foundation to developing EA products. Stage 3 also includes all elements in Stage 2. Attribute: Demonstrates commitment: Element: Written and approved organization policy exists for EA development. An organization should have a documented policy, approved by the organization head, governing the development of the EA. An organization policy is an important means for ensuring enterprisewide commitment to developing an EA and for clearly assigning responsibility for doing so. The architecture policy should define the scope of the architecture as including a description of the baseline ("as-is") and target ("to-be") architecture, as well as a sequencing plan that supports the move between the two. Additionally, the policy should provide for having processes for EA oversight and control, and EA review, validation, and refinement. Further, the policy should identify the major players in the architecture development process, including the chief architect, program office, steering committee, project/system development managers, the organization head, and CIO; it should also identify their roles, responsibilities, and relationships. The policy should address the purpose and value of an EA; its relationship to the organization's strategic vision and plans; and its relationship to capital planning, enterprise engineering, and program management. Reference: CIO Council Practical Guide, Section 3.1.2: Issue an Executive Enterprise Architecture Policy: Attribute: Provides capability to meet commitment: Element: EA products are under configuration management. An organization should ensure the integrity and consistency of the EA products, throughout their life cycles, by placing them under configuration management. Effective configuration management is important for enabling integration among related EA products and for alignment between architecture artifacts. Ensuring that EA products are under configuration management is the responsibility of the EA program office. Typically, an organization will assign a configuration manager to oversee and control the EA product configurations. Through effective configuration management, changes to EA products are identified, tracked, monitored, documented, reported, and audited. Reference: CIO Council Practical Guide, Section 7: Maintain the Enterprise Architecture: Attribute: Demonstrates satisfaction of commitment: Element: EA products describe or will describe both the "as-is" and the "to-be" environments of the enterprise, as well as a sequencing plan for transitioning from the "as-is" to the "to-be.": Consistent with the EA program plans discussed in Stage 2, an organization should ensure that the EA products being developed are enterprisewide in scope and describe both the current ("as-is") environment and the future or target ("to-be") environment, as well as a sequencing plan for moving from the current to the target environment. Reference: CIO Council Practical Guide, Section 5.2: Generate Products and Populate EA Repository; Section 5.2.1: Essentials in Building the Baseline Architecture; Section 5.2.2: Essentials in Building the Target Architecture; Section 5.3: Develop the Sequencing Plan: Element: Both the "as-is" and the "to-be" environments are described or will be described in terms of business, performance, information/data, application/service, and technology. While many details of the EA product may not yet have been defined, the products being developed/drafted should begin to address each of the given terms of reference, or include placeholders for later defining the enterprise in these terms. These terms of reference are business operations, performance management, information/data needs and definitions, application/service delivery vehicles, and technology profiles and standards. Reference: CIO Council Practical Guide, Section 5.2.1: Essentials in Building the Baseline Architecture; Section 5.2.2: Essentials in Building the Target Architecture: Element: Business, performance, information/data, application/service, and technology descriptions address or will address security. An organization should ensure that each of its EA products (including those describing the "as-is" and "to-be" environments in terms of business, performance, information/data, application/service, and technology) explicitly describe how enterprise security is being defined and will be implemented. Reference: CIO Council Practical Guide, Section 5.2.1: Essentials in Building the Baseline Architecture; Section 5.2.2: Essentials in Building the Target Architecture: Attribute: Verifies satisfaction of commitment: Element: Progress against EA plans is measured and reported. To assist in attaining stated EA program goals and objectives, an organization should understand and disclose its progress against plans. As EA products emerge, their content should be assessed against the plans to ensure that expectations are being met. Based on this assessment, plans can be updated to reflect experience to date, while products can be revised to address plan changes. Deviations from expectations contained in plans should be analyzed to determine cause and impact, and appropriate action should be taken to address deviations. Reference: CIO Council Practical Guide, Section 8.2: Identify Where EA Program Expectations Are Not Being Met; Section 8.3: Take Appropriate Actions to Address Deviations; Section 8.4: Ensure Continuous Improvement: Elements Added for Stage 4: Completing the EA Products [See PDF for image] Source: GAO. [End of figure] At Stage 4, organizations move from developing to completing EA products. Stage 4 also includes all elements in Stages 3 and 2. Attribute: Demonstrates commitment: Element: Written and approved organization policy exists for EA maintenance. Because the architecture is a "living" entity, influenced continuously by internal and external change drivers, it needs to be kept current to be relevant. Accordingly, an organization should have a documented policy, approved by the organization head, governing the maintenance of the EA. Such a policy promotes enterprisewide commitment to keeping the EA up to date by, for example, assigning responsibility and accountability for maintenance. The EA policy should provide for establishing a process for architecture maintenance, including oversight and control. Additionally, it should identify the roles, responsibilities, and relationships of key players in the maintenance process, including the chief architect, steering committee, program office, project/system development managers, organization head, and CIO. Reference: CIO Council Practical Guide, Section 3.1.2: Issue an Executive Enterprise Architecture Policy: Attribute: Provides capability to meet commitment: Element: EA products and management processes undergo independent verification and validation. An organization should ensure the quality of its architecture by performing independent verification and validation of both the EA products and the processes used to develop the products. This independent quality determination should be performed by a third party, such as the organization's internal audit function or a contractor not responsible for any architecture development activities. The results of these determinations should be shared with the program office, and reported directly to the EA steering committee. Reference: CIO Council Practical Guide, Section 3.2.5.1: Appoint Key Personnel; Section 5.2.3: Review, Validate, and Refine Models; Section 8.2: Identify Where EA Program Expectations Are Not Being Met: Attribute: Demonstrates satisfaction of commitment: Element: EA products describe both the "as-is" and the "to-be" environments of the enterprise, as well as a sequencing plan for transitioning from the "as-is" to the "to-be.": An organization should complete its EA products according to plans defined in Stage 2. These products should completely and correctly describe both the "as-is" and the "to-be" environments of the enterprise and include a sequencing plan for migrating the organization between these two environments. EA products exhibiting these characteristics and qualities are a logical output of performing the previously discussed core elements. This is a consequence of the hierarchical structure of the EAMMF. That is, if the EA plans developed in Stage 2 and implemented in Stage 3 do not provide for having the "as-is" and "to-be" architectures and a sequencing plan, this core element is unlikely to be satisfied in Stage 4. Reference: CIO Council Practical Guide, Section 5.2: Generate Products and Populate EA Repository; Section 5.2.1: Essentials in Building the Baseline Architecture; Section 5.2.2: Essentials in Building the Target Architecture; Section 5.3: Develop the Sequencing Plan: Element: Both the "as-is" and the "to-be" environments are described in terms of business, performance, information/data, application/service, and technology. An organization's EA products are defined and normalized in terms meaningful to a wide variety of stakeholders, ranging from the organization's chief executive officer and strategic planners to its technology implementers and operators. Accordingly, the "as-is" and the "to-be" architectures need to capture and disclose in meaningful terms business operations, performance measures, information and data needs and definitions, application and service delivery vehicles, and technology profiles and standards. Moreover, these terms set frames of reference that need to be aligned and consistent with one another. Again, performance of the core elements in the previous stages should result in architecture products that satisfy this core element. Reference: CIO Council Practical Guide, Section 5.2.1: Essentials in Building the Baseline Architecture; Section 5.2.2: Essentials in Building the Target Architecture: Element: Business, performance, information/data, application/service, and technology descriptions address security. An organization should explicitly and consistently address security in its business, performance, information/data, application/service, and technology EA products. Because security permeates every aspect of an organization's operations, the nature and substance of institutionalized security requirements, controls, and standards should be captured in the EA products. Reference: CIO Council Practical Guide, Section 5.2.1: Essentials in Building the Baseline Architecture; Section 5.2.2: Essentials in Building the Target Architecture: Element: Organization CIO has approved current version of EA. The current version of the organization's completed EA should be approved by the CIO. This approval is the first in a series of approvals intended to establish the EA as an institutionally endorsed change management and transformation tool. Reference: CIO Council Practical Guide, Section 5.4: Approve, Publish, and Disseminate EA Products: Element: Committee or group representing the enterprise or the investment review board has approved current version of EA. The current version of the organization's completed architecture should also be approved either by the EA steering committee (or comparable body) or by the investment review board. The approval by one or both of these bodies denotes institutional buy-in and thus facilitates the architecture's acceptance and use at all organizational levels as a change management and transformation tool. Reference: CIO Council Practical Guide, Section 5.4: Approve, Publish, and Disseminate EA Products: Attribute: Verifies satisfaction of commitment: Element: Quality of EA products is measured and reported. An organization should ensure that the nature and content of the EA products meet defined quality standards. The ability to demonstrate that these products are of high quality is critical to gaining CIO and subsequent EA approvals. This core element entails developing a set of metrics and assessing the products against those metrics. Such measurement and disclosure of the results to decision-makers mean that timely and appropriate actions can be taken to address deviations from established goals. This measurement and reporting activity is the responsibility of the EA program, supplemented by an independent verification and validation agent. Reference: CIO Council Practical Guide, Section 3.2.5.1: Appoint Key Personnel; Section 5.2.3: Review, Validate, and Refine Models; Section 8.2: Identify Where EA Program Expectations Are Not Being Met; Section 8.3: Take Appropriate Actions to Address Deviations; Section 8.4: Ensure Continuous Improvement: Elements Added for Stage 5: Leveraging the EA to Manage Change [See PDF for image] Source: GAO. [End of figure] At Stage 5, organizations use the EA products in a manner to most effectively achieve results, such as business and systems modernization and organizational transformation. Stage 5 includes all elements in Stages 4, 3, and 2. Attribute: Demonstrates commitment: Element: Written and approved organization policy exists for IT investment compliance with EA. An organization should have a policy governing the implementation of the architecture that is approved by the organization head. Such a policy is important because it is the basis for enforcing the architecture. The EA policy should augment architecture development and maintenance policies by providing for an institutional EA implementation process that is aligned with the organization's capital planning and investment control process. At a minimum, the policy should specify that all IT investments must comply with the architecture unless justified and granted a documented waiver. The policy should also define the roles and responsibilities of the major players in architecture implementation and their relationships. Major players include the investment review board, architecture assessment team, CIO, and chief architect. Reference: CIO Council Practical Guide, Section 3.1.2: Issue an Executive Enterprise Architecture Policy; Section 6.1.2: Establish Enforcement Processes and Procedures: Attribute: Provides capability to meet commitment: Element: A process exists to formally manage EA change. The EA is not a static set of products, but rather a living tool that should change to reflect, for example, new technology opportunities and shifts in organizational constraints and business drivers. Accordingly, a formal process should be defined and implemented for introducing changes to the architecture. This process should recognize both internally and externally prompted change, and it should provide for continuous capture and analysis of change proposals and informed decision-making about whether to make changes. Reference: CIO Council Practical Guide, Section 7.1.1: Reassess the Enterprise Architecture Periodically; Section 7.2: Continue to Consider Proposals for EA Modification: Element: EA is integral component of IT investment management process. An organization should recognize that the EA is a critical frame of reference for making IT investment decisions. Using the EA when making investment decisions is important because the organization should approve only those investments that move the organization toward the target architecture, as defined in the sequencing plan. Our ITIM framework also addresses architecture within the context of ITIM's five stages of investment management maturity.[Footnote 22] For example, at ITIM stage 2, an organization's policies and procedures should provide for identifying the business needs (and the associated users) of each IT project and for ensuring that each IT project fits within the architecture (or be waived from this requirement). The business needs are typically contained in the EA business descriptions. At ITIM stage 3, an organization's policies and procedures should provide for: * specifying the relationship of its architecture to its IT decision- making authority; * specifying the link between the EA and IT portfolio selection criteria, which should take into account the EA so as to (1) avoid unwarranted overlap across investments and (2) maximize systems interoperability; and: * reconciling differences between the organization's EA and its IT investment portfolio. At ITIM stage 4, the organization should periodically analyze its IT investment portfolio to ensure that its investments of IT resources are aligned with the current version of the architecture. Reference: CIO Council Practical Guide, Section 6.1: Integrate the EA with Capital Planning and Investment Control and System Lifecycle Processes: Attribute: Demonstrates satisfaction of commitment: Element: EA products are periodically updated. Depending on the volume and degree of approved changes to the EA, an organization will need to periodically update its EA products. These updates generally reflect an accumulation of individually minor changes that (taken as a whole) represent a material change in the products. Reference: CIO Council Practical Guide, Section 7.1.1: Reassess the Enterprise Architecture Periodically: Element: IT investments comply with EA. An organization's IT investments should be aligned and comply with the applicable components (e.g., business, information/data, and technical) of the current version of the EA, and should not be selected and approved under the organization's capital planning and investment control process unless compliance is documented by the investment sponsor and substantiated by the architect assessment team. Moreover, this compliance is not a one-time event, but rather an integral part of the investment control process and the system life cycle management process. Exceptions to investments being architecturally compliant should be made only on the basis of compelling analytical justifications and should be documented in a waiver to the architecture. These waivers then form the basis for articulating change requests under the formal process for introducing change in the EA. Reference: CIO Council Practical Guide, Section 6.1: Integrate the EA with Capital Planning and Investment Control and System Lifecycle Processes: Element: Organization head has approved current version of the EA. The current version of the EA should ultimately be approved by the head of the organization. Such approval recognizes and endorses the architecture for what it is intended to be--a corporate tool for managing both business and technological change and transformation. Reference: CIO Council Practical Guide, Section 5.4: Approve, Publish, and Disseminate EA Products: Attribute: Verifies satisfaction of commitment: Element: Return on EA investment is measured and reported. The EA is a strategic asset and, as such, should be viewed as an investment in the future. Like any investment, the EA should produce a return (i.e., a set of benefits), and this return on investment should be measured and reported in relation to costs. Measuring return on investment is important to ensure that expected benefits from the EA are realized and to share this information with executive decision- makers, who can then take corrective action to address deviations from expectations. To accomplish this, metrics need to be developed (such as costs avoided through elimination of duplicative or redundant investments) and processes need to be established to collect and report these data. Reference: CIO Council Practical Guide, Section 8.2: Identify Where EA Program Expectations Are Not Being Met; Section 8.3: Take Appropriate Actions to Address Deviations; Section 8.4: Ensure Continuous Improvement: Element: Compliance with EA is measured and reported. Unless the EA is enforced, its value will not be fully realized. Thus, it is not only important to have a process in place to ensure compliance (as described in an earlier core element), it is also important to measure and report on the extent of compliance. To do so effectively, organizations should define metrics, such as number of compliance waivers requested and number granted, to track compliance. Through such measurement and reporting, relevant trends and anomalies can be identified, and corrective action can be taken. Reference: CIO Council Practical Guide, Section 6.1: Integrate the EA with Capital Planning and Investment Control and System Lifecycle Processes: Overall View of EAMMF Matrix: Figure 6 depicts all the core elements and relates them to the applicable stages of maturity and critical success attributes. Figure 6: Summary of EAMMF Version 1.1: Maturity Stages, Critical Success Attributes, and Core Elements: [See PDF for image] Source: GAO. [End of figure] Section 3. Uses of EAMMF Version 1.1: Potential users of the EAMMF include both internal and external stakeholders of a given enterprise. For federal agencies, primary internal stakeholders are agency senior executives, including the agency head, as well as the CIO and chief architect and their staffs. Primary external stakeholders are those with agency oversight responsibilities, such as parent departments, OMB, and congressional committees, as well as independent audit and evaluation organizations. As a model defining ascending levels of EA management maturity, the EAMMF can be used by these stakeholders in two principal ways. First, the framework can be used to provide a set of benchmarks against which to determine where the enterprise stands in its progress toward the ultimate goal: having architecture management capabilities that effectively facilitate institutional change (maturity Stage 5). Second, the framework can be used as the high-level basis for developing specific architecture management improvement plans, as well as for measuring, reporting, and overseeing progress in implementing these plans. Tool for Assessing EA Management Maturity: By describing the elements of an effective EA management program, the EAMMF provides a benchmarking tool for judging an enterprise's efforts to manage architecture development and use. Moreover, because the core elements of this framework are grounded in the CIO Council's Practical Guide, a tool that has been widely accepted across the federal government, some agencies have adopted the EAMMF as a de facto standard for measuring EA management maturity. Using the contents of the EAMMF as criteria, internal and external stakeholders can assess and consistently represent a given enterprise's EA management strengths and weaknesses at a single point in time or over a period of time. Moreover, groups of enterprises can be assessed, represented, and compared. As a result, the framework enables users to identify and understand these strengths and weaknesses in a range of contexts: not only specific to a particular enterprise, but also across a group of related enterprises, such as a given department's component agencies, all independent federal agencies, or sets of federal agencies, such as those that are of a particular size or that share a common mission (e.g., homeland security). When using the EAMMF as an assessment benchmarking tool, it is important to remember that achieving a given stage of management maturity requires the enterprise to satisfy all core elements at that stage, as well as those for each lower stage. The value of the EAMMF, however, goes beyond merely grading a given entity as being at a particular stage. It also extends to identifying the full range of specific strengths and weaknesses of the enterprise's EA management practices (i.e., which core elements are satisfied and which are not). This knowledge allows a given enterprise to build on its collective strengths in addressing its recognized weaknesses. Additionally, the EAMMF allows its users to assess and understand any enterprise, regardless of whether the enterprise is an entire organization (e.g., a federal department) or a component organization (e.g., a branch, bureau, or agency). That is, the EAMMF, like the CIO Council Practical Guide, is enterprise independent. The key consideration, however, is that the unit or scope of assessment needs to be clearly understood and defined before an EAMMF-based assessment is conducted. The amount and depth of the assessment against the EAMMF can vary, depending on the purpose of the assessment and the needs of its users. Accordingly, the EAMMF does not include a methodology or approach for applying the framework; for example, it leaves up to the users the extent to which they verify and validate that each core element is satisfied. EA Management Improvement Planning: The progressive stages of the EAMMF provide a roadmap for incremental improvement of architecture management. In using this roadmap for planning, it is important to recognize that certain core elements are inherently dependent on others, requiring an ordered approach, whereas others do not exhibit such dependencies, so that the timing of their implementation is more flexible. Generally, lower EAMMF maturity stages provide the foundation for higher maturity stages. Some lower stage core elements serve as prerequisites for higher stage core elements. For example, EA plans established in Stage 2 serve as a prerequisite for measuring progress against those plans in Stage 3. However, certain higher stage core elements can be addressed, even though lower stage core elements have not been completely addressed. For example, an organization may have satisfied the Stage 5 core element of having a written and approved policy for EA maintenance without satisfying lower level core elements. Our use of the EAMMF has shown that it is not unusual for federal departments and agencies to have satisfied some core elements at multiple stages, even though not all have been addressed. Additionally, in using the EAMMF for improvement planning, it is important to remember that the framework, like the CIO Council Practical Guide, describes what needs to be done, not how it needs to be done. Thus, when the EAMMF is used for management improvement, the framework remains just that: a framework within which to plan specific EA management steps, activities, processes, authorities, etc., and to subsequently measure, report, and oversee progress on each. To develop an EA management improvement plan that can be actually implemented, an enterprise needs to augment the framework with more detailed criteria, addressing, for example, the appropriate scope of work of an independent verification and validation agent or the attributes of an effective process for assessing a given investment's architectural compliance. Further, in using the EAMMF for improvement planning, it is also important to remember that effective EA management is generally not achieved until an enterprise has a completed and approved architecture that is being effectively maintained and is being used to leverage organization change and support investment decision-making; having an architecture with these characteristics is equivalent to having satisfied many Stage 4 and 5 core elements. At this point in the organization's EA management maturation, management controls and structures are in place for using an approved architecture to guide and constrain its investments in IT. Even if an enterprise is at Stage 4, it is not fully exploiting an architecture unless it is also achieving certain Stage 5 core elements, such as having processes that use the EA in managing the IT investment portfolio and that ensure that IT investments comply with the EA. If these core elements are not in place, the EA will not be a tool for managing IT for institutional results. Appendix. Approach to Developing EAMMF Version 1.1: Our primary goal in developing EAMMF Version 1.1 was to improve the content and usability of Version 1.0. To do this, we solicited comments and suggestions on Version 1.0 from the 116 federal departments and agencies that participated in our 2001 survey of the state of the government's use of enterprise architectures,[Footnote 23] as well as various other internal and external EA stakeholders, such as members of a GAO-sponsored IT management advisory group composed of IT executives from private industry, academia, and state governments. In our 2001 survey of federal departments and agencies, we solicited responses to a questionnaire addressing various EA management topics, and we compared these responses to EAMMF Version 1.0. This comparison showed that 84 percent of the departments and agencies were at maturity stage 1 or 2. Therefore, as a secondary goal in developing Version 1.1, we wanted to avoid invalidating the baseline data obtained in the 2001 survey on the state of EA management in the federal government. Accordingly, in soliciting comments and suggestions from the 116 departments and agencies and various other EA stakeholders, we were mindful to balance the need to introduce missing core elements with the need not to significantly raise the bar for being at Stage 2. To this end, we asked that comments and suggestions for adding core elements be focused on Stages 4 and 5, but we did not restrict any comments and suggestions for the framework. Other areas that we sought respondents' input on were * experience with using the framework; * strengths and/or weaknesses of the framework; and: * ways to improve the framework: * to make it more useful as a tool to define and measure an organization's EA management maturity, * to ensure that the staged structure (and the corresponding core elements) of the framework is not unreasonably demanding, and: * to explain the core elements sufficiently so that they are useful in assessing an agency's enterprise architecture maturity. Of the 116 departments and agencies we contacted, 63 responded. Collectively, they provided about 300 comments and suggestions that we have incorporated as appropriate in Version 1.1. We categorized these comments and suggestions into the eight groups shown in table 2. Table 2. Major Categories of Comments and Suggestions: Link core elements to other relevant guidance (e.g., CIO Council Practical Guide, EA Frameworks). Include EA development, maintenance, and implementation. Include EA return on investment. Add core elements for measuring EA progress. Include security. Include maturity half-stages based on number of core elements satisfied (e.g., Stage 1.5 for satisfying more than half but less than all of the core elements in Stage 2). Better define EAMMF. Comments requiring no change. Source: GAO. [End of table] (310240): FOOTNOTES [1] The first version was introduced in U.S. General Accounting Office, Information Technology: Enterprise Architecture Use Across the Federal Government Can Be Improved, GAO-02-6 (Washington, D.C.: Feb. 19, 2002). [2] A framework can be viewed as a logical structure for classifying and organizing complex information. [3] J. A. Zachman, "A Framework for Information Systems Architecture," IBM Systems Journal 26, no. 3 (1987). [4] See, for example, U.S. General Accounting Office, DOD Business Systems Modernization: Improvements to Enterprise Architecture Development and Implementation Efforts Needed, GAO-03-458 (Washington, D.C.: February 2003); Information Technology: DLA Should Strengthen Business Systems Modernization Architecture and Investment Activities, GAO-01-631 (Washington, D.C.: June 2001); and Information Technology: INS Needs to Better Manage the Development of Its Enterprise Architecture, AIMD-00-212 (Washington, D.C.: August 2000). [5] National Institute of Standards and Technology, Information Management Directions: The Integration Challenge, Special Publication 500-167 (September 1989). [6] U.S. General Accounting Office, Strategic Information Planning: Framework for Designing and Developing System Architectures, GAO/IMTEC- 92-51 (Washington, D.C.: June 1992). [7] U.S. General Accounting Office, Executive Guide: Improving Mission Performance through Strategic Information Management and Technology, GAO/AIMD-94-115 (Washington, D.C.: May 1994). [8] DOD C4ISR Architecture Framework, Version 2.0 (Dec. 18, 1997). [9] Treasury Enterprise Architecture Framework, Version 1.0 (July 3, 2000). [10] Federal Enterprise Architecture Framework, Version 1.1 (September 1999). [11] Clinger-Cohen Act of 1996, Public Law 104-106, section 5125, 110 Stat. 684 (1996). [12] OMB, Information Technology Architectures, Memorandum M-97-16 (June 18, 1997), rescinded with the update of OMB Circular A-130 (Nov. 30, 2000). [13] Office of Management and Budget, Management of Federal Information Resources, Circular No. A-130 (Nov. 30, 2000). [14] Chief Information Officers Council, Architecture Alignment and Assessment Guide (October 2000). [15] Chief Information Officers Council, A Practical Guide to Federal Enterprise Architecture, Version 1.0 (February 2001). [16] E-Government Act of 2002, Public Law 107-347 (Dec. 17, 2002). [17] The E-Government Act of 2002 states that the Administrator of the Office of Electronic Government shall work with the Administrator of the Office of Information and Regulatory Affairs and with other offices within the OMB to oversee, among other things, the development of enterprise architectures. [18] The first version was introduced in U.S. General Accounting Office, Information Technology: Enterprise Architecture Use across the Federal Government Can Be Improved, GAO-02-6 (Washington, D.C.: Feb. 19, 2002). [19] The EAMMF matrix differs from a classical matrix in that each maturity stage includes not only the core elements in the column below that stage, but also the core elements of previous, less mature stages. That is, the core elements are cumulative: the attainment of a particular stage of maturity does not involve dropping any core elements, but rather adding more core elements to the repertoire. [20] U.S. General Accounting Office, Information Technology Investment Management: A Framework for Assessing and Improving Process Maturity, Exposure Draft, GAO/AIMD-10.1.23 (May 2000). [21] Normalization is a process for minimizing the number of redundancies among design or architecture groupings or entities. Designs or architectures that have normalized groupings or entities are better able to accommodate and minimize the impact of future change. [22] U.S. General Accounting Office, Information Technology Investment Management: A Framework for Assessing and Improving Process Maturity, Exposure Draft, GAO/AIMD-10.1.23 (May 2000). [23] U.S. General Accounting Office, Information Technology: Enterprise Architecture Use Across the Federal Government Can Be Improved, GAO-02- 6 (Washington, D.C.: February 2002). GAO's Mission: The General Accounting Office, the investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through the Internet. GAO's Web site ( www.gao.gov ) contains abstracts and full-text files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents using key words and phrases. You can print these documents in their entirety, including charts and other graphics. Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as "Today's Reports," on its Web site daily. The list contains links to the full-text document files. To have GAO e-mail this list to you every afternoon, go to www.gao.gov and select "Subscribe to daily E-mail alert for newly released products" under the GAO Reports heading. Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. General Accounting Office 441 G Street NW, Room LM Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000: TDD: (202) 512-2537: Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov Automated answering system: (800) 424-5454 or (202) 512-7470: Public Affairs: Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S. General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C. 20548: