In October 1996, the CIO Council held its first off-site strategic planning retreat. The session was organized around ten key issues facing the newly established CIO Council. The Council had been created in August, 1996, by Executive Order 13011, "Federal Information Technology," as the principal interagency forum to improve practices on such matters as the design, modernization, use, sharing, and performance of agency information resources.
The ten key issues had been selected from a long list of topics and areas of interest identified by the Council members. They included such topics as reengineering, capital investment, information technology (IT) performance measures, education and training, information assurance, and so on. Working groups on each issue area were tasked with producing an action-oriented report, with specific recommendations on what should be done over the next 12-18 months. At the end of the day long session, the work groups made their presentations. Again and again, the need was expressed for "best practices" culled from case studies of IT success stories. The Outreach Committee of the Council took on the responsibility for this undertaking, aided by the Industry Advisory Council (IAC). The IAC is a nonprofit industry association of IT companies. It had assisted the Council since its earliest days by preparing a report that helped members understand and appreciate the role of the CIO in commercial entities (see Final Report, IAC/CIO Task Force, July 15, 1997).
In December, 1996, IAC created a Task Force made up of 39 volunteer IT professionals drawn from 26 member companies. The Task Force, working under the guidance of the Outreach Committee, was assigned the task of compiling and documenting representative case studies that demonstrated the successful application of IT in Federal departments and agencies.
Case studies have long been used in social science research, including the traditional disciplines (sociology, political science, anthropology, and history) and practice-oriented fields such as urban planning, public administration, and public policy. Initially, we solicited nominations of "success stories" from all Council members. While this call yielded nearly 50 write-ups from various parts of the Government, it highlighted two significant shortcomings of a self-nomination process. First, there was no consistent standard or definition of "success." Second, the "successes" were virtually all initiatives that had been nominated for some agency, departmental, or Government-wide award or another. While strong on hyperbole, they lacked the narrative, context, objectivity, and so on of a case analysis. Furthermore, what about the many other initiatives that, to date, no one had documented?
The IAC volunteers were assigned to one of seven teams, with each team having responsibility for specific agencies or departments. A list of industry volunteers and team listings are included as an appendix. Task Force members were instructed in the case method, internal protocols were established to ensure comparability in the studies, and interview teams were required to be comprised of representatives from multiple companies with no two members from a single firm.
The actual study began with an e-mail message to all CIO Council members about what we were doing and why. Council members were then contacted by the IAC teams and entrance interviews were scheduled, at which the CIOs outlined the priorities and strategies for their respective departments. In the course of these discussions, the CIOs were asked to nominate projects for case study consideration. Thereafter, interview teams worked directly with Federal points of contact on the special projects to develop and refine relevant data on the nominated cases. Overall, more than 100 projects were nominated for consideration, in addition to the nearly 50 initial nominations.
The Task Force, with the guidance of the Outreach Committee, then culled the nominated cases to reach a manageable- but representative- sample of case studies that demonstrated where the application of IT had made a positive difference in Federal departments or agencies. These examples had to be of significant size, materially impact the Governmental unitís mission, and be considered a "success." Success was defined by the Task Force and the Council as:
IT used to:
On May 30, 1997, a "red team" made up of senior IT managers from the Departments of Commerce, State, and Treasury; the National Science Foundation; the Environmental Protection Agency; and the General Accounting Office met in a day-long session to review 32 projects presented by the Best Practices Task Force. From these presentations, the Outreach Committee selected 20 for inclusion in this volume.
We are not so presumptuous as to claim that these cases are the "best" IT projects in the Federal Government. However, they are excellent examples of where and how IT has been applied successfully and in a cost-effective fashion to help achieve a departmentís or agencyís goals and objectives. In addition, these casesóin fact, all those nominatedóprovide valuable insights into how to successfully implement Governmental information systems. The cases came from 19 departments and agencies; only Veterans Affairs had two cases selected. They cover both administrative (payroll/personnel, logistics, communications) and programmatic (weather observations, asbestos standards, edibility, etc.) areas. They come in all sizes, from less than $1 million to almost $150 million. The affected user population or serviced public ranges from the entire population of the nation to fewer than 5,000 individuals. Some took only a few months to complete; one more than a decade. Fully half took two to six years. The common theme, however, is that IT is being applied successfully and contributes immeasurably to mission accomplishment.
When one of us was working on this Preface, an article on the Clinger-Cohen Act came across the desk. In all too typical and common a manner it proclaimed:
Regrettably, the Federal Government does not have a sterling record in delivering quality IT solutions within acceptable cost and schedule. Familiar examples, such as the Federal Aviation Administrationís Advanced Automation System, the National Weather Serviceís modernization program, and the Internal Revenue Serviceís Tax Systems Modernization, are bleak reminders of IT programs that suffer multi-million dollar overruns, schedule slips measured in years, and dismal mission-related results.
We haven't cited the source; such articles are all too common. What does this accretion of obituaries of "runaway systems" (as they were dubbed by a Peat Marwick group) produce? What are the consequences of creating a false stereotype of a federal landscape littered with the remains of failed IT initiatives? What impression(s) do we create?
This isn't a public relations document. It is, however, based on the premise that we need to examine successes as well as failures. Herbert Kaufman (a well-known organizational expert) once argued "...one must understand sickness and death to understand health; pathology contains many of the clues to normality." Of course, he did so in the context of urging study of organizational terminations and eliminations as part of the public administration discipline. Just the opposite is needed in the IT literature. We excel at assembling autopsies. We have too few studies of healthy projects and successful systems implementations. (For a notable exception, see "Executive Guide: Improving Mission Performance Through Strategic Information Management and Technology," GAO/AIMD-94-115, May, 1994.)
Moreover, itís responsible management- not public relations- to report on and qualify (as best we can) return on investment (ROI) and accrued IT benefits. RIOóif reported at allóis too often calculated as systems outputs or activities (for example, so many more gigabits of data storage, or so many more networks installed), rather than in improved mission performance or program results (weather forecasts are 25 percent more accurate and warnings of severe weather conditions are issued 50 percent sooner, allowing more time for preparation or evacuation). We ought to be documenting successful applications as an ongoing part of our responsibilitiesónot as a special publication of the CIO Council and the IAC.
The Task Force developed a number of salient observations in the course of this study; they are presented in the body of this report. Let us note just a couple of key aspects of effective IT management:
Alan P. Balutis Philip J. Kiviat
Chair, Outreach Committee CIO Council Liaison
Chief Information Officersí Council Industry Advisory Council
October, 1997
There is a perception in some Government circles that all information technology (IT) programs are a waste of money. Persistence of such views constitutes a significant challenge for Government Chief Information Officers (CIOs) as they begin to discharge their Information Technology Management Reform Act (ITMRA) responsibilities. For CIOs, such opinions materially inhibit their ability to meet their respective department and agency needs to leverage the power of ITóthe prime means by which Government entities really can "do more with less." Conventional wisdom views IT as a primary means by which tremendous efficiencies can be attained and productivity enhanced. And, more and more, Government needs the benefits that such technology brings.
A volunteer group of IT professionalsódrawn from industry and possessing experience in commercial and Government IT projectsówas asked to evaluate and document representative examples of current, successful IT projects from across the Federal Government. In completing this task, the IAC team is able to offer powerful suggestions about "what works" and suggest pathways for others in Government to emulate.
Early in the study, it was apparent that conditions had changed since the passage of the ITMRA in 1996, which required the establishment of the CIO position within Federal departments and selected agencies and, by Executive Order of the President, the establishment of the CIO Council. There was now a noticeable improvement in the sharing of information, techniques, and approaches (both technical and business oriented) throughout the Federal Government, brought about by the CIO Council itself. The Council provides an essential, high-level opportunity for professionals to "network" among peers and then extend those contacts to their subordinatesóin essence, the power to share information to leverage othersí experience in defining and solving common business problems.
In assessing the numerous case study nominations (more than 100), the IAC team adopted a firm rule of not presenting any projects that were not far enough into their implementation phase to clearly demonstrate and validate their claimed return on investment (ROI). This was adhered to even though several of these projects offered great potential for selection as a success story in some future study.
Viewed as a whole, the nominated Government IT projects share some interesting traits and accomplish a wide variety of business purposes via a diversity of technologies. The task force found that successful IT projects were best understood if grouped into the following categories (alphabetically presented):
Too complex to be described here, a detailed description of these categoriesóalong with their impactócan be found in the analytical chapter that follows this summary.
During this effort, team members had the opportunity to make a number of other observations about successful implementation of IT programs in the Federal Government. These observations are described below.
Web TechnologyóMany of the nominated case studies revealed a strong predilection toward using Web technology. This was not just a case of emulating a "hot" new tool, but rather a very solid match between the functions that Government programs were trying to achieve, the strengths of a specific technology, and the relative ease of implementing solutions using that technology.
CustomersóThere was a heightened sensitivity in attentiveness toward customers in the nominated programs. This sensitivity appeared to be an important factor in assuring program success. In virtually every instance, a serious effort had been made to identify who the customers were, along with some degree of monitoring customer satisfaction after the program was implemented.
Other Human ConsiderationsóThe really successful projects appear to be guided by the views that IT needs to be an enabler of human work and that users have to "embrace" any system in order for it to be successful.
The Endemic Challenges of Technology SelectionóOne of the more frequent reasons that IT projects fail is due to the selection of the technology that did not "mature" as envisioned. Because the IT world is subject to rapid technological change, program leads need to define and then continually update the evolutionary path of their core technologies. Failure to do so assures that a project or program will be left unsupported by the progressive improvements in technology, and that it will be more expensive and painful to "catch up."
The teams found general acceptance of the concept that Government should leverage trends and standards that come from the marketplace in the form of commercial off-the-shelf (COTS) products. Even those rare individuals who philosophically disagreed with the COTS proposition grudgingly acknowledged the tremendous price advantages that COTS brings. But, as with almost any great concept, there are some downside challenges, as noted below.
Part of the interview process included posing what were referred to as the "CIOís Questions" about each nominated project. These questions are also referred to as "Raineís Rules," having been urged on the CIOs by the Director of Management and Budget. The specific questions were:
All three are pointed at trying to ensure that the Government really gets a positive return on its IT investment. The team had two significant, overarching observations as a result of posing these questions: (1) Since many of the nominated projects were initiated before the Clinger-Cohen Act (the ITMRA) or before the emphasis on "reinventing Government" had become a core element of administration policy, the precise questions may not have been posed, but the desired effect appears to have been achieved in virtually every case; (2) the third question is extremely important, but strict adherence to the way it is stated can lead to unnecessary "consumption" of resources. The team observed that in many instances doing the detailed reengineering in conjunction with implementation makes more sense.
The team was asked to evaluate the ROI approaches being used by the respective departments and agencies. However, rather than being evaluators, we saw our role as raising the issue and reporting our observations. The CIO Council has asked the IAC to establish another IAC task force to examine the issue of ROI and render a report later in 1997.
The course of our study and these observations led the team to conclude the following:
The members of this IAC Task Force were afforded a very special opportunity to examine a large number of diverse, successful Government IT programs. In the course of our study, we tried to document the core similarities of the successes. As IT professionals, we all have a tendency to focus on various aspects of the technology, but we need to be mindful that in all successful projects there are only two abiding themes:
"Within the senior leadership of my department there is a widespread perception that all IT programs are a black hole into which we pour money; they all seem to be over budget, behind schedule, and never seem to deliver more than a small fraction of what was promised."
- Government Official
Readers may have encountered this opinion in their own work or experience or perhaps in media presentations about Government information technology (IT) programs and projects. The perception isnít new. The above quoteófrom a 1993 not-for-attribution interview with a senior Government officialówas made to the Task Force Chair for this Industry Advisory Council (IAC) effort, then working on a similar project.
Such opinions probably go back to the first unsuccessful IT implementation in Government. However, the persistence of such views constitutes a significant challenge for Government Chief Information Officers (CIOs) as they begin to discharge their Information Technology Management Reform Act (ITMRA) responsibilities. Any perception that money will be wasted if applied to a particular use serves as a powerful inhibitor to initiating or continuing that kind of project. This tendency seems to be particularly enhanced in times of tight budgets, which seems to be a continuing condition for most of the Federal Government. For CIOs, such opinions materially inhibit their ability to meet their respective department and agency needs to leverage the power of ITóthe prime means by which Government entities really can "do more with less."
Thus, we are confronted with a paradox. Conventional wisdom strongly adheres to the necessity of using information more wisely and views IT as a primary means by which tremendous efficiencies can be attained and productivity enhanced. The commercial sector of the U.S. economy and the economies of other industrial nations abound with remarkable tales of the creation of wealth and enormous growth by clever uses of information and the technologies that facilitate its use. At the same time, there continues to exist a popular belief that Government "just cannot get it right"óyet more and more, Government needs the benefits that such technology brings!
This project was executed by a volunteer group of IT professionals drawn from industry. All the team members had experience in commercial and Government IT projects; all had experience with both successful and not-so-successful IT projects. Our mission was to document representative examples of current, successful IT projects from across the Federal Government. It was not to find the "best" project or projects, if for no other reason that no team could possess the necessary Solomon-like wisdom that the award of such an appellation would require. Instead, we sought to identify the more successful cases and then examine them to find commonalities in their success. There have been many other more scholarly endeavors that have taken the contrary approachólooking at commonalties in unsuccessful endeavors. We chose to accentuate the positive because, as IT professionals, we know there are many more successful endeavors than failuresóeven if prevailing media coverage tends to suggest that project failure is the normal mode. We believe that by highlighting and then focusing on successful trends in Government IT programs, we can make powerful suggestions about "what works" and suggest pathways for others in Government to emulate.
The collective experience of the IAC team represented literally hundreds of individual IT professional years of experience in dealing with the Federal Government. Collectively, we had seen and participated in many different projects involving every part of the Federal Government. Nevertheless, very early in the process of interviewing and gathering data, it was apparent that conditions had changed since the passage of the ITMRA. This 1996 law required establishment of the CIO position within Federal departments and selected agencies; the CIO Council was established by Executive Order of the President.
Very early in our effort the team began to observe an improvement in the sharing of information, techniques, and approaches (both technical and business oriented) throughout the Federal Government. This improvement was not merely the result of some kind of indoctrination or sharing of published materials, but appeared to be in the form of consensus and agreement on approach arrived at after deliberation and debate. Such agreement was most evident when discussing high-level strategy and divergent viewpoints became clear as discussion moved in the direction of specific tacticsówhich may be the result of specific individual department and agency considerations.
As the project continued, our IAC team realized that the CIO Council itself was the source of the improved information sharing. The Council provided an essential, high-level opportunity for professionals to "network" among peers and then extend those contacts to their subordinates. This process of encouraging and facilitating contacts at second, third, and lower levels of leadership appears to have created new contacts that did not exist before. Of course, there were still those who preferred to highlight individual department or agency uniqueness rather than acknowledge commonality of business needs. But for those who choose to avail themselves of its offerings, the CIO Council is a testament to the power in sharing information to leverage othersí experience in defining and solving common business problems.
As expected, the IAC teams were bombarded by recommendations that a certain project be considered, even though still in its early implementation phase or not yet validated as a pilot project. While we listened carefully to such proposals, we adopted a firm rule of not presenting any projects that were not far enough into their implementation phase to clearly demonstrate and validate their claimed return on investment (ROI). Even though several of these projects offered great potential for selection as a success story in some future study, we did not want to appear to endorse any project with insufficiently documented performance data.
As noted in the Preface, Federal departments were asked to nominate the IT projects and practices that they considered their success stories. Viewed as a whole, the nominated Government IT projects share some interesting traits. First, they come in all sizes and shapes. Furthermore, they accomplish a wide variety of business purposes and do so with a wide diversity of technologies. Therefore, to bring some order to this universe of nominations, we grouped them into the following categories (alphabetically presented):
Naturally, several of the nominated projects could apply to more than one category. At the end of this section, the reader will find a matrix displaying how the 20 case studies map to various categories. However, before proceeding to that point, we offer some comments about each of these domains.
This is the classic realm of applying IT in both the business world and Government. In a traditional sense, this was the first office automation. To a considerable extent, office automation has occurred virtually everywhere within the Federal Government. While the teams did find variability among agencies, the fact is that basic office automation has occurred, local area networks (LANs) abound, and efforts to upgrade basic automation tools available to Government professionals are ongoing. There is nothing remarkable about office automation itselfóand no such cases were nominated.
The nominated cases represented new thinking in how information might be better used to accomplish department and agency missions. In most cases, it meant using machines and IT devices to replace human beings (as in the Department of Commerceís Automated Surface Observation System). The basic value being sought tended to be one or a combination of three core concerns:
These kinds of projects tended to have very solid ROIs. As with many efficiency-oriented projects, there can also be some real "human" challengesódisplaced workers, disappointed customers, or both. These considerations can defeat even the best conceived plans unless care is taken to "sell" the project and keep it "sold."
This category was a surprise. Many of us expected that we would find the most successful uses of IT oriented toward the core business issues faced by the departments and agencies. Early on, however, we began realizing that a relatively large number of the nominated projects appeared to be wholly new services for agency customers and constituents. In some cases, they appeared to be in response to expressed needs. However, in many cases, the emergence and maturation of specific technology appears to have brought about a recognition of a previously unmet need among customers/clients/constituents and a fulfillment of that need.
More often than not, the technology behind this "discovery" has been Web-based technology. Similarly, the majority of the nominated projects in this category also tend to be members of the Information Sharing category. The rapid maturation of this technology and the explosive growth of the Internet have given thoughtful Government professionals a whole new "tool kit" for providing access and information to interested publics. In some instances, this has led to the definition of new missions and the subsequent accomplishment of those missions.
With such programs, it becomes difficult to firmly establish and validate an ROI if one limits the definition of ROI to traditional, accounting parameters. However, for many Government programs, such limitation does not really appear valid andóas addressed belowóthis ROI challenge appears particularly applicable to Information Sharing type projects.
A considerable number of nominated projects were in this category, with the team coming to the realization that they very likely represent the next, major wave of automation in the Government sector. The combination of being able to manage to the version level of documents and to avoid the extensive replication of individual files makes such systems extremely attractive to Government organizations that are essentially "document centric." When the power of an easily configurable workflow manager is added, this technology becomes almost essential in that it goes a long way toward ensuring that processes become absolutely replicable. Although several of these types of projects were nominated but did not make the final "cut," we believe they will be well represented in future study efforts.
Implementation of these kinds of projects offers the potential of extremely strong ROI. However, as with the projects in the Automating Manual Processes category, there are some serious downside risks that need to be controlled. As with any efficiency improvement program, unless exceptional care is taken in the execution of the implementation, there can be enormous pressure to redirect the effort and ameliorate any "pain." For these kinds of systems, the "pain" is internal to the organizational entity and involves job changes, position realignments, and organizational restructuring. Unless a continuous process of "internal selling" is maintained, the pain can cause redirection and very rapidly thwart all the expected savings from the ROI.
These can be very powerful programs, and they require extremely well developed and executed implementation plans. The IAC team is confident that in the very near future programs in this category will be the representative examples of successful IT in the Federal Government.
This was an area in which the team expected to see the application of IT and was not disappointed. However, an unexpected aspect of the nominated projects was that many of these projects tended to be very smallóalbeit quite special.
These cases essentially focused on internal agency/department operations. They also tended to be initiated/driven/completed by highly motivated, committed Government professionals who were seeking a "tool" to help them do their jobs. Another similarity was that pressures to economize or reduce expenses in an organization appeared to spark an interest in finding/developing these "tools." These projects were usually focused on monitoring usage of some "utility" within an organization, such as telephones. Alternatively, they might be used to track assignment or reassignment of personnel to/from an agency or locale to ensure the start or demise of some service, or some combination of these. Typically, systems that are the "engine" of such a program are interfaced to a core agency system (personnel, billing, etc.). Finally, these programs had very attractive ROIs, with respectable savings having been attained with reportedly very inexpensive development and implementation costs.
Other such nominated projects tended to use IT to conserve or reduce some resource. Usually, the conserved resource was time. Typically, the time savings came from eliminating steps and was achieved by sharing information. A prime example of such a project can be found in one of the case studiesóthe Department of Justiceís Joint Automated Booking System. As one would expect, ROI for such programs tends to be very high.
One of the more impressive categories among those nominated was the Executive Information Systems (EIS). Certainly within commercial entities there are a wide variety of EIS, but the demands and uses within Government for EIS offer entirely new challenges and opportunities. Within both venues, such systems are used to inform senior officials and to support decision making. However, with Government, there is the added consideration of needing to be responsive to both Congress and to external constituencies (both citizens and organizations).
The EIS the teams reviewed tended to be focused on the needs of either one or the other of these special audiences. In both cases, one of the key issues was the ability to respond quickly either to questions or to changing conditions. Typically, the nominated projects tended to be replacements for much earlier technology which required the use of specialized query tools, provided very limited access, and had response times measured in days. The replacement projects offered much wider access, rapid response, and user friendly interfaces. In other words, the replacement projects were "customer centric."
Almost without exception, one of the key issues in the decision to move forward with these kinds of projects was the agencyís recognition that its inability to access and provide information rapidly constituted a major threat to its credibility and continued support by Congress or interested public. We believe that these are very serious and legitimate considerations for public institutions, thereby making it more challenging for such projects to assess ROI.
Typically, these kinds of programs use the power of IT to compare data (drawn from multiple databases or prior transactions) to test and determine if specific criteria or conditions have been met (or violated). As one would expect, ROIs are very high for such programsóand the privacy issues challenging. For detailed descriptions of such programs, refer to the case studies on the Department of Veteran Affairsí CHAMPVA and the Department of Housing and Urban Developmentís Tenant Eligibility Verification System.
By far and away, this was the category with the most nominated projects. In fact, there were so many nominations that we had to create internal categories:
For Fee. The Government provides a service and charges those accessing the data. Typically, the database that the public is accessing is extracted from public records or filings. The "for fee" service may make a profit, but usually only a fraction of the total systems cost is covered. A prime example in the case studies is the Security and Exchange Commissionís Electronic Data Gathering and Retrieval (EDGAR) system.
Cross Agency. Several different departments or agencies jointly share data from their internal databases to create a collective product. That product may then be shared with external entities (rare) or only be used for Governmental purposes (more common). The information sharing tends to be driven by mission necessity. A good example in the case studies is the Intelligence Communityís InteLink.
Free. These cases were the more common of those nominated. In general, these types of information sharing are with the general public or special publics. The data being shared may be data products developed within or by the agency, compilations of data drawn from diverse sources but presented in a form the public will find useful, or almost any other combination one can think of. Again, these instances are best thought of as "customer centric"ódoing something or making available data that the public wants, needs, or finds useful. The case studies contain several of these, such as the Department of Laborís OSHA Asbestos Advisor and the Environmental Protection Agencyís Envirofacts.
As noted in several other categories, the emergence of Web technology has proven to be a powerful stimulus for these kinds of programs. Indeed, it is not too much of a stretch to note that Web technology is really what has made these kinds of "outreach" programs feasible to the general public. Without Web technology, it is likely that interest in availing oneself to such extensive databases and data sets would be confined only to the technologically adroit. ROI on such systems represents a significant challenge, especially if one confines oneís definition of ROI to quantitative factors, as discussed below.
There were relatively few projects nominated that were focused solely on infrastructure upgrade, and none made the final list of case studies. Neither of these conditions should be considered indicators that infrastructure is not important. The fact is, adequate infrastructure is absolutely essential; without it, all the other grand plans and applications are just "pipe dreams." Unfortunately, infrastructure doesnít seem to have dynamic appeal and thus tends to compete poorly right up to the point where disaster threatens.
It should be noted that each of the projects became possible either because the necessary investment in infrastructure was done before the start of or in conjunction with the program, or the technical solution was adapted to the available infrastructure. In either case, the infrastructure was a powerful influence on the final result.
Given the rate of growth of IT usage, it is almost impossible to conceive of infrastructure expenditures as being wasted. Nevertheless, in many instances that appears to be the perception of functional users when infrastructure projects are competing for fundingóor so many Government IT professionals reported to us.
This is another domain that tends to be challenged with regard to dynamic appeal. Nevertheless, the teams did receive several nominations for e-mail systems which tie together whole departments. To individuals outside the Government arena, the importance of this issue may be somewhat difficult to comprehend and appreciate. However, a very large fraction of routine Government business can be conducted by e-mail communications.
Unfortunately, when individual e-mail systems were being purchased by individual agencies and even subelements within agencies, there were extremely large numbers of available e-mail systemsómost incompatible with other e-mail systems. The net result was that, until recently, it was not uncommon to find major Federal departments that had up to 40 different, incompatible e-mail systems. Over the last few years, virtually all of the departments have made major strides in resolving this challenge of electronic connectivity for the conduct of routine business and written communications.
Of the several mail systems nominated, only one is included as a case studyóthe Department of the Treasuryís Treasury Communications System. Like all the other case studies, it was included because of other factors, not because it is the bestóalthough the professionals who worked so hard on it surely believe it to be so.
It is unlikely that a future study like this will again include a Mail Solutions category. While there are certainly still incompatible mail systems within and between Government organization, the requirement for pervasive electronic connectivity has been widely accepted. Consequently, it is reasonable to expect that over the next few years the specific, technical problems for mail will be resolved. However, that future study will likely have some similar category that is achieving the same purposeóresolution of market derived standards for "early adopters."
Several of the teams received nominations for the internal processes that specific departments had instituted to track either their IT programs or the approval process used to plan and execute their investment programs. While all very interesting and aligned with ITMRA objectives, none are included in the selected case studies. The rationale for this is not to suggest that such processes are not important. Rather, it is the recognition that virtually every Federal organization has such processes; they are required by law. As vital as they may be, these processes do not ensure that projects will be successful. It may be considered heresy in the Government environment, but the best process in the world for selecting or tracking IT projects does not appear to be either necessary nor sufficient for delivering a successful, implemented IT program. Therefore, the team did not recommend any such projects as candidates for case studies.
This was another unexpected category for which we received several nominations. In many ways, it could also be considered a special subset of the Information Delivery/ Sharing category. Nominated projects in this category tended to fall into either of two major subcategories:
Information Standardization. In these cases, the power of IT was harnessed to deliver standard data to all interested individuals. Typically, these were internal regulations or other policy or precedent establishing documents. Virtually all of the nominated cases used Web technology to deliver, via an Intranet, the desired information. In some cases, it represented the means by which new employees were able to learn about their agency or department. The very powerful element that recommends such solutions is the ability to eliminate the challenge of posting changes to "paper documents" and the incredible ordeals of distributing such "change packages," etc. Using Web technology means a single posting ensures that the new data is available to all.
An alternative approach in this same category is to use such systems to ensure the currency of information on regulations or eligibility criteria applicable to a specific audience or the public in general. Web-based technology remains key, but access is granted via the Internet or on a subscription basis. The ROI from such projects literally depends on both how inadequate the legacy system may be and the "costs" of people operating on the basis of outdated information.
Process Standardization. As we have seen, modern IT has attained the capability to actively support the work processes in the office and in other settings. A prime means of providing this support is via a functionality commonly referred to as "workflow." This functionality is resident in many upper-end application suites such as some Document Management systems and Product Data Management systems, as well as stand-alone workflow applications. The power of these systems is that an internal business process, once established, can be routinely enforced by the automated information system itself. For those not familiar with this technology, this may sound like, "Big Brother has arrived!" However, Government leaders increasingly speak very favorably about such concepts/approaches as "ISO 9000" oróin the software development worldóthe Software Engineering Instituteís "Capability Maturity Model" (e.g., Level 3). The attractiveness of these methodologies is that they offer the prospect of promising replicable processes. Workflow management systems are able to achieve that via the automated systems internal to an organization.
Strictly speaking, we did not receive for nomination any projects that were specifically process standardization cases; however, several of them were very close. The ROI and power of these kinds of tools is enormous, given the proper work circumstances. One of the case studies, the Department of Health and Human Services, FDAís Operational and Administrative System for Import Support (OASIS), is a good representative example of this category.
This is another category that consists of several different subcategories, given the nature, range and diversity of nominated projects. Without belaboring the issues concerning this category, these are the salient observations the team made from the nominated projects:
Training is a frequent recipient of the application of IT, and the team received many nominations for training related projects. In general, training solutions tended to be one of three different types.
Computer Based Training. The first, and most prevalent, uses the power of IT to replicate and deliver training. These instances can include computer based training/instructionówhere automated programs are used to train individualsóand simulationówhere the student uses a computer to perform exercises or activities that directly relate to job specific skills. The ROI of these kinds of solutions is highly dependent on several factors, including the frequency with which the materials need to be changed, the size of the serviced training audience, etc. Included in the case studies is an excellent example of this category, the Department of Energyís FETCís Computer Based Training.
Interactive Distance Learning/Video. A second use of IT involves the application of video teleconferencing and may also include some computer-based training. Typically, these kinds of solutions are selected when dealing with one or more of the following conditions: highly dispersed trainee population, the need for frequent training updates, when the material presented changes frequently or the risk/cost of applying out-of-date data is high, instances where commonality of standards needs to be assured, and having to rely on a very small and highly skilled training staff. ROI in these conditions is partially achieved by avoiding the travel expense for the dispersed students (and/or the few instructors) and the savings associated with risk reduction or error avoidance. The Social Security Administrationís Interactive Distance Learning Studio is a good example of this category.
Special Cases. The third use of IT for training relates to using the power of the technology to meet the special needs of trainee audiences. These instances tend to be focused on persons with special needs that computer-based devices can optimally meet, e.g., the ability to endlessly repeat specific exercises, or the ability to rapidly reformat and change presentation media, etc. An example of this is the Department of Veteran Affairís Adaptive Unit.
In the course of conducting this effort, team members had the opportunity to make a number of other, essentially qualitative observations about successful implementation of IT programs in the Federal Government:
As previously noted, virtually everywhere we looked we encountered a strong predilection toward using Web technology. At first, some of the more skeptical members believed that there may have been an element of emulation across programsóusing Web technology because it was "hot." However, over time and after gathering more information, the team came to the conclusion that what we were observing was a very solid match between the functions that Government programs were trying to achieve, the strengths of a specific technology, and the relative ease of implementing solutions using that technology. One would hope that, as new technologies emerge on the scene and are embraced by Government, this tight link between functional requirements and the features of the selected technology will continue.
The teams found a heightened sensitivity in attentiveness toward "customers" in the nominated programs. Our belief is that this sensitivity was an important factor in assuring success for programs. As part of our interview methodology, we asked program managers to identify their customers and what special steps had been taken to monitor customer satisfaction. We posed these questions to test whether the old myths of Government agency inattention to customers could be documented. In virtually every instance, a serious effort had been made to identify who the customers were, along with some degree of monitoring customer satisfaction after the program was implemented. In some cases, full blown customer satisfaction surveys are conducted; at a minimum, specific monitoring of negative reactions (e.g., adverse e-mail comments) is conducted.
This was a significant change from conditions some of us had experienced while working on Government projects on earlier occasions. We also believed that attentiveness to oneís customers is a key element of any successful project (in this sense, the customer is the user of the system, not the organizational entity paying for development). Team members were very sensitive to the reality that in Government there can be such a disconnect, posing serious challenges to effective program management and program success. In virtually all of the nominated cases, we detected a strong focus on the user-customer. We believe this validates the belief that this is the true path to long-term successful projects. However, we must also acknowledge that if a program cannot meet its sometimes short-term organizational demands, it probably will not "live" to see the long term.
Over and over again, the team encountered instances where the enthusiasm and zeal to move forward with a specific project had not been extended to addressing and solving implementation issues, especially the human ones. As noted in several of the discussions of the groupings of the projects, special care needs to be devoted to both "internally keeping the project sold" and dealing with usersí concerns. Failure to attend to either of these can be fatal for even the best conceived project or process that may be introducing the very latest technology. Members of the task force concluded our assignment with a new appreciation for the reality that all too frequently IT projects get so focused on matters of solving technical issues that people issues become ignored. The really successful projects appear to be guided by the views that IT needs to be an enabler of human work and that users have to "embrace" any system in order for it to be successful.
Among the nominated projects were a few that might best be referred to as "turnaround" candidates. These were instances where early developmental efforts did not go smoothly and the projects got so far off-track that they were seriously considered for cancellation. No such projects made it into the final set of case studies. Nevertheless, we believe some relevant observations apply.
All of us who are involved in technology based projects need to appreciate that not all projects will be successfulófor many reasons. One of the more frequent reasons in IT projects is due to the selection of the technology that is key to the project. Most of us have heard or know of instances in which an immature technology was selected but did not "mature" as envisioned by its advocates. Such challenges can become major embarrassments as schedules slip, costs mount, and valid Government needs remain unmet. We would like to believe that these kinds of horror stories are confined to history and the early days of ITóthe 1960s and 1970s. The hard reality is that IT technology keeps changing at what seems to be progressively faster rates. The challenge of making technology choices is going to continue; mistakes are going to be made.
The increasing tendency to accept COTS solutions in the Government should reduce the incidence of error in technology selection. But, when attempting bold advances or electing to become an "early adopter," one needs to be prepared to invoke risk mitigation strategies. Emerging standards can change or move in wholly unexpected directions. Similarly, what is COTS today can become commercially unsupported in a fairly short time, due to significant market changes.
The summary observation is that the IT world is subject to rapid technological change. All participants need to define and then continually update the evolutionary path of their core technologies. Failure to do so assures that your project or program will be left unsupported by the progressive improvements in technology, and it will be more expensive and painful to "catch up."
As previously noted, the teams found general acceptance of the concept that Government should leverage trends and standards that come from the marketplace in the form of COTS products. Even those rare individuals who philosophically disagreed with the COTS proposition grudgingly acknowledged the tremendous price advantages that COTS brings. When trying to achieve oneís missionówhether in the business world or in Governmentócost considerations tend to have dominant influence, especially when there are no evident performance differences. But, as with almost any great concept, there are some downside challenges. In the course of this study, we believe we encountered several worthy of note:
One of the more interesting parts of the interview process was posing what were referred to as the "CIOís Questions" about each nominated project. These questions are also referred to as "Raineís Rules," having been urged on the CIOs by the Director of Management and Budget. The specific questions were:
All three are first rate questions and are pointed at trying to ensure that the Government really gets a positive return on its IT investment. The questions are also aimed at trying to reinforce the "reinvention of Government" and ensure that past practices do not totally dictate future practices. The team had two significant, overarching observations as a result of posing these questions:
The difficulty seems to stem from those instances where there is widespread lack of familiarity or understanding of the power and capabilities of the technology about to be adopted. Seeking to comply with the dictum to reengineer first, some agencies have rushed into a demanding process of defining their processes and detailing "to be" conditions, etc. There are many willing hands available to assist in this processófor a price. Scores of process diagrams are produced. Unfortunately, in some instances, the reengineering then essentially gets redone in the implementation phase of the project because of the earlier lack of familiarity with the technology being applied.
Our team received reports of extended re-reengineering periods and associated worker dislocation and dissatisfaction. These uniformly appeared to be in cases where major technology steps were being attempted and/or where workflow or process management tools were being used for the first time. The observation being made should not be misunderstood. The team firmly believes in reengineering. We do, however, question the utility of total reengineering when the majority of users are totally unfamiliar with the capabilities of the new technology (picture the reengineering of an office environment from IBM Selectrics to a Novell LAN). It would be more prudent to acknowledge that all processes will eventually be touched, but not at the outset. Then, begin the reengineering effort, confine it to high-level business processes, decide on the technology that will be used, and then do the detailed reengineering in conjunction with implementation, recognizing that the real reengineering will continue over time as new insights are gained by users.
As noted in the Preface, as we began this project the team was asked to examine the issue of ROI. All team members were familiar with the term and the guiding precepts used by our respective companies, but very few could be considered ROI experts. However, we did not define our mission as "evaluating" the ROI approaches being used by the respective departments and agencies. Rather, we saw our role as raising the issue and reporting our observations. During the course of our study, the CIO Council asked the IAC to establish another IAC Task Force to examine the issue of ROI. That Task Force is completing its work and will render its report later in 1997.
Our observations on ROI are not intended to foreshadow the findings of the ROI Task Force, which are expected to be of a more overarching nature. Instead, we confined our effort to projects and processes deemed successful and nominated by Federal departments. Given this narrow universe, our key observations on ROI are as follows:
The course of our study and these observations led the team to conclude:
The members of this IAC Task Force were afforded a very special opportunity to examine a large number of diverse, successful Government IT programs. In the course of our study, we tried to document the core similarities of the successes. As IT professionals, we all have a tendency to focus on various aspects of the technology, but we need to be mindful that in all successful projects there are only two abiding themes: