Appendix G Electronic Government Information Products Assessment Agency Meetings Held and Discussion Questions Electronic Government Information Products Assessment Agency Meetings Held Agency Number of Attendees Department of Health and Human Services 10 Meeting Date: September 14, 1998 Supreme Court of the United States 11 Meeting Date: September 15, 1998 Environmental Protection Agency 12 Meeting Date: September 17, 1998 Department of Education 5 Meeting Date: September 22, 1998 Department of Commerce 8 Meeting Date: September 23, 1998 National Archives and Records Administration 7 Meeting Date: September 24, 1998 Electronic Government Information Products Assessment Agency Meeting Discussion Questions 1. Does your agency have preferred medium and format standards for Government electronic information products at the permanent public accessibility stage? If so, what are the top three? What factors does your agency consider in determining preferred standards (e.g., user needs, agency's dissemination requirements or policies, cost, security etc.)? What about specific standards for CD-ROMs as they relate to user documentation, installation, search software, etc.? 2. Can you give us any examples of particularly innovative and creative product formats, mediums and/or online approaches? We have in mind formats, mediums, or online approaches that may well point to the wave of the future, not only for a particular product but also for other kinds of products, yet is neither an agency-mandated standard nor even a common agency practice. 3. Is there any difference between your agency's preferences for mediums and formats as opposed to the preferences of intermediary distributors? If so, what are those differences, and why are the two preferences different? 4. Has your agency involved external user groups in assessing the value and effectiveness of the dissemination of Government electronic information products? If so, are there formats and mediums that seem particularly appropriate for public dissemination to users who may be economically, technically, or physically disadvantaged? 5. Does your agency follow any internally or externally prescribed guidelines for the presentation and organization of products in online formats? If so, what are they (e.g., WWW Federal Consortium, FIPS Guidelines, agency or departmental publication specifications or guidelines)? 6. Has your agency undertaken any kind of cost benefit analyses for producing or creating products in preferred or emerging formats, mediums, or online approaches for distribution to the FDLP? If so, which ones appear to be the most cost-effective? 7. What factors does your agency consider in deciding to create or retain products in more than one medium? Is this a common agency practice? 8. Are there trends with respect to migrating specific families of products from pre-electronic mediums to electronic mediums or formats? For example, are loose-leaf publications, training manuals, annual reports, conference proceedings, newsletters, rules and regulations, scientific journals, etc., targeted for migration to a particular medium? If so, which mediums and formats are used for specific families of products? 9. Has your agency identified any medium and format standards that seem particularly appropriate for use throughout a product's entire information life cycle, not just at one stage (i.e., creation, storage and retrieval, communication and dissemination, archiving and disposition) for electronic Government information products? If so, which ones? 10. How do you determine whether a product should be made permanently publicly accessible when you create or produce it? If so, what criteria do you use to determine which products will be permanently publicly accessible? Can you give us any examples of how you ensure permanent public accessibility for a given product? 11. Does your agency routinely provide locator tools (e.g., GILS or specific agency locators) to enhance access to information sources and services available to external users and customers? If so, is this an official policy, common agency practice, or both? 12. Are there trends for facilitating public access to your agency products by including them in broad electronic Federal Government information services such as GPO Access, LOC Thomas, and NTIS FedWorld? Are you using any particular guidelines to facilitate that decision, and if so, what are they? Summary of Responses to Agency Meeting Questions 1. Does your agency have preferred medium and format standards for Government electronic information products at the permanent public accessibility stage? If so, what are the top three? What factors does your agency consider in determining preferred standards. Agencies reported using the following electronic mediums most often: CD-ROM, Internet, and Bulletin Board System (BBS). The most frequently used formats include: * HTML, PDF, ASCII * TIFF, GIF, JPEG * Lotus/Domino In determining medium and format standards, agencies consider the amount of information or files used, the timeliness of the information (e.g., more recent products or publications often placed on the web), and user needs for easy and quick access to information. For example, some agencies are looking to Windows as an interface for CD-ROM products since the public is used to seeing and using Windows. 2. Do you have any examples of particularly innovative and creative product formats, mediums, and/or online approaches? We have in mind formats, mediums or online approaches that may well point to the wave of the future for not only a particular product but also for other kinds of products, yet is neither an agency-mandated standard nor even a common agency practice. Almost all the agencies interviewed are exploring a wide range of innovative and creative format, medium, and web applications. Below is a sampling of some of the interesting online approaches and formats used by the agencies interviewed. * Data warehousing: Taking information not previously publicly accessible and integrating it into an online format. The format used is an Oracle database using SQL to query. * Interactive Geographic Information System (GIS). Provide mapping capability through GIS combined with regulatory information to create dynamic maps. * Online catalog of all products on the agency's website, using Oracle with a ColdFusion interface with their search engine (Verity). * Creating user guides for CD-ROMs as pop-up HELP or short Read-Me files so users will be more inclined to use HELP. * Live "real-time" web casting of selected speeches. Format: RealPlayer software, available free from the web. * Radio news broadcast news service. Provides daily radio sound-bits for news reports. Format: RealAudio RealPlayer software and WAV file format for downloading. * Real-time forecasting of air pollution levels for 22 states from one site. Format: animated GIFs created by nonproprietary software designed by computer center in North Carolina. * Multimedia CD-ROM (i.e., art, music, animation, film, and video). * Searchable electronic inventory of all proposals funded over the last 30 years so the agency can analyze its own information and make it available to others. Format: legacy database put into WAV database using HTML on the fly. * Documents stored in TIFF format for image and textual data. As customers request documents, the agency converts them to PDF so customer can download. 3. Is there any difference between your agency's preferences for mediums and formats as opposed to the preferences of intermediary distributors? Generally, agencies indicated that intermediary distributors do not find agency formats to be restrictive. However, the distributors often modify formats (e.g., from HTML to ASCII, or reformat data using compression technology). 4. Has your agency involved external user groups in assessing the value and effectiveness of the dissemination of electronic Government information products? If so, are there formats and mediums that seem particularly appropriate for public dissemination to users who may be economically, technically, or physically disadvantaged? Yes, all agencies reported that they involve users in assessing some aspects of their products, per OMB Circular A-130. Examples include the following: Focus groups are used to: * Determine the capability of new products (e.g., Can you use the same technology for the newer version of a product? Do users lose anything (e.g., Macros) when they update a product?). * Determine ways to create more user-friendly CD-ROMs that resulted in the agency establishing three principles for producing CD-ROMs: make them simple to use, intuitive, and self-tutorial. * Determine how information is presented on the web (Alpha and Beta testing). * Solicit feedback on usability and accessibility; focus groups conducted with tribal Governments, teachers, librarians, children, etc. * Learn about expectations, behavior, and problems in accessing products on the web experienced by the elderly. * Interview and videotape users to assist agency redesign of website. * Solicit ideas from educators at professional conferences about topics that they would like to see presented in online products. For technologically or economically disadvantaged customers, one agency sub-unit faxes free copies of information printed from the CD-ROM or the Internet. Another agency sub-unit reported they try to reach economically disadvantaged customers by training rural community leaders in isolated areas on ways to access health-related information on the web . 5. Does your agency follow any internally or externally prescribed guidelines for the presentation and organization of products in online formats? If so, what are they (e.g., WWW Federal Consortium, FIPS Guidelines, agency or departmental publication specifications or guidelines)? Most agencies have developed guidelines or "best practices" for presentation of products in online formats. However, several agency representatives indicated that the real challenge is in convincing agency staff to comply with the guidelines. The following agencies provided information on their guidelines: 1. NARA: NARA Guidelines for Digitizing Archival Materials for Electronic Access (not to be considered a standard for digital imaging). (http://www.nara.gov/nara/vision/eap/eapspec.html) 2. Federal Web Consortium's guidelines are based on Dept. of Education's guidelines (http://www.ed.gov/internal/wwwstds.html). One sub-unit, NCES, also has developed guidelines. 3. EPA used WWW Federal Consortium guidelines (http://www.dtic.mil/staff/cthomps/guidelines/) to develop their own guidelines for presentation. 4. Census uses a process and structure for submitting items for the web, but it is not yet formalized. 5. No departmental guidelines exist for the fifth agency, but most sub-units have some kind of guidelines for presentation and organization, although they might vary among sub-units. 6. Has your agency undertaken any kind of cost-benefit analyses for producing or creating products in preferred or emerging formats, mediums, or online approaches for distribution to the FDLP? If so, which ones appear to be the most cost-effective? Generally, agencies have not conducted a formal cost-benefit analysis. Agency representatives made the following observations, however: * One agency sub-unit tracks the number of customers who purchase a product and compares this amount against the cost of producing it. They discontinued a product on CD-ROM because so few people could afford to buy it. * The web reduces administrative costs for printing and mailing hard copies of publications. * One agency reported a dramatic decrease (by 25,000) in the number of publications requested under the Freedom of Information Act due to the web. * One agency tracks the number of people that a product can potentially reach (e.g., they sell 2,000 paper copies through GPO, but have 9,000 hits on the website). * One agency reported that they order fewer publications to fill customer requests as a result of the web. 7. What factors does your agency consider in deciding to create or retain products in more than one medium? Is this a common agency practice? Agency representatives reported that they consider a variety of factors in creating and retaining products in more than one medium, although they did not characterize these factors as a common agency practice. Several agencies reported that these issues are considered on a case-by-case basis or by the individual program unit. The key factors considered are: 1. Budget (e.g., some products in CD-ROM are too expensive to make available to a small audience). 2. Cost (e.g., cost to print and mail product as opposed to make it available on the web). 3. Needs of technologically disadvantaged users (e.g., one agency maintains its Fax on Demand service, even though it is not cost-effective). 4. Accessibility (e.g., one sub-unit stores products in TIFF image format so they can produce them in whatever medium of output customers want). 5. Size of audience (e.g., agencies survey users and use web software to track use). 6. Number of queries or type of customer requests. Customers will often request a publication or product in more than one medium. 7. Regulatory requirements. Some products must be produced in paper regardless of whatever other medium it is produced in. 8. Type of product/publication (e.g., searchable databases are only suitable for electronic mediums). 8. Are there trends with respect to migrating specific families of products (e.g., loose-leaf publications, training manuals, annual reports, conference proceedings) from pre-electronic mediums to electronic mediums or formats? If so, which mediums and formats are used for specific families of products? The general trend reported by agency representatives is to migrate more products to the web, especially recent ones. Some examples include: * Conference proceedings and presentations online in PowerPoint or PDF. * Newsletters in HTML. * Training manuals and annual reports in HTML and PDF. * Information for records managers are posted to Gopher, but will move to the agency's website in 1999. 9. Has your agency identified any medium and format standards that seem particularly appropriate for use throughout a product's entire information life cycle, not just at one stage (i.e., creation, storage and retrieval, communication and dissemination, archiving and disposition) for electronic Government information products? If so, which ones? Most agencies have either not addressed this issue of information life cycle or are struggling with it. * NARA has established medium and format standards for transferring permanent records to the National Archives in 36CFR 1228.188. * One sub-unit reporting putting documents in ASCII, but using Oracle for database management. * One sub-unit is beginning to think about standardization for some documents. They draft documents in Lotus Notes (GroupWare) and publish final document in another database that goes onto the web. They use Rich Text Format (RTF) to accommodate images and text. * Several sub-units indicated that the technology is changing so rapidly they cannot establish standards. 10. How do you determine whether a product should be made permanently publicly accessible when you create or produce it? If so, what criteria do you use to determine which products will be permanently publicly accessible? Can you give us any examples of how you ensure permanent public accessibility for a given product? No agencies could provide responses to this question or indicated that this issue has not been resolved. Some observations: * One sub-unit is committed to making paper and CD-ROM-based products available for permanent access, but they are less clear about their commitment to products on the Internet. * Some agency representatives did not understand the differences between permanent public access and permanent records. (The experts interviewed for this study provide some insight into the reasons that agencies are not addressing this issue.) 11. Does your agency routinely provide locator tools (e.g., GILS or specific agency locators) to enhance access to information sources and services available to external users and customers? If so, is this an official policy, common agency practice, or both? Most agencies indicated that the web format supercedes the original GILS concept. However, most agencies have their own locators: * NTIS has a catalog and maintains some GILS records. * EPA's website has a GILS record and they put all Internet products on one server so there is one access point for all their products. * Development and maintenance of GILS records is official agency policy for NARA. 12. Are there trends for facilitating public access to your agency products by including them in broad electronic Federal Government information services such as GPO Access, LOC Thomas, and NTIS FedWorld? Are you using any particular guidelines to facilitate that decision, and, if so, what are they? About half of the agencies use GPO Access or NTIS FedWorld. The other half relies more heavily on individual agency websites with good links.