Both e-learning and the amended Section 508 of the Rehabilitation Act are at the beginning stages of implementation. This can make the implementation of either difficult to understand for both the program office responsible for e-learning and the contracting office. This paper is written to aid them in their understanding of the issues and processes in purchasing the best e-learning solution while being in compliance with the provisions of Section 508 as amended.
Section 508 of the Rehabilitation Act was substantially amended in 1998 and now requires Federal agencies to procure only electronic and information technology that meets a set of very specific standards. This new law has the promise to revolutionize the technology used by the Federal government--- and the world. Even though the IT industry is responding, there is still sophisticated technology that remains inaccessible to persons with disabilities. This problem is compounded by a lack of understanding regarding how the Section 508 standards are to be considered during the Federal procurement process. As a consequence, many agencies are either reluctant to procure the most effective e-learning solutions for their workforce or choose not to implement e-learning at all. This end result helps neither the agencies nor their employees with disabilities.
Such a negative effect is unnecessary. The authors of the Section 508 standards realized that incorporating accessibility into electronic and information technology (EIT) would take time. Therefore, the procurement process permits the full implementation of Section 508 by the agencies while allowing them to procure the technology that they need today. Thus, it is vitally important to understand how the procurement process works with the Section 508 standards.
The most important step is for agencies to fully describe all of the training requirements of their organization in the e-Learning /EIT solution they hope to implement. This should include objective measures of learning effectiveness. These objective measures must be justifiable based upon agency needs. Once the agency has identified its needs, then the relevant Section 508 standards are taken into consideration. This allows agencies to fully meet their needs while giving industry a strong incentive to make the best technology into the most accessible technology. For commercial off-the-shelf products, this means that agencies can procure e-learning solutions that fully meet their training needs. For custom-developed solutions, the procurement process is slightly more complicated but still permits agencies to fully meet their needs while allowing vendors to make their products as accessible as possible.
For decades, Congress has enacted legislation to make work opportunities equally available to all segments of the population. Over the last decade, our society has increasingly relied on information systems and the Internet. The promise of the World Wide Web is easy and instant access to a vast world of information, services, and communications.
Ironically, while people with disabilities may seem to be the greatest beneficiaries of this new ubiquitous technology, they are often shut out of government services and employment because the technology was not designed to be accessible. For instance, just as a set of stairs may be a barrier to a person who uses a wheelchair, a computer program or web site that does not include basic programming needed by a screen reader (a computer program that reads out text on a computer screen using voice) will be useless to a person who is blind. Worse yet, designing EIT to accommodate a wide-range of disabilities is usually easy-to-do and “invisible” to non-disabled users demanding the most interactive and compelling interfaces.
In 1998, Congress took a major step in responding to this problem by amending Section 508 of the Rehabilitation Act. Both Section 508 of the Rehabilitation Act of 1973 (29 U.S.C. 794(d)) and the accompanying standards (36 CFR Part 1194) are key steps towards making EIT accessible by making the Federal government a leader in EIT accessibility for people with disabilities. Since its amendment, industry has responded quickly and we are already seeing much of this new technology permeating the marketplace at all levels and affecting the electronic and information technology that we all use. As amended, Section 508 requires comparable access to the data and information provided by Federal agency electronic and information technology (EIT). Section 508’s biggest effect, however, will likely be in the technology that the Federal government purchases--- thus affecting the entire information technology infrastructure over time. Congress required the Access Board, a small independent Federal agency, to develop standards for EIT accessibility and required the Federal Acquisition Regulations (FAR) to incorporate these standards. As of June 2001, all EIT (including computer software applications) procured by Federal agencies must meet the requirements set forth in the FAR for EIT.
Some people believe that Section 508 creates a dilemma in the context of e-learning. On the one hand, there is a documented and direct relationship between e-learning interactivity and the robustness and effectiveness of the overall learning experience. In general the more interactive an e-learning experience is, the greater the effectiveness and efficiency of the courseware. On the other hand, highly interactive and engaging web enabled learning fails to meet all of the applicable Section 508 standards. Therefore there tends to be a conflict between robust, interactive web enabled training courses and conformance with the Section 508 standards.
Many agencies fail to understand, however, either the potential of e-learning solutions or the procurement process envisioned by Section 508. This failure means that the Federal government loses both the value of web-enabled learning and the vast potential it has to impact human capital performance (HCP). Therefore, agencies need simple and effective guidance to help them in their procurement and implementation of e-learning solutions.
The e-Training Best Practices Committee (BPC) has written this paper to provide such guidance. It is written to aid those individuals who have the responsibility of implementing e-learning (the program office) and the contracting officers who work with them. It addresses the specific issues surrounding Section 508 and e-learning initiatives within the Federal government and for Federal employees. It concentrates on e-learning as applied to the Federal workplace. Its intent is to provide meaningful and useful background on Section 508 and the procurement process and to make concrete recommendations for procuring and implementing e-learning initiatives using commercial off-the-shelf (COTS) products within Federal agencies. This paper first proposes an objective measure of the effectiveness of any e-learning solution--- the interactivity of that solution. Then, the paper outlines how to effectively procure solutions that meet these requirements while also meeting the legal requirements of Section 508.
Interactivity is one of the most important elements in a successful e-learning program. Dr. Martie Buzzard of the University of Oklahoma defines interactive e-learning as “The acquisition of information, knowledge and skills through any computer-delivered electronic system that allows the user to control, combine, and manipulate different types of media, such as text, sound, video, computer graphics, and animation.”
The July 16, 2002 e-Learning Developers’ Journal discussed the importance of interactivity and cited two studies; the first going back as far as July 1990 when J.D. Fletcher of the Institute for Defense Analyses published a paper entitled. “Effectiveness and Cost of Interactive Videodisc Instruction in Defense Training and Education”. This study was quoted in the Instructional Effectiveness Study carried out by Oklahoma University and published in 1992 by Skill Dynamics, an IBM company. The article notes that there are two reasons why the Fletcher study is important. First it shows that certain types of e-learning can be more effective than traditional instruction and second that the direction to go in order to make e-learning effective is to create flexible e-learning systems that adapt to the learner through interactivity. For this reason, the key criterion recommended by the BPC is interactivity.
Dr. Conrad Gottfredson in his Checklist for Evaluating the Instructional Integrity of an IT Self-Learning Systemhas identified and quantified through example the factors in interactivity. Those factors can be found within the Methodology Considerations of the Checklist and are:
How effectively do the lessons gain and maintain the attention of the learner?
How clear is the presentation of content? (Concise, focused, logical)
How thought-provoking are the questions?
To what degree does the learning system engage the learner?
How useful are the analogies?
How useful are the literal examples?
How useful are the scenarios?
To what degree are skills modeled appropriately?
How appropriate is the cognitive practice?
How appropriate is the hands-on-guided practice?
How appropriate is the hands-on-unguided practice?
How meaningful is the integrated review?
To what degree is there meaningful feedback?
To what degree are there meaningful mastery checks?
In the Checklist, each of these questions gives specific examples for each level of interactivity. For a printout of the Methodology Considerations section of the Checklist, please refer to the Resources section of this paper.
A simulation, wherein a user would actually have to perform tasks with multiple branched outcomes, is considered a high degree of e-Learning interactivity; answering questions would be considered a relatively low degree of interactivity. In the Checklist above these factors are weighted according to their relative impact on interactivity and learning effectiveness. Indications are that key measures of training effectiveness, such as actual learning, learning rate, consistency, and retention, dramatically improve with improving levels of interactivity. Therefore, specific requirements for training effectiveness should be incorporated into the procurement requirements for e-Learning systems.
The Rehabilitation Act has been law since 1973. Two important sections of the Act affect agencies when they are procuring and implementing e-learning solutions for their workforce. They are sections 501 and 508 of the Rehabilitation Act.
Because Section 508 is enforceable only with respect to newly procured EIT, its initial impact will be at the procurement stage. The Federal Acquisition Regulation (FAR) governs the Federal procurement process. While the Section 508 standards for EIT accessibility are quite rigorous, both the FAR and the Access Board standards incorporate considerable flexibility that both meets the long-term needs of Federal employees with disabilities and allows EIT developers and manufacturers freedom to design innovative technologies and solutions for accessibility. For instance, because many of the EIT standards are “forward looking” and may not be readily available in technologies when the standards were written, Federal agencies are not required to meet all of the standards when procuring off-the-shelf EIT if such EIT is not commercially available. Instead, agencies are only required to purchase the EIT that “best meets the standards.” In addition, the EIT standards recognize the innovative nature of the high-tech industry; the standards permit deviations from specific provisions through alternative techniques (called “equivalent facilitation”) that meet the goals of a particular provision while not meeting its literal wording. This flexibility creates economic incentives for manufacturers (through natural market forces) to make their products increasingly accessible over time. This solution helps the IT industry continue to innovate and sell to the Federal government while ensuring that people with disabilities gain greater and greater access to the data and information provided through the Federal government’s EIT. It is also designed to have the least impact on how Federal procurement officers do their jobs.
Section 508 permits agencies to procure products that meet agency needs but that are not the most accessible only when the agency determines that compliance imposes an “undue burden.” In this context, “undue burden” is a legal term that means “significant difficulty or expense.” In making an undue burden determination, agencies must consider all resources available to the program or component procuring the EIT--- not simply the cost of the EIT being procured. In addition, the documentation by the agency supporting the procurement must explain why, and to what extent, compliance with each such provision creates an undue burden. The Access Board’s standards provide some guidance on the definition of undue burden. Specifically, section 1194.4 of the standards define undue burden as:
Undue burden means significant difficulty or expense. In determining whether an action would result in an undue burden, an agency shall consider all agency resources available to the program or component for which the product is being developed, procured, maintained, or used.
Obviously, undue burden is a very difficult threshold to meet.
Another concept that is important to understanding how the Section 508 standards are considered during the procurement process is the concept of “nonavailability” or commercially unavailable products. The Access Board’s FAQ’s provide guidance on this term:
An agency may conclude that EIT meeting the applicable technical provisions of the Access Board's standards is not available (and purchase EIT that does not meet those provisions) when it cannot find a commercial item that both meets applicable Access Board's technical provisions and can be furnished in time to satisfy the agency's delivery requirements. If products are available that meet some, but not all, applicable provisions, agencies cannot claim a product as a whole is nonavailable just because it does not meet all of the applicable provisions. Agency acquisitions must comply with those applicable technical provisions that can be met with supplies or services that are available in the commercial marketplace in time to meet the agency's delivery requirements. Nonavailability determinations must be documented. The concept of nonavailability is recognized in the Access Board's standards (at 36 CFR 1194.2(b)) and the FAR (at 39.203(c)) because agencies may find that some of their needs cannot be satisfied with EIT that meets all the applicable technical provisions. However, as manufacturer offerings of products that meet the applicable technical provisions increase over time, incidents of nonavailability will decrease.
Nonavailability is an independent basis for acquiring EIT that does not meet the applicable Access Board's technical provisions. See FAR 39.204(e)(2)(ii). Undue burden documentation would be required if products meeting all (or some) of the standards are commercially available but would create an undue burden on the agency to acquire. (In other words, when a product meeting the standards is commercially available, it is treated no differently than a non-commercial item with respect to application of the undue burden justification.) Agencies should document commercial nonavailability in accordance with agency procedures to assist in defending the agency's decision in the event of a complaint.
It is important to carefully consider the Section 508 standards, regardless of whether your agency is procuring a commercial off-the-shelf e-learning solution or a custom-developed solution. Sometimes, however, e-learning solutions may meet the intent--- but not the literal wording--- of a particular provision. Because the IT Industry is constantly innovative, the Section 508 standards are designed to encourage such solutions for accessibility. Section 1194.5 of the Access Board’s standards includes the provision regarding equivalent facilitation.
Nothing in this part is intended to prevent the use of designs or technologies as alternatives to those prescribed in this part provided they result in substantially equivalent or greater access to and use of a product for people with disabilities.
Again, the FAQ’s on the Section 508.gov site provide useful guidance.
The Access Board's standards provide that agencies may accept EIT offered by vendors which uses designs or technologies that do not meet the applicable technical provisions in Subpart B but provide substantially equivalent or greater access to and use of a product for people with disabilities. (See 36 CFR 1194.5.) This is referred to as "equivalent facilitation." Equivalent facilitation is not an exception or variance from the requirement to provide comparable access. Rather, it is a recognition that technologies may be either developed or used in ways not envisioned by the technical provisions in Subpart B but still result in the same or better functional access as would be provided by strictly meeting the provisions in Subpart B. Functional outcome, not form, is the key to evaluating whether a technology results in "substantially equivalent or greater access." In effect, meeting the functional performance criteria in Subpart C of the Board's standards is the test for equivalent facilitation. For example, an information kiosk that is not accessible to a person who is blind might be made accessible by incorporating a telephone handset connected to a computer that responds to touchtone commands and delivers the same information audibly that is provided on the screen. In short, the concept of equivalent facilitation is designed to allow the marketplace to offer innovative solutions. For this reason, agencies must draft their solicitations for EIT so that products offering equivalent facilitation are considered along with those that strictly meet the technical provisions of Subpart B of the standards.
Where courseware draws upon underlying databases or programmatic logic to produce the user interfaces, equivalent facilitation in the context of e-learning courseware may be courseware that provides alternate pages that draw upon those same underlying databases or programmatic logic in order to provide a learning experience that adequately accommodates a user’s disabilities. Because solutions relying on equivalent facilitation are in lieu of meeting the exact standards applicable to specific technologies, it is important to consider the specific needs addressed by each applicable standard (in light of subpart C of the standards) to determine if equivalent facilitation is really being achieved.
This is a term embodied in the federal contract procedure that allows offerors of commercially available product to address all needs of the agency and for government evaluator(s) to determine which product “best meets” those needs. In order to determine products that best meet the agency’s needs, the needs should be stated in the most quantifiable terms possible. In the case of e-learning, the use of interactivity is quantifiable as well as the Section 508 standards. The use of “best meets” is invoked only after market research is completed and products are evaluated using a criteria designed to determine which products “best meets” the agency’s requirements.
The Section 508 Steering Committee has developed an extensive set of Frequently Asked Questions (FAQ’s) available on the section508.gov website. These FAQ’s spell out the steps necessary in order to integrate Section 508 into the procurement process:
First, the requirements (program) office must read the Access Board standards, and determine which technical provisions apply.
Second, the requiring official must perform market research to determine the availability of products and services that meet the applicable technical provisions. In determining availability, requiring officials should consider, among other things, information on vendor web sites and the government's Section 508 web site (www.section508.gov).
Third, the requiring official must identify which technical provisions, if any, do not apply due to an exception, such as nonavailability or undue burden.
Fourth, technical specifications and minimum requirements must be developed considering the results of market research and agency needs. The requiring official must submit this information along with the purchase request, including nonavailability or undue burden documentation as appropriate, to the contracting officer.
Finally, the contracting officer must draft and issue a solicitation to receive offers from interested sources or consider placing an order under a delivery order or task order contract. Proposal evaluation may yield additional information that could require reconsideration of the need for an exception (either retracting or invoking an exception, such as nonavailability).
As noted above, an agency must consider EIT that offers equivalent facilitation and should state in its solicitation that it will do so.
The process explained below integrates the Section 508 requirements as defined in the Frequently Asked Questions and the procurement steps into one step-by-step process for e-learning procurements.
The following diagram outlines the steps of the procurement process.
Step One: Identify Needs and Applicable 508 Standards. The first and most important step is to carefully decide your needs. In the context of e-learning, this step will likely require the most careful consideration on the part of the e-learning program office or official. There are three specific areas of needs that should be considered. They are learning objectives (content), interactivity and electronic and information technology (EIT) needs. In addition to identifying the learning objectives of your e-learning solution, you will also want to objectively quantify the level of interactivity and the learning effectiveness of the solution. You should be careful at this stage to ensure that ANY product meeting your stated requirements would truly meet the needs of your agency. The program office must also identify the applicable 508 standards. Most vendors are award of these standards and will submit their compliance with the standards in their proposals. A direct link to these standards can be found in the Resources section of this paper.
Step Two: Identify Pool of Products Meeting Those Needs. The second step is identifying those products that meet the needs you identified in step one. This is market research. The FAR states, “Agencies must ensure that legitimate needs are identified and trade-offs evaluated to acquire items that meet those needs.” Market research is simply looking at all the products available through your contracting vehicle of choice and deciding which meet your defined program needs. If you are buying products through a GWAC (government-wide acquisition contract), such as GSA's Federal Supply Schedule contracts, or similar procurement vehicle, evaluation of those sources constitute market research. Alternatively, if you are soliciting proposals, this stage may involve collecting offers from vendors.
Step Three: Identify Those Products That Meet Needs and 508 Standards. From the group of products that meet your program needs, you should identify those products that fully meet the Section 508 standards and then choose from among those products. If none of the products identified in step two fully conform to the Section 508 standards, then you must choose the product that best meets the standards. In this third step, you must always choose the product that either fully meets or that best meets the Section 508 standards, unless choosing that product would impose an undue burden. Whatever your product decision, if you choose a product that does not meet all the 508 standards, you must justify your choice using the undue burden exception or commercially unavailable documentation. All of this documentation is then given to the contracting officer for final disposition.
Agency procurement officials already follow a process similar to this one in making procurement decisions every day in a wide variety of contexts. Section 508 does not disturb this procurement process--- it only affects the ultimate choice of products selected.
Based on this analysis, the BPC believes that agencies should not be hindered at all in the use of commercially available e-learning products. When agencies are procuring commercially available e-learning solutions, they should identify their agency needs and the applicable Section 508 standards. If products exist that meet the Section 508 standards, agencies should choose among those products in the normal way procurements are made. If products meeting the Section 508 standards are not commercially available, then agencies should carefully consider and document which of those products “best meet” the standards and then proceed to procure that product. If procuring that product, however, imposes significant difficulty or expenses (in light of the resources available to the component or program making the procurement), then the agency should carefully document the undue burden associated with procuring the product. This solution allows the marketplace to drive accessibility while still allowing vendors to develop effective and compelling e-learning materials.
When agencies contract for the development of custom e-learning products (including the use of in-house contractors to develop custom e-learning solutions), a different analysis applies. In this case, the commercial availability defense disappears and agencies must ensure that their products conform to all of the Section 508 standards.
The procurement process is key in meeting both the e-learning requirements and the requirements of Section 508. This section gives three practical examples of the points made in the previous sections and shows how an agency can evaluate off-the-shelf e-learning software that meets the agency’s learning needs and Section 508 responsibilities.
In each example, agency representatives have compared the software of four vendors using the following criteria:
In each example a scale from 1 to 100 has been used, with 100 meaning that the software meets all of the agencies requirements for content, interactivity and technical specifications.
Prior to selecting the software the agency developed a list of content requirements, interactivity requirements, and technical specifications. The agency gave the software to a team to rank the products. After a review of the criteria, the team developed a justification and decided that no product with a total score of less than 210 on these three items would be selected. The team members’ average scores are listed below.
Based on this initial review, Vendor A and Vendor B were selected. Section 508 requirements were used to review the software. A scale of 1 to 100 was used. A score of 100 indicates that the software fully meets all Section 508 requirements.
In this case, Vendor B. was selected. It had the highest score on content and interactivity and the highest score for Section 508 compliance.
In example 2, none of the scores for the content, interactivity, and technical requirements change.
However, there is a change in the Section 508 scores.
In this case, Vendor A was selected. Even though Vendor A had a lower score than Vendor B on the initial content and interactivity requirements, it still surpassed the agency’s threshold for those items (210). Since it has the higher 508 rating it is selected.
The question may be asked, what if Vendor C and D were fully 508 compliant?
In example 3, none of the scores for the content, interactivity and technical requirements change.
However, Vendor C and D are also rated on Section 508 scores.
In this case, Vendor A was selected. Since the agency had established both a content and interactivity standard that Vendor C and Vendor D did not meet, neither can meet the agency’s needs even if they are fully 508 compliant.
These examples amply demonstrate the importance of objectively identifying agency needs early in the procurement process.
Both the Federal government and private industry are working towards making e-learning available to all segments of the Federal workforce. Given the constant improvements in technology, meeting Section 508 requirements will probably be an on-going process. What is important is that both groups work together towards that goal. This shared responsibility is at the heart of this paper.
As we move towards a more accessible EIT infrastructure in the Federal government, it is important to keep both the goals of Section 508 and the procurement process in perspective. By amending Section 508, Congress neither intended to stymie Federal agency e-learning programs nor disregard the needs of Federal employees with disabilities. This paper outlines how both of these needs can be met.
The following information is available on the Internet:
Section 508: Information about Section 508 is available on the Internet. The General Services Administration (GSA) oversees the Section 508 website at http://www.section508.gov which includes links to the text of the Section 508 amendments, Section 508 Standards, and the FAR amendments incorporating the 508 standards. The Section 508 standards are also available from the Access Board’s web site at http:///www.access-board.gov/sec508/508standards.htm. The Access Board has created additional guidance available at http://www.access-board.gov/sec508/guide/.
VPAT 508 Standards: The applicable standards are listed in the Voluntary Product Accessibility Template and can be found on the web at: http://www.itic.org/policy/508/vpat.html. You should refer to Section 11942.22 – Web-based Internet information and applications and possibly Section 1194.24 – Video and Multi-media Products.
Federal Acquisition Regulation: The web site for the Federal Acquisition Regulation (FAR) can be found at http://www.arnet.gov/far
The E-Learning Guild: http://www.elearningguild.com
Checklist for Evaluating the Instructional Integrity for an IT Self-Learning System, Specific
Training Methodology Considerations, Reprinted with the permission of Conrad Gottfredson.
Link to PDF Version
This paper would not have been possible without the assistance of:
Members of the Interagency Section 508 Steering Committee and Working Group
Staff of the Department of Transportation’s Virtual University
Members of the Department of Justice’s e-Learning Initiative Working Group
Members of the Office of Management and Budget’s e-Training Best Practices Committee
In fact, the use of measures and targets for product effectiveness should be considered for all types of purchases.
The e-Training Best Practices Committee (BPC) was formed by the Office of Management and Budget (OMB) for the express purpose of obtaining recommendations from both Federal and private industry on best approaches to implementing the eGov, e-Training initiative.
This paper only addresses e-learning in the context of the Federal workplace. It does not address the additional issues raised by e-learning by Federal-funded entities, such as colleges and universities, research institutions, and state and local governments.
Communication via e-mail from Dr. Martie Buzzard, University of Oklahoma, September 10, 2002
Checklist for Evaluating the Instructional Integrity of a Self-Learning System, Bob Mosher and Conrad Gottfredson, 1999
Federal employment is also affected by Section 504 of the Rehabilitation Act, 29 U.S.C. 794d
Providing an interpreter or instructor is not a means of achieving compliance with Section 508. It does not provide equivalent facilitation, because it is not a means of making the actual product meet the standards. These accommodations may, however, greatly assist an agency in meeting its separate requirements under other laws, such as sections 501 and 504 of the Rehabilitation Act.
The Section 508 Steering Committee is an interagency steering committee comprising the General Services Administration, Access Board, Department of Justice, and other key agencies involved in implementing Section 508.
For commercial products, GSA has worked closely with the community of Federal IT vendors to help them provide a systematic and efficient means of providing objective information to procurement officers to aid them in their decisions. Using the so-called VPAT (voluntary product accessibility template), vendors are now providing checklists that greatly streamline the procurement process, thus alleviating the need for procurement officers becoming programmers or accessibility specialists. In order to stay competitive in the IT industry, more and more vendors are completing their VPAT’s and some are using them as internal benchmarks for the continual refinement of the accessibility of their products. Information about the VPAT program, including links to the VPAT’s of many EIT vendors, is available through the section508.gov website.
There have been other initiatives, both in the public and private sectors, that will further help make EIT accessible while simplifying the life of Federal procurement officers. Several Federal agencies have laboratories that have been established to test products for compliance. Vendors have submitted their products for testing and received feedback on deficiencies and possible solutions. One agency is working with an e-learning vendor that has a particularly robust product. The vendor is developing timelines and specific actions that will be done to make the product accessible to all learners without sacrificing the interactivity. Several agencies are working together to share information on their concerns with the interpretation of Section 508. With the common goal of providing e-learning to all Federal employees that choose to take advantage of it, the continued cooperation between vendors and government and between Federal agencies will facilitate clearer practices and products both now and in the future.
 FAR Section 10.001
 FAR Section 39.203