• Increase automation and productivity in software development enabling
• quality and reliability improvements in software
• reduced time to develop and test software
• cost amortization through software component reuse
• Increase productivity of software users by enabling
• increased quality through specialization
• improved systems dependability
• improved focus on core business not software development
• Broaden software markets for US software producers by enabling
• creation of systematically reusable software componentsv
• increased software interoperation
• easy adaptation for use in foreign markets

• Practical technology for automated semantic based composition
• automated composition from independently developed software components
• component interoperation in any functionally compatible context
• systematic software reuse based on semantic characteristics alone
• Automated methods and tools for a sustainable commerce in software components
• increase the portion of the software process subject to automation
• reduce the direct involvement of software experts in custom development
• enabling tools and processes for component-based software
• Technology to overcome other potential barriers to component-based software
• commercial security
• location and use of relevant components
• component usability
• assurance of component reliability

This program proposes to accelerate the development of technologies enabling an emerging commerce in software components and movement to semantic-based automation in software development. The program focus is on software development technology for complex, large-scale commercial and industrial applications. This program seeks strategic software technologies that apply to a wide spectrum of industries or application domains. The program includes development of software tools, infrastructure and critical components needed in a variety of complex information processing application areas.

As a commerce in software components emerges, most software developers focus will shift from design and implementation processes to a focus on solving the application problem. Only the specialized group of component and infrastructure developers will focus on functionality, performance, interoperation, dependability, and security issues with the help of new tools increasing the level of automated support available. A new market for fine-grained software components and specialized high-performance tools would allow new software technologies to move rapidly into widespread use, expand markets, and reduce both time to market and cost of market entry.

A viable commerce in software components could allow the software industry to accelerate its evolution into increasingly specialized software suppliers which better understand specific industry needs. A software development tools industry, analogous to the machine tools industry would supply automated high-performance software tools and a software architectural consulting services. A software components industry analogous to a parts manufacturing industry would supply components incorporating generic capabilities needed across many industries. A component assembly industry would use these components in combination with domain or industry customized components.

Semantic-based software has its greatest potential in enabling a wide range of users with different needs and skills to effectively use the tools to customize software. Semantic-based development tools are tools that allow the developer to focus on the problem that he is solving and the functionality available not require him to focus on the programming process and syntax of the result. Spreadsheet applications provide a good example of an enabling semantic-based software tool for a relatively simple application area. Accounts can write applications using spreadsheets today that would have required teams of programmers 15 years ago. With more capabilities placed for industry and domain experts, specialized industry and enterprise application development could result in higher productivity and higher quality output available to industry.

Program Plan and Scope

All projects in this solicitation must fall within the technical scope defined by the technical goals. ATP will not fund proposals that are limited to personnel management techniques, manual approaches, data collection, or are incompatible with semantic approaches and fine-grained software components.

ATP expects semantic-based software development approaches and fine-grain software components to be key elements of the program vision and, therefore, key elements of its scope. Semantic approaches are those that allow the user to focus on the functionality of the components instead of syntax issues empowering users with new capabilities. Fine-grain components incorporate functionality which can be reused in multiple applications providing the potential economic benefits of widespread reuse.

Since ATP announced this program in early 1994, there has been a cultural shift as well as rapid technical innovation in the software industry. A robust information infrastructure, and the software needed to support it, is critical to all industries today. The World Wide Web, Java and changing software development methods have opened industries to new ideas in acquiring and distributing all information products including software. New business models are still catching up with this technical revolution.

This change has provided a tremendous opportunity for the software industry and for the Component-Based Software program. Today, in contrast to two years ago at the start of the program, there is wide discussion of what a software 'component' might usefully consist of and how a commerce in software components might evolve. Because the 'shrink-wrap' or package software industry represents a visible, low cost market, a component-based software industry may evolve to customize applications or supply new applications in this area. If this commerce in components is to evolve to meet industrial demands for reliability, security, scalability and extension with new functionality in large, complex systems, addressing the goals of this program quickly is critical. Less ambitious approaches aimed at the package software industry which do not address industrial requirements may preclude more innovative options.

While most of the funded projects focus on tools supporting semantic-based development and composition of components in an application domain or industrial area, the Component-Based Software program lacks sufficient projects in the following areas:

• automated semantic-based composition of cross-domain  components
• automated semantic-based composition for enterprise-wide systems
• infrastructure for commerce in semantic-based components
• commercial security
• location and use of relevant components from semantic characteristics
• revenue collection for fine-grain components
• component usability
• sufficiently abstracting non-semantic details of component implementation

These areas are important for long term program success and have not been sufficiently addressed in current funded projects.

Program management of the funded projects will emphasize four points: synergy among projects, effective transfer to widespread use, response to opportunities created by newly available infrastructure, and relevance to end-user applications. Coordination among the projects and with the impacted industries is especially important. Overcoming nontechnical barriers requires involvement and visibility by a wider community of informal participants and affected industries, platform providers, potential horizontal product producers, and custom and vertical market providers. The Component-Based Software program has held two annual workshops to supplement bilateral interproject interactions. These workshops have provided an opportunity for projects to report progress toward program goals, to discuss new and continuing technical and business opportunities, and for ATP to seek advice for updating program plans.

It is important that participants be willing to cooperate where appropriate to achieve interoperability and other benefits of synergy. Projects that are critical to overall program success require the highest degree of commitment to interproject coordination and cooperation. Joint ventures involving normally competing companies are welcome.

Benefit to the US economy

For complex large-scale applications, custom software has three important characteristics: design costs are frequently over a million dollars, marginal costs of producing additional units are near zero, and typical production quantities are one. Because these complex systems are custom products, low replication costs are not exploited to recover design costs. Industry and government can no longer afford to wait years for the development of large systems which all too often are operating failures.

Large custom systems are also notoriously hard to maintain and evolve to meet changing requirements. The recent wave of concern over the year 2000 problem is a prime example of this problem. Because large systems embed their use of dates, identifying those portions of any large system relying on a two digit date is an expensive, error prone operation for many legacy systems.

Market pressures have motivated a multilevel supply chain structure in other industries to recover costs, improve quality through specialization, and enable rapid assembly and reconfiguration from purchased components. These pressures increasingly apply to the software industry. Software has to change as quickly as all other elements of the evolving virtual corporation to meet industry's needs. The primary barriers to commercial reliance on software components are technical: we do not know how to independently produce reusable software components meeting industrial needs.

Alone, technology developed in this program is unlikely to change the entire basis of competition within the software industry. To have a lasting impact, technical success must lead to products incorporating methodologies and approaches that can sustain significant use. In 1994, when this program started, ATP identified four nontechnical barriers which might delay the emergence of a software component marketplace:

• initial benefits of reduced costs, increased reliability and interoperation, or promises of horizontal markets may be inadequate to overcome 'first user' risk.
• the greatest benefits require a sustained marketplace in specialized components supporting a change to predictable costs and schedules from component assembly
• value in legacy software systems will not be easily abandoned
• pricing and revenue collection schemes may be inadequate

Since 1994, trends in the use of "applets" and "plug-ins" for extending consumer-oriented software and in the use of large-scale components (e.g., databases) among commercial users have emerged. These software trends illustrate that the demand exists for software components if the cost and quality of the components meet user needs and if the software components can be easily and safely integrated to meet the user's needs. The technical goals of the Component-Based Software program address these barriers and should help accelerate these trends by establishing specific quality improvement expectations and encouraging development of an effective infrastructure for commerce in software components.

Once a component-based marketplace achieves critical mass, large economic benefits will accrue to customers of the software industry. Automated semantic-based fine-grained composition generates widespread reuse, which enables technical specialization and competition, which in turn encourages quality improvement. Increasing automation and declining duplication of effort should decrease costs while raising industry capacity. Automation eliminates the error-prone aspects of software implementation and can assist in requirements validation. With widespread commerce in software components, the entrepreneur with a great technical idea could rapidly distribute an innovative component in a broadly reusable product. A multilevel value added supply chain in software would provide a channel for rapid transition of new technologies to widespread use. In this sense, the software components industry would come to resemble the electronic components industry with its rapid infusion of new technology and seemingly endless succession of order of magnitude advances.

Although potential economic benefits from this program derive primarily from interoperation and quality improvements through specialization, benefits from reuse alone are significant and more easily calculated. Conservatively, assuming that development of a reusable component would double its cost and that each reuse would cost a quarter as much as redeveloping the component, reusing a component ten times would lower overall costs by 60 percent. With even a five percent market share, the savings in software development today from reuse would exceed be in the billions annually. Because technologies sought in this program have the potential to change the entire basis of competition in software, estimating, long term benefit from the program would be as difficult as attempting to value the electronics industry of today based on the cost of transistors in 1960.

If the US is first to move to software based on automated semantic processing of fine-grained components, it would gain a substantial advantage that would be difficult for another nation to duplicate. The software expertise currently concentrated in the US together with a first producer advantage accruing to US developers, and an effective competition among US firms in world markets would represent a significant entry barrier for foreign producers of software component and tool products.

The trend toward Component-Based Software is a paradigm shift that will help the US maintain its leading position in software technology. As with other industries, the US faces significant competition from other countries which can offer programming services at a fraction of their cost in the US. The US will not likely retain jobs for low value added software development tasks. The Component-Based Software paradigm could enable US software professionals to concentrate on higher value, more specialized roles in software architecture, design, component development, automated implementation technologies, tool building and domain modeling.

Good Technical Ideas

The purpose and role of software is management of complexity. Hardware technology is the domain of fabrication, production, performance, capacity and physical products; ultimately, its capability is limited by physical reality. Software technology is the domain of problem solving techniques, design methods, decision making, agility, change management, and information products; ultimately, its capability is limited only by imagination.

Deficiencies in software are a reflection of human limitations in understanding and managing complexity during the software development process. These limitations are aggravated by the current emphasis on issues of syntax, representation and completeness, which relate to the artifacts and processes of software construction. The goal of this program is to move the focus to issues of application needs, software architectures, and the semantic aspects of design. This program emphasizes semantic processing, automated composition and synthesis, and fine-grained interoperation. It seeks innovative technologies that will raise the developer's role in building applications to that of specifying requirements, constraints, functionality, and satisfaction; and that will contribute to a critical mass of reusable components The program seeks automated tools and tool fragments that can work cooperatively in a context where the typical software developer has minimal direct involvement with issues of syntax, form, representation, and implementation.

Previous approaches to software development have not produced the demonstrable increases in reliability, software developer productivity or software utility. For a given functionality, the perceived costs in labor, frequency of errors, demands on machine resources, and response time relative to hardware performance has been rising rather than falling. Software systems must incorporate innovations on an automated basis to achieve the improvement needed in a useful timeframe.

The pacing technologies for this program will support the automatic composition of independently developed software and enable interoperation of fine-grained software in any functionally compatible context. Listed below are a few examples of potential ideas in selected areas of software technology. The examples are for illustration ONLY. ATP is not suggesting that they are the best ideas or anticipate that these ideas will be the focus of actual proposals:

Combine proof techniques and other formal methods to identify, retrieve and validate logical compatibility among software components based on formal specification of their functionality. Combine this approach with transformation techniques to automatically convert the components to implementations with compatible interface representations at each point of composition. Extend this approach to include formal specification of performance requirements, resource constraints, and optimization criteria and to employ compiler optimization techniques to automate their processing.

Alternatively, develop automated semantic-based design tools so that software development can begin at the semantic design level. Here the marketplace of components could be design elements such as domain models. The software synthesis process would involve automated inference to generate both the component implementations and the 'glue' for existing components.

Combine simulation, formal methods, expert systems, and derivation technologies to overcome the requirements validation problem and to enable design by exception. This would support rapid prototyping of complete simulations of applications given only a few key requirements.

Industry Commitment

During the development of this focus program, ATP received 56 white papers discussing industry needs for improved software development technologies. Common concerns across these white papers included the need for more systematic reuse, interoperation, automation and semantic approaches.

In early 1994 and again in early 1995, ATP held regional workshops on software technology to discuss white papers, alternative focus programs and industry participation in possible ATP programs within software and information technology. Total attendance exceeded 400 individuals representing over 275 organizations.

Following the first set of regional workshops in 1994, NIST announced the Component-Based Software focus program in April, 1994. ATP held the first solicitation in 1994 and the second in 1995. The first two competitions used an abbreviated proposal process. ATP invited proposal writers to submit abbreviated proposals in a specified format for early feedback during the proposal writing process. There were 110 abbreviated proposals and 96 full proposals submitted in the two competitions combined. The evaluation boards selected 18 of those proposals for award.

Another view of industry commitment to the Component-Based Software paradigm is the rapid acceptance across the diverse elements of the software development and user communities of Java, the World Wide Web, Active X and other initial approaches to a Component-Based Software paradigm. The breadth and depth of the interest generated in these technologies indicate the increasing critical nature of the software development crisis and the need for technologies satisfying the Component-Based Software goals. This program and the software industry recognize the importance of continued advancement in the basic infrastructure for software development technology for industry to take advantage of the economic opportunity available from the use of software components.

ATP Will Make a Difference

Unlike mission agency programs where solving a government problem has been the overriding concern, this effort emphasizes productivity improvement.

Problems addressed in this program are industry wide. There is a broad consensus in the software industry that the key problems of the industry involve issues of poor quality, missed delivery schedules, high cost, poor interoperation, and a lack of a viable software component industry.

The value of software solutions depends as much on the breadth of their use as on any intrinsic value. In an industry dominated by very small companies, the high risk high payoff technical solutions needed for addressing the problems of large, complex systems usually lie beyond the capabilities of any one company or project.

Rewards for innovative software technology development seldom accrue primarily to the developing company. Although it is expensive and high risk to develop new software technologies, there is often minimal risk and relatively low cost for duplicating that effort. This situation could provide incentive for competitors to cooperate within an ATP program to share the cost of solutions for mutual benefit.

Software technology provides infrastructure in virtually all industries today. Thus, software technology can have enormous benefit to the nation and to all industries that make or will make extensive use of software. Most of the ultimate benefits accrue to the users of software intensive products. ATP provides an opportunity for users and platform providers to work with software technology providers to ensure that the results of the project meet industry needs.

This program will build on a number of automated implementations developed under DARPA programs. Its focus, however, differs from other efforts because its technical focus is on semantic methods and automation of the implementation process. ATP also adds three important dimensions not normally considered in mission agency programs: a focus on productivity and cost effectiveness rather than quality alone, planning for commercialization before and during technical development and inclusion of technical requirements that arise from the anticipated commercialization process.

Date created: February 1997
Last updated: April 12, 2005