4.1.3.1. Software
4.1.3.1. Software
Software is a key enabler for almost every system, making possible the achievement and sustainment of advanced warfighting capabilities. Development and sustainment of software is frequently the major portion of the total system life-cycle cost, and factors such as safety, security, reliability, interoperability, and insertion of new technology are considered at every decision point in the acquisition life cycle.
Software engineering requires unique technical management and contracting expertise to address architectures, requirements mapping, integration, technical data rights, assurance, and suitability for intended use. The more critical or complex the software acquisition effort, the more important it is to seek developers with demonstrated experience and knowledge (for more information on the uses of external resources see DAG section 4.1.4. Engineering Resources). The Program Manager should understand software development principles and best practices. To support program protection, program planning, contracting, configuration management, integration, test, and sustainment, the Program Manager should also have a working knowledge of software terms, tools, development models, risks, and challenges. These elements are major cost drivers for software in complex systems. Because key system capabilities are now more frequently implemented in software, techniques that estimate and assess function size, cost, performance, and risk are required for program planning, contract development, and progress assessment. Systems engineering (SE) principles and practices help anticipate, plan for, and mitigate challenges and risks in software development and system integration.
Establish the software acquisition strategy as early as possible to address function and component allocation to software and determine what is to be developed, what is provided as Government off-the-shelf (GOTS) software, commercial-off-the-shelf (COTS) software, or open source software (OSS), and what is a mix or hybrid. The strategy also incorporates plans for associated data and intellectual property rights for GOTS, COTS, and OSS.
Software-intensive acquisitions typically involve modeling and simulation (M&S) in engineering support roles specific to each phase of acquisition. Example uses of M&S in software acquisition are to:
- Study development cost by function,
- Study feasibility of the prospective system in the intended operational environment,
- Conduct engineering trade-offs and analyses of alternatives,
- Study and refine viability of planned software and computers to meet KPPs,
- Simulate undeveloped equipment during software testing, and
- Emulate the interoperability environment of the system during integration.
M&S activities are most valuable earlier in program planning as decision support tools and may be used iteratively to assess evolving functional architectures. The cost of M&S is allocated during initial program planning. Cost basis is the rationale supporting the balance between M&S cost and degree of needed risk reduction. M&S used by a Program Manager to make decisions should be verified and validated to the intended use in a time frame before assessment is needed. Data used by M&S to support assessments should have a known pedigree and should be adequate to the level of assessment. See DAG section 4.3.19.1. Modeling and Simulation for more information.
An incremental software development approach enables the developers to deliver capability in a series of manageable releases or builds to gain user acceptance and feedback for the next increment and reduce the overall level of risk. Frequent requirements and design-validation activities involving the end users and developers can assist the program to define viable increments of capabilities that have operational value for early fielding before the whole system capability is delivered. This incremental approach may not be viable when the end system is not usable until the entire set of essential capabilities is integrated and tested. For example, weapon systems are dependent upon software for real-time controls that can affect life and safety. As such, these weapon systems are required to be qualified and certified for security, safety, and interoperability before being released for operational use. In addition, safety and security assurance certifications and approvals require a predetermined objective level of rigor in verification, validation, and accreditation (VV&A) of these software releases. This VV&A is based on risk, not on the complexity, number of software lines of code (SLOC), or size of each software release. The Joint Software Systems Safety Handbook provides guidance for implementing safety-critical software designs with the reasonable assurance that the software executes within the system context and is at an acceptable level of safety risk.
Iterative development approaches should be planned well in advance and should consider impacts to other system elements of the functional architecture or other interconnecting systems. The program should focus on the allocation of functional architecture elements to the physical architecture and identifying the interdependencies and associated technical risks as part of determining the content for each iteration or build. Incremental or iterative development should be employed to carefully define the final end state of the supporting physical hardware elements when functionality or capability is to be added over time. Memory, processor overhead, and input/output capacity should be designed to support growth in capability. Implementing an open systems architecture (OSA) as part of the software design and development increases design flexibility, supports incremental deliveries, allows for opportunities to use COTS and OSS, facilitates future upgrades and modifications, and supports technology insertion (see DAG sections 4.3.18.4. Commercial-Off-the-Shelf and 4.3.18.15. Open Systems Architecture).
Software SE uses architectural modeling to develop and refine software requirements and to partition a system’s software into components and subcomponents. Software architectural decisions are driven by principles including co-location of external interfaces in one component to reduce risk of vulnerability, aggregating functions having higher mutual interaction, determining components that have well-defined interfaces to other components as candidates for technology insertion or OSS in support of OSA, and allocation of functionality to maximize use of COTS given acceptable risk.
When employing COTS software, criteria for selecting among competitive alternatives may not include details of commercial design or performance but should require ample evidence that the software is adequate for its intended use. Code-scanning tools should be used to help ensure that COTS software does not pose a security or software assurance risk. (See DAG Chapter 7 Acquiring Information Technology, Including National Security Systems and NIST-SP-800 series publications for additional information.) In addition, mitigation of security and information assurance risks associated with COTS software go beyond code-scanning techniques for their solution. Those risk mitigation efforts should be expanded to make use of activities identified in DAG section 4.3.18.24. System Security Engineering, as well as the activities discussed in DAG Chapter 13 Program Protection.
In programs for which software capability is procured as a service, the service-level agreement(s) (SLA) should reflect operational or field performance including all path constraints, such as satellite time delays, low data rate access, and intermittent service, as part of the operational environmental constraints and potential security requirements. These SLA provisions are important because service providers may not be willing to disclose details of their operations and staffing (such as overseas data centers or help desks).
It is not uncommon for weapon system acquisitions to contain a mix of Government-off-the-shelf (GOTS) software with complete technical data and software rights, other software items with restricted Government purpose rights, and software with virtually no rights other than the commercial license to use or access the software (see FAR Subpart 27.4). The Systems Engineer and Program Manager should be aware of the implications of these differences regarding acquisition and sustainment costs, performance, and the consequences on change control and sustainment of deployed systems. For deployed systems, the Systems Engineer should understand the system concept of operations (CONOPS), any maintenance plans, the targeted audience that is expected to use the software application, and level of training of the potential users. This understanding is necessary in order to effectively balance the cost, scheduling and potential risks in maintenance, training, and documentation.
As a best practice, the Systems Engineer for a software-intensive system, defined as “a system in which software represents the largest segment in one or more of the following criteria: system development cost, system development risk, system functionality, or development time” (DAU Glossary of Acronyms and Terms), should be well versed in the technical and management activities of computer programming, software project planning, and software configuration management, including defining computer software configuration items. The SE approach should include software engineers early in the acquisition life cycle to ensure software considerations are included in defining and allocating software-related requirements and generating cost and schedule estimates, especially for software-intensive systems. Software engineers are also needed to evaluate the developer’s software architecture, functional baseline, allocated baseline, and product baseline, documents, plans, and estimates, including M&S capabilities and facilities. Program-independent software engineers should support validation activities.
SE processes should be adapted to address considerations of hardware, software, and human systems integration. For example, risk management activities for hardware, software, and human systems integration should be combined so that risk mitigation plans can address all hardware, software, and human systems integration aspects of individual risks.
For software-related acquisitions the Systems Engineering Plan (SEP) should consider the following, as a minimum, for software SE planning:
- Software-unique risks
- Inclusion of software in technical reviews, with the addition of the Software Specification Review (SSR) as a precursor to the Preliminary Design Review (PDR) for software-intensive systems
- Software organization, integrated product teams, and relationships to interdependent organizations
- Software technical performance, process, progress, and quality metrics (see DAG section 4.3.4. Technical Assessment Process)
- Software safety, security, protection, and similar requirements, including processes, architecture, and interfacing systems
- Configuration management, verification, and validation of software integration labs/facilities used as tools for software development
- Open systems architecture, associated data rights, and sustainment considerations
- Automated test plans, development tools, and pedigreed data to support modeling of requirements, design, and environmental interfaces
- Software problem reporting and assessment, code development, build generation, and regression testing
- Software independent verification and validation (IV&V) to be accomplished, especially as it relates to contractor proprietary software
- Versioning, data control, and testing, especially for GOTS
- Verification of documentation, configuration management, test relevancy, and other considerations for legacy versus new software
Each of the Services provides additional guidance to assist the Program Manager, Systems Engineer, and Software Engineers on software-intensive systems:
Software Integrated within the Acquisition Life Cycle
Software considerations occur and vary throughout the acquisition life cycle, with specific activities associated with each acquisition phase described in Table 4.1.3.1.T1.
Table 4.1.3.1.T1. Acquisition Phase-Specific Software Considerations
|
Phase
|
Software Considerations
|
|
Materiel Solution Analysis
|
Some system requirements map directly to software requirements, while others can be implemented in hardware or firmware, providing opportunities for trade-offs and studies that optimize design and reduce vulnerabilities and risks. The ability to analyze and model options, and articulate the pros and cons of each, can have long-range impacts on the delivered system, suitability for intended use, and ultimate life-cycle cost.
|
|
Technology Development
|
Competitive prototyping of software-intensive systems helps to identify and mitigate technical risks. System prototypes may be physical or math models and simulations that emulate expected performance. High-risk concepts may require scaled models to reduce uncertainty too difficult to resolve purely by mathematical emulation. On occasion, competitive full-scale prototypes are needed to resolve cost/benefit alternatives between competing software-intensive system designs. Software programs typically conduct a Software Specification Review (SSR) to assess the software requirements and interface specifications for computer software configuration items, in support of the Preliminary Design Review (PDR). The software trouble reporting system is in operation and may be used to track any remediation in design and software code and unit testing.
|
|
Engineering and Manufacturing Development
|
To demonstrate that the detailed software design is complete at Critical Design Review (CDR), software documentation should represent the design, performance, and test requirements, along with the development and software/systems integration facilities to be employed in coding and integrating the deliverable software. Software and systems used for computer software configuration item development such as simulations and emulations, should be validated, verified, and ready to begin coding upon completion of the CDR, starting the implementation and synthesis of the software products. Software trouble reporting is used extensively to track problems and problem criticality levels. Problem report metadata should be selected so that the reports are relevant in development, test, and in operation to tracking and assessments. Typically, software functions vary in mission criticality so that problems reported in those functions are more critical to the system. There is legacy problem report tracking information that can be used to generally profile and predict which types of software functions may accrue what levels of problem reports. Program progress decisions can be made based on assessments of patterns of problem reports among software components of the system.
|
|
Production and Deployment
|
Software may be refined as needed in response to operational test and evaluation activities and in support of the Full-Rate Production and/or Full Deployment Decision and Initial Operational Capability.
|
|
Operations and Support
|
The In-Service Review (ISR) assesses user acceptance and potential upgrades on delivered software systems. A block change or follow-on incremental development may be defined that delivers maintenance, safety, or urgent builds and upgrades to the field in a controlled manner. Procedures for updating and maintaining software on fielded systems can require operators to download new builds or to install them from physical media, and may require more training. Procedures should be in place to support effective configuration management and control. There are inherent risks involved in modifying software on fielded systems upon which warfighters depend while engaged in frontline activities. Another aspect of the hardware-software interaction is that maliciously altered devices or inserted software can infect the supply chain, creating unexpected changes to systems. Vigilance is needed as part of supply chain risk management (see DAG Chapter 5 Life-Cycle Logistics and Chapter 13 Program Protection). Upon completion of development, the problem report tracking system can be used with other factors as legacy information to inform system and component upgrades. During Operations and Support phase, software problem reporting is continued.
|
Factors for Managing Software-Intensive Systems
Programs consider several factors when managing software-intensive systems, including the following:
Software Development Plan (SDP): The SDP as a best practice provides details below the level of the Systems Engineering Plan (SEP) and the contractor’s Systems Engineering Management Plan (SEMP) for managing software development and integration. The SDP Data Item Description (DID) DI-IPSC-81427A is a tailorable template and a useful starting point in defining a software development plan. The SDP provides the Systems Engineer with insight into, and a tool for monitoring, the processes being followed by the developer for each activity, the project schedules, the developer’s software organization, and resource allocations.
Post-Deployment Software Support (PDSS): The management of the software development process and the implementation of a process that ensures software supportability are among two of the most difficult challenges facing the Program Manager in management of software-intensive systems. The Program Manager should effectively address the issues of software supportability, the software test environment, and other equipment, material, and documentation, including data rights that are required to provide PDSS for those end users identified in the SDP or in other documents similar to the Computer Resources Life Cycle Management Plan. (For more information on PDSS see MIL-HDBK-347.) Successful PDSS planning should assist the Program Manager in controlling software life-cycle costs.
Data Protection and Software Assurance: These factors are defined as the level of confidence that software functions as intended and is free of vulnerabilities, either intentionally or unintentionally designed or inserted as part of the software code, throughout the acquisition life cycle. The Program Manager is responsible for protecting system data and software, whether the data are stored and managed by the program office or by the developer (see DAG Chapter 13 Program Protection).
Software Data Management and Technical Data Rights: Rights associated with commercial products can be highly restrictive and are defined in licenses that may restrict the number of copies made and ability to alter the product. Often there is no assurance of suitability for intended purposes and no recourse to the vendor. Open source, sometimes referred to as “freeware,” may not be free and may also have restrictions or carry embedded modules that are more restrictive than the overall package. The Program Manager, Systems Engineer, software engineer, and contracting officer should be familiar with the restrictions placed on each software item used in the contract or deliverable to the Government. The Program Office should determine the necessary intellectual property rights to computer software and should ensure that the intellectual property right should be determined in advance of the RFP and contract award and that they are acquired as needed, including:
- All requirements tools and data sets;
- All test software and supporting information necessary to build and execute the tests;
- All other software test tools such as interface simulators and test data analyzers whether custom-developed or not; and
- All information for defects remaining in the software upon delivery to the Government.
Software Reuse: The reuse of any system, hardware, firmware, or software should be addressed in multiple plans and processes throughout the acquisition life cycle, including the SEP, SDP, firmware development plan, configuration management plan, test plans (Test and Evaluation Master Plan, Software Test Plan, Independent Verification and Validation Plan), and quality assurance plans (system and software). (Note: Software reuse has traditionally been overestimated in the beginning of programs, and software reuse has often proven to be more costly than new software development. Software reuse plans should be monitored as a potential risk.) For more discussion of the reuse of software, see DAG section 4.3.18.15. Open Systems Architecture.
Software Acquisition and Sustainment Costs: Related costs should be accurately estimated in advance and then tracked to monitor execution within program cost constraints using relevant metrics (size, complexity, productivity factors, quality, development organization’s past performance/productivity, etc.).
Government and Industry Teaming: Teaming is needed in order for the Government to successfully acquire software-reliant systems with industry as a partner. As a result of the teaming agreement, the Government may be able to use the experience and expertise of its industry partner. Extensive teaming with industry makes it incumbent on the Government to ensure that it maintains current and applicable software engineering expertise.
Software Safety: Software safety is applicable to most DoD systems as a factor of the ubiquitous nature of software-driven functions, network connectivity, and systems of systems (SoS). Specific mandatory certifications such as “air worthiness certification” require attention early in the development cycle to ensure adequate documentation and testing are planned and executed to meet certification criteria. Systems Engineers are encouraged to check with certification authorities frequently because rules can change during development.
DAG Tutorial 
