- The Requirement
for a Systems Engineering Plan (SEP)
- Q: Who directed and where
is the requirement or policy?
- A: At the time, the Acting Undersecretary
of Defense for Acquisition, Technology, and Logistics,
issued the policy in a USD(AT&L) systems
engineering policy memorandum, dated Feb 20, 2004,
requiring:
"All programs responding to a capabilities or
requirements document, regardless of acquisition category,
shall employ a robust systems engineering (SE) approach
that balances total system performance and total ownership
costs within the family-of-systems, system-of-systems
context. Programs shall develop a Systems Engineering
Plan (SEP) for Milestone Decision Authority (MDA) approval
in conjunction with each Milestone review, and integrated
with the Acquisition Strategy. This plan shall describe
the program's overall technical approach, including
processes, resources, metrics, and applicable performance
incentives. It shall also detail the timing, conduct,
and success criteria of technical reviews."
- Q: Is DoDI 5000.2 going to
be updated to reflect the SEP requirement?
- A: The USD(AT&L) systems
engineering policy memorandum, dated Feb 20,
2004, states, "Toward that end, I am establishing
the following policy, effective immediately and to
be included in the next revision of the DoD 5000
series acquisition documents: …." OUSD(A&T)
Systems and Software Engineering has drafted recommended
language for inclusion in the DoDI 5000.2 update.
- Q: Does the SEP requirement
apply to AIS and MAIS programs?
- A: Yes, the USD(AT&L) systems
engineering policy memorandum, dated Feb 20,
2004, states, "All programs responding to a
capabilities or requirements document, regardless
of acquisition category, shall employ a robust systems
engineering (SE) approach that balances total system
performance and total ownership costs within the
family-of-systems, system-of-systems context. Programs
shall develop a Systems Engineering Plan (SEP) for
Milestone Decision Authority (MDA) approval in conjunction
with each Milestone review, and integrated with the
Acquisition Strategy."
- Q: Why does my program need
to write a SEP, the next milestone (MS) is not for many
years? [This answer addresses the requirement for MS
A, B, and C.]
- A: The SEP is required because it
captures the program's overall technical approach and
integration to program planning required to achieve
the next MS. A good SEP captures and shares a roadmap
of the technical execution of your program. Program
personnel should use the SEP as a reference to access
information relative to the program. The SEP should
be continuously updated as plans are solidified or
modified, activities move from plans to historical
facts, known risks are mitigated or significantly change,
new risks are identified, new tools and technologies
are adopted, and as a myriad of other factors cause
an adjustment to the way the program technical activities
should best be managed.
- Q: Do we need a SEP for post-MS
C programs?
- A: Yes. A fundamental change in DoD
policy is the designation of the program manager as
the life cycle manager (Total Life Cycle Systems Management
(TLCSM)), responsible for effective and timely acquisition
and sustainment of the system throughout its life cycle.
Systems engineering continues through production, fielding,
and deployment. Manufacturers will propose changes
through Engineering Change Proposals and users will
introduce new requirements that must be analyzed as
part of the SE process. In addition, the program manager
is responsible for providing the needed product support
capability to maintain the readiness, sustainment,
and operational capability of a system. Effective sustainment
of a system during operations and support includes
a multi-disciplined effort to: collect and triage all
service use information, analyze root causes of problems,
assess suitability and effectiveness trends, develop
solutions, and continually manage the fielded system
configuration and associated processes. Thus, the need
for technical planning to ensure the application of
a disciplined, systems engineering approach continues
beyond MS C.
- SEP Approval/Approval
Procedures
- Q: How often do we need to
update and resubmit the SEP?
- A. The USD(AT&L) systems
engineering policy memorandum, dated Feb 20,
2004, states, "Programs shall develop a Systems
Engineering Plan (SEP) for Milestone Decision Authority
(MDA) approval in conjunction with each Milestone
review,…." Notionally that would be a
new or updated SEP prior to Milestone (MS) A, MS
B, and MS C. If there are major program changes,
such as changes to the acquisition strategy or technical
approach between scheduled Milestones, the SEP should
be updated to reflect such changes. SEP status should
also be reported at progress reviews held by the
MDA in between MS reviews.
- Q: Who is the approval official
for the SEP?
- A: The Milestone Decision Authority
is the SEP approving official. SEPs for ACAT ID and
IAM programs are approved by the USD(AT&L) with
recommendations from the appropriate Overarching Integrated
Product Team (OIPT) Lead. The component acquisition
executives set their
own SEP approval policies for programs under their
decision authority.
- Q: Who signs the SEP from
OSD?
- A: The SEP approving official is the
designated Milestone Decision Authority. For certain
ACAT ID programs that is USD(AT&L) and for certain
ACAT ID and ACAT IAM programs that is ASD(NII).
- Q: What are the signature
blocks for OSD approval?
- A: Names change as appointments are
confirmed or career personnel are reassigned. Either
leave the signature blocks blank and let the OSD staff
insert them during staffing or consult with the OSD
staff immediately prior to submission.
- Q: What is the relationship
between the CAE SE, PEO SE, PM SE, and ODUSD(A&T)
SSE staff?
- A: Organizationally, senior component
SE representatives participate with ODUSD(A&T)
SSE at the Systems
and Software Engineering (SSE) Forum. The Forum
was established in response to the dictum in the USD(AT&L) systems
engineering policy memorandum, dated Feb 20, 2004.
The Forum meets monthly (initial meeting was April
22, 2004) to discuss SE revitalization and develop
collaborative SE guidance. Programmatically, program
managers are encouraged to establish a Systems Engineering
(SE) IPT to perform the technical planning necessary
to develop a comprehensive SEP. The SE IPT should include
all of the program's engineering resources and stakeholders.
- Q: What is the relationship
of the ODUSD(A&T) SSE and ASD(NII) SE staff?
- A: Systems and Software Engineering
(SSE) provides ASD(NII) systems engineering support
across all ACAT ID and IAM and special interest programs
assigned to ASD(NII). The ASD(NII) staff performs a
specialized function relating to net-centric aspects
of programs, focusing on interfaces and the Global
Information Grid (GIG). The SSE and NII staff collaborate
on program support activities, providing the necessary
insights in understanding network, communications,
information assurance, and compliance with the Net-Ready
Key Performance Parameter (NR-KPP).
- Q: What is the timeframe
for OSD to approve our SEP?
- A: Formal SEP submissions for staff
approvals should be completed within 6 weeks from receipt
of the document. Early involvement and participation
in Systems Engineering (SE) IPTs, as well as reviews
of draft SEPs, should reduce turn-around time and rework.
- Q: Why is the SEP a "living" document?
- A: A good SEP provides a roadmap to
the technical execution of a program. Program personnel
should use the SEP as a reference to understand the
overall technical approach to a program. A SEP should
be continuously updated as plans are solidified or
modified, activities move from plans to historical
facts, known risks are mitigated or significantly changed,
new risks are identified, new tools and technologies
are adopted, and as a myriad of other factors cause
an adjustment to the program's overall technical approach.
- Q: If the SEP is a living
document, do I need to get it re-signed by OSD after
every change?
- A: No, SEP changes should be tracked
and kept under local configuration control, and do
not require resubmittal to the MDA for approval. It
is recommended that a courtesy copy be made available
when major changes are made, so that the cognizant
approving organization is kept current on the program's
systems engineering plans, and is available to assist
if requested. However, SEP updates are required as
formal submission to the Milestone Decision Authority
(MDA) prior to each Milestone review, or if there are
major program changes between scheduled Milestone reviews.
- Q: Can we get feedback from
ODUSD(A&T) SSE on SEPs submitted?
- A: As part of its support role, ODUSD(A&T)
Systems and Software Engineering, Assessments and Support
(ODUSD(A&T) SSE/AS) provides feedback for out-of-cycle
draft SEPs directly to the requesting organization.
If a program manager sends a draft SEP for review,
the program manager will get a direct response. If
a Component Acquisition Executive (CAE) provides a
SEP for approval, the response is provided to that
CAE.
- Q: Who do I contact to start
early informal dialogue and coordination?
- A: Contact the Program Support Team
Lead (PSTL) in ODUSD(A&T) Systems and Software
Engineering, Assessments and Support (ODUSD(A&T)
SSE/AS) whose portfolio includes your program or type
of program. SSE/AS program support teams are organized
around product lines. There are currently eight PSTLs
focused on the following product lines: Fixed Wing
Aircraft; Rotary Wing Aircraft; Missiles and Unmanned
Aircraft; Land Combat Systems; Ships; Command, Control,
Intelligence, Surveillance, and Reconnaissance (C2ISR);
Business Systems; and Communications Systems. You may
contact these teams through the following address: atl-as@osd.mil.
- SEP Development/Format
- Q: Do you have an example
of a good SEP?
- A: Not at this time. There are a select
number of SEPs that are candidate examples, and we
are in the process of working with the cognizant PMs
to sanitize their documents to be used as training
aids. These, and others to follow, will be available
as examples in the future.
A few cautions about using "examples:"
** Programs in different domains will use different
processes.
** Any given SEP is an operating plan for how a program
will execute its systems engineering. This plan can
vary greatly from program to program based on scope,
risk, life cycle phase, implementing organization,
and multiple other factors. Without knowing these underlying
influences, the technical planning in a SEP may or
may not apply to another program.
- Q: How much detail is needed?
- A: Only as much detail as is required
to convey to program stakeholders what the operational
plan is for the technical management of the program.
There should be sufficient detail for the reader to
understand who does what, when, how, and where—and
what control mechanisms are used to manage it all.
- Q: Do we need an SE IPT to "work" the
development of our SEP?
- A: Program managers are encouraged
to establish a Systems Engineering (SE) IPT to perform
the technical planning necessary to develop a comprehensive
SEP. The SE IPT should include all of the program's
engineering resources and stakeholders.
- Q: If the SEP or an update
is required at each Milestone, what is the expected content
for each Milestone?
- A: For the first SEP submittal, the
content should focus on a plan of action for how the
program office intends to manage its technical efforts
for the life cycle phase following the Milestone (MS).
Past SE activities should be briefly addressed to demonstrate
how you got to this point. Long-term planning considerations
for future phases should be briefly covered. For example,
if approaching MS B the plan should summarize systems
engineering activities used in selection and refinement
of the material concept, maturation of enabling technologies,
and the status of the systems engineering products
that are in place to support the milestone decision.
It would then provide detailed discussions of the systems
engineering activities planned for the system development
and demonstration (SDD). Finally, it should address
production, deployment, operations and support considerations
to the extent these phases are influencing systems
engineering in the SDD phase (See the SEP
Preparation Guide, v2.0, for more specifics). For
a re-submittal, the content should include an update
of the plan of action for program technical planning.
For example, assuming the schedule has changed since
the last submitted SEP and more details have come available
regarding technical reviews etc., this information
would be updated accordingly.
- Q: How many pages should
the SEP be?
- A: SEP length will vary by program
scope, maturity, and risk. It should cover the overall
technical planning effort for the program and may cite
subordinate plans when the program determines that
additional planning detail is required.
- Q: How should related documents
be referenced vice including everything in the SEP?
- A: Summarize the referenced material
in the SEP, discussing all linkages and specifics of
how other documents support the SEP. The referenced
material should become part of the overall SEP, included
as attachments, annexes, hyperlinks, or as a minimum,
mapped under a single configuration control scheme
so that all information will be maintained current
and remain consistent with the program's overarching
technical planning.
- Q: Can we use existing related,
relevant documentation rather than redeveloping or reformatting
existing documentation?
- A: Existing documentation can be used
to the extent it captures relevant systems engineering
technical planning. For example, the program may have
a contractor's plan for technical planning. However,
it generally does not include the Government program
manager's plan for technical management of the program.
We have also found that prime contractor's technical
plans are often weak in flowing down systems engineering
processes and requirements to subcontractors and vendors.
Follow the guidelines provided in the question above
regarding referencing other documents, as to proper
referencing and maintaining the supporting information.
- SEP/SEMP Relationship
- Q: What is the difference
between the SEP and the SEMP?
- A: In the USD(AT&L) systems
engineering policy memorandum, dated Feb 20,
2004, we chose to call the document a "Systems
Engineering Plan (SEP)." While the Systems Engineering
Management Plan (SEMP) is a traditional term used
previously in DoD and more common with industry,
the SEP
Preparation Guide, v2.0, clearly states we have
no preference regarding the title of your technical
planning document. The important point to keep in
mind is the content of the document, not the title
of it.
- Q: Do I need both a SEP and
SEMP, or can they be combined into one document?
- A: This question assumes the SEP is
a "government" product and that the SEMP
is a "contractor" document; however, this
is not a valid distinction. That said, the decision
on combining the program office's technical planning
with the contractor's technical planning is an individual
program decision based on resource limitations and
other factors unique to the situation. If you choose
to have one document, it must capture the technical
planning for the entire program. If you choose to have
two documents, for example, a "SEP" for the
program office responsibilities and a "SEMP" for
the contractor effort, the two documents must have
a common systems engineering thread, linking the technical
planning from the program office down to the lowest
level supplier. The bottom line is that we expect a
program to have a "shared vision" of the
overall technical planning effort, regardless of how
many documents contain that shared vision.
- Q: Is the contractor SEMP
adequate to meet the requirement?
- A: A contractor's systems engineering
plan is acceptable only if it addresses the overall
program's systems engineering technical approach across
the government, prime, and sub-contractor domains.
Prime contractor systems engineering plans typically
only address the technical planning associated with
the prime contractor's effort, and do not necessarily
encompass the entire technical effort associated with
a program, such as requirements trade space for which
the government retains authority, government furnished
equipment, integrated developmental and operational
test requirements and planning, etc. We have also found
that prime contractor's documents do not adequately
flow down the systems engineering process and requirements
to subcontractors and vendors. However, if your program
has an existing contractor systems engineering plan,
you may cross reference that technical planning to
the overall technical planning for the program described
in the SEP.
- SEP Technical
Issues
- Q: Where do I find the statutory
and regulatory requirements for my program? How do I
know if I have found them all?
- A: The DoDI 5000.2 is the primary reference and starting
point for these types of requirements. Service-unique
or commodity-oriented related requirements should
also be addressed. However, simply providing a listing
of these requirements in the SEP is not sufficient.
The technical planning for the program, as documented
in the program's SEP, should include discussion of
how the program plans to interpret, apply, and comply
with these kinds of requirements from a technical
perspective. Also key in the SEP is discussion of
how these requirements will be integrated with other
requirements (e.g., KPPs, specified and derived requirements)
in an integrated framework to ensure requirements
traceability throughout development, test, and evaluation.
- Q: Most, if not all, programs
employ requirements
management (RM) tools (e.g. Dynamic Object Oriented
Requirements System (DOORS) or similar RM tools) to provide
requirements management and traceability for stakeholder,
statutory, regulatory, and derived requirements. Why
must this tool be described in detail again in the SEP
rather than referenced?
- A: Specifics about any particular
tool can be referenced and need not be described in
detail in the SEP. What should be discussed in detail
is how the tool will be used to manage requirements
as they are iteratively and recursively analyzed, traded,
decomposed, and allocated across the system. How the
program will ensure requirements traceability (both
up and down the system) should also be discussed in
detail.
- Q: How do we get the contractor
to support the added SEP workload if they are already
under contract and have no funding slack?
- A: It is expected that most, if not
all, industry best practices includes technical planning
as part of their overall engineering and technical
effort. It is this planning that is envisioned as being
captured in the SEP. If the contractor has not conducted
technical planning or has not captured it in written
form, then there are likely more serious issues on
the program beyond "added SEP workload."
- Q: What is the definition
of technical baseline referenced in the Defense Acquisition
Guidebook and the SEP Preparation Guide?
- A: Virtually all industry best practices
and systems engineering standards include establishment
and control (configuration management) of system instantiations
during development. These instantiations could include
logical solution representations and physical solution
representations at various levels of the system hierarchy—functional
and allocated baselines. During development, establishment
and multi-disciplined review of these baselines is
a key element of event-driven technical reviews. Ultimately,
the technical baseline evolves to include a product
baseline that addresses the build-to specifications
and drawings together with the associated production
and support processes.
The specific definitions follow:
** Technical baseline: 1. A specification or product
that has been formally reviewed and agreed upon, that
thereafter serves as the basis for further development,
and that can be changed only through formal change
control procedures. 2. A document or a set of such
documents formally designated and fixed at a specific
time during the life cycle of a configuration item.
3. Any agreement or result designated and fixed at
a given time, from which changes require justification
and approval. (Adapted from IEEE STD 610.12 (1990))
** Functional baseline: The initially approved documentation
describing a system's or item's functional characteristics
and the verification required to demonstrate the achievement
of those specified functional characteristics. (MIL-STD-480B)
** Allocated baseline: The initially approved documentation
describing an item's functional and interface characteristics
that are allocated from those of a higher level configuration
item (CI), interface requirements with interfacing
configuration items, additional design constraints
and the verification required to demonstrate the achievement
of those specified functional and interface characteristics.
(MIL-STD-480B)
** Product Baseline: The initially approved documentation
describing all of the necessary functional and physical
characteristics of the configuration item (CI); any
required joint and combined operations; the selected
functional and physical characteristics designated
for production acceptance testing; and tests necessary
for deployment/installation, support, training, and
disposal of the CI. This baseline is typically initiated
at the Critical Design Review (CDR) and finalized at
the Physical Configuration Audit (PCA), and normally
includes product, process, and material specifications;
engineering drawings; and other related data. (INCOSE)
- Q: How should the program
office describe their technical reviews when they are
using agile software development methods?
- A: To be of value in determining the
technical maturity of a system, technical
reviews must be event driven. Event-driven reviews
are not only required by DoD policy (reference the
USD(AT&L) policy
memorandum, dated Oct 22, 2004), but are also a
fundamental part of most industry best practices. Programs
using any development method should ensure that method
provides for event-driven technical reviews (that is,
reviews having entry and exit criteria quantifiable
in technical baseline completion terms). Methods that
do not have such provisions, while seemingly "agile," often
lack the ability to measure technical maturity to objective
measures.
- Q: If we truly convert to
non-schedule driven milestones, how do we manage against
contractor burn rates, fixed budgets, and iron schedules
in the acquisition program baseline (APB)?
-
A: The purpose of event-based technical
reviews is to provide an objective measure of
technical maturity to provide the sound, technical
basis against which to manage technical progress
achieved at the cost and schedule. It is expected
that having this insight to technical maturity against
objective measures, the program will be better able
to understand the true implications of contractor
burn rates, fixed budgets, and iron schedules.
- Q: Are PEOs, CAEs/SAEs, and
OSD really signed up to the idea that schedule is malleable?
- A: Knowledge of schedule, isolated
from the understanding of technical maturity (maturity
of the technical baseline) at a point in time, is not
viewed as good systems engineering or program management.
Schedule is no more malleable than maturity of the
technical baseline, hence the emphasis (and policy)
for event-based technical reviews.
- Q: Do you have a rule of
thumb based on historical evidence, regarding the time
between technical reviews?
- A. The time between technical reviews
is wholly dependent on the scope and risk of a program
and how a program lays out its technical baseline approach.
Functionally complex programs will generally require
more time to evolve their technical baseline—reviewed
at a system functional review (SFR). Programs having
many configuration items (CIs) will have a more complex
system decomposition and allocation, requiring more
time to achieve the entry criteria to meet multiple
preliminary design reviews (PDRs) and will require
more time to get to build-to (or code-to) specifications
(product baseline) for those CIs' critical design reviews
(CDRs).
- Q: Is there an example of
how Earned Value Management can take advantage of good
systems engineering planning?
- A: Too often, systems engineering
is viewed as not having a defined "product." This
results in systems engineering being reflected as level
of effort (LOE) in many Earned Value Management (EVM)
plans and cost accounts. Good systems engineering planning,
including the identification and management of the
technical baselines (functional baseline, allocated
baseline, and product baseline) should call out specific
systems engineering products in the form of these baselines.
When aligned to the system work breakdown structure
(WBS), the technical baselines should be the basis
for earned value cost accounts and provide a product-driven
view of managing systems engineering costs (and value).
Additionally, when the maturity of these technical
baselines are entry criteria for event-based technical
reviews, earned value can provide critical insight
to technical progress against accumulated cost and
schedule measures.
- Q: How should Engineering
and Manufacturing Readiness Levels (EMRLs) be addressed
in the SEP?
- A: EMRLs and Technology Readiness
Levels (TRLs) are both ways of looking at technical
maturity in an objective way. Good systems engineering
uses the technical reviews and their entry criteria
with the same goal—to assess technical maturity
against objective measures. EMRLs and TRLs can best
be integrated into a strong systems engineering approach
by aligning these views with the technical reviews
planned for the program.
Additionally, any review of technology or manufacturing
readiness should occur in the integrated context of
a technical review, where other driving aspects such
as requirements maturity, earned value, hazard risks,
supportability, human factors, integrated test, etc.
are also addressed. Any program planning to use EMRLs
and/or TRLs should review these criteria in light of
planned technical review entry criteria (e.g., required
percentage of build-to and code-to packages complete
for CDR entry). The SEP should detail all of the criteria
to be applied at the technical review, including the
EMRL and TRL views.
- Q: How should the SEP be
used to help with development of a program's Integrated
Master Plan/Integrated Master Schedule or Single Acquisition
Management Plan?
- A: Sound technical planning, including
systems engineering best practices, should be the basis
for a strong program plan and schedule. From this standpoint,
a program's SEP would serve as a key input to planning
and scheduling of overall program activities in the Integrated
Master Plan/Integrated Master Schedule (IMP/IMS) or
Single Acquisition Management Plan (SAMP). An example
of this relationship of systems engineering to program
planning is the definition of the program's critical
path. While a program's critical path is defined in
the IMS, many of the key events that comprise that
critical path would be derived from the detailed technical
planning found in the SEP.
|