Contracting Scorecard

        Click here to download the MS Word (231 KB)        

This document describes the background of a contracting scorecard approach and identifies a list of common factors that are applied to the contracting scorecard. The list, which is summarized for convenience in two tables, contains a wide range of factors, and you should tailor it for your situation by deletion, extension, specialization, or addition.

The contracting scorecard uses the dimensions of a balanced scorecard. The balanced scorecard approach was chosen because it is designed to explore both short-term value creation and long-term business success of the many aspects of business and provide traceability between business objectives and a "vital few" outcome and output measures.

The contracting scorecard is essentially a method of representing and working with the wide range of strategic, operational, and financial implications that your organization or project has as a consequence of contracting and the supplier relationship. The actual representation and use of the data will change and evolve through the life of the contracting process An example of a template is given in Table 1, but the concept is intended to be flexible and should not be limited by this template; evolve it to represent the information you require to support each activity of the contracting process and to maintain traceability back to the original business objectives.

The contracting scorecard representation does not imply that you have to use a sophisticated balanced scorecard mechanism to follow this contracting process, neither is it intended to impose a level of sophistication on your analysis or the number of factors you define. It is primarily intended as a way that you can expose, evaluate, and maintain focus on the most important factors that are affected by contracting a product or service to a supplier. Therefore, you should use the scorecard from the earliest point of developing the essential criteria in the make or buy analysis to managing the relationship. Choose an approach consistent with your organization and limit analytic use of the scorecard unless you have reliable measures with which to work. Focus on what is essential to consider and keep it as simple as possible.

The standard balanced scorecard of a business is generally an optimization of four sometimes-conflicting business issue areas: financial, customer, internal process, and learning and innovation. To this general approach, the authors added the sourcing liability issue area, which exposes the risks unique to a contracting arrangement, for example, misconduct, technology diffusion, end-product degradation, supply disruption, and inability to delivery.

Table 1. Example of a Contracting Scorecard Template

Business Issue Area/Factor		Performance Indicators		Gaps/Actions
		Outcome Measures (Lag indicators)	Output Measures (Lead indicators)
Financial Performance
F1	Increased profitability	Return on equity		Reduce overhead costs
Customer Satisfaction
C1	Increased satisfaction "after sale"	Customer retention	Satisfaction survey results	Institute "Customer Satisfaction First" initiative
C2
Internal Process
I1
Learning and Innovation
L1
Sourcing Liability
S1

Once you have recognized a potential need or goal that might be satisfied by contracting, you can use the list of common factors within each business issue area given in the following sections as a starting point for determining the factors that will drive the achievement of the business objectives. Table 2 summarizes the factors within the business issue areas of financial performance factors, customer satisfaction factors, internal process factors, and learning and innovation factors. Sourcing liability factors are further classified as contractual factors, technical factors, and environmental factors and summarized in Table 3. This classification, however, cannot represent how you should interpret that factor in your situation. For example, you might evaluate the major impact of a factor to be in financial performance rather than in internal process, or you may consider this factor in both business issue areas and weight them appropriately. Review each factor for relevance to the characteristics of the contract, and use comments to document strategies at each stage through the contracting process to deal with your issues and contract objectives.

Financial Performance Factors

Financial performance is the most mature of the business issue areas, the most quantitative, and the one most frequently applied to analysis in terms of the cost performance ratio of different alternatives. For most profit-making organizations, this business issue area contains both making more money and increasing effectiveness of current expenditure.

This business issue area is composed of costs, budgets, expenses, investments, and return on investment (ROI). There are two challenges in developing factors in this area. The first is to develop measures to represent factors such as quality or value. The second is to not lose sight of the business objectives, by decomposing the financial issue area into separate elements and then locally trying to optimize each.

Table 2 summarizes the contracting decision factors and the strategies for optimizing the performance of the supplier in achieving business objectives related to each of the factors.

Table 2. Contracting Decision Factors

Factor Identification		Optimization Strategy
Financial Performance Factors
	Cost or expense	Agree to specific measures and processes as part of the contract or management plan to allow you to control cost.
	Investment	Evaluate the investment by building a series of scenarios that consider alternative uses of capital.
	ROI	Establish well-defined measures and monitor cost benefits.
Customer Satisfaction Factors
	Quality and reliability	Establish quality and reliability performance measures and process for collection, analysis, and resolution.
	Brand equity	Establish customer satisfaction feedback processes. Define the supplier's participation and responsibility.
	Delivery	Establish delivery performance measures and processes for collection, analysis, and resolution.
	Agility	Establish the contract type that gives you the flexibility you require.
	Price	Define the supplier's contribution to the cost of supplying your customers and the value derived from it.
	Product capability	Establish well-defined performance measures and processes for collection, analysis, and resolution.
Internal Process Factors
	Internal relations	Establish or update employee survey process. Validate effectiveness of the survey.
	Process capability	Establish an effective improvement plan for you and the supplier. Acquire staff with process skills.
Learning and Innovation Factors
	External relations	Establish customer satisfaction feedback processes. Define the supplier's participation and responsibility.
	Transaction planning and coordination	Establish well-defined process performance measures (e.g., costs and processes for collection, analysis, and resolution).
	Transaction flexibility	Establish performance measures for demand fulfillment. Establish expected technology improvements from supplier. Monitor your market segment trends.
	Functional intelligence	Define expected market intelligence to be provided by the supplier.

The following list describes the financial performance factors:

*   Cost or Expense. This factor is the total cost of the contract, which is the sum of the direct and indirect costs allocable to the contract, incurred or to be incurred. The accuracy and estimating techniques vary when the cost is estimated within the different activities of the contracting process. In the Perform Make or Buy Analysis activity, the evaluation generates an initial "Should pay cost," which you refine and carry forward as part of the supplier agreement If reduced cost is the objective of the contract, then you and the supplier must agree to specific measures and processes as part of the SMP to allow you to control cost.

*   Investment. An investment is something you pay for that continues to give you a benefit long after you paid for it. Contracting allows you an opportunity to transfer assets and ongoing investments, such as training or process improvement, to the supplier. Your funds now can be used for areas that have a higher significance to the business. Evaluate the investment by building a series of scenarios that consider alternative uses of capital. You must consider not just the first-order consequences but also the 'orders' of consequences following your project decision. Build a scenario of contingencies given the projected decision. Look at the downstream effects of the decision and their side effects:

A. Are there hidden costs? If so, add them to the analysis.

B.   Will the new project steal market share from ongoing investments? What is the expected effect of these losses?

C.   How flexible are the beliefs upon which you have established the project parameters? Can these beliefs be factored into the calculation? Accounting for this now keeps the value of the project in real terms.

D. If the project is a complete failure, is wiped out by unforeseen contingencies, or hampered by personnel problems, what will be the effect on your organization?

*   Return on Investment. Use net present value analysis in any supporting decisions to initiate, renew, or expand programs or projects, which would result in a series of measurable benefits or costs extending into the future. Calculating present value may involve receipts as well as expenditures. For example, the alternatives may have some salvage value after their useful life has ended. You must consider the estimated receipt from the sale of the item in your analysis. The difference between the present value of the receipts and the present value of the expenditures is net present value. The best financial choice is the alternative with the highest net present value. Regardless of the application, use a four-step process to determine net present value:

1.       Select the discount (interest) rate.

2.       Identify the cost/benefits to be considered in the analysis.

3.       Establish the timing of the costs/benefits.

4.       Calculate the net present value of each alternative.

Customer Satisfaction Factors

The customer satisfaction issue area represents the customer and market segment in which you compete. Factors address fundamental issues such as: What business are we in? What is our market share potential? Are our customers satisfied with quality, service, and price? What is our customer profitability profile?

A product's position is the view customers have of its worth in a particular market segment. Product attributes, services, personnel, or image characteristics can all differentiate positions; however, it is not to the organization's advantage to compete in all these dimensions concurrently. Strong competitive advantage can be developed from one of four competitive strategies.

*   Market leaders have the largest market share and often lead in price changes, introductions, distribution, and promotion.

*   Market challengers fight to increase market share from either leaders or smaller firms.

*   Market followers try to avoid leader retaliations.

*   Market nichers serve small segments overlooked by larger firms.

The customer issue area is the link between the contracting choice and the overall competitive priorities of the organization for the product or service. Looking at the characteristics of a product that impel customers to select it over competing products identifies these competitive priorities. Differentiation is the key to competitive advantage in your chosen market to finding new users, new uses, and more usage. Therefore, you should align the contracting analysis with the market segment and organization, product line, or product strategies.

The following list describes the customer satisfaction factors:

*   Quality and Reliability. Determine whether product or service quality and reliability are important differentiators for you. If they are not differentiators, because the market is driven for example by cost, then it is likely that this factor will not weigh highly in your evaluation. If they are differentiators, evaluate which approach will deliver the best continued competitive advantage, and analyze the risk that the competitive advantage will be eroded or that it will cease to be a differentiator. For example, if many competitors will enter the market soon, then price and value may rise in importance over quality and reliability. So, unless you can maintain your profit with a price premium, you would need to stop doing those things that created the perception of superior quality and reliability. On the other hand, if you are building a life-critical system, quality and reliability will always be important factors.

*   Brand Equity. This factor evaluates the effect on the most important assets of any business. These intangibles include a base of loyal customers, brands, symbols and slogans, and the brand's underlying image, personality, identity, attitudes, familiarity, associations, and name awareness. These assets, along with patents, trademarks, and channel relationships, compose brand equity and are a primary source of competitive advantage and future earnings. Brand equity initially is built by laying a foundation of brand awareness, eventually forming positive brand images, and is maximized ultimately by high levels of brand loyalty. Your contracting decision can affect all these intangibles, both positively and negatively. For example, contracting with a supplier, who has a known name in a new market for you, enables you to gain positive leverage in that market. Contracting with someone who is perceived as a second rank or minority player in a market also can affect you depending on your market segment and customer buying behavior.

*   Delivery. Use this factor to evaluate both "speed to market" and on-time delivery performance. You need to evaluate delivery if either of these issues differentiates you in the market segment and weight it by how important the issues are to the contract decision. Often, these are an optimization with quality and cost. You may also use this factor to represent geographic delivery capability, which could be enhanced by the choice of the right contractor. For example, by selecting a supplier close to your major customers, you potentially improve customer satisfaction by shortening delivery or turnaround times.

*   Agility. This factor evaluates how quickly you could make changes to meet customer demand, enter a new market, or react to competitive pressure. This is a key factor framing the success or failure of a contracting relationship, and you should aim to determine how much flexibility would be lost or gained through the life of the contract. Contracting could be used to increase your flexibility by:

E.   Allowing you an opportunity to transfer investments and future costs in assets that do not differentiate or give you competitive advantage and use the funds to focus on parts of the product or service that you consider core competency.

F.   Allowing you to play catch up by selecting a supplier with more advanced capability than you have internally. This can be driven either by competitive necessity or by a desire to tap an interim solution as you build internal capability.

G.   Allowing you to ride the market leader or owner of the innovation curve.

*   Price. Price refers here to the end product or service for which the customer pays (cost of the contract is evaluated under financial performance factors). Price is most often a value proposition: How much will the customer pay for a level of capability of the product or service? There are several techniques to evaluate this, such as conjoint analysis or discrete choice modeling. Conjoint value analysis is used for pricing products and product features, which simulates the likely impact on product acceptance of different pricing scenarios. This is done through collecting individuals' preferences from interviews or surveys on product features at various prices and simulating pricing scenarios. The specific methods are outside the scope of this scorecard.

*   Product Capability. If you contract, the supplier now will be responsible for meeting your product capability requirements. In some industries, product capability is fixed by the customer; in others, it is another variable optimized primarily with cost, quality, and delivery. You also may use this factor to represent interoperability with current systems, which is a significant area of product capability to assess when contracting.

Internal Process Factors

The internal process issue area identifies the critical business processes in which your organization or project must excel. These processes focus on the capabilities that have the greatest impact on customer satisfaction, financial performance, and learning and innovation.

This area is where you develop factors that not only seek to improve existing processes through a supplier relationship but also to identify new processes that might be required to meet business objectives. This includes factors such as expanded infrastructure or reduced cycle time for new or revised products.

The following list describes the internal process factors:

*   Internal Relations. This factor revolves around the people who make up the organization. This factor is described in terms of employee morale (how the employee feels about the company) and employee development (how the employee matures with the company). This factor also covers employment stability, which clearly becomes important if the sourcing decision results in layoffs or high levels of overtime.

*   Process Capability. This factor refers to the maturity of processes within your organization and the supplier's organization, the effectiveness of those processes, and the ability of the organization to follow them. This might be measured through one of the capability maturity models that are available. The factor might be important because of a customer requirement or be driven by your concern for the supplier meeting their performance commitments. You may consider doing a formal evaluation against one of the available models, or you may require that the supplier engage in some form of process improvement activity as part of the contract.

Learning and Innovation

The learning and innovation issue area deals with identifying and managing the infrastructure, people, information systems, procedures, policies, and culture that are necessary to create and sustain the long-term health of the organization or project. It is a measure of how effectively the organization coordinates contracting and how the approach to different contracting alternatives impacts the internal and external relations.

Typically, objectives in this area are driven by large gaps between the organization's abilities and those required to achieve the targets for future success. Examples of gaps could be identified from other business issue areas, for example:

*   During an ROI assessment, where you are looking at optimizing use of investments, to provide better customer value and improve profits

*   During strategic repositioning or realignment, where responding to competitive forces cause you to reevaluate what you do or adjust how you do it so that it better matches the changed external environment

*   During an operational assessment, where you are trying to improve process efficiency as determined through process benchmarking, value chain analysis, or activity-based costing

This issue area includes employee-based factors, such as employee satisfaction, retention, skills inventory, training, and knowledge of organizational process and technology. Your employees can see contracting as a threat or a benefit. Examine how their viewpoint will affect them and how contracting is aligned to your culture; then look at alignment of employee incentives with overall project or organizational success criteria.

The following list describes the learning and innovation factors:

*   External Relations. This factor refers to an organization's suppliers and customers. This is relevant if a relationship with a supplier was being built or changed. In addition, the organization should evaluate the effect of the sourcing alternatives on their customers. For example, contracting may be seen negatively by the customer as lack of long-term commitment by the organization or seen positively as a move to open standards by use of industry-standard components.

*   Transaction Planning and Coordination. This factor looks at the degree of complexity in managing transactions for the make or buy alternatives. It weighs the advantages of contracting to an organization and not having to directly manage activities, with the increase in formal coordination and planning that comes with the complexity and interrelatedness of the product or service being contracted. Obviously, this performance depends on the organization's ability to plan and coordinate and would be related to their capability maturity. This would also have dependencies on geographic distribution, stability of requirements, and the number of suppliers whose products and services need to be coordinated.

*   Transaction Flexibility. This factor refers to how easily the organization can absorb market-driven technological innovation, marketplace realignment, and demand fluctuations. Contracting may allow an organization to leverage the innovative capabilities of the supplier; however, contracting may reduce the organization's ability to use technology innovation as a competitive advantage if it does not control the technologies' development in the marketplace. This factor takes into account the stability of the supplier and the customer's position in the marketplace (e.g., acquisitions or mergers, such as contractor being bought by major competitor, or major competitor developing in-sourced capability as a competitive advantage). Also, the transaction flexibility covers the ability of the specific management structures and operation to absorb demand fluctuations.

*   Functional Intelligence. This factor allows the organization to measure the level of competitive intelligence derived in a make or buy alternative. It is the ability of the organization to obtain competitive data, through industry watching or benchmarking. An organization that is evaluating two contracts may differentiate the suppliers by their ability to provide better intelligence about market trends or operational performance and therefore give the organization the potential to derive competitive advantage.

Sourcing Liability Factors

The sourcing liability issue area identifies factors that depend on the supplier relationship type, for example:

*   Contracting of all your internal management information systems

*   A strategic partnership with another supplier to co-develop a commercial product

*   A contract with a system developer to produce a part of a system you are developing under contract to a customer

*   An agreement with another group within your own company for them to provide either a system component or functional support (such as maintenance) for your system

*   Purchase of a COTS product to be part of your system

The sourcing liability issue area identifies factors that depend on your relationship with the supplier for example, good prior history, financially linked through holding company or widely used by competitors. In addition identify factors that relate to the stability of the supplier such as, relative financial size, debt or newness in the market.

While the issue area uses the term 'liability' implying negative factors, it does not mean that you always attempt to minimize these areas, for example when using the scorecard to select between suppliers. You must evaluate the scorecard as a whole, rather than trying to optimize each business objective. The size of the business opportunity may completely overshadow any sourcing liability factors.

In general the sourcing liability factors are manifested as risks with associated risk mitigation strategies in your contracting process. For example, if a supplier is not sufficiently stable, you may negotiate as part of agreeing to terms that all intellectual property will revert to you at no addition cost if the supplier goes out of business. Or if you suspect that the supplier may reallocate staff away from your project, you may require visibility into their resource allocation and usage as part of the management process you establish.

These sourcing liability factors are characterized into three groups as contractual, technical, and environmental liabilities. Table 3 summarizes the liability factors.

Table 3. Contracting Liability Factors

Factor Identification		Optimization Strategy
Contractual Liability Factors
	Misconduct	Specify expected behaviors and include misconduct penalties. Agree to all costs and criteria for increasing costs.
	Inability to deliver	Specify key personnel minimum qualifications and require approval of personnel changes. Develop cross-hiring guidelines.
	Lack of visibility	Build trust and reduce barriers to cooperation. Establish formal status reviews and technical interchange meetings and informal social events. Specify well-defined performance measures.
	Poor or nonperformance	Establish minimum and maximum required performance thresholds and penalties for performance below the minimum (and incentives for performance better than the maximum).
	Unacceptable product	Specify well-defined acceptance criteria in the contract. Establish early testing capability and frequent capability reviews and observe the acceptance tests before agreeing to accept the final product.
	Endless contract modifications	Use the SMP, rather than the agreement for frequently changing items. Perform solid requirements analysis and establish strong change control. Consider short-term renewable contracts.
	Failure to manage contract	Assign contract manager. Ensure adequate training of affected personnel. Establish progress reviews and corrective action tracking.
Technical Liability Factors
	Technology diffusion	Develop well-defined intellectual property rights with effective duration. Require supplier personnel to agree to confidentiality.
	End-product degradation	Establish performance measures of supplier's responsiveness to your customers.
	Interface/integration problems	Develop solid interface specification prior to supplier development. Schedule early validation. Require test harness as part of contract.
	Licensing and maintenance issues	Develop licensing and usage monitoring. Agree to supplier's long-term role, responsibility, and costing structure.
	Lack of support	Define contingencies for supplier being unable to supply agreed-upon support, such as the ability to take over support or transfer it to another supplier.
	Incompatible products	Develop a solid specification prior to supplier's development, including conformance to required standards. Schedule early validation.
	Lack of supplier QA and/or CM	Perform role for supplier, train supplier, or require that supplier meet a certain standard within an agreed timeframe.
Environmental Liability Factors
	Geographically distributed development	Establish mechanisms for collaboration at different organizational levels. Establish frequent reviews. Architect solution to minimize consequence of distribution. Get distributed organizations to agree on project objectives and priorities.
	Foreign country issues	Establish mechanisms for collaboration at different organizational levels. Establish frequent reviews. Architect solution to minimize consequence of distribution. Get distributed organizations to agree on project objectives and priorities.
	Capability maturity differences	Measure the supplier against your model. Perform roles for supplier. Train supplier. Require effective improvement plan as part of agreement.
	Classified system development	State requirement in RFP. Validate supplier's conformance. Evaluate testing and integration impacts.
	Supply disruption	Establish multiple supplier contracts. Establish capability to take over from the supplier.

Contractual Liability Factors

The following list describes the contractual liability factors:

*   Misconduct. This factor evaluates the supplier's behavior. For example, if the contractor's capabilities are of great strategic value, there may be opportunistic rising of prices or other behaviors to exploit that dependency. In addition, your organization may build an infrastructure to support a specific supplier, for example, setting up of clean rooms for certification, which creates switching costs to change contractor. These costs are higher when the degree of infrastructure specialization is higher and there are few alternative suppliers.

*   Inability to Deliver. This factor refers to the capability to provide qualified staff and maintain that staff throughout the life of the project. This takes into account bait-and-switch tactics relative to key personnel and the depth of experience and knowledge in the technical domain of personnel assigned to the project.

In an effort to win the contract award, the supplier may offer resumes of their most experienced personnel in the proposal. Once the contract is underway, however, the supplier may use those personnel for other work and staff the contract with junior people who cannot perform at the promised level of competence. You can mitigate this risk by specifying in the contract that the supplier's identified key personnel must work on the job, and any change of staff must meet some minimum qualifications (such as equivalent to or greater than the key personnel they are replacing) and must be preapproved by you. Then it is up to your contract manager to disapprove inexperienced staff if their ability to meet program requirements is questionable.

A similar but potentially more destructive tactic is when the supplier works closely with members of your staff, learns their capabilities, and then hires them away to do similar work. If this is a concern, the supplier agreement should include an enforceable arrangement whereby neither you nor supplier is allowed to hire personnel away from the other.

*   Lack of Visibility. When the supplier is not co-located with the acquirer at the development site, it can be difficult for the acquirer to know exactly what progress is being made and what problems are surfacing. This lack of visibility makes it impossible to project future costs and schedules or to be proactive in managing the risks on the contract. A cooperative attitude by both organizations is essential to establishing open communication of problems as well as progress. This spirit of cooperation must begin with the initial contract negotiations and continue throughout the contract duration in status reviews and technical interchange meetings.However, the main way to mitigate this risk is to include in the contract well-defined performance measures that must be collected and reported by the supplier. Performance measures include progress-related data, such as costs, schedules, and milestone completion, and product-related data on meeting technical requirements, such as display times, reliability, or storage capacity. These measures should be aligned with the business needs of the project and be consistent with the performance measures that the acquirer collects and analyzes about his own efforts on the project. For further detail on defining an appropriate measurement program, see the Practical Software and Systems Measurement guidebook and course (Department of Defense and U.S. Army 2000).

*   Poor or Nonperformance. Despite your best efforts to monitor the supplier, they may still fail to meet the contract or product requirements. They may have done their best and were not able to meet their own expectations, or perhaps they did not consider this contract important enough to their business to focus attention and the right resources on the job. In any case, you do not want to be left without recourse. To deal with this situation, the contract needs to include minimum and maximum required performance thresholds, penalties for performance below the minimum, and incentives for performance better than the maximum. If the supplier's performance does not reach or exceed the thresholds specified in the contract, then the contract manager must take the appropriate action to penalize poor performance or to reward excellent performance. This needs to be done in a timely manner to affect the final outcome positively.

*   Unacceptable Product. A delivered product that does not work as promised is closely related to unacceptable supplier performance. Frequently, this situation is due to a lack of understanding of what the other party specified. It is incumbent upon the acquirer to specify clear and complete acceptance criteria in the contract and to personally run or closely observe the acceptance tests before agreeing to accept the final product.

*   Endless Contract Modifications. Change will happen, and the longer the contract duration, the more changes are likely to occur during its existence. Although the contract cannot define future changes in requirements or business needs, you can build flexibility into the agreement by anticipating that changes may be needed and specifying a process for dealing with them. Front-end gathering of all stakeholders' needs, performing a solid requirements analysis early in the project, including both the users and the suppliers on the requirements analysis team, and having a strong change management strategy can minimize the number of requirements changes. Instead of long-term contracts with a service provider, you can consider short-term, renewable contracts allowing changing business needs to be accommodated with each contract renewal. For a single project or product purchase, the contract should specify some limits for additional supplier charges due to requested changes (such as fixed hourly rates for changes) or enhancements and some limits for the extent and timely notice of acquirer-requested changes.

*   Failure to Manage Contract. If your management staff is not experienced or trained in managing contracts, there is a risk that the supplier agreement will not be adequately managed. It is essential that someone be designated as contract manager. This manager is responsible for monitoring the supplier's progress and work products, holding periodic status reviews and technical interchange meetings, reviewing and analyzing performance measurements, identifying and managing contract risks, and taking corrective actions when needed to keep the project on track.

Technical Liability Factors

The following list describes the technical liability factors:

*   Technology Diffusion. This factor considers the likelihood of a firm losing the confidentiality of a particular value-creating asset to a contractor. An example would be specialized knowledge about a component's design or manufacturing process. With the loss of internal control, there is the opportunity of transfer of this knowledge to the organization's competitors. The risk is highest when there is close contact, and significant intellectual property is transferred to the contractor.

*   End-Product Degradation. This factor means that important parts of the contract could be distorted or ignored. It appraises the extent to which a contractor is closer to the end user than the organization and evaluates the probability and consequences that the contractor will ignore or degrade product attributes and/or ignore customer needs. This risk is highest when the contractor forms the interface to the organization's customers so that it is out of direct control. For example, the contractor is responsible for supporting its component.

*   Interface or Integration Problems. Regardless of whether a system component is hardware or software, COTS or newly developed, the interfaces of that component to other components, subsystems, systems, and humans must be well-defined to ensure proper integration. Even when the interfaces are well-defined, there still may be integration difficulties, but they will be easier to resolve if the interfaces were specified in detail and the component was built to specifications. Delineate the interface definitions clearly and completely in the contract or in a project deliverable. Involve the supplier in a joint effort to specify the interface requirements and agree to the final specification prior to developing the component. For further detail regarding requirement definition, see Software Productivity Consortium (2000b).

*   Licensing and Maintenance Issues. If the supplier incorporates COTS components into a delivered product, the applicable software licenses or hardware maintenance agreements need to cover your organization (or the end users). If this is not done, you will be in violation of the licensing or maintenance agreements and will have to incur the cost of purchasing the license again or establishing a new maintenance agreement in order to obtain product updates or technical support from the manufacturer. If required software licenses or intellectual property rights are not transferable, this could be a major factor against contracting. The contract should address the extent to which the supplier will be responsible for and involved in long-term maintenance and their responsibilities during the warranty period. Fees for maintenance can be part of the contract, negotiated later, or renegotiated at periodic intervals (Klepper and Jones 1998).

*   Lack of Support. If suppliers provide software or hardware components with inadequate documentation to allow another company to support the system, then you will be totally dependent on that supplier for long-term maintenance. Even if the documentation is sufficient for maintenance, you may find that over time, some of the hardware parts are no longer available in the marketplace or that the software language used is no longer supported by compilers and other development tools. If the supplier is your maintainer and goes out of business, drops that line of work, or simply refuses to support the product any longer, your system will be left without technical support. Mitigating this risk requires one or more of the following approaches:

H. Consider the long-term viability of each candidate supplier in your initial source selection criteria.

I.    In the contract negotiations, try to secure the rights to the data associated with any delivered products so that you retain the option to use your own staff or other suppliers for maintenance. If this is not possible for a deliverable such as source code, require that the item be placed in escrow for a specified period of time.

J.    Require complete maintenance manuals to be delivered with any system components. The manuals must be sufficiently detailed for others to provide maintenance.

K. Specify in the product requirements that the product must be supportable for some specified number of years. Do not be too optimistic about how many years are reasonable for ongoing support because technology changes so rapidly.

L.   Prepare a plan to replace each system component if and when it should become obsolete. This should be anticipated when defining interfaces, so that you are not locked into a particular component.

*   Incompatible Products. When products are developed without following any particular standards, those products may be incompatible with other system components or may not meet your quality expectations. The suppliers may have their own standards to follow, which could be sufficient. If they do not, your organization may have standards that you will want the supplier to follow. There are also numerous industry standards involved with system development, such as for design, manufacture, testing, communications, or documentation. The supplier may not need to conform to a standard for each work product they build or activity they perform. However, when the quality of a delivered product is critical or when you know that a product will have to interface with other products at some point in the future, specifying one or more standards to follow will help ensure a certain level of quality and compatibility.

*   Lack of Supplier Quality Assurance and/or Configuration Management. If the supplier lacks capability maturity in the areas of QA or CM, there may be an increased risk of poor product quality or lack of properly baselined versions of products. Mitigate these risks by planning for your organization's QA or CM personnel to monitor, review, and audit the supplier's QA or CM processes or to perform those activities for the supplier.

Environmental Liability Factors

The following list describes the environmental liability factors:

*   Geographically Distributed Development. When a system development team is located at more than one site, coordination of efforts becomes more difficult and potentially problematic. Management visibility into the progress and issues at multiple sites is limited. Interfaces between products built at different sites may fail to keep up-to-date with changes in either product. Minor issues are frequently discussed informally between engineers who are co-located, but at different sites there is no opportunity for this kind of unscheduled interchange of ideas. Conventions, techniques, methods, and tools can vary significantly from site to site leading to incompatible work products and more rework than planned.

As you cross boundaries of space, time, culture and nationality, you must take several steps to mitigate these risks. Technology advances of the past decade, especially regarding communications and interoperability, are key. Essentially, management and engineering processes and supporting environments must be either adjusted to minimize spatial, temporal, organizational, and cultural distance or optimized to take advantage of the gap. The Consortium Distributed Development Technical Report provides guidance, best practices, and lessons learned in dealing with these issues (Software Productivity Consortium 2001b).

*   Foreign Country Issues. When development is conducted in a foreign country, not only do you have the distributed development issues, but you may have to deal with additional problems associated with time zone and cultural differences, as well as regulatory and product control issues, such as exporting technologies. Scheduling a telephone conference can be difficult when 2 sites are 12 hours apart. The use of collaborative technologies and tools is essential to making such a geographical separation workable. As with any distributed development effort, the contract manager must perform more detailed planning and coordination to ensure frequent interactions so that the off-site development team stays on track. In addition, cultural problems and misunderstandings can surface in the use of language, in individual and corporate values, and in customs and traditions regarding work environments, reward systems, risk avoidance, holidays, vacations, personnel interactions, organizational hierarchy, and job motivations. Misunderstanding someone's intent and meaning can lead to staff relationship problems or work being done incorrectly. All project members on both teams who will interface with members of the other team must be trained in as many of these cultural differences as possible before work begins. This training is an additional cost that must be factored into contracting plans.

For details about the latest Department of Commerce regulations regarding exports, see http:www.bxa.doc.gov/ For a compilation of data on cultural differences between nations, see Carmel (1999).

*   Capability Maturity Differences. Many organizations are engaged in pursuing higher-level process maturity by following one or more of the capability maturity models that are available. When two organizations involved in an acquirer-supplier relationship have different maturity levels, some basic incompatibilities result. For instance, a Level 3 organization can readily put together a development plan or management plan because there is a standard process and previous project assets that can be tailored for this project. Lower-level organizations have to struggle more with creating such plans from scratch. Collecting and analyzing performance measures will be a normal part of business for a higher-level organization but very difficult for most low maturity organizations that do not have the infrastructure required for a measurement program. Invariably, the personnel in both organizations become frustrated in dealing with each other. For instance, the more mature organization might expect more detail in plans or process descriptions and feel that the other team is deliberately not being cooperative by not supplying that information. The lower maturity team might not be familiar with defining processes and feel that the other team is too demanding to ask for so much worthless detail. Consider capability maturity in evaluating candidate suppliers and put plans in place to deal with the maturity differences.

*   Classified System Development. If the product or service to be procured requires the supplier's staff or facility to have a certain type and level of government clearance, you must state these requirements clearly in the solicitation request. When the supplier is providing only an unclassified portion of a classified system, carefully design plans for acceptance testing, integration, and any associated rework and retesting and define them in the supplier agreement and SMP because the supplier will not have direct access to the classified system during integration and testing.

*   Supply Disruption. This factor is an evaluation of the possibility of supply disruption and its impacts on the organization and end users. This factor applies to both in-sourced and contracted components. It takes into account items such as the contractor's financial stability, delivery stability (e.g., history of shutdowns), geographic considerations, shipping stability, and political instability. This factor often leads an organization to adopt a multisource strategy.

Top