Ch 4 - Developing and Using Cost Estimating Relationships

• 4.0 - Chapter Introduction
• 4.1 - Identifying Situations For Use
• 4.2 - Identifying And Using Rules Of Thumb
• 4.3 - Developing And Using Estimating Factors
• 4.4 - Developing And Using Estimating Equations
• 4.5 - Identifying Issues And Concerns

4.0 - Chapter Introduction

In this chapter, you will learn to use cost estimating relationships to estimate and analyze estimates of contract cost/price.

Cost Estimating Relationship Definition. As the name implies, a cost estimating relationship (CER) is a technique used to estimate a particular cost or price by using an established relationship with an independent variable. If you can identify an independent variable (driver) that demonstrates a measurable relationship with contract cost or price, you can develop a CER. That CER may be mathematically simple in nature (e.g., a simple ratio) or it may involve a complex equation.

The goal is to create a statistically valid cost estimating relationship using historical data. The parametric CER can then be used to estimate the cost of the new program by entering its specific characteristics into the parametric model. CERs established early in a program’s life cycle should be continually revisited to make sure they are current and the input range still applies to the new program. In addition, parametric CERs should be well documented, because serious estimating errors could occur if the CER is improperly used.

It is important to make sure that the program attributes being estimated fall within (or, at least, not far outside) the CER dataset. For example, if a new software program was expected to contain 1 million software lines of code and the data points for a software CER were based on programs with lines of code ranging from 10,000 to 250,000, it would be inappropriate to use the CER to estimate the new program.

Among the several advantages to using cost estimating relationships are:

• Versatility: If the data are available, parametric relationships can be derived at any level, whether system or subsystem component. And as the design changes, CERs can be quickly modified and used to answer what-if questions about design alternatives.
• Sensitivity: Simply varying input parameters and recording the resulting changes in cost can produce a sensitivity analysis.

• Statistical output: Parametric relationships derived from statistical analysis generally have both objective measures of validity (statistical significance of each estimated coefficient and of the model as a whole) and a calculated standard error that can be used in risk analysis. This information can be used to provide a confidence level for the estimate, based on the CER’s predictive capability.

• Objectivity: CERs rely on historical data that provide objective results. This increases the estimate’s defensibility.

Disadvantages of CERs include:

• Database requirements: The underlying database must be consistent and reliable. It may be time consuming to normalize the data or to ensure that the data were normalized correctly, especially if someone outside the estimator’s team developed the CER. Without understanding how the data were normalized, the analyst has to accept the database on faith—sometimes called the black-box syndrome, in which the analyst simply plugs in numbers and unquestioningly accepts the results. Using a CER in this manner can increase the estimate’s risk.
• Currency: CERs must represent the state of the art; that is, they must be updated to capture the most current cost, technical, and program data. Existing CERs must be validated that the current data still result in the established CER.
• Relevance: Using data outside the CER range may cause errors, because the CER loses its predictive ability for data outside the development range.
• Complexity: Complicated CERs (such as nonlinear CERs) may make it difficult for others to readily understand the relationship between cost and its independent variables.

Steps for Developing a CER . Strictly speaking, a CER is not a quantitative technique. It is a framework for using appropriate quantitative techniques to quantify a relationship between an independent variable and contract cost or price.

Developing a CER is a 6-step process. Follow the six steps whenever you develop a CER. Whenever you evaluate a CER developed by someone else, determine whether the developer followed the six steps properly.

Step 1. Define the dependent variable (e.g., cost dollars, hours, and so forth.) Define what the CER will estimate. Will the CER be used to estimate price, cost dollars, labor hours, material cost, or some other measure of cost? Will the CER be used to estimate total product cost or estimate the cost of one or more components? The better the definition of the dependent variable, the easier it will be to gather comparable data for CER development.

Step 2. Select independent variables to be tested for developing estimates of the dependent variable. In selecting potential independent variables for CER development:

• Draw on personal experience, the experience of others, and published sources of information. When developing a CER for a new state-of-the-art item, consult experts experienced with the appropriate technology and production methods.
• Consider the following factors:
  • Variables should be quantitatively measurable. Parameters such as maintainability are difficult to use in estimating because they are difficult to measure quantitatively.
  • Data availability is also important. If you cannot obtain historical data, it will be impossible to analyze and use the variable as a predictive tool. For example, an independent variable such as physical dimensions or parts count would be of little value during the conceptual phase of system development when the values of the independent variables are not known. Be especially wary of any CER based on 2 or 3 data observations.
  • If there is a choice between developing a CER based on performance or physical characteristics, performance characteristics are generally the better choice, because performance characteristics are usually known before design characteristics.

Step 3. Collect data concerning the relationship between the dependent and independent variables. Collecting data is usually the most difficult and time-consuming element of CER development. It is essential that all data be checked and double checked to ensure that all observations are relevant, comparable, and relatively free of unusual costs.

Step 4. Explore the relationship between the dependent and independent variables. During this step, you must determine the strength of the relationship between the independent and dependent variables. This phase of CER development can involve a variety of analytical techniques from simple graphic analysis to complex mathematical analysis. Simple ratio analysis, moving averages, and linear regression are some of the more commonly used quantitative techniques used in analysis.

Step 5. Select the relationship that best predicts the dependent variable. After exploring a variety of relationships, you must select the one that can best be used in predicting the dependent variable. Normally, this will be the relationship that best predicts the values of the dependent variable. A high correlation (relationship) between a potential independent variable and the dependent variable often indicates that the independent variable will be a good predictive tool. However, you must assure that the value of the independent variable is available in order for you to make timely estimates. If it is not, you may need to consider other alternatives.

Step 6. Document your findings. CER documentation is essential to permit others involved in the estimating process to trace the steps involved in developing the relationship. Documentation should involve the independent variables tested, the data gathered, sources of data, time period of the data, and any adjustments made to the data.

4.1 - Identifying Situations For Use

Situations for Use. You can use a cost estimating relationship (CER) in any situation where you quantify one of the following:

• A relationship between one or more product characteristics and contract cost or price. A product-to-cost relationship uses product physical or performance characteristics to estimate cost or product price. The characteristic or characteristics selected for CER development are usually not the only ones driving cost, but the movement of cost has been found to be related to changes in these characteristics. The following table identifies several product characteristic that have been used in CER development:

• A relationship between one or more elements of contract cost and another element of contract cost or price. A cost-to-cost relationship uses one or more elements of contract cost to estimate cost or product price. If you can establish a relationship between different elements of cost (e.g., between senior engineering labor hours and engineering technician hours), you can use a CER to reduce your estimating or analysis effort while increasing accuracy. If you can establish a relationship between an element of cost and total price (e.g., between direct labor cost and total price), you can use that information to supplement price analysis, without requiring extensive cost information.

4.2 - Identifying And Using Rules Of Thumb

Identifying Rules of Thumb. As you perform your market analysis, you should be on the lookout for cost estimating rules of thumb that are commonly used in the product marketplace. For example, when we compare the prices of houses, we typically do so in price per square foot. Using this rule of thumb, we can compare the cost of different houses or the same house in different parts of the country. There may be ways to develop more accurate estimates, but this rule of thumb is widely accepted, relatively easy to calculate, and it provides reasonably accurate results for many purposes. The same statement can probably be made about most rules of thumb. You may be able to develop better cost estimating relationships, but given the time available and the dollars involved, rules of thumb often provide useful tools for contract pricing.

Validate a Rule of Thumb Before Using. Like any CER, a rule of thumb can be based on another cost, performance characteristic, or physical characteristic of the item being priced. Unlike other CERs, rules of thumb typically have not been validated for use in specific estimating situations. Validation has come from acceptable results produced in a variety of situations over a number of years. Before you use a rule of thumb, consider the 6-step CER development process and ask the following questions:

• Can the rule of thumb reasonably be used to estimate what you are trying to estimate (e.g., cost, hours, or price)?
• Are there other rules of thumb that can be used to estimate the same cost, hours, or price?
• Are the data required to use this rule of thumb readily available?
• Does the rule of thumb provide reasonably accurate results?
• If more than one rule of thumb is available, which one appears to produce the most accurate estimate?
• Have technical experts or other buyers documented the results obtained from using the rule of thumb?

Example of Rule of Thumb Validation. You just received two offers for 500 laboratory tables. Each table is 4' x 6' (24 square feet of surface area), with an oak frame and legs. The work surface is a unique composite material developed to meet Government requirements. The low offer is $425; that offer is $175 less than the second low offer and $180 less than the Government estimate. As a result, you are concerned that the price may be unreasonably low. You have no acquisition history for this item and there are no similar items on the commercial market. As a result, you have been looking for a CER that you can use in your pricing decision. Another buyer, who has acquired similar tables, tells you that he has used a rule of thumb in pricing similar tables -- $19 per square feet of surface area. You want to know the answers to the following questions before you use it in making your own pricing decisions.

• Can the rule of thumb reasonably be used to estimate what you are trying to estimate (e.g., cost dollars, hours, or product price)?

The answer appears to be yes. The buyer who recommended the CER has used it successfully. Additional information shows that he learned of the CER from the scientists who developed the table-top material.

• Are there other rules of thumb that can be used to estimate the same cost or price?

You have asked several "experts" and have been unable to identify any other rules of thumb for estimating the price of these unique tables.

• Are the data required to use this rule of thumb readily available?

Yes, you already know the table surface area.

• Does the rule of thumb provide reasonably accurate results?

You have identified four recent acquisitions of similar tables and recorded the following information comparing the estimates made using the rule of thumb and the actual prices paid:

This sample size is too small to perform an effective statistical analysis, but you can still subjectively evaluate rule of thumb estimate accuracy. All estimates are within 8.8 percent of the actual price. For a rule of thumb, this appears reasonably accurate, especially since our evaluation did not consider other acquisition situation differences (e.g., the number of tables on each contract).

• If more than one rule of thumb is available, which one appears to produce the most accurate estimates?

In this example, there is only one known rule of thumb to consider.

• Have technical experts or other buyers documented the results obtained from using the rule of thumb?

In this case, the buyer documented every contract file when the rule of thumb was used. Such documentation is not only valuable in supporting the contracting officer's decision on price reasonableness; it provides valuable information to any contracting officer considering rule of thumb use in the future.

Example of Using a Rule of Thumb in Estimate Development. Once you have determined that a rule of thumb is acceptable for estimate development, you must apply it to the available data. Using this rule of thumb, your estimate would be $456 (24 x $19). That estimate is about 7.3 percent higher than the low offer. Based on the rule of thumb, the price does not seem unreasonable.

4.3 - Developing And Using Estimating Factors

Situations for Using Estimating Factors. An estimating rate or factor is a simple ratio, used to estimate cost or price. The rule of thumb used to develop table price estimates in the previous section is an example -- $19 per square foot. As the size of the table top increases, the price estimate increases in direct proportion. Most rules of thumb are simple factors. Many CERs developed by Government or industry are also simple factors. They are relatively easy to develop, easy to understand, and in many cases quite accurate.

Development and use of estimating rates and factors involves two important implicit assumptions.

• There is no significant element of the cost or price being estimated that is not related to the independent variable (i.e., there is no "fixed cost" that is not associated with the independent variable).
• The relationship between the independent variable and the cost being estimated is linear.

If you believe that there are significant costs that cannot be explained by the relationship or that the relationship is not linear, you should either try to develop an equation that better tracks the true relationship or limit your use of the estimating factor to the range of the data used in developing the factor.

Example of Estimating Factor Development. Assume that you are negotiating a guard service contract for your facility and you want to develop a CER to assist you in estimating a should-pay contract price. Development should follow the 6-step CER process.

Step 1. Define the dependent variable. The objective is to develop an estimate of the price that the Government should expect to pay for this contract.

Step 2. Select independent variables to be tested for developing estimates of the dependent variable. Logically, the major driver of price in a guard service contract is the wages paid the security guards manning the various posts identified in the contract.

Step 3. Collect data concerning the relationship between the dependent and independent variables. You have collected information on prices, minimum manning requirements, and service contract wage-rate determinations for the guard service contract at your facility for the last three years. The minimum manning requirement for the current contract totals 75,000 (Guard II) hours. The Service Contract Act (SCA) wage rate for the current year is $10.00 per hour. The estimated direct labor cost for each year (Column D) is calculated by multiplying estimated direct labor hours (Column B) by the Service Contract Act wage rate (Column C).

Step 4. Explore the relationship between the dependent and independent variables. The following table demonstrates calculation of the Price to Direct Labor Cost Ratio. The ratio (Column F) is calculated by dividing the contract price (Column E) by the estimated direct labor cost (Column D). In Year 1 for example, price was 1.68 times the estimated direct labor cost.

Step 5. Select the relationship that best predicts the dependent variable. It appears from the information above, that there is a relationship between contract price and the estimated direct labor cost. The price is between 1.67 and 1.69 times the estimated direct labor cost. The average ratio is 1.68.

Average Ratio = (1.68 + 1.67 + 1.69) / 3
= 1.68

You can now use this ratio to estimate the price of similar contracts.

Step 6. Document your findings. Your documentation of CER development should include the information from the 6-step process above. Exact documentation requirements will vary with the analysis involved.

Using an Estimating Factor in Estimated Development. Once you calculate an estimating factor, you can use it to estimate should-pay price for similar products. Using the 1.68 factor from the guard contract example, you can calculate a should-pay price for the current year. Using this factor, the best estimate of a reasonable price would be $1,260,000, as shown in the table below:

Given the data above, you should be reasonably confident of your estimate, because the range of ratios is only from 1.67 to 1.69. Even without statistical analysis, that range might be useful in establishing a range of reasonable prices.

High side: 1.69 x $750,000 = $1,267,500

Mean: 1.68 x $750,000 = $1,260,000

Low side: 1.67 x $750,000 = $1,252,500

4.4 - Developing And Using Estimating Equations

Situations for Using Estimating Equations. Not all estimating relationships lend themselves to the use of simple estimating factors. If there is a substantial element of the cost or price being estimated that is not related to the independent variable (i.e., there is a "fixed cost" that is not associated with the independent variable), you should consider using a linear estimating equation. If the relationship is not linear, consider a nonlinear estimating equation.

Example of Estimating Equation Development. Assume that you are analyzing the costs proposed for the construction of a new house and decide to develop a CER to support your analysis. Development should follow the 6-step CER process described in the chapter Introduction

Step 1. Define the dependent variable. The objective is to estimate the cost of a new base housing model.

Step 2. Select independent variables to be tested for developing estimates of the dependent variable. A variety of house characteristics could be used to estimate cost. These include such characteristics as square feet of living area, exterior wall surface area, number of baths, and others.

Step 3. Collect data concerning the relationship between the dependent and independent variables.

Step 4. Explore the relationship between the dependent and independent variables. Analysis of the relationship between the independent variable and house price could be performed using many different techniques. In this situation most analysts would use regression analysis. However, here we will use graphic analysis to demonstrate the thought process involved. Three independent variables will be tested: number of baths, living area, and exterior wall surface area.

• Price and the Number Of Baths. This graph appears to demonstrate that the number of baths is not a good estimating tool, because three houses with a nearly $8,000 price difference have the same number of baths.
  
  

• Price and Square Feet Of Living Area. This graph appears to depict a relationship.
  
  

• Price and Exterior Wall Surface Area. This graph also appears to depict a relationship.
  
  

Step 5. Select the relationship that best predicts the dependent variable. Based on the initial analysis, it appears that square feet of living area and exterior wall surface have the most potential for development of a CER. The two graphs below depict efforts to fit a straight line through the observed values. Note that both graphs demonstrate efforts to fit a line with and without using the data from the Ambassador model.

• Price and Living Area.
  
  

• Price and Exterior Wall Surface Area.
  
  

Consider Analysis Results and Other Data. Viewing both of these relationships, we might question whether the Ambassador model data should be included in developing our CER. In developing a CER, you need not use all available data if all data is not comparable. However, you should not eliminate data just to get a better looking relationship. After further analysis, we find that the Ambassador's size is substantially different from the other houses for which we have data and the house for which we are estimating. This substantial difference in size might logically affect relative construction cost. Based on this information, you might decide not to consider the Ambassador data in CER development.

Final Analysis. If you exclude the Ambassador data, you find that the fit of a straight-line relationship of cost to the exterior wall surface is improved. The relationship between cost and square feet of living area is even closer, almost a straight line.

If you had to choose one relationship, you would probably select living area over exterior wall surface because living area has so much less variance from the trend line. Since the relationship between living area and price is so close, we can reasonably use it for our CER.

If the analysis of these relationships did not reveal a useful predictive relationship, you might consider combining two or more of the relationships already explored or exploring new relationships.

Step 6. Document your findings. In documenting our findings, we can relate the process involved in selecting living area for price estimation. We may then present the following graph developed as an estimating tool.

We might also convert the graphic relationship to an equation. The cost estimating relationship (CER) would be:

Y = $117,750 + ($17.50 x Sq Ft of Living Area)

Using an Estimating Equation to Estimate Cost. Once developed, you can use an estimating equation to contract cost or price in similar circumstances.

For example: Applying our new CER to the estimation of cost for our new 2,600 square-foot house, we would estimate:

Y = $117,750 + ($17.50 x Sq Ft of Living Area)
= $117,750 + ($17.50 x 2,600)
= $117,750 + $45,500
= $163,250 estimated price

CERs, like most other tools of cost analysis, MUST be used with judgment. Judgment is required to evaluate the historical relationships in the light of new technology, new design, and other similar factors. Therefore, knowledge of the factors involved in CER development is essential to proper application of the CER.

4.5 Identifying Issues And Concerns

Questions to Consider in Analysis. As you perform price or cost analysis, consider the issues and concerns identified in this section as you consider use of a cost estimating relationship.

• Does the available information verify the existence and accuracy of the proposed relationship?

Technical personnel can be helpful in analyzing the technical validity of the relationship. Audit personnel can be helpful in verifying the accuracy of any contractor data and analysis.

• Is there a trend in the relationship?

For example, the cost of rework is commonly estimated as a factor of production labor. As production continues, the production effort should become more efficient and produce fewer defective units which require repair. The factor should decrease over time. You should also consider the following related questions: Is the rate distorted by one bad run? What is being done to control the rate? What else can be done?

• Is the CER used consistently?

If an offeror uses a CER to propose an element of cost, it should be used in all similar proposals. Since the CER can be used to estimate the average value, some jobs should be expected to cost more and others less. With a valid CER, you assume the variances will be minor and will average out across all contracts. To use a CER in some cases and a discrete estimate in others destroys it usefulness by over or understating costs across all proposals (e.g., using the average unless a discrete estimate is lower/higher negates the averaging out of cost across all contracts and is clearly unfair to one of the contracting parties).

• Has the CER been consistently accurate in the past?

No matter how extensive the price/cost information or how sophisticated the analysis technique, if a CER does not do a good job of accurately projecting cost, then it is not a useful tool.

• How current is the CER?

Even the most accurate CER needs to be reviewed and updated. While the time interval between updates will differ with CER sensitivity to change, in general a CER should be reviewed and updated at least annually. A CER based on a moving average should be updated whenever new data become available.

• Would another independent variable be better for developing and applying a CER?

If another independent variable would consistently provide a more accurate estimate, then it should be considered. However, remember that the CER may be applicable to other proposals, not just yours. It is possible that a relationship which works well on your contract would not work well across the entire contract population. When assessing CER validity, you should consider all affected contracts.

• Is the CER a self-fulfilling prophecy?

A CER is intended to project future cost. If the CER simply "backs into" a rate that will spread the cost of the existing capacity across the affected contracts, then the CER is not fulfilling its principle function. If you suspect that a CER is being misused as a method of carrying existing resources, you should consider a should-cost type review on the functions represented by the CER.

• Would use of a detailed estimate or direct comparison with actual cost from a prior effort produce more accurate results?

Development of a detailed estimate can be time consuming and costly but the application of the engineering principles required is particularly valuable in estimating cost of efficient and effective contract performance.