# Reliability Growth Planning

### From ReliaWiki

In developmental reliability growth testing, the objective is to test a system, find problem failure modes, and at some point, incorporate corrective actions in order to increase the reliability of the system. This process is continued for the duration of the test time. If the corrective actions are effective, then the system mean time between failures (MTBF) will move from an initial low value to a higher value. Typically, the objective of reliability growth testing is not just to increase the MTBF, but to increase it to a particular value. In other words, we want to increase the MTBF to meet a specific goal or requirement. Therefore, determining how much test time is needed for a particular system is generally always of interest in reliability growth testing. This brings us to the topic of growth planning.

In this chapter, we will address the common questions engineers and managers face at the planning stage of product development, in terms of meeting the design reliability goals. Planning for required test time, setting realistic reliability goals and creating a strategy for meeting those goals are all essential parts of project management. It is therefore necessary to have a tool that can be utilized for such planning purposes. The basis of the tool will be the Duane postulate. A model that incorporates the realities faced during development and generates a plan that can be used as a guideline during developmental testing will be developed.

## Background

The Duane postulate is based on empirical observations, and it reflects a learning curve pattern for reliability growth. This identical learning curve pattern forms the basis of the Crow-AMSAA (NHPP) model. The Duane postulate is also reflected in the Crow extended model in the form of the discovery function .

The discovery function is the rate in which new, distinct problems are being discovered during reliability growth development testing. The Crow-AMSAA (NHPP) model is a special case of the discovery function. Consider that when a new and distinct failure mode is first seen, the testing is stopped and a corrective action is incorporated before the testing is resumed. In addition, suppose that the corrective action is highly effective that the failure mode is unlikely to be seen again. In this case, the only failures observed during the reliability growth test are the first occurrences of the failure modes. Therefore, if the Crow-AMSAA (NHPP) model and the Duane postulate are accepted as the pattern for a test-fix-test reliability growth testing program, then the form of the Crow-AMSAA (NHPP) model must be the form for the discovery function, .

To be consistent with the Duane postulate and the Crow-AMSAA (NHPP) model, the discovery function must be of the same form. This form of the discovery function is an important property of the Crow extended model and its application in growth planning. As with the Crow-AMSAA (NHPP) model, this form of the discovery function ties the model directly to real-world data and experiences.

The use of the Duane postulate as a reliability growth planning model poses two significant drawbacks: The first drawback is that the Duane postulate's MTBF is zero at time equal to zero. This was addressed in MIL-HDBK-189 by specifying a time where growth starts after and the Duane postulate applies [13]. However, determining is subjective and is not a desirable property of the MIL-HDBK-189. The second drawback is that the MTBF for the Duane postulate increases indefinitely to infinity, which is not realistic.

Therefore, the desirable features of a planning model are:

- 1) The discovery function must have the form of the Crow-AMSAA (NHPP) model and the Duane postulate.
- 2) The start time is not required as an input.
- 3) An upper bound on the system MTBF is specified in the model.

All of these desirable features are included in the planning model discussed in this chapter, which is based on the Crow extended model.

The Crow extended model for reliability growth planning is a revised and improved version of the MIL-HDBK-189 growth curve [13]. MIL-HDBK-189 can be considered as the growth curve based on the Crow-AMSAA (NHPP) model. Using MIL-HDBK-189 for reliability growth planning assumes that the corrective actions for the observed failure modes are incorporated during the test and at the specific time of failure. However, in actual practice, fixes may be delayed until after the completion of the test or some fixes may be implemented during the test while others are delayed and some are not fixed at all. The Crow extended model for reliability growth planning provides additional inputs that accounts for specific management strategies and delayed fixes with specified effectiveness factors.

## Growth Planning Inputs

### Management Strategy Ratio & Initial Failure Intensity

When a system is tested and failure modes are observed, management can make one of two possible decisions, either to fix or to not fix the failure mode. Therefore, the management strategy places failure modes into two categories: A modes and B modes. The A modes are all failure modes such that, when seen during the test, no corrective action will be taken. This accounts for all modes for which management determines that it is not economical or otherwise justified to take a corrective action. The B modes are either corrected during the test or the corrective action is delayed to a later time. The management strategy is defined by what portion of the failures will be fixed.

Let be the initial failure intensity of the system in test. is defined as the A mode's initial failure intensity and is defined as the B mode's initial failure intensity. is the failure intensity of the system that will not be addressed by corrective actions even if a failure mode is seen during test. is the failure intensity of the system that will be addressed by corrective actions if a failure mode is seen during testing.

Then, the initial failure intensity of the system is:

The initial system MTBF is:

Based on the initial failure intensity definitions, the management strategy ratio is defined as:

The is the portion of the initial system failure intensity that will be addressed by corrective actions, if seen during the test.

The failure mode intensities of the type A and type B modes are:

### Effectiveness Factor

When a delayed corrective action is implemented for a type B failure mode, in other words a BD mode, the failure intensity for that mode is reduced if the corrective action is effective. Once a BD mode is discovered, it is rarely totally eliminated by a corrective action. After a BD mode has been found and fixed, a certain percentage of the failure intensity will be removed, but a certain percentage of the failure intensity will generally remain. The fraction decrease in the BD mode failure intensity due to corrective actions, , is called the *effectiveness factor* (EF). A study on EFs showed that an average EF, is about 70%. Therefore, about 30%, (i.e., ), of the BD mode failure intensity will typically remain in the system after all of the corrective actions have been implemented. However, individual EFs for the failure modes may be larger or smaller than the average. This average value of 70% can be used for planning purposes, or if such information is recorded, an average effectiveness factor from a previous reliability growth program can be used.

### MTBF Goal

When putting together a reliability growth plan, a goal MTBF (or goal failure intensity ) is defined as the requirement or target for the product at the end of the growth program.

### Growth Potential

The failure intensity remaining in the system at the end of the test will depend on the management strategy given by the classification of the type A and type B failure modes. The engineering effort applied to the corrective actions determines the effectiveness factors. In addition, the failure intensity depends on , which is the rate at which problem failure modes are being discovered during testing. The rate of discovery drives the opportunity to take corrective actions based on the seen failure modes, and it is an important factor in the overall reliability growth rate. The reliability growth potential is the limiting value of the failure intensity as time increases. This limit is the maximum MTBF that can be attained with the current management strategy. The maximum MTBF will be attained when all type B modes have been observed and fixed.

If all the discovered type B modes are corrected by time , that is, no deferred corrective actions at time , then the growth potential is the maximum attainable with the type B designation of the failure modes and the corresponding assigned effectiveness factors. This is called the *nominal growth potential*. In other words, the nominal growth potential is the maximum attainable growth potential assuming corrective actions are implemented for every mode that is planned to be fixed. In reality, some corrective actions might be implemented at a later time due to schedule, budget, engineering, etc.

If some of the discovered type B modes are not corrected at the end of the current test phase, then the prevailing growth potential is below the maximum attainable with the type B designation of the failure modes and the corresponding assigned effectiveness factors.

If all type B failure modes are discovered and corrected with an average effectiveness factor, , then the maximum reduction in the initial system failure intensity is the growth potential failure intensity:

The growth potential MTBF is:

Note that based on the equations for the initial failure intensity and the management strategy ratio (given in the Management Strategy and Initial Failure Intensity section), the initial failure intensity is equal to:

### Growth Potential Design Margin

The Growth Potential Design Margin ( ) can be considered as a safety margin when setting target MTBF values for the reliability growth plan. It is common for systems to degrade in terms of reliability when a prototype product is going into full manufacturing. This is due to variations in materials, processes, etc. Furthermore, the in-house reliability growth testing usually overestimates the actual product reliability because the field usage conditions may not be perfectly simulated during testing. Typical values for the are around 1.2. Higher values yield less risk for the program, but require a more rigorous reliability growth test plan. Lower values imply higher program risk, with less safety margin.

During the planning stage, the growth potential MTBF, can be calculated based on the goal MTBF, and the growth potential design margin, :

or in terms of failure intensity:

## Nominal Idealized Growth Curve

During developmental testing, management should expect that certain levels of reliability will be attained at various points in the program in order to have assurance that reliability growth is progressing at a sufficient rate to meet the product reliability requirement. The idealized curve portrays an overall characteristic pattern, which is used to determine and evaluate intermediate levels of reliability and construct the program planned growth curve. Note that growth profiles on previously developed, similar systems provide significant insight into the reliability growth process and are valuable in the construction of idealized growth curves.

The nominal idealized growth curve portrays a general profile for reliability growth throughout system testing. The idealized curve has the baseline value until an initialization time, when reliability growth occurs. From that time and until the end of testing, which can be a single or, most commonly, multiple test phases, the idealized curve increases steadily according to a learning curve pattern until it reaches the final reliability requirement, . The slope of this curve on a log-log plot is the growth rate of the Crow extended model [13].

### Nominal Failure Intensity Function

The nominal idealized growth curve failure intensity as a function of test time is:

and:

where is the initial system failure intensity, is test time and is the initialization time, which is discussed in the next section.

It can be seen that the first equation for is the failure intensity equation of the Crow extended model.

### Initialization Time

Reliability growth can only begin after a type B failure mode occurs, which cannot be at a time equal to zero. Therefore, there is a need to define an initialization time that is different from zero. The nominal idealized growth curve failure intensity is initially set to be equal to the initial failure intensity, until the initialization time, :

Therefore:

Then:

Using the equation for initial failure intensity:

we substitute to get:

Then:

The initialization time, allows for growth to start after a type B failure mode has occurred.

### Nominal Time to Reach Goal

Assuming that we have a target MTBF or failure intensity goal, we can solve the equation for the nominal failure intensity to find out how much test time, , is required (based on the Crow extended model and the nominal idealized growth curve) to reach that goal:

Note that when or, in other words, the initial failure intensity is lower than the goal failure intensity, then there is no need to solve for the nominal time to reach the goal because the goal is already met. In this case, no further reliability growth testing is needed.

### Growth Rate for Nominal Idealized Curve

The growth rate for the nominal idealized curve is defined in the same context as the growth rate for the Duane postulate [8]. The nominal idealized curve has the same functional form for the growth rate as the Duane postulate and the Crow-AMSAA (NHPP) model.

For both the Duane postulate and the Crow-AMSAA (NHPP) model, the average failure intensity is given by:

Also, for both the Duane postulate and the Crow-AMSAA (NHPP) model, the instantaneous failure intensity is given by:

Taking the difference, between the average failure intensity, and the instantaneous failure intensity, , yields:

- Then:

For reliability growth to occur, must be decreasing.

The growth rate for both the Duane postulate and the Crow-AMSAA (NHPP) model is the negative of the slope of as a function of :

The slope is negative under reliability growth and equals:

The growth rate for both the Duane postulate and the Crow-AMSAA (NHPP) model is equal to the negative of this slope:

The instantaneous failure intensity for the nominal idealized curve is:

The cumulative failure intensity for the nominal idealized curve is:

- therefore:

- and:

Therefore, in accordance with the Duane postulate and the Crow-AMSAA (NHPP) model, is the growth rate for the reliability growth plan.

### Lambda - Beta Parameter Relationship

Under the Crow-AMSAA (NHPP) model, the time to first failure is a Weibull random variable. The MTTF of a Weibull distributed random variable with parameters and is:

The parameter lambda is defined as:

Using the equation for lambda in the MTTF relationship, we have:

or, in terms of failure intensity:

## Actual Idealized Growth Curve

The actual idealized growth curve differs from the nominal idealized curve in that it takes into account the average fix delay that might occur in each test phase. The actual idealized growth curve is continuous and goes through each of the test phase target MTBFs.

### Fix Delays and Test Phase Target MTBF

Fix delays reflect how long it takes from the time a problem failure mode is discovered in testing, to the time the corrective action is incorporated into the system and reliability growth is realized. The consideration of the fix delay is often in terms of how much calendar time it takes to incorporate a corrective action fix after the problem is first seen. However, the impact of the delay on reliability growth is reflected in the average test time it takes between finding a problem failure mode and incorporating a corrective action. The fix delay is reflected in the actual idealized growth curve in terms of test time.

In other words, the average fix delay is calendar time converted to test hours. For example, say that we expect an average fix delay of two weeks. If in two weeks the total test time is 1,000 hours, then the average fix delay is 1,000 hours. If in the same two weeks the total test time is 2,000 hours (maybe there are more units available or more shifts), then the average fix delay is 2,000 hours.

There can be a constant fix delay across all test phases or, as a practical matter, each test phase can have a different fix delay time. In practice, the fix delay will generally be constant over a particular test phase. denotes the fix delay for phase where is the total number of phases in the test. The RGA software allows for a maximum of seven test phases.

### Actual Failure Intensity Function

Consider a test plan consisting of phases. Taking into account the fix delay within each phase, we expect the actual failure intensity to be different (i.e., shifted) from the nominal failure intensity. This is because fixes are not incorporated instantaneously; thus, growth is realized at a later time compared to the nominal case.

Specifically, the actual failure intensity will be estimated as follows:

**Test Phase 1**

For the first phase of a test plan, the actual idealized curve failure intensity, , is :

Note that the end time of Phase 1, must be greater than . That is, .

The actual idealized curve initialization time for Phase 1, is calculated from:

where

Therefore, using the equation for the initialization time, we have:

By obtaining the initial failure intensity for , we get:

**Test Phase **

For any test phase , the actual idealized curve failure intensity is given by:

where and is the test time of each corresponding test phase.

The actual idealized curve MTBF is:

### Actual Time to Reach Goal

The actual time to reach the target MTBF or failure intensity goal, can be found by solving for the actual idealized curve failure intensity:

Since the actual idealized growth curve depends on the phase durations and average fix delays, there are three different cases that need to be treated differently in order to determine the actual time to reach the MTBF goal. The cases depend on when the actual MTBF that can be reached within the specific phase durations and fix delays becomes equal to the MTBF goal. This can be determined by solving for the actual idealized curve failure intensity for phases through , and then solving in terms of actual idealized curve MTBF for each phase and finding the phase during which the actual MTBF becomes equal to the goal MTBF. The three cases are presented next.

**Case 1: MTBF goal is met during the last phase**

If indicates the cumulative end phase time for the last phase, and indicates the fix delay for the last phase, then we have:

Starting to solve for yields:

We can substitute the left term by solving for the nominal time to reach the goal; thus, we have:

- therefore:

**Case 2: MTBF goal is met before the last phase**

The equation for that was derived for case 1 still applies, but in this case and are the time and fix delay of the phase during which the goal is met.

**Case 3: MTBF goal is met after the final phase**

If the goal MTBF, is met after the final test phase, then the actual time to reach the goal is not calculated since additional phases have to be added with specific duration and fix delays. The reliability growth program needs to be re-evaluated with the following options:

- Add more phase(s) to the program.
- Re-examine the phase duration of the existing phases.
- Investigate whether there are potential process improvements in the program that can reduce the average fix delay for the phases.

Other alternative routes for consideration would be to investigate the rest of the inputs in the model:

- Change the management strategy.
- Consider if further program risk can be acceptable, and if so, reduce the growth potential design margin.
- Consider if it is feasible to increase the effectiveness factors of the delayed fixes by using more robust engineering redesign methods.

Note that each change of input variables into the model can significantly influence the results.

With that in mind, any alteration in the input parameters should be justified by actionable decisions that will influence the reliability growth program. For example, increasing the average effectiveness factor value should be done only when there is proof that the program will pursue a different, more effective path in terms of addressing fixes.

## Reliability Growth Planning Examples

### Growth Plan for 3 Phases

A complex system is under design. The reliability team wants to create an overall reliability growth plan based on the Crow extended model. The inputs to the model are the following:

- The requirement or goal MTBF is hours.
- The growth potential design margin factor is .
- The average effectiveness factor is
- The management strategy is .
- The beta parameter for the discovery function, of the type B failure modes is
- The test will be conducted in three phases. The cumulative phase end times are and hours. The average fix delays for each phase are , and test hours, respectively.

Find the following:

- The growth potential MTBF and failure intensity.
- The initial failure intensity.
- The type A and type B initial failure intensity.
- The parameter of the Crow extended model.
- The type B failure mode discovery function.
- The initialization time, for the nominal failure intensity function.
- The nominal idealized failure intensity function.
- The nominal time to reach the MTBF goal.
- The MTBF that can be reached at the end of the last test phase based on the nominal idealized growth curve.
- The actual initialization time for phase 1,
- The MTBF that can be reached at the end of the last test phase based on the actual idealized growth curve.
- The actual time to reach the MTBF goal.
- The actual time to reach the MTBF goal if the cumulative end phase time for the third (last) phase is hours.
- Use the RGA software to generate the results for this example. Consider the cumulative last phase as hours, as given in question 13.

**Solution**

- The growth potential MTBF is:
- the growth potential failure intensity is:

- The initial failure intensity is given by:
- The type A failure mode intensity, is:
- The type B failure mode intensity, is:

- Using the inputs of and in the failure intensity equation for the Crow-AMSAA (NHPP) model, we have:
- or:

- or:

- The type B failure mode discovery function is:
- Since we know the and parameters, we have:

- The initialization time, is:
- or:

- The nominal idealized growth curve is given by the following equation:
- or:

- The nominal time to reach the MTBF goal is:
- For the goal failure intensity we have:

- So the nominal time to reach the MTBF goal is:

- Using the equation for the nominal failure intensity, we find the failure intensity that can be reached at the end of the last test phase based on the nominal idealized growth curve, . The total (cumulative) test time is hours. Therefore we have:
- So the nominal final MTBF that can be reached at 4500 hours of test time is:

- We can find the actual initialization time by using:
- For question 6, we had found that . So we have:

- The failure intensity that can be reached at the end of the last test phase based on the actual idealized growth curve for hours is given by:
- therefore:

- therefore:

- So the actual final MTBF that can be reached at 4500 hours of test time is:

- From question 11, it is shown that the actual final MTBF that can be reached at 4,500 hours of test time is less than the MTBF goal for the program. So, in accordance with the third case that is described in Actual Time to Reach Goal section, this is the scenario where the actual time to meet the MTBF goal will not be calculated since the reliability growth program needs to be redesigned.
- The failure intensity that can be reached at the end of the last test phase based on the actual idealized growth curve for hours is given by:
- therefore:

- therefore:

- So, the actual final MTBF that can be reached at 6000 hours of test time is:

- In this case, the actual MTBF goal becomes higher than the target MTBF sometime during the last phase. Therefore, we can determine the actual time to reach the MTBF goal by using:

- therefore:

- then:

- To use RGA to obtain the results for this example, first we create a growth planning folio by choosing
**Insert > Folios > Growth Planning**. In the growth planning folio, specify the number of phases, the cumulative phase time and average fix delays for each of the three phases, as shown in the figure below.

- We then enter the remaining input needed by the Crow extended model, as shown in the following figure.

- The input provided is the required MTBF goal of 56, the growth potential design margin of 1.35, the management strategy of 0.95 and the discovery beta of 0.7. The results show the initialization time, , the initial MTBF, the final actual MTBF that can be reached for this growth program and the actual time when the MTBF goal is met. Note that in this case we provided the goal MTBF and solved for the initial MTBF.

- Different calculation options are available depending on the desired input and output. For example, we could provide the initial and goal MTBF and solve for the growth potential design margin, as shown in the figure below.

- Click the
**Plot**icon to generate a plot that shows the nominal and actual idealized growth curve, the test termination time, the MTBF goal and the planned growth for each phase, as shown in the figure below.

### Growth Plan for 7 Phases

The reliability team of a product manufacturer is putting together a reliability growth plan for one of the new products under design. The team wants to create an overall reliability growth plan based on the Crow extended model. The inputs to the model are the following:

- The requirement or goal MTBF is hours.
- The growth potential design margin factor is .
- The average effectiveness factor is based on historical information for similar products developed by the company.
- The management strategy is .
- The beta parameter for the discovery function, of the type B failure modes is , based on a data analysis of the previous product development project that was of similar nature.
- The test will be conducted in seven phases. The cumulative phase end times are , and hours. The average fix delay for each phase is one week.

- For the first two phases, the plan is to test 1,000 hours per week, so the average fix delay in terms of test hours for phases 1 and 2 is and hours.
- For phases 3 and 4, the prototype test units are planned to be doubled, so the average fix delay in terms of test hours within one week is also going to be doubled, so we have: and hours.
- For phases 5 and 6, a second shift is going to be added, so the amount of test hours within a week is going to be doubled again. Therefore, the average fix delay in terms of test hours within one week is also going to be doubled: and hours.
- For the last phase, more units and weekend shifts are planned to be added, so the total test hours within one week are going to be 6,000 hours. In accordance, the average fix delay in terms of test hours will be hours.

Construct a reliability growth plan using the RGA software and make sure that the goal MTBF can be met within the total test hours allocated for growth testing. If not, make necessary changes in the phase durations in order to meet the goal.

**Solution**

The following figure shows the planning input in the RGA software.

The next figure shows the planning calculations based on the goal MTBF of 100 hours and the rest of the known inputs into the model. As seen by the planning calculation results, the final actual MTBF at the end of the seventh phase is 98.66 hours, which is less than the MTBF goal of 100 hours.

Also, the next figure shows that the nominal time to meet the MTBF goal is 48,420 hours, which is higher than the total test time that is currently planned to be allocated for reliability growth.

A new plan needs to be considered. The reliability team decides to increase the test time for phases 5 and 6 to be 10,000 hours each, instead of the 5,000 hours that were considered in the initial plan. Also, the team decides to increase the duration of the 7th phase to 15,000 hours. Since this is going to be achieved by testing for more calendar time in phases 5, 6 and 7, the average fix delay during those phases is the same as the one in the initial plan. The test time per week is not going to be affected, only more weeks will be added to the schedule. The next figure shows the revised plan, together with the calculated results. The actual time to meet the MTBF goal is now 54,331 hours. The final MTBF that can be achieved at the end of the 55,000 hours of growth testing is 100.04, which is slightly higher than the goal MTBF of 100 hours. The reliability team considers this plan as acceptable, since the MTBF goal will be met.

The next plot shows the overall reliability growth plan, with the nominal and actual growth curves, the MTBF goal line and the planned MTBF for each of the seven phases.

### Growth Plan for 4 Phases

The reliability team of a product manufacturer has put together a reliability growth plan, based on the Crow Extended model, for one of their new products. The reliability growth model was constructed with the following inputs:

- The requirement or goal MTBF is hours.
- The growth potential design margin factor is .
- The average effectiveness factor is
- The management strategy is .
- The beta parameter for the discovery function, of the type B failure modes is .
- The test is planned to be conducted in four phases. The cumulative phase end times are , and . The average fix delay in terms of test hours for each phase is: , and hours.

- Plot the nominal and actual reliability growth curves for this program, using the RGA software.
- The reliability program was initiated and actual test data from phase 1 is now available. The test data were analyzed using the Crow extended-continuous evaluation model and the following figure shows the results. Note that the
*I*events, which represent the times for implementation of fixes for BD modes, occur at least 10,000 hours after the first occurrence of the specific BD mode. This is a reflection of the average fix delay being equal to 10,000 hours for the first test phase. Of course we could have some exceptions that could have been fixed before 10,000 hours after the first occurrence (i.e., discovery) of the mode, but on average, they should be fixed after 10,000 hours.

The same applies to the delayed fixes at the end of phase 1. On average there should be at least 10,000 test hours between the discovery of the failure mode and the implementation of a fix for that mode, so failure modes discovered after 9,000 hours of testing in the first phase cannot be implemented at the end of the first phase.

The figure below shows the effectiveness factor window. The record shows that the fixes for failure modes BD 11000, BD 12000 and BD 13000 are not implemented at the end of the first phase due to the fix delay. The effectiveness factor used is in accordance with the planning model and has been fixed to 0.7.

Construct a multi-phase graph that shows the analysis points for the test data in phase 1, as compared to the nominal and actual idealized growth curves for this reliability growth plan. Is the program on track, so far, as compared to the plan?

**Solution**

- The following figures show the plan inputs, phase durations and average fix delays as entered in the RGA software, together with the results of the growth model.
- The next figure shows the nominal and actual idealized growth curves for this program, together with the program MTBF goal.

- The next step is to associate the reliability growth plan with the data from phase 1. To do that, right-click the
**Additional Plots**header in the Current Project Explorer and choose**Add MultiPhase Plot**from the shortcut menu, then follow the wizard to specify the data sheet that contains the phase 1 data with the folio that defines the growth plan. The figure below shows the generated multiphase plot, which brings together the reliability growth planning model with the actual test results. This plot shows the demonstrated MTBF for each analysis point during the first phase of testing. Analysis points for projected and growth potential MTBF can also be plotted. For this example, it can be said that the program is on track since the demonstrated MTBF of the first phase is higher than the planned MTBF for that phase.- The multiphase plot can be updated continuously as the growth program progresses and more test data become available. In that manner, the reliability team can assess the actual reliability performance against the established goals.