WhenToStop Frestimate module

This software reliability growth model tool is the only Frestimate component that is used exclusively during software testing or once there are observed failures to extrapolate current failure rate, MTBF, reliability and availability estimates into the future.

Reliability growth models are used late in the software development process. However, they are useful for validating predictions made earlier in the process. One common application of the reliability growth models is to establish "When To Stop" testing in order to meet a specific quantitative reliability objective.  While predictor models use empirical data to forecast reliability, reliability growth models use actual defect data during a system level test.

Software reliability growth models

View the WhenToStop tutorial

Key Capabilities

  • Estimate whether or not the required or predicted failure rate or MTTF objective will be met
  • Determine the test time and/or defects needed to reach an objective failure rate or MTTF
  • Determine When to stop testing given some quantitative objective
  • Determine the best objective for MTTF or failure rate given resources available
  • Extrapolate the MTTF growth over time
  • Determine the objective MTTF or failure rate given resource constraints
  • Determine staff persons needed to correct projected defects
  • Determine current failure rate, MTTF and reliability as well as project it into the future given the current trend.
  • Ability to import data from other defect tracking software programs
  • Easy to use and robust input mechanism for defect data
  • Ability to easily extrapolate growth, growth rates, actual and extrapolated MTTFs.
  • Graphical results that show you the actual versus predicted, relative error of the prediction and the growth expected
  • Detailed help files that describe the algorithms and how to interpret your results
  • Ability to export all plots to file and to clipboards
  • Ability to filter the defects by severity and calculate results only on selected severities
  • Detailed and summary reports
  • Determine the growth rate during testing
  • Automates the recommended Software Reliability Growth Models in the 2016 IEEE Recommended Practices for Software Reliability.

Screen Shots

The first step is to enter the number of observed software defects per day and the number of test/operation hours per day.

software reliability growth models

Next, Frestimate determines if the fault rate is increasing or decreasing. If it's decreasing then the models are able to estimate the reliability parameters. The total inherent defects is estimated by plotting the cumulative faults versus the cumulative fault rate. The y intercept of this graph is the estimated inherent defects. That estimate is used in several of the software reliability growth models. The slope of that graph as well as the x intercept are also used by the software reliability growth models. These are called the software reliability estimation parameters.

software reliability growth model parameter estimation

This shows the estimated software MTBF using various software reliability growth models.

estimate software MTBF

This shows relative accuracy of each of the software reliability growth models for all data points as well as for a user selected number of recent data points.

software reliability growth model accuracy

This shows the software reliability estimation confidence bounds for each of the models for all data points as well as for a user selected number of recent data points.

software reliability growth model confidence bounds

This shows the estimated remaining software defects and time required to reach a reliability objective.

estimate remaining software defects with the software reliability growth models