Seperating uncertainty from randomness and variability

Download a pdf copy of this help file  here

See also: Model design introduction, Building models that are easy to check and modify

The three uses of distributions, for modeling randomness (R), inter-individual variability (V) and uncertainty (U), mean that we can get very confused about how to treat each in the same risk analysis model. Failing to get it right can have a very big impact.

Start with a V/R model

Randomness (R) and inter-individual variability (V) are properties of the real world, and as such form the base of our risk analysis model. Uncertainty (U), i.e. the degree of knowledge we have about the parameters enumerating R and V, does not affect how the real world operates, and so is overlaid onto a V/R model. This section discusses how to put together a V/R model. Then you should consider how to overlay uncertainty for the model's parameters.

A risk analysis model that separates uncertainty from randomness and variability (or uncertainty and randomness from variability) is described as second-order. A V/R model comes in two forms: calculated and simulated.

Overlay uncertainty onto the base model

The base model built of elements of randomness (R) and variability (V) will include parameters (like a binomial probability, a population mean, a Poisson intensity) for which we need values. Almost always we will not know these parameter values precisely and rely on statistics and/or expert judgment to provide estimates and the uncertainty around those estimates. Uncertainty, then, is simply overlaid onto the V/R model.

Including uncertainty about fitted parameters with ModelRisk

If we use the fitting functions of ModelRisk to get our parameter values, overlaying uncertainty can be done easily through the optional uncertainty parameter that is provided for each fitting function (distributions, copulas and time series). When this parameter is set to TRUE, the uncertainty on the parameters of the fitted model is automatically included, using bootstrapping techniques. A new value for the uncertain fitted parameter will then be used on each spreadsheet recalculation.  

The ModelRisk distribution fitting functions (and VoseOgiveU) simulate random values from fitted distributions with uncertainty. These functions can be entered into spreadsheets in two ways:

1.      In one cell, or in many cells, but not as array function

2.      In many cells as array function

In the first case, the uncertainty and variability are mixed, because each random value is sampled form a different distribution. However in the second case, all random values are sampled from the same distribution and the distribution will change only with the next iteration.

The uncertainty parameter is present in both the VoseFit functions to directly simulate from a fitted model, and in the VoseFitP functions that return only the fitted parameters. So they can be used in each of the techniques described below.

There are a number of different approaches in which we can overlay uncertainty onto a V/R model:

1. Variability and Randomness are calculated, and uncertainty is simulated (VC/RC/US model)

This option preserves the separation of subjective uncertainty from the physical model. It is the easiest to understand, but the base V/R model is the most difficult to construct, so it is generally only useful for simple models.

2. Variability is calculated, Randomness and Uncertainty are simulated (VC/RS/US model)

This option will blend uncertainty and randomness together as they are being simulated together. It is quick to construct the model, quick to simulate, but one loses the ability to separately analyze the random and uncertainty components in second order plots. It's a small loss for most applications, and a model that simulates uncertainty and randomness together can easily be extended to make them separate so in general we recommend this approach most. It has the added advantage that one can see the interaction between probability and uncertainty distributions that is not possible with a V/R calculation model.

3. Variability, Randomness and Uncertainty are simulated together (VS/RS/US model)

This option may be necessary when there is a great deal of variability that you need to include in your model, but it is full of potential traps.

4. Variability is calculated, Randomness is simulated and Uncertainty is simulated in second loop (VC/RS/UL model)

Although somewhat unwieldy, this model structure allows you to get the greatest benefit from simulation to avoid complicated probability maths, but keeps you away from the difficult area of simulating Variability.

5. Variability is simulated, Randomness is simulated in 1st loop and Uncertainty is simulated in 2nd loop (VS/RL1/UL2 model)

In principle this type of model makes logical sense:

1. Select an individual with specific characteristics from variability distributions and place values for variability parameters into model (start loop 1);

2. Draw values from each uncertainty distribution for all uncertain parameters and place in risk model (start loop 2);

3. Simulate model and save results;

4. Repeat steps 2 and 3 until to get a 2nd order distribution for the selected individual (end loop2);

5. Select another individual from the variability distributions, and repeat steps 1 to 4 to get a sufficient representation of the variability of the population (end loop1)

In practice, such a model would probably be better undertaken using a modeling platform other that Excel.