Spider plots

 Spider plots describe how sensitive the value of an output variable is to the input variables of the model. ModelRisk uniquely offers a much faster method of producing spider plots than competing software products that is also more technically correct.

Spider plots can be produced in ModelRisk by selecting an output variable in the Simulation Results window and clicking:

Producing a spider plot requires making the following choices:

1. Select the output of interest.

2. Select the statistic of interest by clicking on one of the following icons:  for the conditional mean, conditional standard deviation, conditional coefficient of variation or conditional percentile respectively. If Percentile has been selected, click  to define the required percentile.

3. Select the number of tranches to be used by clicking . The number of tranches define the number of points that will be plotted for each input variable. In the graph above this is 10, for example.

Interpretation

In the plot above, an analysis has been performed of the sensitivity of the mean of the ‘Total Revenue’ output. 10 tranches have been used. This means that an analysis has been performed by splitting up simulation data from input distributions into ten groups in terms of their cumulative probability: 0%-10%, 10%-20%, 20%-30%, …, 90%-100%.

The simulation data are filtered for each of these groups to find the corresponding output values that occurred when the input variable being analyzed lies within each percentile band listed above. The statistic of interest (the mean in the example above) is then calculated for the filtered data. Repeating this analysis across each tranche for each selected input variable produces the spider plot.

In the plot above, the horizontal dashed line shows the mean of the unfiltered output values as a reference (in this case about 1880). The vertical range that an input line covers reflects the degree of sensitivity that output statistic has to this input value. So, for example, when Task 5 lies in its 0%-10% range, the Total Revenue mean is approximately 1180, and when Task 5 lies in its 90%-100% range, the Total Revenue mean is approximately 2840 – a range of 1660. Reviewing the graph, one can easily see that the output mean is least sensitive to Task 1.

Why use spider plots?

The sensitivity analysis tool most commonly offered in Monte Carlo simulation is a tornado plot using rank order correlation, which provides a statistical measure of correlation between the input and output generated values. However, this leaves the user with the task of trying to understand how important a (for example) 0.63 rank order correlation is to their business decision. ModelRisk’s spider plots give a sensitivity scale in terms of the output value, which is far more intuitive to the user.

Moreover, correctly performed spider plots allow one to analyze output sensitivity in situations where rank order correlation, or even regression-type analysis, would fail to pick up any significant relationship. For example, in the plot above the output mean has a ‘U-shaped’ sensitivity to Task 4. A simple regression or correlation analysis would show a very small correlation, yet Task 4 has a very large impact.

Note: Some Monte Carlo software tools competing with ModelRisk offer spider plots in what they describe as ‘Advanced Sensitivity Analysis’. These do not perform the same analysis. They fix individual variables at defined percentiles, require one to predetermine the output and statistic of interest, and rerun the simulation. This means that for say the equivalent of 10 tranches and 7 input variables the model needs to run 7*20 + 1 = 141 times. ModelRisk requires only one single simulation run to complete its spider plot. Moreover, one does not have to predetermine the output or statistic. Most importantly, the ModelRisk approach retains any correlation relationships within the model, which are lost in competing software as they fix input variables to specific values but cannot control variables correlated to them.