 |
Download a pdf copy of this help file here |
|
Download a pdf copy of this help file here |
Spider plots describe how sensitive the value of an output variable is to the input variables of the model. ModelRisk uniquely offers a much faster method of producing spider plots than competing software products that is also more technically correct.
Spider plots can be produced in ModelRisk by selecting an output variable in the Simulation Results window and clicking:
Producing a spider plot requires making the following choices:
1. Select the output of interest.
2. Select the statistic of interest by clicking
on one of the following icons: 
for the conditional mean, conditional standard deviation, conditional
coefficient of variation or conditional percentile respectively. If Percentile
has been selected, click 
to
define the required percentile.
3. Select the number of tranches to be used
by clicking 
. The number of
tranches define the number of points that will be plotted for each input
variable. In the graph above this is 10, for example.
Interpretation
In the plot above, an analysis has been performed of the sensitivity of the mean of the ‘Total Revenue’ output. 10 tranches have been used. This means that an analysis has been performed by splitting up simulation data from input distributions into ten groups in terms of their cumulative probability: 0%-10%, 10%-20%, 20%-30%, …, 90%-100%.
The simulation data are filtered for each of these groups to find the corresponding output values that occurred when the input variable being analyzed lies within each percentile band listed above. The statistic of interest (the mean in the example above) is then calculated for the filtered data. Repeating this analysis across each tranche for each selected input variable produces the spider plot.
In the plot above, the horizontal dashed line shows the mean of the
unfiltered output values as a reference (in this case about 1880). The
vertical range that an input line covers reflects the degree of sensitivity
that output statistic has to this input value. So, for example, when Task
5 lies in its 0%-10% range, the Total Revenue mean is approximately 1180,
and when Task 5 lies in its 90%-100% range, the Total Revenue mean is
approximately 2840 – a range of 1660. Reviewing the graph, one can easily
see that the output mean is least sensitive to Task 1.
We could also have selected the conditional percentile by clicking 
and specified the 90th percentile by clicking
and typing ‘90’ in the dialog box:
The spider plot would then have shown the 90th percentile values for
the output after conditioning on each input lying within each tranche.
So, for example, with 10 tranches each line would describe how the 90th
percentile of the output would look the input corresponding to the line
in the graph were to lie in the 0-10%, 10-20%, …, 90-100% sections of
its distribution. That would tell us how sensitive the output right tail
is to the various inputs: the flatter the line, the less sensitive it
is.
Why use spider plots?
The sensitivity analysis tool most commonly offered in Monte Carlo simulation is a tornado plot using rank order correlation, which provides a statistical measure of correlation between the input and output generated values. However, this leaves the user with the task of trying to understand how important a (for example) 0.63 rank order correlation is to their business decision. ModelRisk’s spider plots give a sensitivity scale in terms of the output value, which is far more intuitive to the user.
Moreover, correctly performed spider plots allow one to analyze output sensitivity in situations where rank order correlation, or even regression-type analysis, would fail to pick up any significant relationship. For example, in the plot above the output mean has a ‘U-shaped’ sensitivity to Task 4. A simple regression or correlation analysis would show a very small correlation, yet Task 4 has a very large impact.
Note: Some Monte Carlo software tools competing with ModelRisk offer spider plots in what they describe as ‘Advanced Sensitivity Analysis’. These do not perform the same analysis. They fix individual variables at defined percentiles, require one to predetermine the output and statistic of interest, and rerun the simulation. This means that for say the equivalent of 10 tranches and 7 input variables the model needs to run 7*10 + 1 = 71 times. ModelRisk requires only one single simulation run to complete its spider plot. Moreover, one does not have to predetermine the output or statistic. Most importantly, the ModelRisk approach retains any correlation relationships within the model, which are lost in competing software as they fix input variables to specific values but cannot control variables correlated to them.