Planning a risk analysis

In order to plan a risk analysis properly you'll need to answer a few questions:

• What do you want to know and why?

• What assumptions are acceptable?

• What is the timing?

• Who is going to do the risk analysis?

Questions and motives

The purpose of a risk analysis is to provide information to help make better decisions in an uncertain world. A decision maker has to work with the risk analyst to precisely define the questions that need answering. You should consider a number of things:

1. Rank the questions that need answering from 'critical' down to 'interesting'. Often a single model cannot answer all questions, or has to be built in a complicated way to answer several questions, so a common recognition of the extra effort needed to answer each question going down the list helps determine a cut-off point;

2. Discuss with the risk analyst the form of the answer. For example, if you wanted to know how much extra revenue might be made by buying rather than leasing a vessel, you'll need to specify a currency, whether this should be as a percentage or in actual currency, whether you want just the mean (that can make the modelling a lot easier) or a graph of the distribution. Explain what statistics you need and to what accuracy (e.g. asking for the 95^th percentile to the nearest $1000) as this will help the risk analyst save time or figure out whether an unusual approach might be needed to get the required precision;

3. Explain what arguments will be based on these outputs. This is a key breakdown area because a decision-maker might ask for specific outputs and then puts them together into an argument that is probabilistically incorrect. Much embarrassment and frustration all round. It is better to explain the arguments (e.g. comparing with the distribution of another potential project's extra revenue) that would be put forward and find out if the risk analyst agrees that this is technically correct before you get started;

4. Explain whether the risk analysis has to sit within a framework. This could be a formal framework, like a regulatory requirement or a company policy, or it could be informal like building up a portfolio of risk analyses that can be compared on the same footing (for example, we are helping a large chemical manufacturer to build up a combined toxicological, environmental, etc. risk analysis database for their treasure chest of compounds). It will help the risk analyst ensure the maximum level of compatibility - e.g. that the same base assumptions are used between risk analyses;

5. Explain the target audience. Sometimes there can be several report versions: the executive summary; the main report; and the technical report with all the formulae and guides for testing. Often others will want to run the model and change parameters, in which case consider putting the model into ModelRisk Cloud, which eliminates the possibility to mess up the mathematics or have various unauthorized, edited versions of the same model around the company. Knowing the knowledge level and focus of each target audience, and knowing what types of reporting will be needed at the offset saves a lot of time;

6. Discuss any possible hostile reactions. The results of a risk analysis will not always be popular and when people dislike the answers they start attacking the model (or, if you're unlucky the modeler). Assumptions are the primary Achilles' heel since we can argue forever about whether assumptions are right. Arguments about statistical analysis of data is also rather draining - it usually involves a couple of very technical people with opposing arguments about the appropriateness of a statistical procedure that nobody else understands. The decision to include and exclude certain data sets can also create a lot of tension. The arguments can be minimized, or at least convincingly dismissed, if people likely to be hostile are brought into the analysis process early, or an external expert is asked to give an independent review;

7. Figure out a timeline. Decision makers have something of a habit of setting unrealistic deadlines. When those deadlines pass, nothing very dramatic usually happens since the deadline was some artificial internal confection. It is best to openly discuss whether a deadline is really that important because if one has to meet a tight deadline (and that happens) the quality of the risk analysis may be lower than was achievable with more time. The decision-maker has to be honest about time limits and decide whether it is worth postponing things for a bit;

8. Figure out the priority level. The risk analyst might have other work to juggle too. The project might be of high importance and justify pulling off other resources to help with the analysis or instructing others in the organization to set aside time to provide good quality input; and

9. Decide on how regularly the decision maker and risk analyst will meet. Things change and the risk analysis may have to be modified, so find that out sooner rather than later.

Determine the assumptions that are acceptable or required

If a risk analysis is to sit within a certain framework, discussed above, it may well have to comply with a set of common assumptions to allow meaningful comparisons between each risk analysis' results. Sometimes it is better not to revise some assumptions for a new analysis because it makes it impossible to compare. You can often see a similar problem with historic data, e.g. calculating crime or unemployment statistics. It seems that the basis for these statistics keeps changing making it impossible to know whether the problem is getting better or worse.

In a corporate environment there will be certain base assumptions used for things like interest and exchange rates, production capacity and energy and raw material prices. The same stochastic assumptions should be used in all models, which can be achieved using SIDs. In a risk analysis world these should be probabilistic forecasts but they are nonetheless often fixed point values. Oil companies, for example, have the challenging job of figuring out what the oil price might be in the future. They can get it very wrong so often take a low price for planning purposes: e.g. $16 a barrel which in 2007 might seem rather unlikely for the future. The risk analyst working hard on getting everything else really precise could find such an assumption irritating, but it allow consistency between analyses where assigning distributions of uncertainty to oil price forecasts would be so large as to mask the differences between investment opportunities.

Some assumptions we make are conservative meaning that if, for example, we need a certain percentile of the output to be above X before we accept the risk as acceptable, then a conservative assumption will bias the output to lower values. Thus, if the output still gives numbers that say the risk is acceptable we know we are on pretty safe ground. Conservative assumptions are most useful as a sensitivity tool to demonstrate that one has not taken an unacceptable risk, but they are to be avoided whenever possible because they run counter to the principle of risk analysis which is to give an unbiased report of uncertainty.

Time and timing

If done properly, risk analysis is an integral part of the planning of a project, not an add-on at the end. One of the prime reasons for doing risk analyses is to identify risks and risk management strategies so the decision-makers can decide how the risks can be managed which could well involve a revision of the project plan. That can save a lot of time and money on a project: if risk analysis is added on at the end, you lose all that potential benefit.

The data collection efforts required to produce a fixed-value model of a project are little different from the efforts required for a risk analysis, so adding a risk analysis on at the end is inefficient and delays a project as the risk analyst has to go back over previous work.

It is good practice for a risk analyst to write up the report as the model develops. It helps keep a track of what one is doing and makes it easier to meet the report submission deadline at the end. Writing down the logic also helps one spot any mistakes early.

Finally, try to allow the risk analyst enough time to check the model for errors and get it reviewed. topic

© Vose Software™ 2017. Reference Number: M-M0918-A