R&D Portfolio Selection: Optimizer Example | Vose Software

ModelRisk Optimizer example · Pharmaceuticals and biotech

R&D Portfolio Chosen by the Optimizer

Which projects to fund for the most value, with limits on budget risk and on the chance of no launch?

  • Complexity: ★★★★★
  • Free Excel workbook (PH-09)
  • Needs ModelRisk with the Optimizer: free trial

From the model’s saved run

The Optimizer’s plan against the starting plan

The two portfolios, and how each performs on the same simulated futures (US$m).

Portfolio NPV — cumulative probability
DecisionStageStarting planOptimizer’s planChange
A ONC-301 solid tumoursPhase IIIfundfund–
B ONC-210 leukaemiaPhase IIfundfund–
C IMM-220 psoriasisPhase IIfundshelveddropped
D IMM-110 Crohn's diseasePhase Ishelvedshelved–
E NEU-120 Alzheimer's diseasePhase Ifundfund–
F NEU-010 Parkinson's diseasePreclinicalshelvedfundadded
G RARE-230 enzyme deficiencyPhase IIfundfund–
H ONC-020 lymphomaPreclinicalshelvedshelved–
MeasureUnitStarting planOptimizer’s planChange
Projects fundedprojects550
Portfolio NPV, meanUS$m556545−12
Portfolio NPV, 1 in 10 (10th percentile)US$m−437−39938✔ better
Launches, expectedproducts1.341.22−0.13
Chance of no launchchance18.4%21.2%2.8%
Chance of breaking the budgetchance35.2%19.7%−15.4%✔ better
R&D cost, mean (present value)US$m515468−47✔ better
Budget risk accepted, at mostchance25%25%
No-launch risk accepted, at mostchance25%25%

Breaking the budget: spending above US$170 million in at least one of the next 5 years.

Spending in the busiest budget year — cumulative probability

The business question

A mid-sized pharma company has eight development projects, from preclinical work to a Phase III cancer drug. Each phase costs money and may fail, and three pairs share a mechanism, so their Phase II results tend to agree. PH-04 compared four portfolios: one project per mechanism kept most of the value but broke the US$170 million budget about one time in three.

Here the ModelRisk Optimizer searches all 256 portfolios.

Who it is for
R&D portfolio committees, heads of development and strategy, finance teams in pharma and biotech.
Main outputs
The Optimizer's portfolio against a starting plan (PH-04's one project per mechanism) · for both: NPV (mean and 1 in 10), expected launches, the chance of no launch, the chance of breaking the budget, spending in the busiest year and R&D cost · for each project: chance of launch, mean NPV and value per dollar · spending against the budget.

What the Optimizer decides

1 = fund the project, 0 = shelve it. The starting plan is PH-04's portfolio of one project per mechanism: A, B, C, E and G.

DecisionStarting planOptimizer’s plan
A ONC-301 solid tumoursYesYes
B ONC-210 leukaemiaYesYes
C IMM-220 psoriasisYesNo
D IMM-110 Crohn's diseaseNoNo
E NEU-120 Alzheimer's diseaseYesYes
F NEU-010 Parkinson's diseaseNoYes
G RARE-230 enzyme deficiencyYesYes
H ONC-020 lymphomaNoNo
Goal
Maximise the mean NPV
Limit
Chance of breaking the budget at most 25% (the chance of overspending in a budget year)
Limit
Chance of no launch at most 25% (the chance that no project launches)

The Optimizer’s run: Vertex Detector, 256 plans tried, each on 10,000 samples, in 86.3 seconds.

How the model works

Six sheets after this one, read from left to right. Each one feeds the next.

  1. Inputs sheet

    The annual R&D budget and its horizon, the correlation between the Phase II results of projects that share a mechanism, the discount rate, and the two limits: the budget risk and the no-launch risk accepted.

  2. Projects sheet

    The duration and yearly cost of each phase. Each project's current stage, chance of success in each phase and value at launch (P10 and P90).

  3. Decisions sheet

    The eight cells the Optimizer changes — fund (1) or shelve (0) each project — beside the starting plan, with the goal (the highest mean NPV) and the limits (the chance of breaking the budget and the chance of no launch).

  4. Pipeline sheet

    Three bivariate Normal copulas give correlated uniforms for the Phase II results of each pair; VoseBernoulli decides every remaining phase. The last phase each project works on, the launch year and the value at launch, discounted to today, follow.

  5. Spend sheet

    The phase each project works on in each year gives its spending. Each plan adds up the projects it funds: its NPV, its launches, and whether it spends more than the budget in a year of the budget horizon.

  6. Results sheet

    Both plans on the same simulated outcomes: NPV, launches, budget risk, spending in the busiest budget year and R&D cost; each project's value per dollar; spending of the Optimizer’s plan against the budget; the shared mechanisms.

ModelRisk techniques

  • ModelRisk Optimizer: eight yes/no decisions
  • Budget risk and no-launch risk as probability limits
  • Correlated Phase II results (copula)
  • Both portfolios on the same samples

Find similar models

The results in detail

The projects

Each project as if funded: chance of launch, value and R&D cost (US$m).

Spending of the Optimizer’s plan against the budget
ProjectStageLaunch chanceMean NPVR&D cost PV, meanValue per US$ of R&D
A ONC-301 solid tumoursPhase III55%4291593.71
B ONC-210 leukaemiaPhase II19%40921.43
C IMM-220 psoriasisPhase II18%18911.19
D IMM-110 Crohn's diseasePhase I12%−3680.95
E NEU-120 Alzheimer's diseasePhase I8%25621.41
F NEU-010 Parkinson's diseasePreclinical5%6441.14
G RARE-230 enzyme deficiencyPhase II34%441121.40
H ONC-020 lymphomaPreclinical8%−2480.96

Value per US$ of R&D: the expected present value of the launch ÷ the expected R&D cost. Below 1, the project destroys value.

Shared mechanisms

Projects that share a mechanism succeed — and fail — together.

PairBoth launch simulatedBoth launch if independentRatio
B and H3.3%1.5%2.1
C and D4.3%2.2%2.0
E and F1.1%0.5%2.5

How to read these results

  • The starting plan, PH-04's one project per mechanism (A, B, C, E and G), is worth about US$555 million on average but breaks the budget about one time in three: when two of its three Phase II projects succeed, their Phase III trials start while A's is still running.
  • The Optimizer keeps A, B, E and G, shelves C, the psoriasis drug, and funds F, the early Parkinson's project, which spends little in the budget years. It gives up about US$12 million of mean NPV, about 2%, and cuts the chance of breaking the budget to about one in five.
  • The budget limit decides: every portfolio worth more on average breaks the budget at least a third of the time. The chance of no launch rises a little, to about 21%, inside its limit. The portfolio is not one of the four PH-04 compared: it adds B to PH-04's within-budget portfolio.
  • F shares E's amyloid mechanism, so their Phase II results tend to agree: in the third table, E and F launch together two to three times as often as if they were independent. When both succeed, their Phase III trials overlap in year 5, which adds a little to the budget risk. The Optimizer judges every portfolio with that correlation included.
  • Try a one-in-ten budget risk on the Inputs sheet and run the Optimizer again: it can then keep only one of the three Phase II projects, and the chance of no launch becomes the limit that binds.

Run the Optimizer yourself. Open the workbook, open the Optimizer from the ModelRisk ribbon (the saved task opens with it) and press Start; then simulate to compare the plans.

Inputs and assumptions

Every assumption in the model, with its uncertainty and where it comes from. The figures are illustrative.

Budget and finance 4 inputs
AssumptionUnitValue
R&D budgetFor each year of the budget horizon below.US$m a year170
Budget horizonThe years the budget is set for.years5
Correlation of Phase II results within a mechanismOn the Normal scale: if the mechanism fails in one project, it probably fails in the other.copula ρ0.80
Discount rateper year10.0%
Limits for the Optimizer 2 inputs

What the portfolio committee will accept. The Optimizer must respect both.

AssumptionUnitValue
Budget risk acceptedThe highest chance of spending more than the budget in a year of the horizon.chance25%
No-launch risk acceptedThe highest chance that none of the funded projects launches.chance25%

Units and conventions

Money
US$ million (US$m); costs discounted from the middle of each year, values from the launch year.
Time
Years from now; year 1 is the coming year.
Value at launch
The NPV, at launch, of the product's sales after all commercial costs.
Breaking the budget
A portfolio's spending exceeds the budget in at least one year of the budget horizon (five years as shipped).
No launch
None of the funded projects reaches the market in the ten years modelled.

Key assumptions and limitations

Phases
Whole years, back to back; a project stops spending when a phase fails.
Correlation
Only through the Phase II results of projects sharing a mechanism; all other results are independent.
Unfunded projects
Shelved: no cost and no value. Partnering them out could recover some value.
Optimizer
Each portfolio the Optimizer tries is judged on the same 10,000 simulated futures (common random numbers, seed 1) as the simulation on the Results sheet, so portfolios are compared fairly and the limits met in the run hold on the Results sheet too.
Data
The company, the projects and all figures are invented for this example.

How to use it

  1. Enter your budget, correlation, discount rate and the two limits on the Inputs sheet, and your projects on the Projects sheet.
  2. On the ModelRisk ribbon, open the Optimizer: the task is saved in this workbook. Press Start. When it finishes, apply the best solution: the funding cells on the Decisions sheet change.
  3. Run a simulation from the ModelRisk ribbon (10,000 samples with a fixed seed are set) to compare the new portfolio with the starting plan on the Results sheet.
  4. To watch the search from the beginning, copy the starting plan onto the Optimizer’s plan first.

ModelRisk functions used

ModelRisk Optimizer
Chooses the projects to fund: highest mean NPV, chance of breaking the budget and chance of no launch at most the limits on the Inputs sheet.
VoseCopulaBiNormal
Correlated uniforms for the Phase II results of two projects sharing a mechanism (array).
VoseBernoulli
Whether each phase succeeds; the U argument takes the copula's uniform for Phase II.
VoseLognormalAlt
Each project's value at launch from its P10 and P90.
VoseOutput, VoseSimMean, VoseSimPercentile, VoseSimProbability
Outputs and their statistics on the Results sheet.

Further reading

  • Monte Carlo simulation explained (Vose guide)
  • Paul, S. M. et al. (2010). How to improve R&D productivity: the pharmaceutical industry's grand challenge. Nature Reviews Drug Discovery 9, 203–214.
  • PH-04 R&D portfolio which projects to fund: the same eight projects, with four portfolios compared side by side.