Industry: Utilities Product: ModelRisk Application: Energy efficiency optimization under uncertainty
An engineering study for a lighting, HVAC, and controls retrofit at a municipal water-treatment plant produced a confident number: 3,200 MWh/yr of electricity savings. On the strength of it, the energy-service company (ESCO) signed a performance guarantee promising 2,720 MWh/yr — 85% of the engineered figure — and agreed to write a shortfall cheque if the meters fell short. The trouble is that engineering models, measured against verified savings across thousands of retrofits, systematically over-predict. The number that actually decides whether the guarantee is met is not the engineered saving; it is the realisation ratio — the fraction of the engineered saving the meters confirm.
Rebuilt in ModelRisk, the realisation ratio comes out right-skewed with a mean of 0.88, a P10 of 0.75, and a P90 of 1.01 — centred well below the 1.00 the deterministic model silently assumes. The verified saving therefore has a mean of 2,807 MWh but only a 59% chance of clearing the 2,720 MWh guarantee. The headline distribution shows exactly where the guarantee lives.
The deterministic study reports 3,200 MWh and stops. Implicitly, it assumes the realisation ratio is exactly 1.00 — that every assumption about run hours, occupant behaviour, model fidelity, and post-install commissioning holds perfectly. The simulation shows that 1.00 sits near the 88th percentile of the realisation-ratio distribution: it is an optimistic outcome dressed up as the expected one. A guarantee priced off it looks free; priced off the real distribution, it carries a 41% chance of a shortfall payment. The point estimate does not just lose the spread — it anchors the contract to the wrong number.
The realisation ratio is constructed as a product of multiplicative effects, which keeps it strictly positive and gives it the right-skew the M&V literature reports (skewness ≈ +0.28 in the simulation):
A shared commissioning-quality factor, drawn Beta(6,2), multiplies several of these at once. This is the modelling decision that matters: a badly commissioned job is bad on rebound and persistence and run-hours together, so the downside scenarios are coherent rather than implausibly diversified away. Without that common factor, the product of near-1 terms would collapse toward its mean and understate exactly the left tail the guarantee is exposed to.
The guarantee level is a negotiation, and the simulation turns it into a probability curve instead of a hunch.
Sweeping the guarantee from 70% to 100% of the engineered saving shows the trade-off precisely: a guarantee at 80% of engineered (2,560 MWh) is met 76% of the time, at 85% it is 59%, at 90% it drops to 40%, and at 95% only 24%. The ESCO had signed at 85% — a level the meters confirm barely better than a coin flip. Pulling the guarantee back to the 80% point lifts the meet probability to 76% and moves the contract onto defensible ground, with the 90% planning threshold reached only below the engineered figure.
The guarantee is exposed to the lower tail of verified savings, so the drivers are ranked by their effect on the P10 verified saving (2,387 MWh).
Commissioning quality is the dominant driver at ±560 MWh of P10 verified saving — more than the engineering-model bias (±300 MWh) and operating-hours uncertainty (±250 MWh) combined. This reframes the whole risk: the largest lever on whether the guarantee is met is not refining the engineering calculation but funding a proper commissioning and fault-detection regime, which compresses the left tail the contract pays out on.
For the ESCO, the guarantee is an option written on the realisation ratio: pay a shortfall when verified savings fall short, earn a gain-share when they over-deliver.
Across simulated M&V outcomes the ESCO's net position averages roughly break-even, with a P10 of about −$37k and a P90 near +$30k, and a 41% probability of paying a shortfall. The mean hides the asymmetry the contract creates: the downside is a real cheque in two years out of five, driven almost entirely by the realisation-ratio left tail that the deterministic 3,200 MWh never acknowledged.
In efficiency retrofits, the engineered saving is the question, not the answer — the realisation ratio is the answer, and Monte Carlo is what puts a number on it before the guarantee is signed rather than after the first reconciliation.