Industry: Biotech
Product: ModelRisk
Application: Protein engineering — sizing a directed-evolution screening campaign by probability of beating the activity target

A Directed-Evolution Campaign That Was Funded to Hit 8x and Had a 17% Chance of Getting There

A protein-engineering group ran directed-evolution campaigns to improve enzyme activity: each round, a library of variants is built and screened, the rare functional hits are identified, the best is carried forward, and the cycle repeats. The lead program needed an 8-fold improvement over wild-type activity to be commercially viable, and the campaign was budgeted around a back-of-envelope plan: screen 900 variants per round for three rounds, expect roughly 1% of variants to be hits, and assume the best of them compounds up to the target. The plan was funded on the strength of a single deterministic chain that quietly multiplied median gains together and arrived at "we should get there."

That chain is the wrong object for a discovery process. Hits arrive as a Binomial count — a random number of successes from a large library at a low per-variant hit rate. Each hit's activity gain is a right-skewed LogNormal fold-change, and what the campaign keeps is the maximum across all hits, not a typical one. The deterministic plan, by compounding median per-round gains, predicts a "typical variant" reaches only 1.83x — and has no way to represent the screening luck that finds something far better. When the group rebuilt the campaign in ModelRisk and ran 40,000 simulated campaigns, both numbers turned out to be misleading in opposite directions.

Screening the best of many hits does far better than the deterministic "typical variant" of 1.83x: the median best variant reaches 5.56x (mean 5.95x), because the maximum of a right-skewed distribution lands well into the tail. But it still falls short of the goal. Against the 8x activity target, the probability that any variant in the whole campaign beats it is just 17%. The deterministic plan understated what good screening achieves and, by treating "we should compound up to 8x" as a foregone conclusion, badly overstated the chance of actually clearing the bar.

Why a point estimate cannot size a screen

A deterministic activity number answers neither question the program needs answered. The campaign does not need to know the typical variant's gain; it needs to know the probability that the best variant beats 8x, and how much screening it takes to make that probability acceptable. Three features of screening break the point estimate:

• Hits are a count, not a fraction. "1% hit rate on 900 variants" is not nine guaranteed hits; it is a Binomial draw that ranges from the low twenties to the high fifties across the campaign. Treating the expected count as certain hides the campaigns that draw few hits and find nothing.
• What you keep is the maximum, not the mean. The campaign retains the single best variant. The distribution of a maximum is nothing like the distribution of a typical draw, and only a simulation that screens many hits and takes the best reproduces it.
• Gains are right-skewed and compound with a tax. Each beneficial mutation's fold-change is LogNormal, and only part of each round's gain carries into the next (epistasis limits stacking). Multiplying medians ignores both the skew and the rare large-effect variants that actually clear the target.

How big a library is needed to make success likely

The decision the program actually faces is a budget one: how many variants per round, over how many rounds, to reach a 90% chance of beating 8x. Sweeping library size shows the trade-off directly. At the funded 900 variants per round, the campaign clears 8x with probability 0% over two rounds, 17% over three rounds, and 68% over four rounds — none of the funded configurations reach the 90% line. To get a three-round campaign to 90% the library has to grow well beyond the grid; a four-round campaign reaches 90% at roughly 2,400 variants per round. Either way, the funded plan was several-fold short of the screening effort its own target demanded.

What changing the chemistry buys versus just screening more

The group compared four designs on the probability of beating 8x, each run with the same seed:

• Baseline (900/round, random library). P(beat 8x) = 17%, median best 5.56x.
• ML-guided library design. A higher hit rate (2.0%) and better median gain lift P(beat 8x) to 47%, median best 7.77x.
• 3x larger library (2,700/round). Brute-force screening reaches 53%, median best 8.22x.
• ML + 3x library. Combining the two reaches 85%, median best 11.22x.

The lesson is that neither lever alone clears 90%, but they are complementary: smarter libraries and more of them together take a 17%-likely campaign to 85%. Brute-force screening (3x library) slightly out-performs ML alone on probability of success, but ML wins on cost per plate — and the program chose the combined design as the only route that made the 8x target a likely outcome rather than a long shot.

What drives the best variant

Ranked by their effect on the P50 best-variant fold improvement, the per-variant hit rate dominates, followed by the hit fold-change median and the shared library-quality factor. Round-to-round carry-over and library size rank below them. The ordering is what pointed the program at ML-guided library design first: lifting the hit rate and the quality of hits moves the outcome more than simply screening more of a random library — which is why the ML lever, not the brute-force lever, anchored the chosen design.

What the model changed

• The campaign was resized before launch. The funded 900-variants × 3-rounds plan was shown to clear the 8x target only 17% of the time; the program moved to the ML-guided, larger-library design that reaches 85%.
• Success is now a probability, not an assumption. "We should reach 8x" was replaced with an explicit P(beat target) that the team tracks as library size and round count are decided.
• The screening budget is sized to a 90% confidence bar, using the library-size sweep to read off the variants-per-round and round-count needed rather than funding a round number of plates.
• ML-guided library design was prioritised over brute-force screening, because the tornado showed hit rate and hit quality — the levers ML moves — drive the best variant more than raw library size does.

ModelRisk Functionality Used

• Binomial hit-count modelling per round, treating the number of functional variants as a random draw from a large library at a low per-variant hit rate rather than a fixed expected count.
• LogNormal activity-gain distribution for each hit, capturing the right skew in beneficial-mutation effect sizes that median-based planning ignored.
• Maximum-of-hits extraction via the order-statistic of the per-round fold gains, reproducing the distribution of the best variant — the thing a screen actually keeps — rather than a typical one.
• Shared per-round library-quality factor correlating hit rate and hit quality within a round, so a well-designed library lifts the whole round together instead of each variant independently.
• Epistasis-taxed round-to-round compounding, carrying only part of each round's gain forward to model the diminishing returns of stacking mutations.
• Library-size sweep against a 90% success bar, converting the campaign into a budget decision: variants per round and number of rounds needed to make the target likely.
• Comparative scenario simulation and tornado sensitivity, ranking ML-guided design above brute-force screening and sizing the chosen campaign on probability of beating the activity target.

The funded plan assumed an 8x result and had a 17% chance of producing one. Monte Carlo simulation in ModelRisk turns a directed-evolution campaign into a sized, probabilistic plan — how big a library, over how many rounds, to make the activity target a likely outcome instead of a hope.