BLOG

Garbage In, Garbage Out: How to Test Assumptions in an Early-Stage TEA

Garbage In, Garbage Out: How to Test Assumptions in an Early-Stage TEA

A techno-economic analysis is only as good as the assumptions inside it. At FEL-1, most of those assumptions are uncertain by definition. The failure isn't making assumptions. It's losing track of them.

That was the thread Crystal Bleecher, Roebling's SVP of Process Engineering, kept pulling on in a recent webinar with Frederick Twigg from UC Berkeley. Crystal has spent 20 years scaling first-of-a-kind technologies. She's seen good technology get shelved because the business case was built on numbers nobody went back to verify.

The recurring theme? Most teams don't fail because their assumptions are wrong. They fail because nobody tracked which ones to worry about.

One scenario, one answer, one problem

Most early-stage feasibility studies evaluate one process configuration under one set of assumptions and produce one answer. Looks good? The project advances. Doesn't look good? The project is shelved or redesigned.

The inputs at FEL-1 are necessarily uncertain:

  • R&D performance data from bench scale

  • Vendor budgetary quotes

  • Construction cost benchmarks from analogous projects in different geographies

  • Raw material pricing that fluctuates quarterly

A single-scenario model treats all of these as fixed values and produces an output that looks precise. That precision is an illusion.

The industry standard for an FEL-1 cost estimate (an AACE Class 5) is –50% to +100%. That standard does a good job of telling you what engineering deliverables you need to land within that range. What it doesn't account for, Crystal pointed out, is uncertainty in the design basis itself.

"The design basis we start with in FEL-1 is not the same design basis we end up with in FEL-3."


A wrong assumption caught in FEL-1 is a few hours of rework. A wrong assumption caught in FEL-3, after construction documents and P&IDs are developed, can cost hundreds of thousands of dollars in completed engineering. Sometimes it kills the project.

So you need to track two kinds of uncertainty: uncertainty in your CapEx numbers, and uncertainty in the design basis those numbers are built on.

Assumptions don't sit still — they compound

Single-variable sensitivity analysis is the most common way teams test assumptions. It's also not enough on its own. It moves one input at a time while everything else stays put, which masks real risk, because assumptions interact.

Here's what Crystal walked through. A recent project modeled a downstream chromatography step with three uncertain inputs:

  • Resin binding capacity

  • Resin cost per liter

  • Resin longevity (cycles before replacement)

Single-variable sensitivity on resin cost alone, across a wide range, looked modest. The model assumed a long resin life, so even an expensive resin amortized over many cycles. No big deal.

Then they ran resin cost and resin longevity together. Higher cost compounded with shorter life broke the OpEx model. The sensitivity wasn't visible until both moved at once. The technology developer ended up going back to reevaluate the entire downstream process and selecting different unit operations.

After running single-variable sensitivity to identify which inputs have any impact, run multi-variable scenarios on the inputs that interact. Resin cost and resin longevity are correlated in the real world. Higher-performance resins cost more but last longer. Cheaper ones degrade faster.

Testing them independently misses the failure mode that matters.

This applies far beyond chromatography. Equipment cost and installation factors. Feedstock price and conversion yield. Labor rates and operating model complexity. Anywhere two assumptions feed the same cost line and are independently uncertain, they need to be tested together.

Not every input is worth losing sleep over

A common trap is treating every uncertain input as equally important. Being off by 50% on the cost of a pump at commercial scale probably doesn't move the needle. Being off by 50% on a spray dryer or a chromatography system can break the project.

Why? Those are the high-cost equipment items where sizing, material selection, and operating parameters drive both CapEx and OpEx.

So rank every uncertain input by its impact on the key economic outputs: unit production cost, IRR, payback period, whatever metric drives the decision. In Roebling's analysis, for example, purchase equipment uncertainty at 30% can land at an 85% impact on project economics, while debt-to-equity ratio uncertainty barely registers.

That visibility tells you where to spend limited de-risking budget. If resin longevity is the highest-impact uncertain input, accelerated aging studies are a better use of R&D dollars than refining a low-impact utility cost.

Process constraint or equipment constraint?

One of the most common sources of bad assumptions in a bioprocess TEA is performance data from bench-scale equipment that doesn't translate to commercial scale. Frederick raised it as a rookie mistake he sees a lot in companies under 10 people: running 2L bioreactors at maximum RPM and gas flow, hitting great mass transfer at the lab bench, and assuming that performance projects out to 10,000 or 100,000 liters.

It doesn't.

Surface-area-to-volume ratios drop. Heat removal becomes the limiting constraint. The operating window narrows. A TEA built on KLA data from a 2L reactor without accounting for those scale-dependent physics produces commercial-scale economics that are wrong from the start.

The model won't flag it. It doesn't know the data came from equipment operating under fundamentally different physical constraints.

Crystal gave a second example from a recent tech transfer package. The R&D team handed over data on chromatography column feed time that looked reasonable on paper. Her team dug into it with R&D and found something surprising: the feed time wasn't long because the process required it. It was long because the column at bench scale was physically small.

The constraint was the equipment, not the process. At commercial scale with appropriately sized equipment, the feed time was entirely different — which changed equipment sizing, throughput, and cost.

“The diagnostic question for every data point coming out of the lab: is this a process constraint, or an equipment constraint?”


If the operating parameter is limited by the physical characteristics of bench-scale equipment (vessel geometry, column dimensions, heat transfer surface area), it can't be used as a commercial-scale design basis without adjustment.

"Noisy biology" is sometimes just noisy instruments

Frederick made a point that's worth pulling out on its own.

Bioprocess teams often write off variability as "biology is noisy." Sometimes it is. But often what gets called noisy biology is actually the pooled coefficient of variation stacking up from analytical instruments that haven't been control-charted properly.

GCMS error of around 5% per measurement, multiplied across the assays in a development workflow, can easily produce total measurement variance above 10%. If you're trying to chase a 5% process improvement against that backdrop, your signal is smaller than your noise. You can't see your own progress.

That's not biology being inherently stochastic. It's lab discipline. And it shows up in TEAs the same way other assumption errors do: as uncertainty nobody quantified entering the model, and a surprise on the way out.

A spreadsheet cell can't tell you where a number came from

Most TEA tools treat input data as a number in a cell. There's no record of where it came from, when it was validated, what scale or equipment configuration it was measured on, or whether it applies to the current project.

That matters because TEA inputs come from very different sources:

  • Thermodynamic properties from literature

  • Equipment sizing from a vendor budgetary quote (six months old)

  • Construction costs from an analogous project in a different country

  • Labor rates from an industry survey

  • Performance data from a bench-scale experiment

A placeholder value from an early literature review looks identical to a validated vendor quote once they're both sitting in a spreadsheet. And the file gets renamed final final v3 four times before anyone notices.

Tracking provenance means documenting, for every significant input: source, date collected or validated, conditions it applies to (scale, geography, equipment type, technology configuration), and an estimated uncertainty range.

It doesn't require a complex system. It requires discipline, and a model structure where the metadata is visible instead of buried.

AI is a multiplier — on whatever foundation you give it

Testing assumptions thoroughly has always been possible. It's also been prohibitively time-consuming. Running 20 or 30 scenarios in Excel, where each one requires manual input changes and separate output tracking, takes days to weeks. Most teams settle for two or three configurations and hope they picked the right ones.

AI shifts the constraint. When the process model, equipment sizing, cost estimation, and financial analysis all live in a single integrated environment, and AI agents handle the mechanical execution, the question moves from "how many scenarios can we afford to run" to "which scenarios should we run and how do we interpret the results."

Sweeps across every uncertain input simultaneously. 100 configurations compared instead of 3. Cost drivers and viability thresholds identified in hours instead of weeks.

But here's the catch, and it's the line Crystal kept coming back to.

"AI on top of a bad foundation just amplifies that. Garbage in, garbage out — but faster, and maybe with more confidence, which is dangerous."


Cost data from a project that doesn't apply to this one. Process data from a literature paper on different equipment. If those are the inputs, AI confidently amplifies the noise. That's worse than producing wrong answers slowly, because the speed creates false rigor.

The foundation has to be deterministic. Same inputs, same outputs, every time. No interpolation. No black boxes. Mass balance closes. Equipment sizing is constrained to real commercial capacity ranges. Every cost assumption visible and modifiable. Uncertainty ranges attached to every significant input.

With that foundation in place, AI is a real multiplier on engineering judgment. The engineer stays in the loop, validates inputs, questions outputs, and applies the judgment no model replicates.

Without the foundation, AI is just faster garbage in, faster garbage out.

Want more? Crystal and Frederick's full conversation on testing TEA assumptions, scaling pitfalls, and where AI actually earns its keep. Watch the full session →

See Roebling in Action

Explore how AI-powered process engineering accelerates design, cost analysis, and infrastructure decisions.

See Roebling in Action

Explore how AI-powered process engineering accelerates design, cost analysis, and infrastructure decisions.

Designed for those who build.

Roebling is where the most ambitious industrial projects start. Roebling offers a first-of-its-kind platform for industrial process engineers and R&D teams in biomanufacturing, chemicals, critical minerals, and beyond.

Copyright © 2026 Roebling. All Rights Reserved.

Designed for those who build.

Roebling is where the most ambitious industrial projects start. Roebling offers a first-of-its-kind platform for industrial process engineers and R&D teams in biomanufacturing, chemicals, critical minerals, and beyond.

Copyright © 2026 Roebling. All Rights Reserved.

Designed for those who build.

Roebling is where the most ambitious industrial projects start. Roebling offers a first-of-its-kind platform for industrial process engineers and R&D teams in biomanufacturing, chemicals, critical minerals, and beyond.

Copyright © 2026 Roebling. All Rights Reserved.

We value your privacy

We use cookies to improve site functionality. Some cookies help us understand how visitors use our site.