Quality Control

Every fit lands with a quality tier and, often, one or more flags. These are not subjective grades. They are derived from the fitted parameters and residuals using the rules below.

The quality tier

Each result carries an excellent / good / fair / poor tier, derived from R² (the fraction of variance the model explains):

R² alone never decides whether a fit is “good”. A perfectly smooth sigmoid through twelve flat-bottomed points can hit R² of 0.99 without the IC₅₀ being identified at all. The flags below catch the cases R² doesn't.

The flags Gradiance raises

Each flag has a severity (warning or critical) and shows up in the results panel next to the affected parameter. They are not blockers. They tell you which part of the fit to look at first.

R² below threshold

Fires when: R² < 0.95. Severity escalates to critical at R² < 0.90.

What to check: the residual plot. If residuals show a clear shape (S, U, or funnel), the model is wrong. If they're flat but noisy, your replicates are scattered.

IC₅₀ extrapolated

Fires when: the fitted IC₅₀ falls outside the concentration range you tested.

What it means: the model is guessing where the midpoint would be if the curve continued past your data. The CI usually balloons accordingly. Add doses to bracket the inflection, or report the IC₅₀ with the explicit caveat that it is extrapolated.

Low dynamic range

Fires when: Top − Bottom is less than 50% of the observed response range.

What it means: the assay barely responded. The fit will still produce numbers, but small absolute changes in noise look like big fractional changes in signal. Validate at higher and lower doses to confirm there is a curve, not a straight line.

Shallow Hill slope

Fires when: |Hill| < 0.5.

What it means: the curve is unusually gradual. Either the response really is shallow (some receptor systems are), or the algorithm is forcing a sigmoid through scattered data. Check whether the dose spacing covers two orders of magnitude — if not, you may not have enough range to detect a steeper slope.

Steep Hill slope

Fires when: |Hill| > 5.

What it means: the fit thinks the response flips almost vertically. Real cooperativity does this; so does having only one or two intermediate points. Add doses near the inflection to confirm.

Asymptote pinned at a constraint

Fires when: Top or Bottom is stuck at the bound you set (e.g. you fixed Bottom to 0 and the optimizer would have gone negative).

What it means: the constraint is doing the work, not the data. That is fine if the constraint reflects assay biology (no compound = full viability). It is not fine if you set the constraint hoping the data would justify it.

Wide confidence interval

Fires when: a parameter's 95% CI spans more than an order of magnitude (or is unbounded).

What it means: the data don't actually identify that parameter. Most often the curve hasn't plateaued on one side. Add doses past where you stopped, or constrain the parameter if you have prior information about its value.

How replicates change the picture

With three or more replicates per dose, Gradiance can separate between-dose structure from within-dose noise. The practical consequence: a fit with high replicate scatter at a single dose may still score “good” because the curve through the means is excellent, even though the residuals look loud. Read the CI on the IC₅₀, not just R², when replicates are noisy.

Workflow when a flag fires

1. Open the residual view. Look for a shape.
2. Look at the parameter table. Is the wide CI on the asymptote, or on the IC₅₀?
3. Decide whether to refit (different model, exclude an outlier, fix an asymptote) or to add data (more doses, more replicates).
4. Ask the agent in plain language. “Try 5PL,” “exclude the 100 µM point,” “fix bottom to zero” all work.