Quality Control
The specific thresholds Gradiance uses to grade a fit, and what each warning is actually telling you
Fits with a reported R² receive a heuristic quality tier and may carry additional flags. These prompts guide review; they do not certify scientific validity, publication readiness, or reliable parameter estimates.
The quality tier
Each successful fit with a finite R² carries an excellent / good / fair / poor display tier:
Excellent
R² ≥ 0.99A high R² describes agreement with the observed responses. Still check residuals, parameter uncertainty, and the experimental design.
Good
0.95 ≤ R² < 0.99Review residual patterns and whether the data identify the parameters. This tier is not a publication acceptance criterion.
Fair
0.90 ≤ R² < 0.95Investigate replicate variation, residual patterns, and model assumptions. R² alone cannot establish the cause of the mismatch.
Poor
R² < 0.90Check model suitability, measurement variation, and parameter uncertainty before interpreting the estimates. The tier does not diagnose the cause.
A high R² can coexist with poorly identified parameters. Review whether the tested range constrains the midpoint, slope, and asymptotes. Automated flags are incomplete and do not replace inspection of the experiment.
The checks recorded for supported fits
The fixed calculator records method-specific warnings. It also records explicitly excluded source rows and empty mapped roles that were not used. Review the saved message and its context; a numeric threshold is not a universal acceptance criterion for every assay.
Low R²
Fires when: R² is below 0.95 for the supported 4PL or below 0.90 for Michaelis-Menten. Linear and quadratic calibration do not apply a generic low-R² warning. The warning does not automatically reject the fit.
What to check: residual patterns, replicate variation, model assumptions, and parameter uncertainty. Curvature may suggest model mismatch; a funnel may suggest nonconstant variance. Neither pattern alone identifies a measurement error. Negative R² values, including values below −1, are possible for R² = 1 − SSE/SST when predictions are worse than the response mean.
Do not exclude observations solely to raise R². Investigate potential errors, retain the original data, document exclusion criteria, and report sensitivity to justified exclusions. See the NIST guidance on model adequacy and outlier investigation. Older saved runs retain the guidance recorded when they were generated.
IC₅₀ extrapolated
Fires when: the fitted IC₅₀ falls outside the concentration range you tested.
What it means: the midpoint is a model estimate beyond the observations. Its interval may be wide or unavailable. Add doses to bracket the inflection, or report the IC₅₀ with the explicit caveat that it is extrapolated.
Possible missing plateaus
Fires when: the fitted 4PL span, Top − Bottom, is less than 70% of the observed response span.
What it means: the fitted curve may not represent both plateaus. Inspect the measured range and the plotted curve; this check does not by itself diagnose weak assay response.
Shallow Hill slope
Fires when: |Hill| < 0.5.
What it means: the fitted curve is unusually gradual. The warning cannot distinguish an experimental response from model mismatch or scattered measurements. Check the dose range and residuals.
Steep Hill slope
Fires when: |Hill| > 4.
What it means: the fitted transition is nearly vertical. The value alone does not establish cooperativity or another mechanism. Inspect intermediate doses and residuals.
Parameter bounds
The supported 4PL uses free parameters within recorded numerical safety bounds. It does not emit an automatic “asymptote pinned” check, and the public supported method does not accept a user-fixed Top or Bottom.
Inspect the executed-method record and whether the fitted value is near a safety bound. A boundary estimate needs scientific review even when no automatic warning appears.
Confidence interval unavailable
Fires when: the supported 4PL or Michaelis-Menten calculator cannot estimate a reported interval because there are too few residual degrees of freedom or the covariance is near-singular.
What it means: the local covariance calculation did not support that interval. The warning does not identify the cause or quantify biological and across-experiment uncertainty. Gradiance does not apply an automatic order-of-magnitude interval-width threshold.
How replicates change the picture
A supported fit records whether included observations were fitted individually or explicitly averaged by x value. It does not infer biological independence or partition technical and biological variance. Preserve replicate identifiers and review the executed-method record before interpreting a parameter interval.
Workflow when a flag fires
- Open the residual view. Look for a shape.
- Look at the parameter table. Is the wide CI on the asymptote, or on the IC₅₀?
- Decide whether a documented source-row exclusion is justified or whether the experiment needs more doses or replicates.
- Ask the agent to review the mapping or a specific observation. The supported dose-response models include unweighted 4PL, asymmetric 5PL, and 3PL with a zero lower plateau. Other fixed parameters and weighting are unsupported. Review midpoint uncertainty and identifiability warnings even when R² is high.