Skip to content

Missing Data: Mechanisms, Methods, and Estimand-Driven Strategy

Definition

Missing data occurs when a patient's post-baseline outcome measurement is not available. The missing mechanism—why data are missing—fundamentally determines which statistical methods are valid and what assumptions must hold for valid inference.

ICH E9(R1) Framework (Final Guidance): The missing data strategy must be aligned with the estimand. The estimand specifies the target quantity; the intercurrent event (IE) strategy and estimation method must address how missing data resulting from IEs (e.g., treatment discontinuation, study withdrawal, rescue therapy) are handled.

Missing Data Mechanisms: Definitions and Practical Testability

MCAR: Missing Completely at Random

Definition: The probability that a measurement is missing is independent of both observed and unobserved outcomes.

  • Formally: P(missing | Y_obs, Y_unobs, X) = P(missing | X), where X denotes baseline covariates
  • Example: A patient's blood sample is lost in transit before assay, unrelated to their disease status

Testability: MCAR is not directly testable from the data. It requires clinical judgment:

  • Is missingness due to administrative/logistical reasons (schedule conflict, relocation) vs. clinical response?
  • If dropouts cluster in the control arm, MCAR is implausible

Practical implication: Any unbiased method (MMRM, MI, complete-case analysis) yields consistent estimates under MCAR.

MAR: Missing at Random

Definition: The probability that a measurement is missing is independent of the unobserved outcome, conditional on observed data and baseline covariates.

  • Formally: P(missing | Y_obs, Y_unobs, X) = P(missing | Y_obs, X)
  • Example: A patient drops out because their last observed tumor size was large, but given that observation, dropout is unrelated to their unobserved future progression

Testability: MAR is not testable from observed data alone, but subject-matter expertise can make it plausible:

  • Are all confounders of missingness observed and adjusted for?
  • Did the trial protocol mandate follow-up assessments for all randomized subjects (strengthens MAR)?
  • Inverse probability weighting (IPW) assumes MAR; if diagnostics suggest violations, sensitivity analyses (MNAR) are warranted

Practical implication: MMRM and multiple imputation under MAR (FCS, Rubin's rules) yield unbiased estimates.

MNAR: Missing Not at Random

Definition: The probability that a measurement is missing depends on the unobserved outcome, conditional on observed data and baseline covariates.

  • Formally: P(missing | Y_obs, Y_unobs, X) depends on Y_unobs
  • Example: A patient with rapidly progressing disease stops attending clinic visits; dropout is correlated with their worsening (unobserved) future tumor burden

Testability: MNAR cannot be tested from the observed data; it must be specified via:

  • Clinical reasoning (e.g., "Progressive disease drives dropout in this trial")
  • Sensitivity parameter (delta adjustment, tipping point range)
  • Reference group assumption (e.g., "Assume post-dropout trajectory matches control arm")

Practical implication: Standard MMRM and MI are biased under MNAR. Sensitivity analyses (reference-based, tipping point) quantify the magnitude of bias under different MNAR assumptions.

Estimand-Driven Missing Data Strategy

ICH E9(R1) Principle (Final Guidance): The missing data strategy is determined by the estimand, not the other way around.

Treatment Policy Estimand → Observed Data

Definition: Estimate the effect of assignment to treatment, including all intercurrent events as observed.

Missing data implication:

  • Dropouts (who did not have a final measurement) are handled as "missing"
  • No imputation is needed for the IE itself; just handle the resulting missing outcome
  • Analysis uses all observed data (MMRM, IPW, repeated measures)
  • Assumption: MAR (conditional on observed outcomes and baseline)

SAP language: "The primary estimand is the treatment policy: we estimate the difference in [endpoint] between arms, counting observed data only. Patients who discontinued study participation contribute observations up to their last assessment."

Hypothetical Estimand → Imputation Required

Definition: Estimate the effect of treatment in a hypothetical scenario where intercurrent events do not occur.

Missing data implication:

  • Dropout after an IE (e.g., treatment discontinuation) represents missing data
  • Must impute post-discontinuation outcomes under the hypothesis that the IE did not happen
  • Assumption: MAR or MNAR (depending on method), with explicit specification of how unobserved values would have evolved

Common hypothetical scenarios:

  1. Patients stay on study drug → Impute missing data assuming they remained compliant
  2. Patients do not receive rescue therapy → Impute under the counterfactual

SAP language: "The hypothetical estimand specifies the effect of [treatment] in the scenario where all patients remain on assigned treatment without rescue therapy. Missing data after treatment discontinuation are imputed using [method], assuming the patient's trajectory would follow [reference/assumption]."

Methods for Handling Missing Data

LOCF and BOCF: Limitations

Last Observation Carried Forward (LOCF):

  • Copies the last observed value forward to fill missing timepoints
  • Bias mechanism: If missingness is correlated with worsening disease, LOCF underestimates true decline
  • Example: A patient with mOS=12 months stops scans at month 6 (disease progression); LOCF assumes month 12 tumor burden = month 6 tumor burden, biasing toward smaller treatment effect
  • Consequence: In oncology trials with informative dropout, LOCF is biased for the true treatment effect

Baseline Observation Carried Forward (BOCF):

  • Imputes missing values with baseline value
  • Bias mechanism: Highly unrealistic for progressing disease; assumes no patient change
  • Not recommended in modern oncology trials

Regulatory stance (ICH E9(R1), Final): LOCF/BOCF are acceptable only for sensitivity analyses, not primary analysis, unless the estimand is explicitly a "baseline observation carried forward" (rare in oncology).

Multiple Imputation Under MAR

Framework:

  1. Specify a multivariate model for the missing data mechanism (e.g., fully conditional specification, FCS)
  2. Draw m plausible values for each missing cell from the posterior predictive distribution
  3. Analyze each completed dataset separately
  4. Pool estimates using Rubin's combination rules:
    • Point estimate: average of m analysis-specific estimates
    • Variance: within-imputation variance + between-imputation variance

Recommended m: Rubin (1987) and modern guidance suggest m ≥ 20 for general use; m = 40–100 if fractions of missing information (γ) exceed 0.3.

Fully Conditional Specification (FCS) Algorithm: - Iteratively fill each variable with missing data by regressing on all others - Cycle through variables until convergence - Draw from posterior predictive distribution

R Package: mice, miceadds (multivariate imputation by chained equations)

SAP template: "Missing data are imputed using multiple imputation with fully conditional specification (m = 40 imputations). The imputation model includes treatment arm, baseline covariates, and all longitudinal outcome measurements. Estimates are pooled using Rubin's combination rules."

Assumptions: Conditional MAR (given included covariates and observed outcomes).

Multiple Imputation Under MNAR: Delta Adjustment

Framework: Assumes missing data are MAR, but applies a sensitivity shift (delta) to the imputed values to quantify bias under MNAR.

Method:

  1. Perform standard MI under MAR
  2. For each imputed dataset, shift the imputed values by delta: Y_imputed_adjusted = Y_imputed_MAR + δ
  3. δ represents the difference between the unobserved mean (given the IE) and the MAR-predicted mean
  4. Analyze each shifted dataset; pool results

Interpretation:

  • δ = 0 → MAR (no bias)
  • δ > 0 → Imputed values are higher than MAR assumes (e.g., better outcomes than predicted)
  • δ < 0 → Imputed values are lower (e.g., worsening progression despite dropout)

Oncology example: In a survival trial, if patients drop out due to progression, set δ = -2 (log HR) to assume missing post-dropout log(HR) is 2 units worse than MAR would predict.

R Package: rbmi (reference-based multiple imputation, supports delta adjustment)

SAP template: "Sensitivity analysis: Delta adjustment applied to imputed post-discontinuation values (δ = {-1, -0.5, 0, +0.5, +1}), where negative delta reflects worsening outcomes among patients who discontinued due to progression."

MMRM: Mixed Model Repeated Measures for Longitudinal Data

Model Specification:

E[Y_ij] = μ + treatment + visit + treatment × visit + baseline + other covariates
Cov(Y_ij, Y_ik) = specified covariance structure

Key feature: MMRM includes all available data (observed timepoints for each subject), without explicit imputation.

Covariance structure choice:

  • Unstructured (UN): No constraints; each pairwise correlation estimated independently. Most flexible, requires more parameters (~n(n+1)/2 where n = number of visits). Preferred when sample size permits (typically n ≥ 200 total).
  • Compound Symmetry (CS): All observations equally correlated; 2 parameters. Restrictive; often inadequate for longitudinal oncology data.
  • Autoregressive (AR1): Correlation decays with visit distance; 2 parameters. Often fits longitudinal trends well; computationally stable.

Assumption: MAR — missing outcomes are independent of unobserved values, conditional on baseline and observed outcomes.

Bias in oncology: If dropout is driven by disease progression (MNAR), standard MMRM is biased even with all subjects included.

Regulatory stance (ICH E9(R1), Final): MMRM with unstructured covariance is acceptable for primary analysis when MAR is plausible.

R Packages:

  • nlme (lme function)
  • lmerTest
  • gls (generalized least squares)

SAP template: "Longitudinal analysis of [endpoint] is performed using a mixed model for repeated measures (MMRM) with unstructured covariance, baseline adjustment, visit, treatment, treatment-by-visit interaction, and demographic covariates."

Reference-Based Imputation for MNAR

When appropriate: Trials where dropout is informative (e.g., driven by disease progression, toxicity). The reference group (e.g., placebo, control treatment) provides the basis for imputing missing values under the assumption that missing post-IE outcomes follow the reference pattern.

J2R: Jump to Reference

Mechanism: After the intercurrent event (IE), the individual's trajectory jumps to the reference group's mean trajectory.

Assumption: Post-IE outcomes equal the reference group's observed profile (ignoring individual baseline differences).

Use case: When you assume that after discontinuation, the patient's disease trajectory reverts to the natural history of untreated disease (reference group pattern).

Limitations:

  • Assumes homogeneous post-IE evolution regardless of pre-IE status
  • Can be unrealistic if individual baseline severity differs substantially from reference average

CIR: Copy Increments in Reference

Mechanism: The individual's increment profile (change per visit) after the IE equals the reference group's observed increment profile.

Assumption: Post-IE change = reference group's observed change; individual baseline status is preserved.

Advantage over J2R: Respects individual baseline severity; only the slope is assumed to follow the reference pattern.

Example: If reference arm shows average monthly decline of 2 units, a discontinued patient with baseline = 50 is imputed as 50 − 2×(months post-IE).

Use case: Common in oncology when baseline disease burden varies; CIR is more flexible than J2R.

CR: Copy Reference

Mechanism: Post-IE, the individual's entire profile (including baseline shift) copies the reference group's observed average profile.

Assumption: Strongest; assumes treatment-discontinuing patients resemble reference group untreated patients in both baseline and trajectory.

Use case: Rare; typically not appropriate unless treatment groups are already balanced and dropout is completely random.

R Package: rbmi (Reference-Based Multiple Imputation)

Installation: install.packages("rbmi")

Key features:

  • Implements J2R, CIR, and CR
  • Combines reference-based imputation with multiple imputation framework
  • Outputs point estimates, 95% CIs, and sensitivity analyses

Example SAP language:

# Impute using CIR (Copy Increments in Reference)
impute_cir <- impute(data, method = "CIR", reference_arm = "placebo")

SAP template: "Sensitivity analysis: Reference-based multiple imputation (rbmi package, method = CIR) assumes that patients discontinuing due to [IE type] have post-discontinuation trajectories matching the control arm's observed change rates, adjusted for individual baseline."

Tipping Point Analysis

Framework: Quantifies how large a departure from MAR (specified by a sensitivity parameter) would be needed to flip the trial conclusion.

Procedure:

  1. Primary analysis: Conduct under MAR (MMRM or MI)
  2. Define sensitivity parameter: δ = difference between unobserved mean and MAR-predicted mean
  3. Sweep δ: Reanalyze across a range of δ values (e.g., δ ∈ {−1, −0.5, 0, +0.5, +1})
  4. Tipping point: The δ value at which the 95% CI crosses the null (treatment effect = 0)
  5. Interpretation: If tipping point is clinically implausible, primary conclusion is robust

Oncology example:

  • Primary analysis: HR = 0.75 (95% CI: 0.60–0.95) under MAR; statistically significant
  • Tipping point analysis: Conclusion flips (CI crosses null) if δ = −0.8 (log scale), meaning missing post-dropout values must be 80% worse than MAR predicts
  • Interpretation: If clinical judgment deems δ = −0.8 implausible (patients wouldn't drop out for reasons that severe), the finding is robust

SAP template: "Tipping point analysis quantifies the magnitude of MNAR departure (parametrized by delta) required to render the treatment effect null. The primary conclusion is robust if the tipping point is clinically implausible."

PRO/QoL with Informative Dropout: Competing Risks

Challenge: Patient-Reported Outcomes (PRO) and Quality of Life (QoL) assessments may have a unique missing data problem: informative dropout due to death or disease progression.

Why Standard Methods Fail

  • A patient who dies cannot report QoL
  • If death is more frequent in the control arm, control-arm QoL may appear higher (survivor bias)
  • MMRM and MI are biased because dropout is not MAR; it is MNAR driven by a competing event (death)

Competing Risks Framework

Key principle: Model the joint distribution of the primary outcome (QoL/PRO) and the competing event (death, progression).

Strategies:

  1. Estimand clarity: Specify whether QoL is estimated:

    • Among all randomized patients (including counterfactual QoL for deceased)
    • Among survivors only (explicit survivor analysis)

  2. Joint model: Jointly estimate QoL progression and time-to-death (or progression)

    • Uses multi-state modeling (alive with QoL score → dead, or alive with different score)
    • Accounts for correlation between QoL trajectory and mortality risk

  3. Sensitivity analysis: Impute post-death QoL under different assumptions (e.g., "assume worst QoL if deceased")

R Package: JM (joint models for longitudinal and survival data), joint models in rstan

SAP template: "PRO/QoL analysis accounts for informative dropout due to death using joint modeling of PRO trajectory and mortality. Estimates reflect QoL among the [entire population / survivors only, as pre-specified in the estimand]."

SAP Template: Primary and Sensitivity Hierarchy

Primary Analysis

**Primary Estimand**: [Treatment policy / Hypothetical]

**Primary Method**: 

- [MMRM with unstructured covariance / Multiple imputation under MAR]
- Baseline and [specify covariates]
- Includes all randomized subjects with ≥1 post-baseline measurement (ITT-evaluable set)

**Assumption**: Missing data are Missing at Random (MAR) conditional on observed measurements and baseline covariates.

**Software/Package**: [nlme, mice, rbmi, etc.]

**Null hypothesis**: H₀: Treatment effect = 0
**Alternative**: H₁: Treatment effect ≠ 0
**Test statistic and α**: [e.g., Wald test, α = 0.025 one-sided]

Sensitivity Analyses Hierarchy

Tier 1 (Always pre-specified):

  1. Tipping point analysis: Quantify delta (departure from MAR) required to flip the conclusion
  2. Delta-adjustment MI: Re-analyze with δ ∈ {−1, −0.5, 0, +0.5, +1}
  3. Reference-based MI (CIR): Assume post-dropout trajectory matches control arm increments

Tier 2 (If MNAR concern is substantial):

  1. Per-protocol set (subset of patients with ≥80% compliance): Assess robustness of ITT finding
  2. Best/worst case imputation: Impute missing values as best/worst plausible outcomes
  3. Jump-to-reference (J2R): More aggressive assumption than CIR

Tier 3 (Regulatory expectation):

  • Multiple imputation with different covariance structures (compound symmetry, AR1)
  • Inverse probability weighting (IPW) to adjust for informative dropout

Regulatory Requirements & ICH E9(R1) (Final Guidance)

Alignment with Estimand

  • Treatment policy: Specify observed data only; justify MAR assumption
  • Hypothetical: Specify IE handling and imputation method; justify MNAR assumptions explicitly

Intercurrent Events (IE) Strategy

The SAP must define the intercurrent event strategy for:

  1. Treatment discontinuation (on/off-study)
  2. Rescue therapy (concomitant medications)
  3. Study withdrawal
  4. Death (if composite endpoint, specify whether counted as failure or censored)

Sensitivity Analysis Expectations

  • Regulatory position (FDA/EMA): Primary analysis + hierarchical sensitivity analyses
  • ICH E9(R1) expectation: At minimum, one MNAR sensitivity analysis (tipping point or delta-adjustment)
  • Transparency: All pre-specified in SAP; post-hoc additions require justification

Source: ICH E9(R1) Addendum (Final, 2025); literature on reference-based imputation (Carpenter et al., Molenberghs et al.); rbmi R package documentation
Status: ICH E9(R1) — Final Guidance; reference-based methods — Standard practice; ICH E20 — Draft (June 2025 revision pending)
Compiled from: FDA ICH E9(R1) guidance; reference-based imputation literature; MMRM/MI methodology reviews