ML-NMR Methodology

ML-NMR Methodology

Comprehensive methodological guidance for conducting rigorous Multilevel Network Meta-Regression following NICE DSU guidance and multinma package documentation.

When to Use This Skill

• Deciding whether ML-NMR is appropriate
• Setting up integration points for AgD
• Specifying priors and models
• Understanding marginal vs conditional effects
• Predicting to target populations
• Reviewing ML-NMR code or results

When to Use ML-NMR

ML-NMR is Appropriate When:

1. Network Structure

   • Multiple treatments form (partial) network
   • Some studies have IPD, others only AgD
   • Want to leverage all available evidence

2. Population Differences

   • Effect modifiers differ across populations
   • Standard NMA transitivity violated
   • Need population-adjusted estimates

3. Target Population

   • Want predictions for specific population
   • Different from any single trial population
   • Policy-relevant population definition

ML-NMR vs Alternatives

Key Concepts

Individual-Level vs Study-Level

ML-NMR Models Both:
├── Individual-level (within IPD studies)
│   - Patient-level outcomes
│   - Patient-level covariates
│   - Exact covariate-outcome relationships
│
└── Study-level (for AgD studies)
    - Aggregate outcomes
    - Covariate summaries
    - Integration over covariate distribution

Population Adjustment

Problem: AgD studies provide aggregate summaries, but we need individual-level predictions.

Solution: Numerical integration over the AgD population's covariate distribution.

For AgD study:
Expected outcome = ∫ f(outcome | covariates, treatment) × p(covariates) d(covariates)

Where:
- f(): Individual-level outcome model (from IPD)
- p(): Covariate distribution in AgD population

Integration Points

What Are Integration Points?

Discrete approximation to the integral over AgD population:

# Specify covariate distribution
add_integration(
  network,
  age = distr(qnorm, mean = 62, sd = 10),
  sex = distr(qbinom, prob = 0.55),
  n_int = 500
)

# Creates 500 "pseudo-individuals" sampled from
# the specified covariate distribution

Choosing Number of Integration Points

Best Practice: Test sensitivity to n_int by running with different values.

Specifying Distributions

# Continuous: Normal distribution
age = distr(qnorm, mean = 62, sd = 10)

# Binary: Bernoulli (using binomial with size=1)
sex = distr(qbinom, prob = 0.55)

# Categorical: Discrete distribution
# May need special handling

# Correlated covariates: Copula methods
# More complex setup required

Model Specification

Regression Component

nma(
  network,
  regression = ~ age + sex + age:sex,  # Covariate effects
  ...
)

# Interprets as:
# Linear predictor = trt_effect + β_age × age + β_sex × sex + β_age:sex × age × sex

Effect Modifier vs Prognostic Factor

In ML-NMR regression formula:
├── Effect modifiers: Interact with treatment
│   - regression = ~ age
│   - Creates age × treatment interaction
│
└── Prognostic factors: Affect baseline risk only
    - Handled through study random effects
    - Or explicit prognostic regression

Prior Specification

nma(
  ...,
  prior_intercept = prior_normal(0, 10),    # Baseline risk
  prior_trt = prior_normal(0, 5),           # Treatment effects
  prior_reg = prior_normal(0, 2),           # Regression coefficients
  prior_het = prior_half_normal(1)          # Heterogeneity
)

# Considerations:
# - Scale depends on link function
# - Log-odds: 2-3 is large effect
# - Informative priors from Turner et al. for het

Marginal vs Conditional Effects

Conditional Effects

• Effect at specific covariate values
• "Effect for a 65-year-old male"
• Directly from model coefficients

Marginal (Population-Averaged) Effects

• Effect averaged over population
• "Average effect in UK population"
• Obtained via integration

# Predict to target population
target <- data.frame(
  age = seq(50, 80, 5),
  sex = 0.5  # 50% male
)

predictions <- predict(fit, newdata = target)

Why the Difference Matters

For non-collapsible effect measures (OR, HR):

• Marginal effect ≠ Average of conditional effects
• Must integrate properly over population
• ML-NMR handles this correctly

Consistency Assessment

Node-Splitting in ML-NMR

# Fit node-split model
nodesplit_fit <- nma(
  network,
  consistency = "nodesplit",
  ...
)

# Check for direct vs indirect disagreement
summary(nodesplit_fit)

Interpretation with Population Adjustment

• Inconsistency could be due to true treatment effect heterogeneity
• Or due to population differences not captured
• Node-splitting should be done after population adjustment

Treatment Rankings

Posterior Rank Probabilities

rank_probs <- posterior_rank_probs(fit)

# Returns probability matrix:
# P(treatment j has rank r)

Interpretation Cautions

Same as standard NMA:

• Rankings have uncertainty
• Small effect differences → large rank uncertainty
• Consider clinical significance alongside ranks

Prediction to Target Population

Specifying Target Population

# Method 1: Point prediction
target <- data.frame(age = 62, sex = 0.5)

# Method 2: Distribution prediction
# Provide many points representing target distribution
target <- data.frame(
  age = rnorm(1000, 60, 12),
  sex = rbinom(1000, 1, 0.45)
)

Types of Predictions

# Relative effects (log scale)
predict(fit, type = "link")

# Relative effects (natural scale)
predict(fit, type = "response")

# Absolute outcomes
predict(fit, type = "response", baseline = ...)

Convergence Diagnostics

Essential Checks

# 1. Print summary (shows R-hat, ESS)
print(fit)

# 2. Trace plots
plot(fit, pars = "d")

# 3. R-hat should be < 1.05
# 4. ESS should be > 400 per parameter

Addressing Convergence Issues

1. Increase iterations: More warmup/​sampling
2. Adjust adapt_delta: Higher (0.95, 0.99) for divergences
3. Reparameterize: Different model specifications
4. Informative priors: If posterior too diffuse
5. Check data: Sparse comparisons cause issues

Reporting Requirements

Methods

• Network structure description
• IPD vs AgD studies identified
• Covariate selection for adjustment
• Integration point specification
• Prior specification with justification
• Target population definition
• Convergence criteria

Results

• Network diagram
• Convergence diagnostics (R-hat, ESS)
• Relative effects for all comparisons
• Treatment rankings with uncertainty
• Consistency assessment
• Predictions to target population
• Sensitivity analyses

Common Pitfalls

1. Insufficient Integration Points

• Results may be unstable
• Check sensitivity to n_int
• Increase until results stabilize

2. Ignoring Convergence

• Must check R-hat and ESS
• Divergent transitions indicate problems
• Don't trust results without convergence

3. Wrong Covariate Distributions

• Must match AgD population
• Extract from publications carefully
• Consider correlation between covariates

4. Misinterpreting Marginal Effects

• Non-collapsible measures need care
• OR/​HR: Marginal ≠ conditional
• Use predict() for proper marginalization

5. Not Specifying Target Population

• Default may not be policy-relevant
• Explicitly define target
• Sensitivity to target specification

Quick Reference Code

library(multinma)

# 1. Set up IPD studies
ipd_net <- set_ipd(ipd_data,
                   study = study, trt = treatment, r = response)

# 2. Set up AgD studies
agd_net <- set_agd_arm(agd_data,
                       study = study, trt = treatment,
                       r = responders, n = sampleSize)

# 3. Combine network
network <- combine_network(ipd_net, agd_net)

# 4. Add integration points
network <- add_integration(
  network,
  age = distr(qnorm, mean = age_mean, sd = age_sd),
  sex = distr(qbinom, prob = sex_prop),
  n_int = 500
)

# 5. Fit ML-NMR
fit <- nma(
  network,
  trt_effects = "random",
  regression = ~ age + sex,
  prior_intercept = prior_normal(0, 10),
  prior_trt = prior_normal(0, 5),
  prior_reg = prior_normal(0, 2),
  prior_het = prior_half_normal(1),
  adapt_delta = 0.95,
  chains = 4,
  iter = 4000,
  warmup = 2000,
  seed = 12345
)

# 6. Check convergence
print(fit)

# 7. Relative effects
rel_eff <- relative_effects(fit)
plot(rel_eff)

# 8. Rankings
ranks <- posterior_rank_probs(fit)
plot(ranks)

# 9. Predict to target
target <- data.frame(age = 60, sex = 0.5)
pred <- predict(fit, newdata = target)

# 10. Node-splitting
nodesplit_fit <- nma(network, consistency = "nodesplit", ...)

Resources

• NICE DSU TSD 18: Population-adjusted comparisons
• Phillippo et al. (2020): ML-NMR methods paper
• multinma package: https://dmphillippo.github.io/multinma/
• Stan User's Guide (for MCMC diagnostics)