Selective bound testing at interim analyses

Introduction

All examples use a common time-to-event design. Enrollment lasts 16 months, with the relative enrollment rate increasing over the first 6 months and then remaining constant. The trial lasts 36 months. Control-group survival is 50%, 40%, and 35% at 1, 2, and 3 years, respectively. We convert each interval’s change in log survival to a piecewise exponential failure rate. Dropout is exponential at an annual rate of 0.02, and the hazard ratio under the alternative is 0.7. Analyses occur after 40%, 70%, and 100% of planned events.

survival_time <- c(0, 12, 24, 36)
control_survival <- c(1, .50, .40, .35)
control_failure_rate <- -diff(log(control_survival)) / diff(survival_time)
enrollment_duration <- c(rep(1, 5), 11)
relative_enrollment_rate <- 1:6

survival_args <- list(
  k = 3,
  alpha = .025,
  beta = .1,
  hr = .7,
  timing = c(.4, .7),
  sfu = sfLDOF,
  sfl = sfHSD,
  sflpar = 1,
  lambdaC = control_failure_rate,
  S = c(12, 24),
  eta = .02 / 12,
  gamma = relative_enrollment_rate,
  R = enrollment_duration,
  T = 36,
  minfup = 20
)

Equivalently, gsDesign2::s2pwe(times = c(12, 24, 36), survival = c(.50, .40, .35))$rate performs this survival-to-rate conversion.

Every example below inherits timing = c(.4, .7) and these survival and enrollment assumptions from survival_args. The gamma input changes from 1 to 6 over the first 6 months; its sixth value then applies through month 16. Because total duration and minimum follow-up are fixed, gsSurv() scales this relative enrollment-rate pattern to attain 90% power. Designs with different stopping rules can therefore show different fitted enrollment rates while retaining the same timing, survival, dropout, and enrollment-ramp assumptions.

The Lan–DeMets O’Brien–Fleming spending function, sfLDOF, sets a high bar for early efficacy stopping that is generally acceptable to regulators. The first interim is delayed until 40% of planned events to balance useful early futility stopping against power loss. We initially use sfHSD with sflpar = 1; this is a relatively aggressive starting point that should be examined in sensitivity analyses before adopting the design.

In many clinical trial designs, it is desirable to test only certain boundaries at specific interim analyses. For example:

• Futility testing at early interims only: In some trial designs, a futility assessment is only meaningful at the first interim analysis. Later analyses focus solely on efficacy.
• No efficacy testing at the first interim: Some regulatory or operational considerations may preclude testing for efficacy at the very first interim.
• Selective harm monitoring: For designs with harm bounds (test.type = 7 or 8), harm monitoring may be needed only at certain analyses.

The testUpper, testLower, and testHarm parameters in gsDesign(), gsSurv(), and gsSurvCalendar() allow fine-grained control over which bounds are active at each analysis. When a bound is inactive at a given analysis, it is set to an extreme value (\(\pm 20\) on the Z-scale) so that it cannot be crossed. Every characteristic for that bound, including cumulative crossing probability, is displayed as NA at the inactive analysis. The underlying probability arrays retain the cumulative crossing information.

Parameters

Each of testUpper, testLower, and testHarm accepts either a single logical value (recycled to all analyses) or a logical vector of length k (the number of analyses):

Validation: At every analysis, at least one of the active bounds must be TRUE. If all three are FALSE at any analysis, an error is raised.

Example 1: Futility testing only at the first interim

A common scenario is to test for futility only at the first interim analysis. This is useful when the trial’s data monitoring committee wants an early “go/no-go” decision, but not ongoing futility monitoring. For this non-binding test.type = 4 design, efficacy testing also begins at IA2 rather than IA1.

# Non-binding futility only at IA1; efficacy starts at IA2
x1 <- do.call(gsSurv, c(survival_args, list(
  test.type = 4,
  testUpper = c(FALSE, TRUE, TRUE),
  testLower = c(TRUE, FALSE, FALSE)
)))

The active futility bound depends on the beta spending function as well as the testing schedule. Here, sfl = sfHSD and sflpar = 1 specify the spending pattern. Changing sfl or sflpar changes how beta is allocated and therefore changes the futility bound at IA1. With sflpar = 1, the futility rule is relatively aggressive: the IA1 bound is approximately HR 0.912, and the probability of stopping there under the design alternative, HR 0.7, is 5.2%. Its impact on power, planned events, and expected trial duration should be compared with less aggressive choices.

The lower bound is active only at IA1. At IA2 and the final analysis, the futility bound shows as NA:

gsBoundSummary(x1, exclude = NULL) |> gt::gt()

The probabilities under the null and alternative are recomputed accounting for the inactive bounds. The underlying cumulative futility crossing probability does not increase after IA1 since no further futility testing occurs; the later inactive futility rows in gsBoundSummary() are displayed entirely as NA.

Using exclude = NULL includes conditional power at the current trend (CP), conditional power under the design alternative (CP H1), and predictive power (PP). Predictive power averages conditional power over the posterior formed from the interim result and the prior argument; it is sometimes referred to as average conditional power. By default, gsBoundSummary() uses a normal prior centered halfway between the null and alternative, with standard deviation 10 / sqrt(x$n.fix). Its variance is equivalent to observing 1% of the fixed-design information, so it is intended to be relatively weak. This means that the posterior used to determine predictive power is dominated by the interim data and primarily reflects variability in the interim observed treatment effect.

Example 2: Asymmetric lower testing

For test.type = 6, astar controls Type I error allocated to the lower bound. To be somewhat aggressive, we set astar = 0.15. As with the other non-binding designs considered here, we skip efficacy testing at IA1:

xs <- do.call(gsSurv, c(survival_args, list(
  test.type = 6,
  astar = .15,
  testUpper = c(FALSE, TRUE, TRUE),
  testLower = c(TRUE, TRUE, FALSE)
)))
gsBoundSummary(xs, exclude = NULL) |>
  gt::gt()

Example 3: Selective harm monitoring (test.type 7/8)

For designs with a separate harm bound, the testHarm parameter controls which analyses include harm monitoring. This is the same concept as testLower for test.type = 5 or 6, in that the harm bound tests whether experimental treatment is worse than control. For test.type = 7 or 8, the futility bound instead examines differences from the hypothesized effect hr. This can be useful when harm monitoring is most critical during early enrollment, before longer-term safety data are available. Continuing with the common survival assumptions, we change the design type and add a harm spending function and testing schedule. We use the same Hwang–Shih–DeCani parameter, 1, for harm spending and skip efficacy testing at IA1:

# Harm bound design with harm monitoring only at IA1 and IA2
xh <- do.call(gsSurv, c(survival_args, list(
  test.type = 8,
  astar = .15,
  sfharm = sfHSD,
  sfharmparam = 1,
  testUpper = c(FALSE, TRUE, TRUE),
  testHarm = c(TRUE, TRUE, FALSE)
)))
gsBoundSummary(xh, exclude = NULL) |> gt::gt()

The harm bound is NA at the final analysis.

common_harm_args <- c(survival_args, list(
  test.type = 8,
  astar = .15,
  testUpper = c(FALSE, TRUE, TRUE),
  sfharm = sfHSD,
  sfharmparam = 1
))

x_all_lower <- do.call(gsSurv, c(
  common_harm_args,
  list(
    testLower = c(TRUE, TRUE, TRUE),
    testHarm = c(TRUE, TRUE, TRUE)
  )
))

x_final_harm <- do.call(gsSurv, c(
  common_harm_args,
  list(
    testLower = c(TRUE, TRUE, FALSE),
    testHarm = c(TRUE, TRUE, TRUE)
  )
))

x_matched_lower <- do.call(gsSurv, c(
  common_harm_args,
  list(
    testLower = c(TRUE, TRUE, FALSE),
    testHarm = c(TRUE, TRUE, FALSE)
  )
))

harm_comparison <- data.frame(
  Analysis = seq_len(x_all_lower$k),
  `All lower bounds active` = x_all_lower$harm$bound,
  `Final harm only` = x_final_harm$harm$bound,
  `Both final lower bounds skipped` = x_matched_lower$harm$bound,
  check.names = FALSE
)
kable(harm_comparison, digits = 3, caption = "Harm bounds by final-analysis testing schedule")

testLower = c(TRUE, TRUE, FALSE)
testHarm = c(TRUE, TRUE, FALSE)

Example 4: Combining selective efficacy and futility

Both testUpper and testLower can be specified simultaneously. For example, a design with futility-only at IA1 and efficacy-only at IA2:

# Futility only at IA1; efficacy at IA2 and Final
x5 <- do.call(gsSurv, c(survival_args, list(
  test.type = 4,
  testUpper = c(FALSE, TRUE, TRUE),
  testLower = c(TRUE, FALSE, FALSE)
)))
gsBoundSummary(x5, exclude = NULL) |> gt::gt()

Note the NA values: efficacy is NA at IA1, and futility is NA at IA2 and the final analysis.

Validation rules

The following rules are enforced:

1. testUpper must be TRUE at the final analysis (the trial must always be able to reject \(H_0\)).
2. At least one bound must be active at every analysis. For example, setting testUpper = c(FALSE, TRUE, TRUE) and testLower = c(FALSE, TRUE, TRUE) would fail because no bound is active at IA1.
3. test.type = 1: Only one-sided efficacy testing. testLower is ignored (set to FALSE internally).
4. test.type = 2: Symmetric two-sided testing. Both testUpper and testLower are overridden to all TRUE.
5. test.type 3–8: testLower must be TRUE for at least one analysis.
6. test.type 7 and 8: testHarm must be TRUE for at least one analysis.

# This fails: testUpper must be TRUE at the final analysis
try(do.call(gsSurv, c(survival_args, list(
  test.type = 3,
  testUpper = c(TRUE, TRUE, FALSE)
))))
#> Error in gsTestBoundsCheck(x$k, x$test.type, testUpper, testLower, testHarm) : 
#>   testUpper must be TRUE at the final analysis

# This fails: no bound active at analysis 1
try(do.call(gsSurv, c(survival_args, list(
  test.type = 4,
  testUpper = c(FALSE, TRUE, TRUE),
  testLower = c(FALSE, TRUE, TRUE)
))))
#> Error in gsTestBoundsCheck(x$k, x$test.type, testUpper, testLower, testHarm) : 
#>   At analysis 1 at least one of testUpper, testLower, or testHarm must be TRUE

Accessing stored flags

The testUpper, testLower, and testHarm logical vectors are stored on the returned gsDesign object:

x1$testUpper
#> [1] FALSE  TRUE  TRUE
x1$testLower
#> [1]  TRUE FALSE FALSE
x1$testHarm
#> [1] FALSE FALSE FALSE

These can be inspected programmatically for downstream analyses or reporting. For gsSurv() and gsSurvCalendar() objects, x$call records the original call and x$inputs retains evaluated survival-model inputs plus the applicable testing-schedule arguments. The normalized logical vectors above remain the authoritative schedule used in calculations and summaries.

Type I Error Preservation

A key property of the selective bounds implementation is that Type I error is preserved at the nominal level regardless of which analyses are skipped.

# Baseline non-binding design; efficacy starts at IA2
x_nb <- do.call(gsSurv, c(survival_args, list(
  test.type = 4,
  testUpper = c(FALSE, TRUE, TRUE)
)))

# Remove futility at IA2 and final
x_nb_sel <- do.call(gsSurv, c(survival_args, list(
  test.type = 4,
  testUpper = c(FALSE, TRUE, TRUE),
  testLower = c(TRUE, FALSE, FALSE))
))

# Non-binding alpha (computed ignoring lower bounds)
nb_alpha_base <- sum(gsDesign:::gsprob(0, x_nb$n.I, rep(-20, 3), x_nb$upper$bound, r = x_nb$r)$probhi)
nb_alpha_sel  <- sum(gsDesign:::gsprob(0, x_nb_sel$n.I, rep(-20, 3), x_nb_sel$upper$bound, r = x_nb_sel$r)$probhi)
cat("Baseline non-binding alpha: ", nb_alpha_base, "\n")
#> Baseline non-binding alpha:  0.025
cat("Selective non-binding alpha:", nb_alpha_sel, "\n")
#> Selective non-binding alpha: 0.025
cat("Upper bounds identical:     ", all.equal(x_nb$upper$bound, x_nb_sel$upper$bound), "\n")
#> Upper bounds identical:      TRUE

nb_alpha_eff <- sum(gsDesign:::gsprob(
  0, x_nb$n.I, rep(-20, 3), x_nb$upper$bound, r = x_nb$r
)$probhi)
cat("Non-binding alpha (skip IA1 efficacy):", nb_alpha_eff, "\n")
#> Non-binding alpha (skip IA1 efficacy): 0.025

# Baseline binding design
x_b <- do.call(gsSurv, c(survival_args, list(test.type = 3)))
cat("Baseline alpha:", sum(x_b$upper$prob[, 1]), "\n")
#> Baseline alpha: 0.02496047

# Remove futility at IA2 and final
x_b_sel <- do.call(gsSurv, c(survival_args, list(
  test.type = 3,
  testLower = c(TRUE, FALSE, FALSE))
))
cat("Selective alpha:", sum(x_b_sel$upper$prob[, 1]), "\n")
#> Selective alpha: 0.02500004
cat("Selective power:", sum(x_b_sel$upper$prob[, 2]), "\n")
#> Selective power: 0.9000007

# Remove efficacy at IA1
x_b_eff <- do.call(gsSurv, c(survival_args, list(
  test.type = 3,
  testUpper = c(FALSE, TRUE, TRUE))
))
cat("Skip IA1 efficacy alpha:", sum(x_b_eff$upper$prob[, 1]), "\n")
#> Skip IA1 efficacy alpha: 0.025
cat("Skip IA1 efficacy power:", sum(x_b_eff$upper$prob[, 2]), "\n")
#> Skip IA1 efficacy power: 0.8999983