library(ssPilot)This document develops and validates the R package ssPilot that implements the sample size calculations described by Whitehead et al. (2016).
library(ssPilot)The methods considered in this project are:
The final calculations will be implemented as functions in the ssPilot R package.
For a two-arm trial with a continuous normally distributed outcome, the standard sample size per treatment arm is
$$ n = \frac{(r+1)(z_{1-\beta}+z_{1-\alpha/2})^2\sigma^2} {rd^2}. $$
where:
Suppose that
σ = 1,
and
d = 0.5.
We use a two-sided significance level of 5% and 90% power.
standard_sample_size(
sd = 1,
effect = 0.5,
power = 0.90,
alpha = 0.05
)[1] 85Whitehead et al. describe an approach in which the uncertainty in the estimate of the population variance is incorporated into the main-trial sample-size calculation.
The method uses an upper confidence limit for the variance estimated from the pilot trial. This produces a more conservative estimate of the variance than simply using the pilot estimate itself.
The upper confidence limit for the variance is given by
$$ s^2_{UCL} = \frac{k s^2} {\chi^2_{1-X,k}}, $$
where s2 is the variance estimated from the pilot trial, k is the degrees of freedom associated with the variance estimate, and X is the confidence level used for the upper confidence limit.
The quantity χ1 − X, k2 is the lower-tail chi-squared quantile with k degrees of freedom.
For a two-arm pilot trial with equal allocation and m participants per treatment arm, the degrees of freedom for the pooled variance estimate are
k = 2m − 2.
Therefore, the degrees of freedom depend directly on the number of participants included in the pilot trial.
For example, if the pilot trial contains 12 participants per treatment arm, then
k = 2(12) − 2 = 22.
We can calculate this directly in R.
pilot_n <- 12
k <- 2 * pilot_n - 2
k[1] 22The next step is to calculate the upper confidence limit for the variance.
Suppose that the standard deviation estimated from the pilot trial is 1. The corresponding variance is therefore
s2 = 12 = 1.
Whitehead et al. consider an 80% confidence level for the upper confidence limit. Thus, we set
X = 0.80.
Using the UCL equation,
$$ s^2_{UCL} = \frac{k s^2} {\chi^2_{1-X,k}}, $$
we can calculate the upper confidence limit for the variance in R.
sd <- 1
variance <- sd^2
conf_level <- 0.80
variance_ucl <-
k * variance /
qchisq(1 - conf_level, df = k)
variance_ucl[1] 1.348532The UCL calculation above produces an upper confidence limit for the variance.
Because the main-trial sample-size equation is expressed in terms of the standard deviation, we take the square root of the upper confidence limit for the variance.
Thus,
$$ s_{UCL} = \sqrt{s^2_{UCL}}. $$
We calculate this quantity in R as follows.
sd_ucl <- sqrt(variance_ucl)
sd_ucl[1] 1.161263The purpose of the UCL method is to incorporate the uncertainty in the pilot estimate of the standard deviation into the sample-size calculation for the main trial.
The upper confidence limit for the standard deviation is therefore used in place of the original pilot estimate of the standard deviation.
The standard sample-size equation is
$$ n_M = \frac{(r+1)(z_{1-\beta}+z_{1-\alpha/2})^2s^2_{UCL}} {rd^2}. $$
Under the UCL approach, the variance estimate is replaced by the upper confidence limit for the variance. Equivalently, we can use sUCL as the standard deviation in our implementation of the standard sample-size function.
For illustration, suppose that the standardized treatment difference is
$$ \delta = \frac{d}{\sigma} = 0.50. $$
We can represent this using
s = 1
and
d = 0.50.
We use 90% power, a two-sided 5% significance level, and equal allocation between the two treatment arms.
effect <- 0.50
power <- 0.90
alpha <- 0.05
allocation <- 1
main_n <- standard_sample_size(
sd = sd_ucl,
effect = effect,
power = power,
alpha = alpha,
allocation = allocation
)
main_n[1] 114The previous calculations can be combined into a single sequence.
Starting with the pilot sample size and the pilot standard deviation, we first calculate the degrees of freedom. We then calculate the upper confidence limit for the variance, obtain the corresponding upper confidence limit for the standard deviation, and finally use that value in the main-trial sample-size calculation.
This sequence represents the computational structure that we will later implement as the ucl_sample_size() function in the ssPilot package.
pilot_n <- 12
sd <- 1
conf_level <- 0.80
effect <- 0.50
power <- 0.90
alpha <- 0.05
allocation <- 1
# Degrees of freedom
k <- 2 * pilot_n - 2
# Upper confidence limit for the variance
variance_ucl <-
k * sd^2 /
qchisq(1 - conf_level, df = k)
# Upper confidence limit for the standard deviation
sd_ucl <- sqrt(variance_ucl)
# Main-trial sample size
main_n <- standard_sample_size(
sd = sd_ucl,
effect = effect,
power = power,
alpha = alpha,
allocation = allocation
)
main_n[1] 114The calculations above demonstrate the complete UCL procedure.
The pilot sample size determines the degrees of freedom. The pilot standard deviation is then used to obtain an upper confidence limit for the variance. The square root of this quantity gives the upper confidence limit for the standard deviation, which is subsequently used to calculate the required main-trial sample size.
The next step is to translate this sequence into the ucl_sample_size() function in the R package.
ucl_sample_size(
pilot_n = 12,
sd = 1,
effect = 0.50,
power = 0.90,
alpha = 0.05,
conf_level = 0.80
)$pilot_n_per_arm
[1] 12
$degrees_of_freedom
[1] 22
$variance_ucl
[1] 1.348532
$sd_ucl
[1] 1.161263
$main_n_per_arm
[1] 114The UCL method requires a pilot study to estimate the population standard deviation. Increasing the pilot sample size improves the precision of the variance estimate, but it also increases the total number of participants required.
The optimized approach considers this trade-off.
For each possible pilot sample size, we:
The optimal pilot sample size is the value that minimizes the total sample size.
The optimization is subject to a minimum pilot sample size of 10 participants per treatment group.
We begin by defining a sequence of possible pilot sample sizes.
For illustration, we consider pilot sample sizes from 10 through 50 participants per treatment arm.
pilot_sizes <- 10:50
pilot_sizes [1] 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
[26] 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50For each candidate pilot sample size, the degrees of freedom are
k = 2m − 2.
The upper confidence limit for the variance is then
$$ s^2_{UCL} = \frac{k s^2} {\chi^2_{1-X,k}}. $$
We can therefore calculate the UCL standard deviation for every candidate pilot sample size.
sd <- 1
conf_level <- 0.80
variance_ucl <- sapply(pilot_sizes, function(m) {
k <- 2 * m - 2
k * sd^2 /
qchisq(1 - conf_level, df = k)
})
sd_ucl <- sqrt(variance_ucl)
sd_ucl [1] 1.183225 1.171277 1.161263 1.152723 1.145336 1.138867 1.133146 1.128042
[9] 1.123452 1.119299 1.115517 1.112056 1.108874 1.105936 1.103212 1.100679
[17] 1.098315 1.096103 1.094027 1.092074 1.090234 1.088494 1.086848 1.085286
[25] 1.083802 1.082390 1.081044 1.079760 1.078532 1.077358 1.076232 1.075153
[33] 1.074116 1.073120 1.072161 1.071238 1.070348 1.069489 1.068660 1.067859
[41] 1.067084Each upper confidence limit for the standard deviation is then used to calculate the corresponding main-trial sample size.
Thus, every candidate pilot sample size produces a corresponding main-trial sample size.
effect <- 0.50
power <- 0.90
alpha <- 0.05
main_sizes <- sapply(sd_ucl, function(s) {
standard_sample_size(
sd = s,
effect = effect,
power = power,
alpha = alpha,
allocation = 1
)
})
main_sizes [1] 118 116 114 112 111 110 108 107 107 106 105 104 104 103 103 102 102 101 101
[20] 101 100 100 100 100 99 99 99 99 98 98 98 98 97 97 97 97 97 97
[39] 96 96 96The objective of the optimization is to minimize the combined number of participants in the pilot and main trials.
Therefore, for each candidate pilot size m, we calculate
Ntotal = m + nM,
where m is the pilot sample size per treatment arm and nM is the resulting main-trial sample size per treatment arm.
total_sizes <- pilot_sizes + main_sizes
total_sizes [1] 128 127 126 125 125 125 124 124 125 125 125 125 126 126 127 127 128 128 129
[20] 130 130 131 132 133 133 134 135 136 136 137 138 139 139 140 141 142 143 144
[39] 144 145 146The optimal pilot sample size is the candidate that produces the smallest total sample size.
optimal_index <- which.min(total_sizes)
optimal_pilot_n <- pilot_sizes[optimal_index]
optimal_main_n <- main_sizes[optimal_index]
optimal_total_n <- total_sizes[optimal_index]
optimal_pilot_n[1] 16optimal_main_n[1] 108optimal_total_n[1] 124The candidate pilot sample sizes and their corresponding main-trial and total sample sizes can be displayed together in a table.
optimization_results <- data.frame(
pilot_n_per_arm = pilot_sizes,
main_n_per_arm = main_sizes,
total_n_per_arm = total_sizes
)
optimization_results pilot_n_per_arm main_n_per_arm total_n_per_arm
1 10 118 128
2 11 116 127
3 12 114 126
4 13 112 125
5 14 111 125
6 15 110 125
7 16 108 124
8 17 107 124
9 18 107 125
10 19 106 125
11 20 105 125
12 21 104 125
13 22 104 126
14 23 103 126
15 24 103 127
16 25 102 127
17 26 102 128
18 27 101 128
19 28 101 129
20 29 101 130
21 30 100 130
22 31 100 131
23 32 100 132
24 33 100 133
25 34 99 133
26 35 99 134
27 36 99 135
28 37 99 136
29 38 98 136
30 39 98 137
31 40 98 138
32 41 98 139
33 42 97 139
34 43 97 140
35 44 97 141
36 45 97 142
37 46 97 143
38 47 97 144
39 48 96 144
40 49 96 145
41 50 96 146We can now compare the sample-size calculations implemented in the package.
For the following example, we use a standard deviation of 1, a treatment effect of 0.50, 90% power, and a two-sided significance level of 0.05.
The UCL and UCL optimized methods use an 80% upper confidence limit.
sd <- 1
effect <- 0.50
power <- 0.90
alpha <- 0.05
conf_level <- 0.80
standard_result <- standard_sample_size(
sd = sd,
effect = effect,
power = power,
alpha = alpha
)
ucl_result <- ucl_sample_size(
pilot_n = 12, # as suggested by Julious (2005)
sd = sd,
effect = effect,
power = power,
alpha = alpha,
conf_level = conf_level
)
optimized_ucl_result <- optimized_ucl_sample_size(
sd = sd,
effect = effect,
power = power,
alpha = alpha,
conf_level = conf_level
)The results from the four calculations can be summarized in a single table.
comparison <- data.frame(
method = c(
"Standard",
"UCL",
"Optimized UCL"
),
main_n_per_arm = c(
standard_result,
ucl_result$main_n_per_arm,
optimized_ucl_result$main_n_per_arm
)
)
comparison method main_n_per_arm
1 Standard 85
2 UCL 114
3 Optimized UCL 108Whitehead AL, Julious SA, Cooper CL, Campbell MJ. Estimating the sample size for a pilot randomised trial to minimise the overall trial sample size for the external pilot and main trial for a continuous outcome variable. Stat Methods Med Res. 2016 Jun;25(3):1057–73. doi:10.1177/0962280215588241
Browne RH. On the use of a pilot sample for sample size determination. Statistics in Medicine. 1995 Sep 15;14(17):1933–40. doi:10.1002/sim.4780141709
Julious SA, Owen RJ. Sample size calculations for clinical studies allowing for uncertainty about the variance. Pharmaceutical Statistics. 2006 Jan;5(1):29–37. doi:10.1002/pst.197
Sim J, Lewis M. The size of a pilot study for a clinical trial should be calculated in relation to considerations of precision and efficiency. Journal of Clinical Epidemiology. 2012 Mar;65(3):301–8. doi:10.1016/j.jclinepi.2011.07.011
Julious SA. Sample size of 12 per group rule of thumb for a pilot study. Pharmaceutical Statistics. 2005 Oct;4(4):287–91. doi:10.1002/pst.185