Abstract
Purpose
Clinical Trial Simulations (CTS) are a valuable tool for decision-making during drug development. However, to obtain realistic simulation scenarios, the patients included in the CTS must be representative of the target population. This is particularly important when covariate effects exist that may affect the outcome of a trial. The objective of our investigation was to evaluate and compare CTS results using re-sampling from a population pool and multivariate distributions to simulate patient covariates.
Methods
COPD was selected as paradigm disease for the purposes of our analysis, FEV1 was used as response measure and the effects of a hypothetical intervention were evaluated in different populations in order to assess the predictive performance of the two methods.
Results
Our results show that the multivariate distribution method produces realistic covariate correlations, comparable to the real population. Moreover, it allows simulation of patient characteristics beyond the limits of inclusion and exclusion criteria in historical protocols.
Conclusion
Both methods, discrete resampling and multivariate distribution generate realistic pools of virtual patients. However the use of a multivariate distribution enable more flexible simulation scenarios since it is not necessarily bound to the existing covariate combinations in the available clinical data sets.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
Whilst simulations have existed for many years as a statistical technique, their use as tool to evaluate treatment response in clinical trials has become possible thanks to the integration of disease progression and pharmacokinetic-pharmacodynamic (PKPD) models. From a drug development perspective, clinical trial simulation (CTS) became a useful tool to support decision making and reduce clinical trial failure (1–3). Among the numerous possible applications, CTS has been used to characterise the behaviour of a biological systems, explore drug properties as well as understand response in different populations. It can also be used to rank or prioritise options for a drug development program, thereby providing an integrated overview of potential designs and outcomes to relevant stakeholders (e.g., clinical experts, regulatory authorities). Moreover, it offers the opportunity to test different “what if” scenarios from first-time-in-human studies throughout post-marketing studies in phase IV (4).
The technical definition of CTS includes the generation of a response for a virtual subject by reproducing the trial design, the disease progression, the drug and the patient’s behaviour using mathematical models and numerical methods (5, 6) (Fig. 1). Even though much attention has been given to disease and drug models, limited efforts have been made to ensure the accurate evaluation of trial design factors, patient behaviour and individual characteristics in clinical trial simulations. The trial execution model represents the design variables of interest in a simulation exercise (e.g. dosing regimens, selection criteria, stratification rules, study duration); the patient’s behaviour comprises those factors that determine the trial execution features, such as adherence and missing records. Individual characteristics include demographic, clinical and physiological measures that altogether describe individual patients in the target population.
Since the objective of a CTS exercise is to evaluate how multiple interacting factors in a specific scenario affect the outcome of the study, it is essential to properly simulate patient populations which reflect individual characteristics in a realistic manner, including an evaluation of the impact of individuals who do not meet inclusion/exclusion criteria, as defined for evidence arising from controlled clinical trials. Thus far, in most cases, published literature regarding the use of simulations in clinical pharmacology has been limited to the evaluation of the effect of changes in some specific design factors, such as the study duration, dose or sampling frequency (8–11). By contrast, there is very little work done on the impact of design factors related to patient selection and protocol stratification. Yet, these publications often do not consider the implication of such factors on trial outcome. This is particularly important if one wants to understand the role of physiological or demographic factors as covariates on response parameters and assess the possibility for personalised therapy or dose adjustment (12).
In this context, covariate distribution models play an essential role. They describe patient-specific aspects defining e.g., patient demographic information, baseline disease characteristics, co-morbidity and concomitant medication (13). For each patient, these details may be considered as a vector containing the patient information (e.g., male, 70 kg, smoker, severe disease status, etc.). This information vector is usually associated with the differences between patients in terms of pharmacokinetics and pharmacodynamics, and it is often used to explain the variability on individual parameter values. Consequently, the simulation of covariates for a virtual patient population assumes great importance as the individual vectors of the covariates for each patient will determine the outcome of the simulated study.
Statistically, the approaches used to simulate virtual populations may be divided in nonparametric and parametric simulations. Nonparametric bootstrap** or re-sampling is one of the most straightforward methods for constructing a population of virtual patients, often used to determine the uncertainty on estimated or predicted quantities (14, 15). Using this approach, a pool of patients can be created by randomly selecting patient vectors for inclusion into the virtual pool of patients. This first random selection of individuals can then be refined by applying inclusion–exclusion criteria in order to obtain the desired population. In general, numerous covariates are correlated, and as such these correlations need to be maintained during the creation of virtual patients. For instance, in paediatric patients, age and weight are generally related and any re-sampling technique will need to account for that correlation (16). In this sense, re-sampling methods provide an opportunity to create patients with a realistic combination of covariates. Nevertheless, the technique suffers an important limitation, i.e., the constraint of the observed covariate matrix in the real data (empirical distribution). Consequently, one should be aware of the fact that the simulated population will not include patients with any other combination of covariates other than the one observed in the source data.
Parametric methods, on the other hand, may be used as alternative to re-sampling in order to generate new combinations of vectors or matrices from an existing distribution. The two main methods for generating covariate vectors with this approach are represented by a series of univariate distributions or by using a unique multivariate distribution for the whole population. The advantage of this latter technique consists in accounting for the correlation between covariates in the simulation process, which allows for the simulation of more realistic covariate vectors. Whilst it can be anticipated that a mechanistic or physiologically-based model may be required with increasing number of interacting factors (17, 18), in many cases, these interactions may be defined using existing knowledge or may be extrapolated from an existing population.
Independently from which choices are made, the method for obtaining simulated data sets should be carefully considered and its performances verified afterwards (19). The simulated data sets should resemble reality if the results are meant to reflect real-life conditions. The correlation between covariates can be estimated from real data to ensure that the simulated data closely reflect the underlying pool of patients. Subsequently, the distribution of the simulated data should be verified to confirm they resemble what is being simulated. This can be done, for example by using summary measures for the covariate distributions, Kaplan–Meier survival curves for survival data or by fitting appropriate regression models.
The objective of this exercise was therefore to compare the use of discrete re-sampling and multivariate normal distribution (MVND) methodologies in the creation of virtual patient populations showing plausible, realistic vector of clinical and demographic characteristics. The MVND methodology developed by Tannenbaum and collaborators (20), which is usually applied to the simulation of continuous variable, was adapted to allow the simulation of categorical data. The approach consists in treating all categorical covariates as if they have continuous values; new covariate values are sampled from a distribution built using the real data and the simulated data are then converted back again into categorical ones. Chronic obstructive pulmonary disease (COPD) was selected as a case study for the comparison of the two methodologies. A previous investigation performed by our group on the progression of COPD has shown that symptoms and severity in this population are closely related to a combination of multiple covariates, continuous as well as categorical ones, including demographics and disease-specific characteristics (21, 22). Moreover, considering the large number of COPD trials failed in the past few years, we anticipate that covariate effect profiling will be critical for the design and evaluation of efficacy trials in this field.
Methods
As the objective of this exercise was to compare the use of discrete re-sampling and multivariate normal distribution (MVND) methodologies in the creation of virtual patient populations, attention was paid to their performance in generating a plausible, realistic vector of clinical and demographic characteristics. Continuous and categorical covariates were simulated using a real set of COPD patients from the Dutch TI Pharma database. Patient demographics and data set details are reported in Table I. The data set used in the simulation consisted of 3498 patients (32% women), and the covariates included in the analysis were COPD severity, smoking status, gender, weight, height and age. The covariate simulation was performed with R 2.12.0 (23). This patient population was used as empirical distribution for the subsequent steps in our analysis.
CTS with Re-Sampling and MVND Patient Generation
Discrete re-sampling was performed using the R package MStoolkit (Mango Solutions, UK). The data set used in the simulations contains only one line per subject, i.e., where applicable multiple time measures were removed and each patient was randomly sampled as a vector of covariates. Each patient could be sampled multiple times in order to obtain pools of patients greater than the original data set, and to introduce variability when an equal number of patients was simulated.
In order to generate continuous and categorical covariates in parallel, a set of pre-defined data manipulation steps is required before the simulations are performed. The MVND methodology was based on the approach described by Tannenbaum et al. (20). Briefly, with this technique the continuous and categorical covariates are treated as continuous variables. All the covariates are log-transformed and normal distribution is then assumed during the simulation. Subsequently, the simulated values are converted back into the normal space. The simulated values describing categorical covariates are linked to pre-defined thresholds within the simulated distribution; these thresholds are calculated from the inverse cumulative distribution function of the empirical distribution.
The MVND covariate simulations were performed according to a user-defined function in R (see Electronic supplementary material). This function takes into account the distribution of covariates in the empirical distribution with a tolerance level of 5% for categorical covariates, whereas continuous covariates are retained for values within the range of the real data. These procedures can be adjusted to different acceptance criteria and correspond to simulating from truncated distributions when compared to the empirical (real) covariate distributions. All simulations were based on study design scenarios with group sizes of 100, 1000 and 3500 patients. Since the MVND method consists in the generation of new subjects, these simulated covariates were also compared with the real data. In this evaluation, each simulation was performed in duplicate in order to evaluate the reproducibility of the method.
Drug-Disease Model for Trough FEV1
In order to evaluate the performance of re-sampling and MVND methodologies in the context of CTS, covariates were simulated in conjunction with a KPD model describing the treatment effects of a bronchodilator on forced expiratory volume 1 sec (FEV1). The analysis was performed using NONMEM 7.1.2 (24, 25).
The changes in FEV1 over time were described by an indirect response model, as expressed by the following differential equation:
where the change in the observed FEV1 over time (dFEV1/dt) is controlled by a zero-order process parameterised as a synthesis rate constant (Kin) and first order elimination process (Kout). The disease status at baseline (steady state conditions) was defined by the ratio between Kin and Kout. The disease progression was modelled as a linear decline in the disease status as follows:
With
where Dis is the disease status, Int_Dis is the disease status at the start of the clinical trial, and Slope_Dis is the daily decline in Dis due to the disease progression.
As described by Eq. (2), the disease status (Dis) reflects not only the natural disease progression over time, but also time-dependent processes (Eq. 3) associated with the changes in airway function, as determined by spirometry (FEV1). Both processes were found to be affected by an individual patient’s clinical and demographic characteristics.
The KPD model consisted of a nonlinear Emax function in which the maximum effect is proportional to the apparent potency parameter (EDK50). Further details on the parameterisation and model validation can be found elsewhere (24). Dropout was not included in this simulation exercise to avoid confounding effects. Each trial was simulated 100 times with a different set of patients.
In spite of the availability of FEV1 response data in the original trial population, we have decided to use a set of simulated responses from the model as reference for the evaluation of the performance of the re-sampling and MVND methods. This choice was made to ensure accurate comparison of results and avoid interference of study-specific aspects (e.g. dropout rate) or model-related issues (e.g. the difference between predicted and observed response) when comparing the different scenarios. The response data from the model are referred in the article as “real data” since they are generated with real covariates. Drug and placebo effect were simulated using the same KPD model as for the reference population. In this case, 100 different data sets were simulated, mimicking the situation observed in 100 different clinical studies with the same inclusion–exclusion criteria, as observed for the empirical distribution. The simulations were performed with a series of ad hoc R functions (26) aimed at automating run execution and obtaining basic statistical summaries. The simulations for treatment and placebo arms were performed independently in different data sets to avoid the interference may arise from patient assignment to the different arms.
MVND to Simulate New Patient Populations
To illustrate the implications of covariate correlations on trial outcome, additional simulation scenarios were considered in which a new patient cohort was created by screening a posteriori subsets of patients simulated according to the steps described above. Using the same ad hoc functions 3500 patients were selected to generate different stratification levels for disease severity, as compared to the empirical distribution (i.e., from 1:2 to 2:1 for mild and moderate severity).
Results
To assess the performance of the two methods for the simulation of covariates, the first step consisted in verifying the impact of different sample sizes on the covariate distributions and their correlations. Figure 2 depicts the covariate correlations for the real pool of patients and for the different populations simulated with 100, 1000 and 3500 patients.
An initial evaluation of the performance of the MVND method consisted in comparing qualitatively and quantitatively the correlation and proportion profiles of real and virtual patients. In all the simulations scenarios, the correlation between covariates was found to be maintained, for both continuous and categorical covariates. This is warranted also for small sample sizes, although slight deviations were observed for simulations with very low numbers of patients.
As shown in Fig. 3 for age and smoking status, a similar pattern is observed for the distribution of the simulated continuous covariates and the proportions for the categorical ones irrespective of sample sizes. Additional details can be found for other continuous and categorical covariates in the Electronic supplementary material (Figure S1 and S2). As depicted in Figs. 3, S1 and S2, the simulated covariates present a median and interquartile range which reflect the covariate distribution in the real population. On the other hand, in some cases the high variability in the original data set has yielded outlier values, which are not captured by the simulations. The concordance between real and simulated data is particularly important to ensure that the simulated data reflect the range of values observed in real life. The proportion of the categorical covariates also reflects the structure of the original pool of patients, showing that this technique can be used to reproduce the patient stratification used in the real trial.
CTS with Re-Sampling and MVND Patient Generation
Figure 4 shows the results for FEV1 at the end of the trial based on the re-sampling (bootstrap**) and MVND methods. The former reflects the impact of covariate correlations in the original population whereas the latter relies on the estimated multivariate distribution. No formal statistical hypothesis test is required to conclude that different results are predicted for trough FEV1 values, depending on the method used and characteristics of the simulated population. The magnitude of such differences may vary with the sample size. As expected the simulated FEV1 vs. time profiles were similar to the original trial when considering comparable or exchangeable patient populations (i.e., bootstrap**). The similarities in the results hold also after varying sample size. By contrast, multivariate methods also generate realistic covariate distributions, but provide insight into the impact of the interaction between covariates (i.e., covariate correlations) in the overall population. These differences unravel variation which is often assigned to mere randomness when comparing clinical trial results in the presence of covariate effects.
Given the presence of time-dependent covariate effects, a comprehensive summary of the two methods is provided in Figs. 5, 6 and 3S, in which FEV1 changes in 100 clinical trials are depicted over the period of 1 year. It should be underlined that despite good prediction of the mean profiles for FEV1, the variability of the simulated data was significantly higher than the real for scenarios based on smaller sample sizes (Figs. 5 and 3S). By contrast, this issue was not evident when results from scenarios with similar sample sizes are compared with each other (Fig. 6).
Overall, it appears that the trials simulated with the multivariate distribution present slightly smaller variability as compared with those obtained by discrete re-sampling.
MVND to Simulate New Patient Populations
The main advantage of MVND method compared to discrete re-sampling is the possibility to create a new pool of patients across a different range of values or stratification levels, whilst ensuring the correlation structure and/or colinearity are maintained.. An example of this feature is presented in Figure 4S (Electronic supplementary material), which shows a new study population stratified by disease severity using a different proportion of patients, as compared to the original patient pool. The FEV1 vs. time profile is similar to the original study, but the FEV1 at baseline is higher for the simulated population.
Discussion
CTS is a powerful resource for the prediction of the outcome of hypothetical clinical trials. However to be effective as a design tool, it is essential to ensure that the patient populations used in the simulation process reflect real patients and that covariate correlations are well described. Particularly important is the covariate distribution defining the correlation between patient demographic and disease characteristics, given that they are often related to the study outcome (12, 27). Such correlations have so far been overlooked in previous investigations in which covariate effects are treated as structural parameters in pharmacokinetic or pharmacokinetic-pharmacodynamic models. The need for identifying covariate correlations and incorporating them into a matrix for the simulation of virtual patients is disregarded in most publications (18). Our investigation has shown the practical implications of discrete re-sampling and MVND for the creation of a virtual patient population.
The concept of re-sampling data – more commonly referred to as bootstrap** – has been in use for more than four decades. Bootstrap** has been shown to have considerable theoretical advantages when it is applied to non-Gaussian data (28). However, its application in CTS presents an important limitation, which is crucial for prospective evaluation of drug response in drug development, which often involves changes in protocol design, doses and most importantly different inclusion and exclusion criteria, making the patient population in a trial not necessarily exchangeable with the subsequent one, as for instance during Phase II and Phase III studies (29, 30). By contrast, the MVND technique allows simulation of new vectors of covariates, which can differ from the ones present in the initial pool of subjects. A limitation of this methodology has been however its use for continuous variables only. We have implemented the procedure described by Tannenbaum et al. (20) into an R function which allows the application of such method for the simulation of continuous as well as categorical covariates (31).
Advantages and Limitations
Historically in pharmacometric research, bootstrap** and other re-sampling techniques have been applied as a statistical tool for model diagnostics and more specifically parameter estimation, i.e., to assess its accuracy and bias. Some of the elementary uses of bootstrap** include the calculation of confidence intervals, hypothesis testing, linear regression, and correlations when exploring the association between variables (32). The technique may also be useful for analysing smallish expensive-to-collect data sets, where prior information is sparse, distributional assumptions are unclear, and where further data may be difficult to acquire (33). These applications do not address the requirements for characterising the impact of covariate effects on a prospective, potentially different population. The increasing focus on personalised medicines and the use of adaptive protocol designs in clinical research impose the need for trial models in which patient population characteristics are clearly reflected in a simulation scenario, especially when making predictions from one group to another (34). From a statistical point of view, such differences in experimental conditions and varying population characteristics can limit the predictive performance of the models used in clinical trial simulations.
In fact, the possibility to evaluate the impact of greater heterogeneity and variability in the study population, as compared to randomised controlled trials, is critical for the use of clinical trial simulations as a quantitative tool for regulatory and clinical decision making. The ability to make inferences and draw conclusions about treatment response in a larger population involves assumptions about the representativeness of the cohort for the ‘target’ population (35, 36). In conjunction with the appropriate model parameterisation, it may also provide the basis for identifying the physiological and clinical mechanisms underlying statistical correlations (37). It should be noted, however, that when using MVND all the covariates are assumed to follow the same distribution, in this particular case, the log-normal distribution. A potential limitation of this assumption is that the multivariate distribution is based on only one vector of covariates per subject, which restricts the simulations to time invariant covariates or baseline values.
COPD as a Paradigm Population
The assessment of response to an intervention should be based on sensitive clinical measures or endpoints that reflect differences in disease severity and/or patient population characteristics. This has been previously illustrated for HIV infection, where CD4 counts and mRNA viral load are correlated with clinical status, antiviral treatment effect, and risk of AIDS (38). Despite ongoing efforts on the evaluation of novel treatments for COPD, little has been made to characterise how differences in patient characteristics, i.e., inclusion and exclusion criteria, correlate with the clinical status and overall treatment response (21, 22).
Our investigation shows that covariate correlations, as defined by the underlying correlation structure, can have clear implications for the outcome of a trial (24, 39). In addition, we have found that for small sample sizes, simulated FEV1 profiles show larger variability as compared to the real data. This is a natural consequence of the smaller sample size of the simulated scenarios (with MVND and re-sampling). On the other hand, when the sample size is comparable to the sample size of the empirical distribution, the FEV1 profiles mimic those of the real population. Another point worth mentioning is that trials simulated with the MVND method show lower variability as compared to those using re-sampling. This may be due to the inclusion of distribution boundaries in the simulation step with the MVND (40). In fact, to avoid unrealistic covariates values, continuous covariate values were retained if within the range of real data. Likewise, for categorical covariates a 5% tolerance was applied as acceptance criterion for differences in the covariate distribution in the simulated and real data sets. In addition, one should bear in mind that the values simulated by re-sampling are driven by the available data, which leads to empirical distributions that are closer or similar to the source data. By contrast, the data simulated with the MVND include a well-defined variance-covariance structure under the assumption that all covariates are log-normally distributed and this distribution propagates across all subgroups (e.g. male and female).
Perspectives
In summary, CTS needs to account for the main sources of variability affecting the variables of interest during an actual trial. In contrast to traditional re-sampling techniques, the MVND method allows simulation of new patient pools in which patient distributions are generated from a pre-defined covariate correlation model, irrespective of inclusion and exclusion criteria. Moreover, since new covariate vectors can be generated, it is possible to evaluate patient distributions which may not have been tested experimentally. In spite of current views on the role of genotypical and phenotypical markers as the basis for stratification in clinical trials (41, 42), our findings suggest that stratification procedures are also necessary in clinical trial simulations whenever moderate and strong covariate effects have been shown to affect pharmacokinetics and pharmacodynamics.
Abbreviations
- COPD:
-
Chronic obstructive pulmonary disease
- CTS:
-
Clinical trial simulation
- Dis:
-
Disease status
- EDK50 :
-
Apparent potency
- Emax:
-
Maximum effect
- FEV1 :
-
Forced expiratory volume in 1 second
- Int_Dis:
-
Disease status at the start of the clinical trial
- Kin:
-
Zero-order synthesis rate constant
- Kout:
-
First order elimination process
- KPD:
-
Kinetic-pharmacodynamic
- MVND:
-
Multivariate normal distribution
- PKPD:
-
Pharmacokinetic-pharmacodynamic
- Slope_Dis:
-
Daily decline in disease status
References
Chabaud S, Girard P, Nony P, Boissel J-P. Clinical trial simulation using therapeutic effect modeling: application to ivabradine efficacy in patients with angina Pectoris. J Pharmacokinet Pharmacodyn. 2002;29(4):339–63.
Kimko HC, Reele SSB, Holford NHG, Peck CC. Prediction of the outcome of a phase 3 clinical trial of an antischizophrenic agent (quetiapine fumarate) by simulation with a population pharmacokinetic and pharmacodynamic model. Clin Pharmacol Ther. 2000;68:568–77.
Lemmens HJM, Wada DR, Munera C, Eltahtawy A, Stanski DR. Enriched analgesic efficacy studies: an assessment by clinical trial simulation. Contemp Clin Trials. 2006;27(2):165–73.
Bonate PL. Clinical trial simulation in drug development. Pharm Res. 2000;17(3):252–6.
Kimko H, Duffull S, editors. Simulation for designing clinical trials. A pharmacokinetic-pharmacodynamic modeling perspective. New York: Marcel Dekker; 2003. p. 396.
Girard P. Clinical trial simulation: a tool for understanding study failures and preventing them. Basic Clin Pharmacol Toxicol. 2005;96:228–34.
Gobburu JVS, Lesko LJ. Quantitative disease, drug and trial models. Annu Rev Pharmacol Toxicol. 2009;49:291–301.
Yim DS, Zhou H, Buckwalter M, Nestorov I, Peck CC, Lee HJ. Population pharmacokinetic analysis and simulation of the time-concentration profile of etanercept in pediatric patients with juvenile rheumatoid arthritis. J Clin Pharmacol. 2005;45(3):246–56.
Agoram B, Sutjandra L, Sullivan JT. Population pharmacokinetics of darbepoetin alfa in healthy subjects. Br J Clin Pharmacol. 2007;63(1):41–52.
Pasipanodya J, Gumbo T. An oracle: Antituberculosis pharmacokinetics-pharmacodynamics, clinical correlation, and clinical trial simulations to predict the future. Antimicrob Agents Chemother. 2011;55(1):24–34.
Lala M, Burckart GJ, Takao CM, Pravica V, Momper JD, Gobburu JVS. Genetics-based pediatric warfarin dosage regimen derived using pharmacometric bridging. J Pediatr Pharmacol Ther. 2013;18(3):209–19.
Sahota T, Della Pasqua O. Feasibility of a fixed dose regimen of pyrazinamide and its impact on systemic drug exposure and liver safety in patients with tuberculosis. Antimicrob Agents Chemother. 2012;56(11):5442–9.
Holford NHG, Kimko HC, Monteleone JPR, Peck CC. Simulation of clinical trials. Annu Rev Pharmacol Toxicol. 2000;40(2):209–34.
Aarons L, Karlsson MO, Mentré F, Rombout F, Steimer JL, van Peer A. Role of modelling and simulation in phase I drug development. Eur J Pharm Sci. 2001;13(2):115–22.
Efron B. Bootstram methods: another look at the jacknife. Ann Stat. 1979;7(1):1–26.
Anderson BJ, Woollard GA, Holford NH. A model for size and age changes in the pharmacokinetics of paracetamol in neonates, infants and children. Br J Clin Pharmacol. 2000;50(2):125–34.
Achour B, Rostami-Hodjegan A, Barber J. Protein expression of various hepatic uridine 5′-diphosphate glucuronosyltransferase (UGT) enzymes and their inter-correlations: a meta-analysis. Biopharm Drug Dispos. 2014;35(6):353–61.
Jamei M, Dickinson GL, Rostami-Hodjegan A. A framework for assessing inter-individual variability in pharmacokinetics using virtual human populations and integrating general knowledge of physical chemistry, biology, anatomy, physiology and genetics: a tale of ‘bottom-up’ vs ‘top-down’ recognition of covariates. Drug Metab Pharmacokinet. 2009;24(1):53–75.
Burton A, Altman DG, Royston P, Holder RL. The design of simulation studies in medical statistics. Stat Med. 2006;25:4279–92.
Tannenbaum SJ, Holford NHG, Lee H, Peck CC, Mould DR. Simulation of correlated continuous and categorical variables using a single multivariate distribution. J Pharmacokinet Pharmacodyn. 2006;33(6):773–94.
Franciosi LG, Page CP, Celli BR, Cazzola M, Walker MJ, Danhof M, et al. Markers of disease severity in chronic obstructive pulmonary disease. Pulm Pharmacol Ther. 2005;19:189–99.
Franciosi LG, Page CP, Celli BR, Cazzola M, Walker MJ, Danhof M, et al. Markers of exacerbation severity in chronic obstructive pulmonary disease. Respir Res. 2006;7:74.
R Development Team. (2010) R: a language and environment for statistical computing. R Foundation for Statistical Computing. R Foundation for Statistical Computing. Vienna, Austria. Retrieved from http://www.r-project.org.
Musuamba FT, Teutonico D, Maas HJ, Facius A, Yang S, Danhof M, et al. Prediction of disease progression, treatment response and dropout in chronic obstructive pulmonary disease (COPD). Pharm Res. 2015;32(2):617–27.
Beal S, Sheiner LB, Boeckmann A, Bauer RJ. NONMEM user guide. (1989–2009). Ellicott City: Icon Development Solutions; 2009.
Teutonico, D., Musuamba, F., Maas, H.J., van Kesteren, C., Facius, A., Yang, S., Danhof, M., Della Pasqua, O. (2010). Development of a template for clinical trial simulations in COPD. PAGE meeting.
Piana C, Danhof M, Della Pasqua O. Influence of covariate distribution on the predictive performance of pharmacokinetic models in paediatric research. Br J Clin Pharmacol. 2014;78(1):145–57.
Wehrens R, Putter H, Buydens LMC. The bootstrap: a tutorial. Chemom Intell Lab Syst. 2000;54:35–52.
Morita S, Yamamoto H, Sugitani Y. Biomarker-based Bayesian randomized phase II clinical trial design to identify a sensitive patient subpopulation. Stat Med. 2014;33(23):4008–16.
Tournoux-Facon C, De Rycke Y, Tubert-Bitter P. Targeting population entering phase III trials: a new stratified adaptive phase II design. Stat Med. 2011;30(8):801–11.
Teutonico, D., Della Pasqua, O. Clinical trial simulation template. https://code.google.com/p/cts-template/.
Henderson AR. The bootstrap: a technique for data-driven statistics. Using computer-intensive analyses to explore experimental data. Clin Chim Acta. 2005;359(1–2):1–26.
Baser O, Crown WH, Pollicino C. Guidelines for selecting among different types of bootstraps. Curr Med Res Opin. 2006;22(4):799–808.
Xu XS, Yuan M, Nandy P. Analysis of dose–response in flexible dose titration clinical studies. Pharm Stat. 2012;11(4):280–6.
Chain AS, Dieleman JP, van Noord C, Hofman A, Stricker BH, Danhof M, et al. Not-in-trial simulation I: bridging cardiovascular risk from clinical trials to real-life conditions. Br J Clin Pharmacol. 2013;76(6):964–72.
Prentice RL. Statistical methods and challenges in epidemiology and biomedical research. In: Rao CR, Miller JP, Rao DC, editors. Handbook of statistics. Amsterdam: Elsevier; 2008. p. 1–27.
Huisinga W, Solms A, Fronton L, Pilari S. Modeling interindividual variability in physiologically based pharmacokinetics and its link to mechanistic covariate modeling. CPT Pharmacometrics Syst Pharmacol. 2012;1:e4.
De Gruttola VG, Clax P, DeMets DL, Downing GJ, Ellenberg SS, Friedman L, et al. Considerations in the evaluation of surrogate endpoints in clinical trials. summary of a National Institutes of Health workshop. Control Clin Trials. 2001;22(5):485–502.
Martinez FJ, Grossman RF, Zadeikis N, Fisher AC, Walker K, Ambruzs ME, et al. Patient stratification in the management of acute bacterial exacerbation of chronic bronchitis: the role of levofloxacin 750 mg. Eur Respir J. 2005;25(6):1001–10.
Harrell Jr FE, Lee KL, Mark DB. Multivariable prognostic models: issues in develo** models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15(4):361–87.
Hesselink AE, van der Windt DA, Penninx BW, Wijnhoven HA, Twisk JW, Bouter LM, et al. What predicts change in pulmonary function and quality of life in asthma or COPD? J Asthma. 2006;43(7):513–9.
Kwon JH, Lee N, Park JY, Yu YS, Kim JP, Shin JH, et al. Actionable gene expression-based patient stratification for molecular targeted therapy in hepatocellular carcinoma. PLoS One. 2013;8(6):e64260.
ACKNOWLEDGMENTS AND DISCLOSURES
The authors declared no conflict of interest. The views presented in this manuscript do not reflect the position of the companies contributing to the TIPharma consortium.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Supplemental material
(PDF 372 kb)
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Teutonico, D., Musuamba, F., Maas, H.J. et al. Generating Virtual Patients by Multivariate and Discrete Re-Sampling Techniques. Pharm Res 32, 3228–3237 (2015). https://doi.org/10.1007/s11095-015-1699-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11095-015-1699-x