| Title: | Efficient Sequential Testing with Evidence Ratios |
|---|---|
| Description: | An implementation of sequential testing that uses evidence ratios computed from the weights of a set of models. These weights correspond either to Akaike weights computed from the Akaike Information Criterion (AIC) or the Bayesian Information Criterion (BIC) and following Burnham & Anderson (2004, <doi:10.1177/0049124104268644>) recommendations, or to pseudo-BMA weights computed from the WAIC or the LOOIC of models fitted with 'brms' and following Yao et al. (2017, <arXiv:1704.02030v3>). |
| Authors: | Ladislas Nalborczyk [aut, cre]
|
| Maintainer: | Ladislas Nalborczyk <[email protected]> |
| License: | MIT + file LICENSE |
| Version: | 0.2.1 |
| Built: | 2024-11-10 04:15:53 UTC |
| Source: | https://github.com/lnalborczyk/ester |
Computes the Akaike Information Criterion (AIC) of a model, or the second-order bias correction for small samples (AICc), as suggested by Burnham & Anderson (2002, 2004).
aic(mod, correction = TRUE)aic(mod, correction = TRUE)
mod |
A fitted model of class |
correction |
Should we apply the second-order correction (default to TRUE) ? |
Ladislas Nalborczyk <[email protected]>
Burnham, K. P., \& Anderson, D. R. (2002). Model Selection and Multimodel Inference: A Practical Information-Theoretical Approach. 2d ed. New York: Springer-Verlag.
Burnham, K. P., \& Anderson, D. R. (2004). Multimodel inference: Understanding AIC and BIC in model selection. Sociological Methods and Research, 33(2), 261-304.
data(mtcars) mod1 <- lm(mpg ~ cyl, mtcars) aic(mod1)data(mtcars) mod1 <- lm(mpg ~ cyl, mtcars) aic(mod1)
Computes the Bayesian Information Criterion of a model (Schwarz, 1978).
bic(mod)bic(mod)
mod |
A fitted model of class |
Ladislas Nalborczyk <[email protected]>
Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461-464.
data(mtcars) mod1 <- lm(mpg ~ cyl, mtcars) bic(mod1)data(mtcars) mod1 <- lm(mpg ~ cyl, mtcars) bic(mod1)
The ESTER package implements sequential testing based on evidence ratios computed from the Akaike weights of a set of models. These weights are being computed using either the Akaike Information Criterion (AIC) or the Bayesian Information Criterion (BIC).
See vignette("ESTER") for a general introduction and overview.
Ladislas Nalborczyk
Maintainer: Ladislas Nalborczyk <[email protected]>
Returns a table with weights of a set of models, based on various
information criteria. Currently, ictab supports the computation of
Akaike weights from the aic or the bic computed on lm
or merMod models, as well as the computation of pseudo-BMA weights,
computed from the WAIC or LOOIC of brmsfit models.
ictab(mods, ic, ...)ictab(mods, ic, ...)
mods |
Should be a named list of models, of class |
ic |
Indicates which information criterion to use. Current supported
information criteria include |
... |
Additional parameters to be passed to the |
An object of class data.frame, which contains the value of the
information criterion (either AIC, BIC, WAIC or LOOIC), the number of parameters
(k for AIC and BIC or p for WAIC or LOOIC), the delta_IC (for AIC and BIC) or the
elpd for models compared with WAIC or LOOIC, and the weight of each
model (Akaike weights for AIC or BIC and pseudo-BMA weights for WAIC or LOOIC).
Ladislas Nalborczyk <[email protected]>
Burnham, K. P., \& Anderson, D. R. (2002). Model Selection and Multimodel Inference: A Practical Information-Theoretical Approach. 2d ed. New York: Springer-Verlag.
Burnham, K. P., \& Anderson, D. R. (2004). Multimodel inference: Understanding AIC and BIC in model selection. Sociological Methods and Research, 33(2), 261-304.
Yao, Y. P., Vehtari, A., Simpson, D., \& Gelman, A. (2017). Using stacking to average Bayesian predictive distributions.
data(mtcars) mod1 <- lm(mpg ~ cyl, mtcars) mod2 <- lm(mpg ~ cyl + vs, mtcars) mod3 <- lm(mpg ~ cyl * vs, mtcars) mods <- list(mod1 = mod1, mod2 = mod2, mod3 = mod3) ictab(mods, aic) ## Not run: library(brms) mod1 <- brm(mpg ~ cyl, mtcars) mod2 <- brm(mpg ~ cyl + vs, mtcars) mods <- list(m1 = mod1, m2 = mod2) ictab(mods, LOO, reloo = TRUE, k_threshold = 0.6, cores = 2) ## End(Not run)data(mtcars) mod1 <- lm(mpg ~ cyl, mtcars) mod2 <- lm(mpg ~ cyl + vs, mtcars) mod3 <- lm(mpg ~ cyl * vs, mtcars) mods <- list(mod1 = mod1, mod2 = mod2, mod3 = mod3) ictab(mods, aic) ## Not run: library(brms) mod1 <- brm(mpg ~ cyl, mtcars) mod2 <- brm(mpg ~ cyl + vs, mtcars) mods <- list(m1 = mod1, m2 = mod2) ictab(mods, LOO, reloo = TRUE, k_threshold = 0.6, cores = 2) ## End(Not run)
Computes sequential evidence ratios, either based on the AIC, BIC, WAIC, or LOOIC.
Supported models currently include lm, merMod, or brmsfit models.
When data involve repeated measures (and so multiple lines per subject),
a column indicating the subject "id" should be provided to the id argument.
If nothing is passed to the id argument, seqtest will suppose
that there is only one observation (i.e., one line) per subject.
seqtest(ic = aic, mod1, mod2, nmin = 10, id = NULL, boundary = Inf, blind = FALSE, nsims = NULL)seqtest(ic = aic, mod1, mod2, nmin = 10, id = NULL, boundary = Inf, blind = FALSE, nsims = NULL)
ic |
Indicates whether to use the aic or the bic. |
mod1 |
A model of class |
mod2 |
A model of class |
nmin |
Minimum sample size from which start to compute sequential evidence ratios. |
id |
If applicable (i.e., repeated measures), name of the "id" column of your dataframe, in character string. |
boundary |
The Evidence Ratio (or its reciprocal) at which the run is stopped as well |
blind |
If true, the function only returns a "continue or stop" message |
nsims |
Number of permutation samples to evaluate (is ignored if blind = TRUE) |
Ladislas Nalborczyk <[email protected]>
## Not run: # A first simple example data(mtcars) mod1 <- lm(mpg ~ cyl, mtcars) mod2 <- lm(mpg ~ cyl + disp, mtcars) seqtest(ic = aic, mod1, mod2, nmin = 10) # Plotting the results seqtest(ic = aic, mod1, mod2, nmin = 10) %>% plot # Example with 10 permutation samples seqtest(ic = aic, mod1, mod2, nmin = 10, nsims = 10) # Example with blinding seqtest(ic = aic, mod1, mod2, nmin = 10, boundary = 10, blind = TRUE) # Example with repeated measures library(lme4) data(sleepstudy) mod1 <- lmer(Reaction ~ Days + (1|Subject), sleepstudy) mod2 <- lmer(Reaction ~ Days + I(Days^2) + (1|Subject), sleepstudy) seqtest(ic = aic, mod1, mod2, nmin = 10, id = "Subject", nsims = 10) # Example with brmsfit models library(brms) mod1 <- brm(Reaction ~ Days + (1|Subject), sleepstudy) mod2 <- brm(Reaction ~ Days + I(Days^2) + (1|Subject), sleepstudy) seqtest(ic = WAIC, mod1, mod2, nmin = 10, id = "Subject") ## End(Not run)## Not run: # A first simple example data(mtcars) mod1 <- lm(mpg ~ cyl, mtcars) mod2 <- lm(mpg ~ cyl + disp, mtcars) seqtest(ic = aic, mod1, mod2, nmin = 10) # Plotting the results seqtest(ic = aic, mod1, mod2, nmin = 10) %>% plot # Example with 10 permutation samples seqtest(ic = aic, mod1, mod2, nmin = 10, nsims = 10) # Example with blinding seqtest(ic = aic, mod1, mod2, nmin = 10, boundary = 10, blind = TRUE) # Example with repeated measures library(lme4) data(sleepstudy) mod1 <- lmer(Reaction ~ Days + (1|Subject), sleepstudy) mod2 <- lmer(Reaction ~ Days + I(Days^2) + (1|Subject), sleepstudy) seqtest(ic = aic, mod1, mod2, nmin = 10, id = "Subject", nsims = 10) # Example with brmsfit models library(brms) mod1 <- brm(Reaction ~ Days + (1|Subject), sleepstudy) mod2 <- brm(Reaction ~ Days + I(Days^2) + (1|Subject), sleepstudy) seqtest(ic = WAIC, mod1, mod2, nmin = 10, id = "Subject") ## End(Not run)