The Full Bayesian Evidence Test, Full Bayesian Significance Test and the e-Value

Description

Provides access to a range of functions for computing and visualizing the Full Bayesian Significance Test (FBST) and the e-value for testing a sharp hypothesis against its alternative, and the Full Bayesian Evidence Test (FBET) and the (generalized) Bayesian evidence value for testing a composite (or interval) hypothesis against its alternative. The methods are widely applicable as long as a posterior MCMC sample is available.

Details

Package for conducting the Full Bayesian Evidence Test (FBET) and the Full Bayesian Significance Test (FBST). The FBST is a Bayesian hypothesis test for testing a sharp hypothesis against its alternative by calculating the e-value, the Bayesian evidence against the null hypothesis. The FBET is a generalization of the underlying Pereira-Stern theory of the FBST and allows for testing interval hypotheses. It provides the Bayesian evidence value, or generalized e-value, which includes the e-value of the FBST as a special case. The Bayesian evidence value is based on the relative surprise function to a reference function. In the FBST, the tangential set corresponding to a sharp null hypothesis serves for quantifying the Bayesian evidence. In the FBET, the Bayesian evidence interval serves for quantifying the Bayesian evidence, which has a strong analogy to the Bayes factor. Next to the core functions, helper functions and visualizations of the results of a FBST and FBET are provided in the package.

Index of help topics:

$,fbet-method           Returns an object from an object of class
                        'fbet'.
$,fbst-method           Returns an object from an object of class
                        'fbst'.
bdm                     bdm
fbet                    fbet
fbet-class              Class '"fbet-class"'
fbst                    fbst
fbst-class              Class '"fbst-class"'
fbst-package            The Full Bayesian Evidence Test, Full Bayesian
                        Significance Test and the e-Value
names.fbet              names.fbet
names.fbst              names.fbst
plot.fbet               plot.fbet
plot.fbst               plot.fbst
show.fbet               show.fbet
show.fbst               show.fbst
summary.fbet            summary.fbet
summary.fbst            summary.fbst

Author(s)

Riko Kelter

Maintainer: Riko Kelter <riko_k@gmx.de>

References

For a details, see: https://arxiv.org/abs/2001.10577 and https://arxiv.org/pdf/2001.10577.pdf.

Returns an object from an object of class fbst.

Description

Returns an object from an object of class fbst

Details

-

Value

-

Author(s)

Riko Kelter

References

For a details, see: https://arxiv.org/abs/2001.10577 and https://arxiv.org/pdf/2001.10577.pdf.

Examples

set.seed(57)
grp1=rnorm(50,0,1.5)
grp2=rnorm(50,0.8,3.2)

p = as.vector(BayesFactor::ttestBF(x=grp1,y=grp2, 
  posterior = TRUE, iterations = 3000, 
  rscale = "medium")[,4])

# flat reference function
res = fbst(posteriorDensityDraws = p, nullHypothesisValue = 0, 
dimensionTheta = 3, dimensionNullset = 2)

# Return the e-value from an fbst object
res$eValue

Returns an object from an object of class fbet.

Description

Returns an object from an object of class fbet

Details

-

Value

-

Author(s)

Riko Kelter

References

For a details, see: https://arxiv.org/abs/2001.10577 and https://arxiv.org/pdf/2001.10577.pdf.

Examples

set.seed(57)
grp1=rnorm(50,0,1.5)
grp2=rnorm(50,0.8,3.2)

p = as.vector(BayesFactor::ttestBF(x=grp1,y=grp2, 
  posterior = TRUE, iterations = 3000, 
  rscale = "medium")[,4])

# flat reference function
res = fbet(p, interval = c(-0.1,0.1), nu=1, FUN=NULL, par=NULL)

# Return the Bayesian evidence value for the interval null hypothesis
res$eValueH0

bdm

Description

Calculates the Bayesian discrepancy measure for a precise null hypothesis.

Usage

bdm(posteriorDensityDraws, nullHypothesisValue=0)

Arguments

Details

The BDM is calculated as \delta_H(x):=2\cdot P(\theta \in I_H(x)|x) where I_H(x):=(m,\theta_0) if m<\theta_0, I_H(x):=\{m\} if m=\theta_0 and I_H(x):=(\theta_0,m) if m>\theta_0, where m denotes the posterior median of the parameter \theta, and the null hypothesis specifies H_0:\theta=\theta_0.

Value

Returns the value \delta_H(x) of the BDM.

Author(s)

Riko Kelter

References

For details, see: https://arxiv.org/abs/2105.13716

Examples

set.seed(57)
grp1=rnorm(50,0,1.5)
grp2=rnorm(50,0.8,3.2)

p = as.vector(BayesFactor::ttestBF(x=grp1,y=grp2, 
  posterior = TRUE, iterations = 3000, 
  rscale = "medium")[,4])

bdm(p,0)

fbet

Description

The function computes the Full Bayesian Evidence Test (FBST) and the Bayesian evidence value (the generalized e-value which obtains the e-value of the FBST as a special case), which is the Bayesian evidence against an interval null hypothesis. The function assumes posterior MCMC draws and constructs a posterior density based on a kernel density estimator subsequently. The Bayesian evidence interval is computed using a linear search based on the evidence-threshold and the calculation of the Bayesian evidence value is performed using numerical integration.

Usage

fbet(posteriorDensityDraws=NULL, interval, nu=1, FUN=NULL, 
par=NULL, posterior=NULL, par_posterior=NULL)

Arguments

Details

If no reference function is specified, a flat reference function r(\theta)=1 is used as default reference function when posteriorDensityDraws are provided.

Value

Returns an object of class fbet if posteriorDensityDraws are provided. When using the posterior argument to pass the posterior as a function, it provides the evidence value for the hypothesis specified in the interval argument.

Author(s)

Riko Kelter

References

For a details, see: https://arxiv.org/abs/2001.10577.

Examples

set.seed(57)
grp1=rnorm(50,0,1.5)
grp2=rnorm(50,0.3,3.2)

p = as.vector(BayesFactor::ttestBF(x=grp1,y=grp2, 
  posterior = TRUE, iterations = 3000, 
  rscale = "medium")[,4])

# flat reference function, nu = 0
res = fbet(p, interval = c(-0.1,0.1), nu=0, FUN=NULL, par=NULL)
summary(res)
plot(res)

# flat reference function, nu = 1
res = fbet(p, interval = c(-0.1,0.1), nu=1, FUN=NULL, par=NULL)
summary(res)
plot(res)

# medium Cauchy C(0,1) reference function, nu = 1
res_med = fbet(posteriorDensityDraws = p, interval = c(-0.1,0.1), nu = 1, 
FUN = dcauchy, par = list(location = 0, scale=sqrt(2)))
summary(res_med)
plot(res_med)

# posterior as function argument
fbet(posterior=dbeta, par_posterior = list(shape1 = 3, shape2 = 4), 
interval = c(0.2,1), nu = 1, FUN=dbeta, par = list(shape1 = 1, shape2 = 1))

Class "fbet-class"

Description

Class for modelling the results of a Full Bayesian Evidence Test

Objects from the Class

Store the results of a FBET

Slots

data:

Object of class "list" holding the results of the Full Bayesian Evidence Test. posteriorDensityDraws holds the posterior MCMC parameter draws, posteriorDensityDrawsSorted stores the sorted posterior MCMC parameter draws, postDensValues stores the posterior density values, indices stores the indices for deciding which values pass the evidence-threshold \nu, interval stores the boundaries of the interval null hypothesis, referenceFunction stores the name of the reference function used, nu specifies the evidence-threshold used for computation of the Bayesian evidence interval, evidenceInterval holds the endpoints of the resulting Bayesian evidence interval, eValueH0 holds the Bayesian evidence value in favour of the interval null hypothesis, eValueH1 holds the Bayesian evidence value in favour of the alternative hypothesis (or equivalently, against the interval null hypothesis)

fbst

Description

The function computes the Full Bayesian Significance Test (FBST) and the e-value, which is the Bayesian evidence against a precise null hypothesis. The function assumes posterior MCMC draws and constructs a posterior density based on a kernel density estimator subsequently.

Usage

fbst(posteriorDensityDraws, nullHypothesisValue, FUN, par, 
dimensionTheta, dimensionNullset, dim, gridSize)

Arguments

Details

If no reference function is specified, a flat reference function r(\theta)=1 is used as default reference function. Note that the posterior dimension dim defaults to 1, and if dim=2, only flat reference functions are supported. Thus, specifying FUN or par has no effect when dim=2.

Value

Returns an object of class fbst.

Author(s)

Riko Kelter

References

For a details, see: https://link.springer.com/article/10.3758/s13428-021-01613-6.

Examples

set.seed(57)
grp1=rnorm(50,0,1.5)
grp2=rnorm(50,0.8,3.2)

p = as.vector(BayesFactor::ttestBF(x=grp1,y=grp2, 
  posterior = TRUE, iterations = 3000, 
  rscale = "medium")[,4])

# flat reference function
res = fbst(posteriorDensityDraws = p, nullHypothesisValue = 0, 
dimensionTheta = 2, dimensionNullset = 1)
summary(res)
plot(res)

# medium Cauchy C(0,1) reference function
res_med = fbst(posteriorDensityDraws = p, nullHypothesisValue = 0, dimensionTheta = 2, 
dimensionNullset = 1, FUN = dcauchy, par = list(location = 0, scale = sqrt(2)/2))
summary(res_med)
plot(res_med)

Class "fbst-class"

Description

Class for modelling the results of a Full Bayesian Significance Test

Objects from the Class

Store the results of a FBST

Slots

data:

Object of class "list" holding the results of the Full Bayesian Significance Test. posteriorDensityDraws holds the posterior MCMC parameter draws, postEffSizeSorted stores the sorted posterior MCMC parameter draws, densZero stores the surprise function value at the sharp null hypothesis parameter value, postDensValues stores the posterior density values, indices stores the indices for deciding which values are located inside the tangential set, nullHypothesisValue stores the sharp null hypothesis parameter value, referenceFunction holds the name of the reference function used, dimensionTheta holds the dimension of the parameter space, dimensionNullset holds the dimension of the null set corresponding to the null hypothesis, eValue holds the Bayesian evidence against the sharp null hypothesis, the e-value, pValue holds the p-value associated with the Bayesian e-value in favour of the sharp null hypothesis, sev_H_0 holds the standardized e-value as a replacement of the frequentist p-value.

names.fbet

Description

Plots the names of the objects stored in the data object of a Full Bayesian Evidence Test.

Usage

## S3 method for class 'fbet'
names(x)

Arguments

Details

Plots the names of the objects stored in the data object of a Full Bayesian Evidence Test.

Value

Returns a list of names.

Author(s)

Riko Kelter

References

For a details, see: https://arxiv.org/abs/2001.10577 and https://arxiv.org/pdf/2001.10577.pdf.

Examples

set.seed(57)
grp1=rnorm(50,0,1.5)
grp2=rnorm(50,0.8,3.2)

p = as.vector(BayesFactor::ttestBF(x=grp1,y=grp2, 
  posterior = TRUE, iterations = 3000, 
  rscale = "medium")[,4])

# flat reference function
res = fbet(p, interval = c(-0.1,0.1), nu=1, FUN=NULL, par=NULL)
names(res)

names.fbst

Description

Plots the names of the objects stored in the data object of a Full Bayesian Significance Test.

Usage

## S3 method for class 'fbst'
names(x)

Arguments

Details

Plots the names of the objects stored in the data object of a Full Bayesian Significance Test.

Value

Returns a list of names.

Author(s)

Riko Kelter

References

For a details, see: https://arxiv.org/abs/2001.10577 and https://arxiv.org/pdf/2001.10577.pdf.

Examples

set.seed(57)
grp1=rnorm(50,0,1.5)
grp2=rnorm(50,0.8,3.2)

p = as.vector(BayesFactor::ttestBF(x=grp1,y=grp2, 
  posterior = TRUE, iterations = 3000, 
  rscale = "medium")[,4])

# flat reference function
res = fbst(posteriorDensityDraws = p, nullHypothesisValue = 0, 
dimensionTheta = 2, dimensionNullset = 1)
names(res)

plot.fbet

Description

Plots the results of a Full Bayesian Evidence Test.

Usage

## S3 method for class 'fbet'
plot(x, ..., leftBoundary = -100, rightBoundary = 100, type = "posterior", 
legendposition = "topleft", main = "")

Arguments

Details

Plots the resulting surprise function, the interval null hypothesis (dotted blue lines), the resulting Bayesian evidence interval (solid blue lines), the evidence-threshold \nu (dotted black line) and the resulting Bayesian evidence values. The Bayesian evidence value in favour of the interval null hypothesis is visualized as the blue area, and the Bayesian evidence value in favour of the alternative hypothesis is visualized as the red area.

Value

Returns a plot.

Author(s)

Riko Kelter

References

For a details, see: https://arxiv.org/abs/2001.10577 and https://arxiv.org/pdf/2001.10577.pdf.

Examples

set.seed(57)
grp1=rnorm(50,0,1.5)
grp2=rnorm(50,0.3,3.2)

p = as.vector(BayesFactor::ttestBF(x=grp1,y=grp2, 
  posterior = TRUE, iterations = 3000, 
  rscale = "medium")[,4])

# flat reference function
res = fbet(p, interval = c(-0.1,0.1), nu=1, FUN=NULL, par=NULL)
summary(res)
plot(res)

plot.fbst

Description

Plots the results of a Full Bayesian Significance Test.

Usage

## S3 method for class 'fbst'
plot(x, ..., leftBoundary = -100, rightBoundary = 100, type = "contour", parNames = NULL, 
xlimleft = NULL, xlimright = NULL, xlabString = "Parameter", ylabString = NULL)

Arguments

Details

Plots the surprise function, the supremum of the surprise function restricted to the null set (blue point) and visualises the Bayesian e-value against the sharp null hypothesis as the blue shaded area under the surprise function. The Bayesian e-value in favour of the sharp null hypothesis is visualised as the red shaded area under the surprise function.

Value

Returns a plot.

Author(s)

Riko Kelter

References

For a details, see: https://arxiv.org/abs/2001.10577 and https://arxiv.org/pdf/2001.10577.pdf.

Examples

set.seed(57)
grp1=rnorm(50,0,1.5)
grp2=rnorm(50,0.8,3.2)

p = as.vector(BayesFactor::ttestBF(x=grp1,y=grp2, 
  posterior = TRUE, iterations = 3000, 
  rscale = "medium")[,4])

# flat reference function
res = fbst(posteriorDensityDraws = p, nullHypothesisValue = 0, 
dimensionTheta = 2, dimensionNullset = 1)
plot(res)
plot(res, xlimleft = -1.5, xlimright = 0.5)

show.fbet

Description

Prints the main results of a Full Bayesian Evidence Test to the console.

Usage

## S3 method for class 'fbet'
show(object)

Arguments

Details

Shows the main results of a Full Bayesian Evidence Test stored in an object of class fbet.

Value

Prints the results onto the console.

Author(s)

Riko Kelter

References

For a details, see: https://arxiv.org/abs/2001.10577 and https://arxiv.org/pdf/2001.10577.pdf.

Examples

set.seed(57)
grp1=rnorm(50,0,1.5)
grp2=rnorm(50,0.8,3.2)

p = as.vector(BayesFactor::ttestBF(x=grp1,y=grp2, 
  posterior = TRUE, iterations = 3000, 
  rscale = "medium")[,4])

# flat reference function
res = fbet(p, interval = c(-0.1,0.1), nu=1, FUN=NULL, par=NULL)
show(res)

show.fbst

Description

Prints the main results of a Full Bayesian Significance Test to the console.

Usage

## S3 method for class 'fbst'
show(object)

Arguments

Details

Shows the main results of a Full Bayesian Significance Test stored in an object of class fbst.

Value

Prints the results onto the console.

Author(s)

Riko Kelter

References

For a details, see: https://arxiv.org/abs/2001.10577 and https://arxiv.org/pdf/2001.10577.pdf.

Examples

set.seed(57)
grp1=rnorm(50,0,1.5)
grp2=rnorm(50,0.8,3.2)

p = as.vector(BayesFactor::ttestBF(x=grp1,y=grp2, 
  posterior = TRUE, iterations = 3000, 
  rscale = "medium")[,4])

# flat reference function
res = fbst(posteriorDensityDraws = p, nullHypothesisValue = 0, 
dimensionTheta = 2, dimensionNullset = 1)
show(res)

summary.fbet

Description

Prints the results of a Full Bayesian Evidence Test.

Usage

## S3 method for class 'fbet'
summary(object, ...)

Arguments

Details

Summarises the results of a Full Bayesian Evidence Test.

Value

Prints the results onto the console.

Author(s)

Riko Kelter

References

For a details, see: https://arxiv.org/abs/2001.10577 and https://arxiv.org/pdf/2001.10577.pdf.

Examples

set.seed(57)
grp1=rnorm(50,0,1.5)
grp2=rnorm(50,0.3,3.2)

p = as.vector(BayesFactor::ttestBF(x=grp1,y=grp2, 
  posterior = TRUE, iterations = 3000, 
  rscale = "medium")[,4])

# flat reference function
res = fbet(p, interval = c(-0.1,0.1), nu=1, FUN=NULL, par=NULL)
summary(res)

summary.fbst

Description

Prints the results of a Full Bayesian Significance Test.

Usage

## S3 method for class 'fbst'
summary(object, ...)

Arguments

Details

Summarises the results of a Full Bayesian Significance Test.

Value

Prints the results onto the console.

Author(s)

Riko Kelter

References

For a details, see: https://arxiv.org/abs/2001.10577 and https://arxiv.org/pdf/2001.10577.pdf.

Examples

set.seed(57)
grp1=rnorm(50,0,1.5)
grp2=rnorm(50,0.8,3.2)

p = as.vector(BayesFactor::ttestBF(x=grp1,y=grp2, 
  posterior = TRUE, iterations = 3000, 
  rscale = "medium")[,4])

# flat reference function
res = fbst(posteriorDensityDraws = p, nullHypothesisValue = 0, 
dimensionTheta = 2, dimensionNullset = 1)
summary(res)