Help for package afc

Version:

1.4.0

Date:

2017-05-18

Title:

Generalized Discrimination Score

Description:

This is an implementation of the Generalized Discrimination Score (also known as Two Alternatives Forced Choice Score, 2AFC) for various representations of forecasts and verifying observations. The Generalized Discrimination Score is a generic forecast verification framework which can be applied to any of the following verification contexts: dichotomous, polychotomous (ordinal and nominal), continuous, probabilistic, and ensemble. A comprehensive description of the Generalized Discrimination Score, including all equations used in this package, is provided by Mason and Weigel (2009) <doi:10.1175/MWR-D-10-05069.1>.

License:

GPL-3

RoxygenNote:

6.0.1

NeedsCompilation:

Packaged:

2017-05-18 15:05:00 UTC; bhj

Author:

Andreas Weigel [aut], MeteoSwiss [cph], Jonas Bhend [cre, ctb]

Maintainer:

Jonas Bhend <jonas.bhend@meteoswiss.ch>

Repository:

CRAN

Date/Publication:

2017-05-18 15:23:57 UTC

Generalized Discrimination Score 2AFC

Description

This package is a collection of routines to calculate the "generalized discrimination score", which is also known as "two alternatives forced-choice score" or short: "2AFC-score". The 2AFC is a generic forecast verification framework which can be applied to any of the following verification contexts: dichotomous, polychotomous (ordinal and nominal), continuous, probabilistic, and ensemble. A comprehensive description of the 2AFC-score, including all equations used in this package, is provided by Mason and Weigel (2009).

Details

The master routine is afc. For a given set of observation and forecast data, and for a specified verification context, afc calls the appropriate functions which are necessary to calculate the 2AFC score.

Why the 2AFC-score? There are numerous reasons for calculating forecast verification scores, and considerable attention has been given to designing and analyzing the properties of scores that can be used for scientific purposes. Much less attention has been given to scores that may be useful for administrative reasons, such as communicating changes in forecast quality to bureaucrats, and providing indications of forecast quality to the general public. The 2AFC test a scoring procedure that is sufficiently generic to be useable on forecasts ranging from simply "yes"/"no" forecasts of dichotomous outcomes to continuous variables, and can be used with deterministic or probabilistic forecasts without seriously reducing the more complex information when available. Although, as with any single verification score, the 2AFC has limitations, it does have broad intuitive appeal in that the expected score of an unskilled set of forecasts (random guessing or perpetually identical forecasts) is 50%, and is interpretable as an indication of how often the forecasts are correct, even when the forecasts are expressed probabilistically and/or the observations are not discrete.

Package:	afc
Version:	1.03
Date:	2010-01-07
License:	GPL-2

Index

afc                     Calculate Generalized Discrimination Score 2AFC
afc-package             Generalized Discrimination Score 2AFC
afc.cc                  2AFC For Continuous Observations And Continuous 
                        Forecasts
afc.ce                  2AFC For Ordinal Polychotomous Observations And
                        Ensemble Forecasts
afc.dc                  2AFC For Dichotomous Observations And
                        Continuous Forecasts
afc.dd                  2AFC For Dichotomous Observations And
                        Dichotomous Forecasts
afc.de                  2AFC for Dichotomous Observations and Ensemble
                        Forecasts
afc.dm                  2AFC For Dichotomous Observations And
                        Polychotomous Forecasts
afc.dp                  2AFC For Dichotomous Observations And
                        Probabilistic Forecasts
afc.mc                  2AFC For Ordinal Polychotomous Observations And
                        Continuous Forecasts
afc.me                  2AFC For Ordinal Polychotomous Observations And
                        Ensemble Forecasts
afc.mm                  2AFC For Ordinal Polychotomous Observations And
                        Ordinal Polychotomous Forecasts
afc.mp                  2AFC For Ordinal Polychotomous Observations And
                        Probabilistic Forecasts
afc.nn                  2AFC For Nominal Polychotomous Observations And
                        Nominal Polychotomous Forecasts
afc.np                  2AFC For Nominal Polychotomous Observations And
                        Probabilistic Forecasts
cnrm.nino34.cc          Example Data of Continuous Observations and
                        Continuous Forecasts
cnrm.nino34.ce          Example Data of Continuous Observations and 
                        Ensemble Forecasts
cnrm.nino34.dc          Example Data of Dichotomous Observations and
                        Continuous Forecasts
cnrm.nino34.dd          Example Data of Dichotomous Observations and
                        Dichotomous Forecasts
cnrm.nino34.de          Example Data of Dichotomous Observations and
                        Ensemble Forecasts
cnrm.nino34.dm          Example Data of Dichotomous Observations and
                        Polychotomous Forecasts
cnrm.nino34.dp          Example Data of Dichotomous Observations and
                        Polychotomous Forecasts
cnrm.nino34.mc          Example Data of Polychotomous Observations and
                        Continuous Forecasts
cnrm.nino34.me          Example Data of Polychotomous Observations and
                        Ensembles Forecasts
cnrm.nino34.mm          Example Data of Polychotomous Observations and
                        Polychotomous Forecasts
cnrm.nino34.mp          Example Data of Polychotomous Observations and
                        Probabilistic Forecasts
rank.ensembles          Rank Ensembles

Author(s)

Andreas Weigel, Federal Office of Meteorology and Climatology (MeteoSwiss), Zurich, Switzerland <andreas.weigel@meteoswiss.ch>

References

Mason, S.J. and A.P. Weigel, 2009: A generic forecast verification framework for administrative purposes. Mon. Wea. Rev., 137, 331-349

Calculate Generalized Discrimination Score 2AFC

Description

This is the master routine for the calculation the Generalized Discrimination Score (aka Two-Alternatives Forced Choice Score - 2AFC) as described in the Paper of Mason and Weigel (2009). This routine requires, as input, datasets of forecasts and corresponding observations, as well as a specification of the verification context. The routine checks whether the input data are consistent with the verification context, then calls the appropriate function to calculate the 2AFC, and finally returns the 2AFC skill value.

Usage

afc(obsv, fcst, obsv.type, fcst.type, m = 0, m2 = 0)

Arguments

obsv

Vector of observations. The required format depends on the specific verification context. More details below.

fcst

Vector or two-dimensional array of forecasts. The required format depends on the specific verification context. More details below.

obsv.type

Character specifying the type of the observations. Possible values: "d" (dichotomous), "m" (polychotomous with ordinal categories), "n" (polychotomous with nominal categories), "c" (continuous).

fcst.type

Character specifying the type of the forecasts. Possible values: "d" (dichotomous), "m" (polychotomous with ordinal categories), "n" (polychotomous with nominal categories), "p" (probabilistic), "c" (continuous) and "e" (ensemble).

m

Number of observation or forecast categories (only required if obsv.type or fcst.type equals "m" or "n")

m2

Number of forecast categories (only required if both obsv.type and fcst.type equal "m"). The number of observation categories is then specified by the argument m above.

Details

Depending on the specific verification context (i.e. the choice for obsv.type and fcst.type), this routine calls the appropriate function(s) to calculate the 2AFC score. The following combinations of obsv.type and fcst.type are possible: (1) "d-d"; (2) "d-m"; (3) "d-p"; (4) "d-c"; (5) "d-e"; (6) "m-m"; (7) "m-p"; (8) "m-c"; (9) "m-e"; (10) "n-n"; (11) "n-p"; (12) "c-c"; (13) "c-e". The required format of the input data obsv and fcst depends on the verification context:

(1) "d-d":
obsv: vector with dichotomous observations (values in 0,1)
fcst: vector of same length as obsv with dichotomous forecasts (values in 0,1)

(2) "d-m":
obsv: vector with dichotomous observations (values in 0,1)
fcst: vector of same length as obsv with polychotomous forecasts (values in 1,..,m)

(3) "d-p":
obsv: vector with dichotomous observations (values in 0,1)
fcst: vector of same length as obsv with forecast probabilities for the event to happen

(4) "d-c":
obsv: vector with dichotomous observations (values in 0,1)
fcst: vector of same length as obsv with real-valued forecasts

(5) "d-e":
obsv: vector with dichotomous observations (values in 0,1)
fcst: two-dimensional array with ensemble forecasts; dim(fcst)[1] = length(obsv); dim(fcst)[2] = ensemble size.

(6) "m-m":
obsv: vector with polychotomous observations (values in 1,..,m)
fcst: vector of same length as obsv with polychotomous forecasts (values in 1,..,m2)

(7) "m-p":
obsv: vector with polychotomous observations (values in 1,..,m)
fcst: two-dimensional array with forecast probabilities for the m categories; dim(fcst)[1] = length(obsv); dim(fcst)[2] = m

(8) "m-c":
obsv: vector with polychotomous observations (values in 1,..,m)
fcst: vector of same length as obsv with real-valued forecasts

(9) "m-e":
obsv: vector with polychotomous observations (values in 1,..,m)
fcst: two-dimensional array with ensemble forecasts; dim(fcst)[1] = length(obsv); dim(fcst)[2] = ensemble size.

(10) "n-n":
obsv: vector with polychotomous observations (values in 1,..,m)
fcst: vector of same length as obsv with polychotomous forecasts (values in 1,..,m)

(11) "n-p":
Same as "m-p".

(12) "c-c":
obsv: vector with real-valued observations
fcst: vector of same length as obsv with real-valued forecasts

(13) "c-e":
obsv: vector with real-valued observations
fcst: two-dimensional array with ensemble forecasts; dim(fcst)[1] = length(obsv); dim(fcst)[2] = ensemble size.

Value

p.afc

Value of Generalized Discrimination Score (2AFC)

Author(s)

Andreas Weigel, Federal Office of Meteorology and Climatology, MeteoSwiss, Zurich, Switzerland

References

S.J. Mason and A.P. Weigel, 2009. A generic verification framework for administrative purposes. Mon. Wea. Rev., 137, 331-349

Examples


  #
  #In all following examples, forecasts of the Nino-3.4 index are evaluated
  #
  #----------------------
  #Example 1: Dichotomous observations, dichotomous forecasts
  # ---------------------
  #Load set of dichotomous observations and dichotomous forecasts
  data(cnrm.nino34.dd)
  obsv = cnrm.nino34.dd$obsv
  fcst = cnrm.nino34.dd$fcst
  #Calculate skill score
  afc(obsv, fcst, obsv.type="d", fcst.type="d")

  # ---------------------
  #Example 2: Dichotomous observations, (ordinal) polychotomous forecasts
  # ---------------------
  #Load set of dichotomous observations and polychotomous forecasts (4 categories)
  data(cnrm.nino34.dm)
  obsv = cnrm.nino34.dm$obsv
  fcst = cnrm.nino34.dm$fcst
  #Calculate skill score
  afc(obsv, fcst, obsv.type="d", fcst.type="m", m=4)

  # ---------------------
  #Example 3: Dichotomous observations, probabilistic forecasts
  # ---------------------
  #Load set of dichotomous observations and probabilistic forecasts
  data(cnrm.nino34.dp)
  obsv = cnrm.nino34.dp$obsv
  fcst = cnrm.nino34.dp$fcst
  #Calculate skill score
  afc(obsv, fcst, obsv.type="d", fcst.type="p")

  # ---------------------
  #Example 4: Dichotomous observations, continuous forecasts
  # ---------------------
  #Load set of dichotomous observations and continuous forecasts
  data(cnrm.nino34.dc)
  obsv = cnrm.nino34.dc$obsv
  fcst = cnrm.nino34.dc$fcst
  #Calculate skill score
  afc(obsv, fcst, obsv.type="d", fcst.type="c")

  # ---------------------
  #Example 5: Dichotomous observations, ensemble forecasts
  # ---------------------
  #Load set of dichotomous observations and 9-member ensemble forecasts
  data(cnrm.nino34.de)
  obsv = cnrm.nino34.de$obsv
  fcst = cnrm.nino34.de$fcst
  #Calculate skill score
  afc(obsv, fcst, obsv.type="d", fcst.type="e")

  # ---------------------
  #Example 6: Polychotomous (ordinal) observations, polychotomous (ordinal) forecasts
  # ---------------------
  #Load set of polychotomous observations (4 categories) and polychotomous forecasts (4 categories)
  data(cnrm.nino34.mm)
  obsv = cnrm.nino34.mm$obsv
  fcst = cnrm.nino34.mm$fcst
  #Calculate skill score
  afc(obsv, fcst, obsv.type="m", fcst.type="m", m=4, m2=4)

  # ---------------------
  #Example 7: Polychotomous (ordinal) observations, probabilistic forecasts forecasts
  # ---------------------
  #Load set of polychotomous observations (4 categories) and probabilistic forecasts
  data(cnrm.nino34.mp)
  obsv = cnrm.nino34.mp$obsv
  fcst = cnrm.nino34.mp$fcst
  #Calculate skill score
  afc(obsv, fcst, obsv.type="m", fcst.type="p", m=4)

  # ---------------------
  #Example 8: Polychotomous (ordinal) observations, continuous forecasts
  # ---------------------
  #Load set of polychotomous observations (4 categories) and continuous forecasts
  data(cnrm.nino34.mc)
  obsv = cnrm.nino34.mc$obsv
  fcst = cnrm.nino34.mc$fcst
  #Calculate skill score
  afc(obsv, fcst, obsv.type="m", fcst.type="c", m=4)

  # ---------------------
  #Example 9: Polychotomous (ordinal) observations, ensemble forecasts
  # ---------------------
  #Load set of polychotomous observations (4 categories) and 9-member ensemble forecasts
  data(cnrm.nino34.me)
  obsv = cnrm.nino34.me$obsv
  fcst = cnrm.nino34.me$fcst
  #Calculate skill score
  afc(obsv, fcst, obsv.type="m", fcst.type="e", m=4)

  # ---------------------
  #Example 10: Polychotomous (nominal) observations, polychotomous (nominal) forecasts
  # ---------------------
  #Load set of polychotomous observations (4 categories) and polychotomous forecasts (4 categories)
  data(cnrm.nino34.mm)
  obsv = cnrm.nino34.mm$obsv
  fcst = cnrm.nino34.mm$fcst
  #Calculate skill score
  afc(obsv, fcst, obsv.type="n", fcst.type="n", m=4)

  # ---------------------
  #Example 11: Polychotomous (nominal) observations, probabilistic forecasts
  # ---------------------
  #Load set of polychotomous observations (4 categories) and probabilistic forecasts
  data(cnrm.nino34.mp)
  obsv = cnrm.nino34.mp$obsv
  fcst = cnrm.nino34.mp$fcst
  #Calculate skill score
  afc(obsv, fcst, obsv.type="n", fcst.type="p", m=4)

  # ---------------------
  #Example 12: Continuous observations, continuous forecasts
  # ---------------------
  #Load set of continuous observations and continuous forecasts
  data(cnrm.nino34.cc)
  obsv = cnrm.nino34.cc$obsv
  fcst = cnrm.nino34.cc$fcst
  #Calculate skill score
  afc(obsv, fcst, obsv.type="c", fcst.type="c")

  # ---------------------
  #Example 13: Continuous observations, ensemble forecasts
  # ---------------------
  #Load set of continuous observations and 9-member ensemble forecasts
  data(cnrm.nino34.ce)
  obsv = cnrm.nino34.ce$obsv
  fcst = cnrm.nino34.ce$fcst
  #Calculate skill score
  afc(obsv, fcst, obsv.type="c", fcst.type="e")

2AFC For Continuous Observations And Continuous Forecasts

Description

Routine to calculate the Generalized Discrimination Score (aka Two-Alternatives Forced Choice Score 2AFC) for the situation of continuous observations and continuous forecasts

Usage

afc.cc(obsv, fcst)

Arguments

obsv

vector with real-valued observations

fcst

vector of same length as obsv with real-valued forecasts

Details

This routine applies Eq.22 of Mason and Weigel (2009) to calculate the 2AFC.

Value

p.afc

Value of Generalized Discrimination (2AFC) Score

Author(s)

Andreas Weigel, Federal Office of Meteorology and Climatology, MeteoSwiss, Zurich, Switzerland

References

S.J. Mason and A.P. Weigel, 2009. A generic verification framework for administrative purposes. Mon. Wea. Rev., 137, 331-349

Examples


  #Forecasts and observations of Nino-3.4 index
  #Load set of continuous observations and continuous forecasts
  data(cnrm.nino34.cc)
  obsv = cnrm.nino34.cc$obsv
  fcst = cnrm.nino34.cc$fcst

  #Calculate skill score
  afc.cc(obsv,fcst)

2AFC For Continuous Observations And Ensemble Forecasts

Description

Routine to calculate the Generalized Discrimination Score (aka Two-Alternatives Forced Choice Score 2AFC) for the situation of continuous observations and ensemble forecasts

Usage

afc.ce(obsv, fcst)

Arguments

obsv

vector with real-valued observations

fcst

two-dimensional array with ensemble forecasts; dim(fcst)[1] = length(obsv); dim(fcst)[2] = ensemble size

Details

This routine first ranks the ensemble forecasts (see rank.ensembles) and then calculates the 2AFC-score with Eq.22 of Mason and Weigel (2009).

Value

p.afc

Value of Generalized Discrimination (2AFC) Score

Author(s)

Andreas Weigel, Federal Office of Meteorology and Climatology, MeteoSwiss, Zurich, Switzerland

References

S.J. Mason and A.P. Weigel, 2009. A generic verification framework for administrative purposes. Mon. Wea. Rev., 137, 331-349

Examples


  #Forecasts and observations of Nino-3.4 index
  #Load set of continuous observations and 9-member ensemble forecasts
  data(cnrm.nino34.ce)
  obsv = cnrm.nino34.ce$obsv
  fcst = cnrm.nino34.ce$fcst

  #Calculate skill score
  afc.ce(obsv,fcst)

2AFC For Dichotomous Observations And Continuous Forecasts

Description

Routine to calculate the Generalized Discrimination Score (aka Two-Alternatives Forced Choice Score 2AFC) for the situation of dichotomous observations and continuous forecasts

Usage

afc.dc(obsv, fcst)

Arguments

obsv

vector with dichotomous observations (values in 0,1)

fcst

vector of same length as obsv with real-valued forecasts

Details

This routine applies Eq.8 of Mason and Weigel (2009) to calculate the 2AFC.

Value

p.afc

Value of Generalized Discrimination (2AFC) Score

Author(s)

Andreas Weigel, Federal Office of Meteorology and Climatology, MeteoSwiss, Zurich, Switzerland

References

S.J. Mason and A.P. Weigel, 2009. A generic verification framework for administrative purposes. Mon. Wea. Rev., 137, 331-349

Examples


  #Forecasts and observations of Nino-3.4 index
  #Load set of dichotomous observations and continuous forecasts
  data(cnrm.nino34.dc)
  obsv = cnrm.nino34.dc$obsv
  fcst = cnrm.nino34.dc$fcst

  #Calculate skill score
  afc.dc(obsv,fcst)

2AFC For Dichotomous Observations And Dichotomous Forecasts

Description

Routine to calculate the Generalized Discrimination Score (aka Two-Alternatives Forced Choice Score 2AFC) for the situation of dichotomous observations and dichotomous forecasts

Usage

afc.dd(obsv, fcst)

Arguments

obsv

vector with dichotomous observations (values in 0,1)

fcst

vector of same length as obsv with dichotomous forecasts (values in 0,1)

Details

This routine applies Eq.2 of Mason and Weigel (2009) to calculate the 2AFC.

Value

p.afc

Value of Generalized Discrimination (2AFC) Score

Author(s)

Andreas Weigel, Federal Office of Meteorology and Climatology, MeteoSwiss, Zurich, Switzerland

References

S.J. Mason and A.P. Weigel, 2009. A generic verification framework for administrative purposes. Mon. Wea. Rev., 137, 331-349

Examples


  #Forecasts and observations of Nino-3.4 index
  #Load set of dichotomous observations and dichotomous forecasts
  data(cnrm.nino34.dd)
  obsv = cnrm.nino34.dd$obsv
  fcst = cnrm.nino34.dd$fcst

  #Calculate skill score
  afc.dd(obsv,fcst)

2AFC For Dichotomous Observations and Ensemble Forecasts

Description

Routine to calculate the Generalized Discrimination Score (aka Two-Alternatives Forced Choice Score 2AFC) for the situation of dichotomous observations and ensemble forecasts

Usage

afc.de(obsv, fcst)

Arguments

obsv

vector with dichotomous observations (values in 0,1)

fcst

two-dimensional array with ensemble forecasts; dim(fcst)[1] = length(obsv); dim(fcst)[2] = ensemble size

Details

This routine first ranks the ensemble forecasts (see rank.ensembles) and then calculates the 2AFC-score with Eq.2 of Mason and Weigel (2009).

Value

p.afc

Value of Generalized Discrimination (2AFC) Score

Author(s)

Andreas Weigel, Federal Office of Meteorology and Climatology, MeteoSwiss, Zurich, Switzerland

References

S.J. Mason and A.P. Weigel, 2009. A generic verification framework for administrative purposes. Mon. Wea. Rev., 137, 331-349

Examples


  #Forecasts and observations of Nino-3.4 index
  #Load set of dichotomous observations and 9-member ensemble forecasts
  data(cnrm.nino34.de)
  obsv = cnrm.nino34.de$obsv
  fcst = cnrm.nino34.de$fcst

 #Calculate skill score
  afc.de(obsv,fcst)

2AFC For Dichotomous Observations And Ordinal Polychotomous Forecasts

Description

Routine to calculate the Generalized Discrimination Score (aka Two-Alternatives Forced Choice Score 2AFC) for the situation of dichotomous observations and (ordinal) polychotomous forecasts

Usage

afc.dm(obsv, fcst, mf = 3)

Arguments

obsv

vector with dichotomous observations (values in 0,1)

fcst

vector of same length as obsv with polychotomous forecasts (values in 1,..,m)

mf

number of forecast categories (default = 3)

Details

This routine applies Eq.5 of in Mason and Weigel (2009) to calculate the 2AFC.

Value

p.afc

Value of Generalized Discrimination (2AFC) Score

Author(s)

Andreas Weigel, Federal Office of Meteorology and Climatology, MeteoSwiss, Zurich, Switzerland

References

S.J. Mason and A.P. Weigel, 2009. A generic verification framework for administrative purposes. Mon. Wea. Rev., 137, 331-349

Examples


  #Forecasts and observations of Nino-3.4 index
  #Load set of dichotomous observations and polychotomous forecasts (4 categories)
  data(cnrm.nino34.dm)
  obsv = cnrm.nino34.dm$obsv
  fcst = cnrm.nino34.dm$fcst

  #Calculate skill score
  afc.dm(obsv,fcst,4)

2AFC For Dichotomous Observations And Probabilistic Forecasts

Description

Routine to calculate the Generalized Discrimination Score (aka Two-Alternatives Forced Choice Score 2AFC) for the situation of dichotomous observations and discrete probabilistic forecasts

Usage

afc.dp(obsv, fcst)

Arguments

obsv

vector with dichotomous observations (values in 0,1)

fcst

vector of same length as obsv with forecast probabilities for the event to happen

Details

This routine applies Eq.5 of Mason and Weigel (2009) to calculate the 2AFC.

Value

p.afc

Value of Generalized Discrimination (2AFC) Score

Author(s)

Andreas Weigel, Federal Office of Meteorology and Climatology, MeteoSwiss, Zurich, Switzerland

References

S.J. Mason and A.P. Weigel, 2009. A generic verification framework for administrative purposes. Mon. Wea. Rev., 137, 331-349

Examples


  #Forecasts and observations of Nino-3.4 index
  #Load set of dichotomous observations and probabilistic forecasts
  data(cnrm.nino34.dp)
  obsv = cnrm.nino34.dp$obsv
  fcst = cnrm.nino34.dp$fcst

  #Calculate skill score
  afc.dp(obsv,fcst)

2AFC For Ordinal Polychotomous Observations And Continuous Forecasts

Description

Routine to calculate the Generalized Discrimination Score (aka Two-Alternatives Forced Choice Score 2AFC) for the situation of polychotomous observations (ordinal) and continuous forecasts

Usage

afc.mc(obsv, fcst, m = 3)

Arguments

obsv

vector with polychotomous observations (values in 1,..,m)

fcst

vector of same length as obsv with real-valued forecasts

m

number of observation categories (default = 3)

Details

This routine applies Eq.18 of Mason and Weigel (2009) to calculate the 2AFC.

Value

p.afc

Value of Generalized Discrimination (2AFC) Score

Author(s)

Andreas Weigel, Federal Office of Meteorology and Climatology, MeteoSwiss, Zurich, Switzerland

References

S.J. Mason and A.P. Weigel, 2009. A generic verification framework for administrative purposes. Mon. Wea. Rev., 137, 331-349

Examples


  #Forecasts and observations of Nino-3.4 index
  #Load set of polychotomous observations (4 categories) and continuous forecasts
  data(cnrm.nino34.mc)
  obsv = cnrm.nino34.mc$obsv
  fcst = cnrm.nino34.mc$fcst

  #Calculate skill score
  afc.mc(obsv,fcst,4)

2AFC For Ordinal Polychotomous Observations And Ensemble Forecasts

Description

Routine to calculate the Generalized Discrimination Score (aka Two-Alternatives Forced Choice Score 2AFC) for the situation of polychotomous observations (ordinal) and ensemble forecasts

Usage

afc.me(obsv, fcst, m = 3)

Arguments

obsv

vector with polychotomous observations (values in 1,..,m)

fcst

two-dimensional array with ensemble forecasts; dim(fcst)[1] = length(obsv); dim(fcst)[2] = ensemble size

m

number of observation categories (default = 3)

Details

This routine first ranks the ensemble forecasts (see rank.ensembles) and then calculates the 2AFC-score with Eq.18 of Mason and Weigel (2009).

Value

p.afc

Value of Generalized Discrimination (2AFC) Score

Author(s)

Andreas Weigel, Federal Office of Meteorology and Climatology, MeteoSwiss, Zurich, Switzerland

References

S.J. Mason and A.P. Weigel, 2009. A generic verification framework for administrative purposes. Mon. Wea. Rev., 137, 331-349

Examples


  #Forecasts and observations of Nino-3.4 index
  #Load set of polychotomous observations (4 categories) and 9-member ensemble forecasts
  data(cnrm.nino34.me)
  obsv = cnrm.nino34.me$obsv
  fcst = cnrm.nino34.me$fcst

  #Calculate skill score
  afc.me(obsv,fcst,4)

2AFC For Ordinal Polychotomous Observations And Ordinal Polychotomous Forecasts

Description

Routine to calculate the Generalized Discrimination Score (aka Two-Alternatives Forced Choice Score 2AFC) for the situation of polychotomous observations (ordinal) and polychotomous forecasts (ordinal)

Usage

afc.mm(obsv, fcst, mv = 3, mf = 3)

Arguments

obsv

vector with polychotomous observations (values in 1,..,mv)

fcst

vector of same length as obsv with polychotomous forecasts (values in 1,..,mf)

mv

number of observation categories (default = 3)

mf

number of forecast categories (default = 3)

Details

This routine applies Eq.14 of Mason and Weigel (2009) to calculate the 2AFC.

Value

p.afc

Value of Generalized Discrimination (2AFC) Score

Author(s)

Andreas Weigel, Federal Office of Meteorology and Climatology, MeteoSwiss, Zurich, Switzerland

References

S.J. Mason and A.P. Weigel, 2009. A generic verification framework for administrative purposes. Mon. Wea. Rev., 137, 331-349

Examples


  #Forecasts and observations of Nino-3.4 index
  #Load set of polychotomous observations (4 categories) and polychotomous forecasts (4 categories)
  data(cnrm.nino34.mm)
  obsv = cnrm.nino34.mm$obsv
  fcst = cnrm.nino34.mm$fcst

  #Calculate skill score
  afc.mm(obsv,fcst,4,4)

2AFC For Ordinal Polychotomous Observations And Probabilistic Forecasts

Description

Routine to calculate the Generalized Discrimination Score (aka Two-Alternatives Forced Choice Score 2AFC) for the situation of polychotomous observations (ordinal) and discrete probabilistic forecasts

Usage

afc.mp(obsv, fcst, m = 3)

Arguments

obsv

vector with polychotomous observations (values in 1,..,m)

fcst

two-dimensional array with forecast probabilities for the m categories; dim(fcst)[1] = length(obsv); dim(fcst)[2] = m

m

number of observation categories (default = 3)

Details

This routine applies Eq.16 of Mason and Weigel (2009) to calculate the 2AFC.

Value

p.afc

Value of Generalized Discrimination (2AFC) Score

Author(s)

Andreas Weigel, Federal Office of Meteorology and Climatology, MeteoSwiss, Zurich, Switzerland

References

S.J. Mason and A.P. Weigel, 2009. A generic verification framework for administrative purposes. Mon. Wea. Rev., 137, 331-349

Examples


  #Forecasts and observations of Nino-3.4 index
  #Load set of polychtomous observations (4 categories) and probabilistic forecasts
  data(cnrm.nino34.mp)
  obsv = cnrm.nino34.mp$obsv
  fcst = cnrm.nino34.mp$fcst

  #Calculate skill score
  afc.mp(obsv,fcst,4)

2AFC For Nominal Polychotomous Observations And Nominal Polychotomous Forecasts

Description

Routine to calculate the Generalized Discrimination Score (aka Two-Alternatives Forced Choice Score 2AFC) for the situation of polychotomous observations (nominal) and polychotomous forecasts (nominal)

Usage

afc.nn(obsv, fcst, m = 3)

Arguments

obsv

vector with polychotomous observations (values in 1,..,m)

fcst

vector of same length as obsv with polychotomous forecasts (values in 1,..,m)

m

number of observation and forecast categories (default = 3)

Details

This routine applies Eq.15 of Mason and Weigel (2009) to calculate the 2AFC.

Value

p.afc

Value of Generalized Discrimination (2AFC) Score

Author(s)

Andreas Weigel, Federal Office of Meteorology and Climatology, MeteoSwiss, Zurich, Switzerland

References

S.J. Mason and A.P. Weigel, 2009. A generic verification framework for administrative purposes. Mon. Wea. Rev., 137, 331-349

Examples


  #Forecasts and observations of Nino-3.4 index
  #Load set of polychotomous observations and polychotomous forecasts (4 categories)
  data(cnrm.nino34.mm)
  obsv = cnrm.nino34.mm$obsv
  fcst = cnrm.nino34.mm$fcst

  #Calculate skill score
  afc.nn(obsv,fcst,4)

2AFC For Nominal Polychotomous Observations Ans Probabilistic Forecasts

Description

Routine to calculate the Generalized Discrimination Score (aka Two-Alternatives Forced Choice Score 2AFC) for the situation of polychotomous (nominal) observations and discrete probabilistic forecasts

Usage

afc.np(obsv, fcst, m = 3)

Arguments

obsv

vector with dichotomous observations (values in 0,1)

fcst

two-dimensional array with forecast probabilities for the m categories; dim(fcst)[1] = length(obsv); dim(fcst)[2] = m

m

number of observation categories (default = 3)

Details

This routine applies Eq.17 of Mason and Weigel (2009) to calculate the 2AFC.

Value

p.afc

Value of Generalized Discrimination (2AFC) Score

Author(s)

Andreas Weigel, Federal Office of Meteorology and Climatology, MeteoSwiss, Zurich, Switzerland

References

S.J. Mason and A.P. Weigel, 2009. A generic verification framework for administrative purposes. Mon. Wea. Rev., 137, 331-349

Examples


  #Forecasts and observations of Nino-3.4 index
  #Load set of polychotomous observations (4 categories) and probabilistic forecasts
  data(cnrm.nino34.mp)
  obsv = cnrm.nino34.mp$obsv
  fcst = cnrm.nino34.mp$fcst

  #Calculate skill score
  afc.np(obsv,fcst,4)

Example Data of Continuous Observations and Continuous Forecasts

Description

Continuous observations (obsv) and continuous forecasts (fcst) of the Nino-3.4 index for January 1961-2000. The forecast data stem from model runs initialized using data for the preceding August 1960-1999.

Description

Routine to rank a set of given ensemble forecasts according to their "value"

Usage

rank.ensembles(fcst)

Arguments

fcst

two-dimensional array with ensemble forecasts; dim(fcst)[1] = number of ensemble forecasts; dim(fcst)[2] = number of ensemble members

Details

This routine ranks a set of ensemble forecasts according to their "value". The higher the "value" of an ensemble forecasts, the higher the rank. The following principle is applied: Assume two ensembles A and B are to be ranked. Without loss of generality, we define A>B if the probability of a random ensemble member of A being larger than a random ensemble member of B exceeds 0.5. This probability is calculated by a 2AFC-like approach based on Eq. 8 of Mason and Weigel (2009). By pairwise comparison of all ensembles, the final ranking is obtained.

Value

ranks

vector with the ranks of the ensemble forecasts

Author(s)

Andreas Weigel, Federal Office of Meteorology and Climatology, MeteoSwiss, Zurich, Switzerland

References

S.J. Mason and A.P. Weigel, 2009. A generic verification framework for administrative purposes. Mon. Wea. Rev., 137, 331-349

Examples


  #Load a set of ensemble forecasts
  data(cnrm.nino34.ce)
  fcst = cnrm.nino34.ce$fcst

  #Rank ensemble forecasts
  rank.ensembles(fcst)

Generalized Discrimination Score 2AFC

Description

Details

Index

Author(s)

References

Calculate Generalized Discrimination Score 2AFC

Description

Usage

Arguments

Details

Value

Author(s)

References

See Also

Examples

2AFC For Continuous Observations And Continuous Forecasts

Description

Usage

Arguments

Details

Value

Author(s)

References

See Also

Examples

2AFC For Continuous Observations And Ensemble Forecasts

Description

Usage

Arguments

Details

Value

Author(s)

References

See Also

Examples

2AFC For Dichotomous Observations And Continuous Forecasts

Description

Usage

Arguments

Details

Value

Author(s)

References

See Also

Examples

2AFC For Dichotomous Observations And Dichotomous Forecasts

Description

Usage

Arguments

Details

Value

Author(s)

References

See Also

Examples

2AFC For Dichotomous Observations and Ensemble Forecasts

Description

Usage

Arguments

Details

Value

Author(s)

References

See Also

Examples

2AFC For Dichotomous Observations And Ordinal Polychotomous Forecasts

Description

Usage

Arguments

Details

Value

Author(s)

References

See Also

Examples

2AFC For Dichotomous Observations And Probabilistic Forecasts

Description

Usage

Arguments