Type: Package
Title: Optimal Item Calibration
Version: 1.7.1
Date: 2023-05-24
Depends: R (≥ 4.1.0)
Description: The restricted optimal design method is implemented to optimally allocate a set of items that require calibration to a group of examinees. The optimization process is based on the method described in detail by Ul Hassan and Miller in their works published in (2019) <doi:10.1177/0146621618824854> and (2021) <doi:10.1016/j.csda.2021.107177>. To use the method, preliminary item characteristics must be provided as input. These characteristics can either be expert guesses or based on previous calibration with a small number of examinees. The item characteristics should be described in the form of parameters for an Item Response Theory (IRT) model. These models can include the Rasch model, the 2-parameter logistic model, the 3-parameter logistic model, or a mixture of these models. The output consists of a set of rules for each item that determine which examinees should be assigned to each item. The efficiency or gain achieved through the optimal design is quantified by comparing it to a random allocation. This comparison allows for an assessment of how much improvement or advantage is gained by using the optimal design approach. This work was supported by the Swedish Research Council (Vetenskapsrådet) Grant 2019-02706.
Author: Mahmood Ul Hassan ORCID iD [aut, cre], Frank Miller ORCID iD [aut]
Encoding: UTF-8
RoxygenNote: 7.2.3
License: GPL (≥ 3)
License_restricts_use: no
Repository: CRAN
Maintainer: Mahmood Ul Hassan <scenic555@gmail.com>
BugReports: https://github.com/scenic555/optical/issues
URL: https://scenic555.github.io/optical/, https://github.com/scenic555/optical
Language: en-GB
Imports: stats
NeedsCompilation: no
Packaged: 2023-05-24 13:46:02 UTC; sceni
Date/Publication: 2023-05-25 20:00:02 UTC

Convergence plot

Description

Convergence plots displaying efficiency of design, violations of equivalence theorem, and alpha(Step length used) vs. iteration number. These plot are suitable for monitoring the convergence of optimal item calibration algorithm.

Usage

convergenceplot(yyy, refline = c(0.002, 1e-05))

Arguments

yyy

a optical object; the output of a call optical()

refline

reference line

Details

Convergence plots have three panel.

Value

A convergence plot is displayed.

See Also

drawdesign

Examples

# 2PL-models for two items; parameters (a, b)=(1.6, -1) and (1.6, 1), respectively
ip <- cbind(c(1.6, 1.6),c(-1, 1))

yyy <- optical(ip)

convergenceplot(yyy, refline=c(0.002, 0.001*0.005/0.45))

Graph for (optimal) design

Description

Generate plot for optimal design with four possible layouts. All layouts have design first incl. efficiency vs. random design; then line with item having minimal directional derivative

Usage

drawdesign(
  yyy,
  ip,
  ablim = 7,
  ylowl = -9999999,
  refline = 0.002,
  textout = TRUE,
  itemnum = NA,
  layout = 1,
  colvec = 1:12
)

Arguments

yyy

a optical object; the output of a call optical()

ip

matrix with item parameters for all items (number of rows determines number of items; number of column 2 (2PL) or 3 (3PL or mixed 2/3-PL with NA for 2PL-items in third column).

ablim

ability limits; plots will be made in the range [-ablim, ablim]

ylowl

y low level (minimum value of directional derivative shown in the plot)

refline

reference line correspond to desired minimum violation of equivalence theorem

textout

If textout=TRUE (default), the item parameters will be printed if number of items $<5$ and the efficiency vs. the random design; if textout=FALSE, no such text is printed

itemnum

number of items

layout

layouts of plots

  • Layout 1: third panel has directional derivatives (cut at ylowl or lowest value of dirdev)

  • Layout 2: third panel has violations of equivalence theorem, should be ideally small. Stopping criterion could be <0.002 (refline)

  • Layout 3: third panel monitors efficiency of design vs. iteration number

  • Layout 4: third panel monitors violations of equivalence theorem vs. iteration number

  • Layout 5: third panel shows item characteristic curves

  • Layout 0: only one panel with design

colvec

vector of color sequence for items (default is the R-default black, red, green, etc.)

Value

An optimal design plot is displayed.

See Also

convergenceplot

Examples

# 2PL-models for two items; parameters (a, b)=(1.6, -1) and (1.6, 1), respectively
ip <- cbind(c(1.6, 1.6),c(-1, 1))

yyy <- optical(ip)

drawdesign(yyy=yyy, ip=ip, ylowl=-1000, refline=0.002, layout=1)

Efficiency of optimal design

Description

This function computes the efficiency of the D, I, and A optimal designs compared to the random design.

Usage

efficiency(
  yyy,
  ip,
  uncert = FALSE,
  ipop,
  oc = "D",
  L = NULL,
  items = FALSE,
  integ = TRUE
)

Arguments

yyy

a optical object; the output of a call optical()

ip

matrix with item parameters for all items (number of rows determines number of items; number of columns is 2 (for 2PL; or 1PL with common a-parameter when NA in first column from second item) or 3 (for 3PL; or mixed 2/3-PL with NA for 2PL-items in third column)

uncert

if false (default), abilities are assumed to be known; if true, handling of uncertainties of Bjermo et al. (2021) is used.

ipop

matrix with item parameters for operational items (used if uncert=TRUE, only).

oc

optimality criterion: "D" (D-optimality, default), "I" (I-optimality with standard normal weight function), "A" (A-optimality).

L

L-matrix (not used for D-optimality)

items

if false (default), only total block efficiency is returned; if true, criteria for optimal and random and the efficiency for each item are reported in each column of output. Last column are then total criteria and efficiency. D-, L-, I-, A-optimality

integ

if true (default), integrate() is used for computation of partial information matrices; if false, Riemann rule is used.

Value

A numerical value is displayed.

See Also

optical

Examples

# 2PL-models for two items; parameters (a, b)=(1.6, -1) and (1.6, 1), respectively
ip <- cbind(c(1.6, 1.6),c(-1, 1))

yyy <- optical(ip)

# Efficiency of A-optimal design compared to random design
efficiency(yyy, ip, oc="A")


# Efficiency of D-optimal design compared to random design
efficiency(yyy, ip, oc="D")

# Efficiency of I-optimal design compared to random design
efficiency(yyy, ip, oc="I")

Optimal item calibration

Description

Calibrate items following a 2PL, 3PL, mixture of 2PL and 3PL model, or 2PL with common discrimination for all items (Rasch-type).

Usage

optical(
  ip,
  oc = "D",
  uncert = FALSE,
  ipop,
  imf = c(0.005, 0.01, 0.02, 0.05, 0.1, 0.2, 0.45),
  maxiter = rep(300, 6),
  eps = rep(0.002, 6),
  nnn = c(0, 50, 50, 200, 200, 200),
  nsp = c(0.001, 1e-04, 1e-04, 1e-05, 1e-05, 1e-05),
  sss = 0.001,
  falpha = 1.08,
  sdr = TRUE,
  ig = 3,
  ex = 0,
  integ = TRUE,
  show_progress = 1
)

Arguments

ip

matrix with item parameters for all items (number of rows determines number of items; number of column is 2 (2PL or Rasch-type with NA from second item in first column) or 3 (3PL or mixed 2/3-PL with NA for 2PL-items in third column).

oc

optimality criterion: "D" (D-optimality, default), "I" (I-optimality with standard normal weight function), "A" (A-optimality).

uncert

if false (default), abilities are assumed to be known; if true, handling of uncertainties of Bjermo et al. (2021) is used.

ipop

matrix with item parameters for operational items (used if uncert=TRUE, only).

imf

the vector of step-lengths; default c(0.005, 0.01, 0.02, 0.05, 0.1, 0.2, 0.45).

maxiter

maximal number of iterations in each inner loop, the length of this vector defines the number of outer loops.

eps

convergence criterion (maximum violation of eq.th.), vector with value for each iteration in the outer loop, but the same number for all iterations is recommended.

nnn

number of new nodes added at each position in the adaptive grid, vector with value for each iteration in the outer loop (nnn [1] not used).

nsp

node spacing between new nodes, vector with value for each iteration in the outer loop (nsp [1] is the spacing between nodes of the starting grid).

sss

step size stopping criterion.

falpha

factor alpha for adjusting the step size vector (should be > 1).

sdr

stop if design repeated (flag TRUE/FALSE).

ig

inner grid between -ig and ig.

ex

intervals of size < ex will be removed (consolidate); if ex=0, no consolidation will be done.

integ

if true (default), integrate() is used for computation of partial information matrices; if false, Riemann rule is used.

show_progress

if 1 (default), no output will be printed for each iteration. If 2, the + symbols will be printed on a line for each Iteration If 3, some output of the function will be printed.

Value

Result of this function is a list with following instances:

dd

directional derivatives of optimal solution.

xi

optimal solution.

t

final grid of ability values which was used.

viomax

largest violation of eq.th. from final solution (if < eps, alg. has converged, otherwise not).

h1

interval boundaries for optimal solution.

ht

Refined table of interval boundaries for optimal design with calibrated items and their corresponding probabilities

mooiter

monitoring iterations; information about each iteration to produce convergence plots.

time

running time of algorithm in minutes.

oc

optimality criterion ("D", "I", "A", "L").

L

L-matrix (not for D-optimality).

Author(s)

Mahmood Ul Hassan (scenic555@gmail.com); Frank Miller (frank.miller@liu.se)

References

Ul Hassan and Miller (2021). An exchange algorithm for optimal calibration of items in computerized achievement tests.Computational Statistics and Data Analysis, 157: 107177.

Ul Hassan and Miller (2019). Optimal item calibration for computerized achievement tests. Psychometrika, 84, 1101-1128.

Bjermo, Fackle-Fornius, and Miller (2021). Optimizing Calibration Designs with Uncertainty in Abilities. Manuscript.

See Also

drawdesign, convergenceplot, efficiency

Examples

# 2PL-models for two items; parameters (a, b)=(1.6, -1) and (1.6, 1), respectively

ip <- cbind(c(1.6, 1.6),c(-1, 1))

yyy <- optical(ip)

# Table of interval boundaries for D-optimal design with items and
# probabilities (expected proportion of examinees in this interval)
yyy$ht



# 1PL-models with common discrimination parameter for two items
# (model assumption is that both have same discrimination);
# parameters (a, b)=(1.6, -1) and (1.6, 1), respectively;
# NA for discrimination means that item has same parameter as preceeding item
ip <- cbind(c(1.6, NA), c(-1, 1))

yyy <- optical(ip)

# Table of interval boundaries for D-optimal design with items and
# probabilities (expected proportion of examinees in this interval)
yyy$ht


# 3PL-models for three items; parameters (a, b, c)=(1, 2, 2.5),
# (-1.5, 0.5, 2) and (0.2, 0.1, 0.05), respectively.
ip <- cbind(c(1, 2, 2.5),c(-1.5, 0.5, 2),c(0.2, 0.1, 0.05))

yyy <- optical(ip)

# Table of interval boundaries for D-optimal design with items and
# probabilities (expected proportion of examinees in this interval)
yyy$ht