Data Class and Tools for Handling Spatial-Temporal Data

Description

Data class for increased interoperability working with spatial-temporal data together with corresponding functions and methods (conversions, basic calculations and basic data manipulation). The class distinguishes between spatial, temporal and other dimensions to facilitate the development and interoperability of tools build for it. Additional features are name-based addressing of data and internal consistency checks (e.g. checking for the right data order in calculations).

Author(s)

Maintainer: Jan Philipp Dietrich <dietrich@pik-potsdam.de>

See Also

Useful links:

• https://github.com/pik-piam/magclass

• doi: 10.5281/zenodo.1158580

• Report bugs at https://github.com/pik-piam/magclass/issues

add_columns

Description

Function adds new columns to the existing magpie object.

Usage

add_columns(x, addnm = "new", dim = 3.1, fill = NA)

Arguments

Value

The extended MAgPIE object

Author(s)

Jan Philipp Dietrich, Benjamin Bodirsky

See Also

add_dimension,dimCode

Examples

a <- maxample("animal")
a2 <- add_columns(a, addnm = c("horse", "magpie"), dim = "species", fill = 42)
getItems(a2, dim = 3)
getItems(a2, dim = 3, split = TRUE)
head(a2[, , "magpie"])

add_dimension

Description

Function adds a name dimension as dimension number "dim" with the name "add" with an empty data column with the name "nm".

Usage

add_dimension(x, dim = 3.1, add = NULL, nm = "dummy")

Arguments

Value

The extended MAgPIE object

Author(s)

Jan Philipp Dietrich, Benjamin Bodirsky

See Also

add_columns,mbind

Examples

a <- maxample("animal")
str(add_dimension(a, dim = 3.2))
str(add_dimension(a, dim = 2.3, nm = paste0("d", 1:3)))

as.RasterBrick

Description

Convert magclass object to a RasterBrick object

Usage

as.RasterBrick(x, res = NULL)

Arguments

Value

A RasterBrick object

Author(s)

Jan Philipp Dietrich

See Also

getCoords

Examples

## Not run: 
if (requireNamespace("raster", quietly = TRUE)) {
   r <- raster::brick(ncols = 36, nrows = 18, nl = 4)
   r[14:18, 25:28] <- (1:20 %*% t(1:4))
   names(r) <- c("y2000..bla", "y2001..bla", "y2000..blub", "y2001..blub")
   m <- as.magpie(r)
   r2 <- as.RasterBrick(m)
}

## End(Not run)

as.SpatRaster

Description

Convert magclass object to a SpatRaster object. Requires the terra package.

Usage

as.SpatRaster(x, res = NULL)

Arguments

Value

A SpatRaster object

Author(s)

Jan Philipp Dietrich

See Also

getCoords

Examples

if (requireNamespace("terra", quietly = TRUE)) {
   r <- terra::rast(ncols = 360, nrows = 180, nl = 4)
   r[85:89, 176:179] <- (1:20 %*% t(1:4))
   r[15:19, 76:79] <-   (10 + 1:20 %*% t(1:4))
   names(r) <- c("y2000..bla", "y2001..bla", "y2000..blub", "y2001..blub")
   m <- as.magpie(r)
   r2 <- as.SpatRaster(m)
}

as.SpatRasterDataset

Description

Convert magclass object to a SpatRasterDataset object. Requires the terra package.

Usage

as.SpatRasterDataset(...)

Arguments

Details

Calls as.SpatRaster and then spatRasterToDataset.

Value

A SpatRasterDataset object

Author(s)

Pascal Sauer

as.SpatVector

Description

Convert magclass object to a SpatVector object. Requires the terra package and requires the magclass object to provide the geometry of the spatial entities as "geometry" attribute in "WKT" format. (see object "m" in example).

Usage

as.SpatVector(x)

Arguments

Value

A SpatVector object

Author(s)

Jan Philipp Dietrich

See Also

as.SpatRaster

Examples

if (requireNamespace("terra", quietly = TRUE)) {
   r <- terra::rast(ncols = 360, nrows = 180, nl = 4)
   r[85:89, 176:179] <- (1:20 %*% t(1:4))
   r[15:19, 76:79] <-   (10 + 1:20 %*% t(1:4))
   names(r) <- c("y2000..bla", "y2001..bla", "y2000..blub", "y2001..blub")
   v <- terra::as.polygons(r)
   m <- as.magpie(v)
   attr(m, "geometry")
   attr(m, "crs")
   v2 <- as.SpatVector(m)
}

~~ Methods for Function as.array ~~

Description

~~ Methods for function as.array ~~

Usage

## S4 method for signature 'magpie'
as.array(x)

Arguments

Methods

list("signature(x = \"ANY\")")

standard as.array-method

list("signature(x = \"magpie\")")

Conversion takes place just by removing MAgPIE-object specific elements

~~ Methods for Function as.data.frame ~~

Description

~~ Methods for function as.data.frame ~~

Usage

## S4 method for signature 'magpie'
as.data.frame(x, rev = 1, raw = FALSE)

Arguments

Methods

list("signature(x = \"magpie\")")

Conversion creates columns for Cell, Region, Year, Data1, Data2,... and Value

Examples

pop <- maxample("pop")
head(as.data.frame(pop))
head(as.data.frame(pop, rev = 2))

a <- maxample("animal")
head(as.data.frame(a, rev = 3))
head(as.data.frame(a, rev = 3, raw = TRUE))
attr(as.data.frame(a, rev = 3), "dimtype")

magpie method for tibble::as_tibble

Description

magpie method for tibble::as_tibble

Usage

as_tibble.magpie(
  x,
  ...,
  .rows = NULL,
  .name_repair = c("check_unique", "unique", "universal", "minimal"),
  rownames = pkgconfig::get_config("tibble::rownames", NULL)
)

Arguments

Value

A tibble object

MAgPIE-Clean

Description

Function cleans MAgPIE objects so that they follow some extended magpie object rules (currently it makes sure that the dimnames have names and removes cell numbers if it is purely regional data)

Usage

clean_magpie(x, what = "all", maindim = 1:3)

Arguments

Value

The eventually corrected MAgPIE object

Author(s)

Jan Philipp Dietrich

See Also

"magpie"

Examples

pop <- maxample("pop")
a <- clean_magpie(pop)

~~ Methods for Function colSums and colMeans ~~

Description

~~ Methods for function colSums and colMeans ~~

Usage

## S4 method for signature 'magpie'
colSums(x, na.rm = FALSE, dims = 1, ...)

Arguments

Methods

list("signature(x = \"ANY\")")

normal colSums and colMeans method

list("signature(x = \"magpie\")")

classical method prepared to handle MAgPIE objects

Collapse dataset dimensions

Description

This function will remove names in the data dimension which are the same for each element (meaning that this data dimension contains exactly one element) or, if forced, remove any other subdimension. It is a generalized version of the function collapseNames

Usage

collapseDim(x, dim = NULL, keepdim = NULL)

Arguments

Value

The provided MAgPIE object with collapsed dimensions

Note

This function has some similarities to dimReduce, but serves a different purpose. While collapseDim only removes dimensions which contain only a single element or which it is specifically told to remove, dimReduce looks whether the entries of a multi-entry dimension are all the same and removes dimensions for which this is the case. In some cases both will lead to the same result but in many other cases the results will differ.

Author(s)

Jan Philipp Dietrich

See Also

getItems "magpie"

Examples

x <- new.magpie(c("GLO.1", "GLO.2"), 2000, c("bla.a", "bla.b"))
collapseDim(x)
collapseDim(x, keepdim = 1:2)
collapseDim(x, dim = 1.1)
collapseDim(x, dim = 3.2)

Collapse dataset names

Description

This function has been superseded by collapseDim which is a more generalized version of this function. Please use this one instead!

Usage

collapseNames(x, collapsedim = NULL, preservedim = NULL)

Arguments

Details

This function will remove names in the data dimension which are the same for each element (meaning that this data dimension contains exactly one element)

Value

The provided MAgPIE object with collapsed names

Author(s)

Jan Philipp Dietrich, David Klein, Xiaoxi Wang

See Also

collapseDim, getItems, "magpie"

complete_magpie

Description

MAgPIE objects can be incomplete to reduce memory. This function blows up a magpie object to its real dimensions, so you can apply unwrap.

Usage

complete_magpie(x, fill = NA, dim = 3)

Arguments

Value

The completed MAgPIE object

Author(s)

Jan Philipp Dietrich, Benjamin Bodirsky

See Also

add_dimension,clean_magpie

Examples

pop <- maxample("pop")
complete_magpie(pop)

ani <- maxample("animal")
complete_magpie(ani)

convergence

Description

Cross-Fades the values of one MAGPIE object into the values of another over a certain time

Usage

convergence(
  origin,
  aim,
  start_year = NULL,
  end_year = NULL,
  direction = NULL,
  type = "smooth",
  par = 1.5
)

Arguments

Value

returns a time-series with the same timesteps as origin, which lineary fades into the values of the aim object

Author(s)

Benjamin Bodirsky, Jan Philipp Dietrich

Examples

pop <- maxample("pop")
population <- add_columns(pop, "MIX")
population[, , "MIX"] <- convergence(population[, , "A2"], population[, , "B1"])

Copy Attributes

Description

This function copies attributes from one object and assigns them to another.

Usage

copy.attributes(
  from,
  to,
  delete = c("names", "row.names", "class", "dim", "dimnames"),
  delete2 = NULL
)

copy.attributes(
  to,
  delete = c("names", "row.names", "class", "dim", "dimnames"),
  delete2 = NULL
) <- value

Arguments

Functions

• copy.attributes( to, delete = c("names", "row.names", "class", "dim", "dimnames"), delete2 = NULL ) <- value: assign attributes from object "value"

Author(s)

Jan Philipp Dietrich

Examples

from <- array(12)
attr(from,"blablub") <- "I am an attribute!"
attr(from,"blablub2") <- "I am another attribute!"

print(attributes(from))

to <- as.magpie(0)
print(attributes(to))

copy.attributes(to) <- from
print(attributes(to))

Copy MAgPIE-files

Description

This function copies MAgPIE-files from one location to another. During the copying it is also possible to change the file type (e.g. from 'mz' to 'csv')

Usage

copy.magpie(input_file, output_file, round = NULL)

Arguments

Author(s)

Jan Philipp Dietrich

See Also

read.magpie,write.magpie

Examples

# copy.magpie("bla.csv","blub.mz")

dimCode

Description

Function converts a dimension name or number to a dimension Code used for MAgPIE objects

Usage

dimCode(dim, x, missing = 0, strict = FALSE, sep = ".")

Arguments

Value

A dimension code identifying the dimension. Either a integer which represents the main dimensions (1=spatial, 2=temporal, 3=data) or a numeric, representing the subdimensions of a dimension (e.g. 3.2 for the second data dimension).

Author(s)

Jan Philipp Dietrich, Kristine Karstens

See Also

mselect, getDim

Examples

pop <- maxample("pop")
dimCode(c("t", "scenario", "blablub"), pop)

dimExists

Description

Function checks whether a dimension exsist in a MAgPIE objects

Usage

dimExists(dim, x, sep = ".")

Arguments

Value

Boolean indicating whether dimension exists or not

Author(s)

Jan Philipp Dietrich

See Also

dimCode

Examples

pop <- maxample("pop")
dimExists(c("t","scenario","blablub"),pop)

dimOrder

Description

Changes the order of the sub-dimension in a magpie object similar to unwrapping and applying the aperm command, but more efficient.

Usage

dimOrder(x, perm, dim = 3)

Arguments

Value

magpie object

Author(s)

Jan Philipp Dietrich, Benjamin Leon Bodirsky

Examples

a <- maxample("animal")
head(a)
head(dimOrder(a, perm = 3:1, dim = 1))
head(dimOrder(a, perm = c(2,1,3), dim = 3))

dimReduce

Description

Remove dimensions which contain identical data for all elements in it

Usage

dimReduce(x, dim_exclude = NULL)

Arguments

Value

The reduced MAgPIE object

Note

This function has some similarities to collapseDim, but serves a different purpose. While collapseDim only removes dimensions which contain only a single element or which it is specifically told to remove, dimReduce looks whether the entries of a multi-entry dimension are all the same and removes dimensions for which this is the case. In some cases both will lead to the same result but in many other cases the results will differ.

Author(s)

Jan Philipp Dietrich

See Also

add_dimension

Examples

# create data with 5 identical scenarios
p <- add_dimension(maxample("pop")[1:3, 1:3, ], nm = paste0("scen", 1:2))
str(p)
str(dimReduce(p))

# set years to same value
p[, , ] <- setYears(p[, 1, ], NULL)
str(p)
str(dimReduce(p))

# set regions to same value
p[, , ] <- setCells(p[1, , ], "GLO")
str(p)
str(dimReduce(p))

Summation over dimensions

Description

This function sums over any (sub-)dimension of a magpie object

Usage

dimSums(x, dim = 3, na.rm = FALSE)

Arguments

Value

A MAgPIE object with values summed over the specified dimensions

Author(s)

Jan Philipp Dietrich

See Also

rowSums, getItems, dimCode

Examples

a <- maxample("animal")
dimSums(a, dim = c(1, 2, 3.2))
dimSums(a, dim = c("x", "y", "cell", "month"))

escapeRegex

Description

Escapes all symbols in a string which have a special meaning in regular expressions.

Usage

escapeRegex(x)

Arguments

Value

The escaped strings.

Author(s)

Jan Philipp Dietrich

See Also

grep

Reconstructs full dimensionality of MAgPIE objects

Description

If a MAgPIE object is created from a source with more than one data dimension, these data dimensions are combined to a single dimension. fulldim reconstructs the original dimensionality and reports it.

Usage

fulldim(x, sep = ".")

Arguments

Value

A list containing in the first element the dim output and in the second element the dimnames output of the reconstructed array.

Author(s)

Jan Philipp Dietrich

See Also

as.magpie,unwrap,wrap

Get cells per region

Description

Counts how often each element of the provided subdimension exists in the given data set. Originally created to count the number of cells in a region (this is also where its name originates from) it can now be used to count elements of any subdimension via the dim argument.

Usage

getCPR(x, dim = 1.1)

Arguments

Value

cells per region

Author(s)

Jan Philipp Dietrich

See Also

getRegions, read.magpie, write.magpie

Examples

getCPR(0.5)
a <- maxample("animal")
getCPR(a, dim = "color")
getCPR(a, dim = 3.2)
getCPR(a, dim = "country")
getCPR(a, dim = c("color", "species"))

Get Cells

Description

Extracts cell names of a MAgPIE-object

Usage

getCells(x)

getCells(x) <- value

setCells(object, nm = "GLO")

Arguments

Details

setCells is a shortcut to use a MAgPIE object with manipulated cell names. setCells uses the variable names "object" and "nm" in order to be consistent to the already existing function setNames.

Value

getCells returns cell names of the MAgPIE-object, whereas setCells returns the MAgPIE object with the manipulated cell names.

Functions

• getCells(x) <- value: set cell names

• setCells(): set cell names

Author(s)

Jan Philipp Dietrich

See Also

getRegions, getNames, setNames, getCPR, read.magpie, write.magpie, "magpie"

Examples

a <- as.magpie(1)
 getCells(a)
 setCells(a, "AFR")

getComment

Description

Extracts the comment from a MAgPIE-object

Usage

getComment(x)

getComment(x) <- value

setComment(object, nm = NULL)

Arguments

Value

getComment returns the comment attached to a MAgPIE-object, NULL if no comment is present. setComment returns the magpie object with the modified comment.

Functions

• getComment(x) <- value: set comment

• setComment(): set comment

Author(s)

Markus Bonsch

See Also

getRegions, getNames, getYears, getCPR, read.magpie, write.magpie, "magpie"

Examples

 a <- as.magpie(1)
 #returns NULL
 getComment(a)
 #set the comment
 getComment(a)<-c("bla","blubb")
 getComment(a)

Get Coordinates

Description

Extracts spatial coordinates of a MAgPIE-object

Usage

getCoords(x, xlab = "x", ylab = "y")

getCoords(x, xlab = "x", ylab = "y") <- value

Arguments

Value

coordinates of the MAgPIE-object

Functions

• getCoords(x, xlab = "x", ylab = "y") <- value: set coordinates

Author(s)

Jan Philipp Dietrich

See Also

as.RasterBrick, getItems, "magpie"

Examples

a <- maxample("animal")
getCoords(a)

getDim

Description

Function which tries to detect the dimension to which the given elems belong

Usage

getDim(elems, x, fullmatch = FALSE, dimCode = TRUE)

Arguments

Value

The name or dimCode of the dimensions in which elems were found.

Author(s)

Jan Philipp Dietrich

See Also

mcalc,dimCode

Examples

 pop <- maxample("pop")
 getDim(c("AFR","CPA"),pop)
 getDim(c("AFR","CPA"),pop,fullmatch=TRUE)
 getDim(c("AFR","CPA"),pop,dimCode=FALSE)
 

Get Items

Description

Extract items of a given (sub-)dimension of a MAgPIE-object

Usage

getItems(x, dim = NULL, split = FALSE, full = FALSE)

getItems(x, dim, full = NULL, maindim = NULL, raw = FALSE) <- value

Arguments

Value

items of the requested dimension in the MAgPIE-object. If split=TRUE and applied to a main dimension (1,2,3) a list of items for each sub-dimension.

Functions

• getItems(x, dim, full = NULL, maindim = NULL, raw = FALSE) <- value: set dimension names

Author(s)

Jan Philipp Dietrich

See Also

dimCode

Examples

x <- maxample("pop")
getItems(x, "scenario")
getItems(x, 3.1)
getItems(x, "i") <- paste0("REG", seq_len(ncells(x)))
getItems(x, "i")
y <- x[, 1, ]
getItems(y, "t") <- NULL

Get dataset names

Description

Extracts dataset names of a MAgPIE-object

Usage

getNames(x, fulldim = FALSE, dim = NULL)

getNames(x, dim = NULL) <- value

Arguments

Details

setNames is a shortcut to use a MAgPIE object with manipulated data names. The setNames method uses the variable names "object" and "nm" in order to be consistent to the already existing function setNames.

Value

getNames returns data names of the MAgPIE-object, whereas setNames returns the MAgPIE object with the manipulated data names.

Functions

• getNames(x, dim = NULL) <- value: set names

Author(s)

Jan Philipp Dietrich

See Also

setNames-methods, getRegions, getYears, getCPR, read.magpie, write.magpie,ndata, "magpie"

Examples

a <- as.magpie(1)
getNames(a)
setNames(a, "bla")

x <- new.magpie("GLO", 2000, c("a.o1", "b.o1", "a.o2"))
getNames(x, dim = 2)

getSets(x, fulldim = FALSE)[3] <- "bla.blub"
getNames(x, dim = "bla")

getSets(x)[4] <- "ble"
getNames(x, dim = "ble") <- c("Hi", "Bye")
x

Get a list of celluare region-belongings

Description

Extracts a vector containing the region of each cell of a MAgPIE-object

Usage

getRegionList(x)

getRegionList(x) <- value

Arguments

Value

A vector with ncell elements containing the region of each cell.

Functions

• getRegionList(x) <- value: set region names

Author(s)

Jan Philipp Dietrich

See Also

getRegions,getYears, getNames, getCPR, read.magpie, write.magpie, "magpie"

Examples

# a <- read.magpie("example.mz")
# getRegionList(a)

Get regions

Description

Extracts regions of a MAgPIE-object

Usage

getRegions(x)

getRegions(x) <- value

Arguments

Value

Regions of the MAgPIE-object

Functions

• getRegions(x) <- value: overwrite region names

Author(s)

Jan Philipp Dietrich

See Also

getYears, getNames, getCPR, read.magpie, write.magpie, "magpie"

Examples

# a <- read.magpie("example.mz")
# getRegions(a)

Get sets

Description

Extracts sets of a MAgPIE-object if available

Usage

getSets(x, fulldim = TRUE, sep = ".")

getSets(x, fulldim = TRUE, sep = ".") <- value

Arguments

Value

Sets of the MAgPIE-object. If no information about contained sets is available NULL

Functions

• getSets(x, fulldim = TRUE, sep = ".") <- value: replace set names

Author(s)

Markus Bonsch, Jan Philipp Dietrich

See Also

getRegions, getNames,getYears, getCPR, read.magpie, write.magpie, "magpie"

Examples

a <- new.magpie("GLO.1", 2000, c("a.o1", "b.o1", "a.o2"))
 getSets(a) <- c("reg", "cell", "t", "bla", "blub")
 getSets(a)

 getSets(a)["d3.1"] <- "BLA"
 getSets(a, fulldim = FALSE)
 getSets(a)

Get years

Description

Extracts years of a MAgPIE-object

Usage

getYears(x, as.integer = FALSE)

getYears(x) <- value

setYears(object, nm = NULL)

Arguments

Details

setYears is a shortcut to use a MAgPIE object with manipulated year names. setYears uses the variable names "object" and "nm" in order to be consistent to the already existing function setNames.

Value

getYears returns years of the MAgPIE-object, whereas setYears returns the MAgPIE object with the manipulated years.

Functions

• getYears(x) <- value: rename years

• setYears(): set years

Author(s)

Jan Philipp Dietrich

See Also

getRegions, getNames, setNames, getCPR, read.magpie, write.magpie, "magpie"

Examples

 a <- as.magpie(1)
 getYears(a)
 setYears(a,1995)

Has Coordinates

Description

Checks, whether object contains coordinates.

Usage

hasCoords(x, xlab = "x", ylab = "y")

Arguments

Value

Boolean indicating whether coordinates were found or not

Author(s)

Jan Philipp Dietrich

See Also

getCoords

Examples

hasCoords(maxample("pop"))
hasCoords(maxample("animal"))

Has Sets

Description

Checks, whether set names have been set

Usage

hasSets(x)

Arguments

Value

Boolean indicating whether coordinates were found or not

Author(s)

Jan Philipp Dietrich

See Also

getCoords

Examples

hasSets(maxample("pop"))
hasSets(maxample("animal"))

head/tail

Description

head and tail methods for MAgPIE objects to extract the head or tail of an object

Usage

## S3 method for class 'magpie'
head(x, n1 = 3L, n2 = 6L, n3 = 2L, ...)

Arguments

Value

head returns the first n1 x n2 x n3 entries, tail returns the last n1 x n2 x n3 entries.

Author(s)

Jan Philipp Dietrich

See Also

head, tail

Examples

 pop <- maxample("pop")
 head(pop)
 tail(pop,2,4,1)

is.temporal, is.spatial

Description

Functions to find out whether a vector consists of strings consistent with the definition for auto-detection of temporal or spatial data.

Usage

is.temporal(x)

Arguments

Value

Returns TRUE or FALSE

Author(s)

Jan Philipp Dietrich

Examples

is.temporal(1991:1993)
is.spatial(c("GLO","AFR"))

isYear

Description

Function to find out whether a vector consists of strings in the format "yXXXX" or "XXXX" with X being a number

Usage

isYear(x, with_y = TRUE)

Arguments

Value

Returns a vector of the length of x with TRUE and FALSE

Author(s)

Benjamin Bodirsky

Examples

x <- c("1955", "y1853", "12a4")
isYear(x, with_y = TRUE)
isYear(x, with_y = FALSE)

log-method for MAgPIE objects

Description

log-method for MAgPIE-objects respectively. Works exactly as for arrays.

Usage

## S4 method for signature 'magpie'
log(x, base = exp(1))

Arguments

Methods

x = "magpie"

works as log(x) for arrays.

Lowpass Filter

Description

Filters high frequencies out of a time series. The filter has the structure x'(n) = (x(n-1)+2*x(n)+x(n+1))/4

Usage

lowpass(x, i = 1, fix = NULL, altFilter = NULL, warn = TRUE)

Arguments

Value

The filtered data vector or MAgPIE object

Author(s)

Jan Philipp Dietrich, Misko Stevanovic

Examples

lowpass(c(1, 2, 11, 3, 4))
# to fix the starting point
lowpass(c(0, 9, 1, 5, 14, 20, 6, 11, 0), i = 2, fix = "start")

magclassdata

Description

General magclass-dataset

Details

Please do not directly access that data. It should be only used by library functions.

Author(s)

Jan Philipp Dietrich

Class "magpie" ~~~

Description

The MAgPIE class is a data format for cellular MAgPIE data with a close relationship to the array data format. is.magpie tests if x is an MAgPIE-object, as.magpie transforms x to an MAgPIE-object (if possible).

Arguments

Objects from the Class

Objects can be created by calls of the form new("magpie", data, dim, dimnames, ...). MAgPIE objects have three dimensions (cells,years,datatype) and the dimensionnames of the first dimension have the structure "REGION.cellnumber". MAgPIE-objects behave the same like array-objects with 2 exceptions:
1.Dimensions of the object will not collapse (e.g. x[1,1,1] will remain 3D instead of becoming 1D)
2.It is possible to extract full regions just by typing x["REGIONNAME",,].

Please mind following standards:
Header must not contain any purely numeric entries, but combinations of characters and numbers are allowed (e.g. "bla","12" is forbidden, wheras "bla","b12" is allowed)
Years always have the structure "y" + 4-digit number, e.g. "y1995"
Regions always have the structure 3 capital letters, e.g. "AFR" or "GLO"

This standards are necessary to allow the scripts to detect headers, years and regions properly and to have a distinction to other data.

Author(s)

Jan Philipp Dietrich

See Also

read.magpie, write.magpie, getRegions, getYears, getNames, getCPR, ncells, nyears, ndata

Examples

showClass("magpie")

pop <- maxample("pop")

# returning PAO and PAS for 2025
pop["PA", 2025, , pmatch = "left"]

# returning CPA for 2025
pop["PA", 2025, , pmatch = "right"]

# returning CPA PAO and PAS for 2025
pop["PA", 2025, , pmatch = TRUE]

# returning PAS and 2025
pop["PAS", 2025, ]

# return all entries for year 2025
pop[2025, dim = 2]

# returning everything but values for PAS or values for 2025
pop["PAS", 2025, , invert = TRUE]

# accessing subdimension via set name

a <- maxample("animal")
a[list(country = "NLD", y = "53p25"), , list(species = c("rabbit", "dog"))]

# please note that the list elements act as filter. For instance, the
# following example will not contain any dogs as the data set does
# not contain any dogs which are black.
a[list(country = "NLD", y = "53p25"), , list(species = c("rabbit", "dog"), color = "black")]

# it is also possible to extract given combinations of subdimensions
# via a data-frame
df <- data.frame(getItems(a, 3, split = TRUE, full = TRUE))[c(1, 3, 4), ][3:2]
getItems(a[df], 3)

# Unknown dimensions to be added in output!
df$blub <- paste0("bl", 1:dim(df)[1])
getItems(a[df], 3)

magpie_expand

Description

Expands a MAgPIE object based on a reference

Usage

magpie_expand(x, ref)

Arguments

Details

Expansion means here that the dimensions of x are expanded accordingly to ref. Please note that this is really only about expansion. In the case that one dimension of ref is smaller than of x nothing happens with this dimension. At the moment magpie_expand is only internally available in the magclass library

You can influence the verbosity of this function by setting the option "magclass.verbosity". By default verbosity is set to 1 which means that only warnings are returned. Setting verbosity to 2 means that warnings as well as additional notes are returned. This is done by options(verbosity.level=2)

With version 5 of the package magpie_expand has been updated to a newer version (currently 2.1) and since version 6 this is the only currently supported version. To switch to the old setup you have to install magclass in a version < 6 and set options(magclass_expand_version=1).

By default expansion is based on the elements in a dimension ignoring the set name of the dimension. To expand based on set names instead of contents (recommended) you can switch options(magclass_setMatching=TRUE). Please be careful with this setting as it alters the behavior of magclass objects quite significantly! For more information have a look at vignette("magclass-expansion").

Value

An expanded version of x.

Author(s)

Jan Philipp Dietrich

See Also

as.magpie, options

Examples

a <- new.magpie(c("AFR", "CPA"), "y1995", c("m", "n"))
b <- new.magpie("GLO", "y1995", c("bla", "blub"))
magpie_expand(b, a)
options(magclass.verbosity = 2)
magpie_expand(b, a)

magpie_expand_dim

Description

Expands a single MAgPIE object dimension

Usage

magpie_expand_dim(x, ref, dim = 1)

Arguments

Details

Expansion means here that the dimensions of x are expanded acordingly to ref. Please note that this is really only about expansion. In the case that one dimension of ref is smaller than of x nothing happens with this dimension. At the moment magpie_expand is only internally available in the magclass library

In contrast to magpie_expand this function is expanding only a single dimension. It is meant as a support function for magpie_expand itself.

Value

An expanded version of x.

Author(s)

Jan Philipp Dietrich

See Also

as.magpie, options

Examples

d <- new.magpie(c("AFR.BLUB.1", "AFR.BLUB.2", "EUR.BLUB.1",
                    "AFR.BLA.1", "AFR.BLA.2", "EUR.BLA.1"), fill = 1)
 getSets(d)[1:3] <- c("reg", "b", "i")
 e <- new.magpie(c("BLA.AFR.A", "BLA.EUR.A", "BLUB.AFR.A", "BLUB.EUR.A",
                    "BLA.AFR.B", "BLA.EUR.B", "BLUB.AFR.B", "BLUB.EUR.B"), fill = 2)
 getSets(e)[1:3] <- c("b", "reg", "a")
 magclass:::magpie_expand_dim(d, e, dim = 1)

MAgPIE-Sort

Description

Brings the spatial and temporal structure of MAgPIE objects in the right order. This function is especially useful when you create new MAgPIE objects as the order typically should be correct for MAgPIE objects.

Usage

magpiesort(x)

Arguments

Value

The eventually corrected MAgPIE object (right order in spatial in temporal dimension)

Author(s)

Jan Philipp Dietrich

See Also

"magpie"

Examples

pop <- maxample("pop")
a <- magpiesort(pop)

magpply

Description

apply command for magpieobjects. Very efficient for replacing loops.

Usage

magpply(X, FUN, MARGIN = NULL, DIM = NULL, ..., INTEGRATE = FALSE)

Arguments

Value

magpie object

Author(s)

Jan Philipp Dietrich, Benjamin Leon Bodirsky

Examples

pop <- maxample("pop")
magpply(pop, FUN = sum, MARGIN = 2)
fourdim <- pop * setNames(pop, c("jkk", "lk"))
magpply(fourdim, FUN = sum, MARGIN = c(1, 3.1))

maxample

Description

A collection of magclass example data sets

Usage

maxample(data)

Arguments

Value

the chosen example data set

Author(s)

Jan Philipp Dietrich

Examples

p <- maxample("pop")
str(p)

a <- maxample("animal")
str(a)
getItems(a, split = TRUE)

mbind

Description

Merges MAgPIE-objects with identical structure in two dimensions. If data differs in the temporal or spatial dimension each year or region/cell must appear only once!

Usage

mbind(...)

Arguments

Value

The merged MAgPIE object

Author(s)

Jan Philipp Dietrich, Misko Stevanovic

See Also

"magpie"

Examples

m <- new.magpie(c("AFR", "CPA", "EUR"), c(1995, 2005), "Data1", fill = c(1, 2, 3, 4, 5, 6))
ms <- dimSums(m, dim = 3.1)
mbind(m, ms)
my <- new.magpie(getRegions(m), 2010, getNames(m), fill = c(6, 6, 4))
mbind(m, my)
md <- new.magpie(getRegions(m), getYears(m), "Data2", fill = c(7, 6, 5, 7, 8, 9))
mbind(m, md)

pop <- maxample("pop")
a <- mbind(pop, pop)
dim(pop)
dim(a)

mcalc

Description

Select values from a MAgPIE-object

Usage

mcalc(x, f, dim = NULL, append = FALSE)

Arguments

Details

This functions only work for MAgPIE objects with named dimensions as the dimension name (set_name) has to be used to indicate in which dimension the entries should be searched for!

Value

The calculated MAgPIE object in the case that append is set to FALSE. Otherwise nothing is returned (as x is appended in place)

Author(s)

Jan Philipp Dietrich

See Also

mselect

Examples

pop <- maxample("pop")
pop
mcalc(pop, X12 ~ A2 * B1, append = TRUE)
pop
mcalc(pop, `Nearly B1` ~ 0.5 * A2 + 99.5 * B1)

MSelect

Description

Select values from a MAgPIE-object

Usage

mselect(x, ..., collapseNames = FALSE)

mselect(x, ...) <- value

Arguments

Details

This functions only work for MAgPIE objects with named dimensions as the dimension name (set_name) has to be used to indicate in which dimension the entries should be searched for!

Value

The reduced MAgPIE object containing only the selected entries or the full MAgPIE object in which a selection of entries was manipulated.

Functions

• mselect(x, ...) <- value: replace values in magpie object

Author(s)

Jan Philipp Dietrich

See Also

collapseNames, "magpie"

Examples

pop <- maxample("pop")
mselect(pop, i = c("AFR", "EUR"), scenario = "A2", t = "y2035")

Count elements

Description

Functions to count the number of cells/years/datasets/regions of an MAgPIE-object

Usage

ncells(x)

ndata(x)

nregions(x)

nyears(x)

Arguments

Value

Functions

• ndata(): count datasets

• nregions(): count regions

• nyears(): count years

Author(s)

Jan Philipp Dietrich

Examples

a <- is.magpie(NULL)
ncells(a)
nyears(a)
ndata(a)
nregions(a)

Count sub-dimensions

Description

Functions to count the subdimensions of an MAgPIE-object

Usage

ndim(x, dim = NULL)

Arguments

Value

Number of subdimensions

Author(s)

Jan Philipp Dietrich

Examples

  a <- maxample("animal")
  ndim(a)
  ndim(a,1)
  ndim(a,2)
  ndim(a,3)

new.magpie

Description

Creates a new MAgPIE object

Usage

new.magpie(
  cells_and_regions = "GLO",
  years = NULL,
  names = NULL,
  fill = NA,
  sort = FALSE,
  sets = NULL,
  unit = NULL
)

Arguments

Value

an empty magpie object filled with fill, with the given dimnames

Author(s)

Benjamin Bodirsky, Jan Philipp Dietrich

See Also

as.magpie

Examples

a <- new.magpie(1:10, 1995:2000)
b <- new.magpie(c("AFR", "CPA"), "y1995", c("bla", "blub"), sets = c("i", "t", "value"))
c <- new.magpie()

population_magpie

Description

Example dataset for a regional MAgPIE object

Value

A2 and B1 population scenario from SRES

Author(s)

Benjamin Bodirsky

print

Description

print method for MAgPIE objects for conventient display of magpie data.

Usage

## S3 method for class 'magpie'
print(x, drop = TRUE, reshape = FALSE, ...)

Arguments

Value

Invisibly, the MAgPIE object x.

Author(s)

Jan Philipp Dietrich, Kristine Karstens, Felicitas Beier

See Also

print

Examples

pop <- maxample("pop")
print(pop)
print(pop[, 1, ], drop = FALSE)
print(pop[, 1, ])

Read MAgPIE-object from file

Description

Reads a MAgPIE-file and converts it to a 3D array of the structure (cells,years,datacolumn)

Usage

read.magpie(
  file_name,
  file_folder = "",
  file_type = NULL,
  as.array = FALSE,
  comment.char = "*",
  check.names = FALSE,
  ...
)

Arguments

Details

See write.magpie for a list of supported file formats.

Value

Note

See write.magpie for the detailed structure of binary MAgPIE formats .m and .mz.

Author(s)

Jan Philipp Dietrich, Stephen Bi, Florian Humpenoeder, Pascal Sauer

See Also

"magpie", write.magpie

Read file in report format

Description

This function reads the content of a reporting file (a file in the model intercomparison file format *.mif) into a list of MAgPIE objects or a single MAgPIE object.

Usage

read.report(file, as.list = TRUE, showSeparatorWarning = TRUE)

Arguments

Details

The Model Intercomparison File Format (MIF) is the default file format for data produced by Integrated Assessment Models. It is based on the common format used for Model Intercomparison Projects such as EMF and SSP with some slight changes/clarifications in its definition. For interactions between models this format should be used. For everything else it is at least recommended to use this format, too.

Aim of this standardization is to achieve a more flexible and smooth communication between models and to facilitate the creation of aggregated outputs from integrated assessment scenario runs which then can easily be uploaded to external databases such as the EMF or SSP database. By using this standard most of the required decisions for a working input output interface between models have already been specified which significantly reduces the required work to get a new interaction running.

Definition

The format is characterized by the following features:

• The file ending is ".mif"

• The file is written in ASCII format

• Entries are separated with ";", every line ends with a ";"

• The file always contains a header

• The format of the header is: Model;Scenario;Region;Variable;Unit;<ADDITIONAL_COLUMNS>;<YEARS>;

The first 5 entries always have to exist, <ADDITIONAL_COLUMNS> is additional information which can be added optionally (e.g. "Description") and <YEARS> are the years for which data is delivered. <YEARS> are always written as 4 digit numbers. In the (very unlikely) case that a year before 1000 is used the number has to start with a 0, e.g. 0950. <ADDITIONAL_COLUMNS> can be anything, there are no further rules at the moment what it can contain. However, there are strict rules for naming these columns. Allowed are single names starting with a capital letter without special characters in it except "_" which is allowed. Examples: "Description" allowed, "More Description" not allowed, "More_Description" allowed, "123Description" not allowed, "Description123" allowed. Scripts using this format must be able to ignore additional columns. For years there are no specific limitations/requirements which years should be reported. Scripts dealing with this data must be able to work with different temporal resolutions. For variables basically everything can be reported here. Missing values have to be marked with "N/A".

Author(s)

Jan Philipp Dietrich

See Also

write.report

Examples

## Not run: 
read.report("report.csv")

## End(Not run)

Replace Non-Finite Data

Description

Replaces all instances of non-finite data (NA, NaN, Inf, and -Inf).

Usage

replace_non_finite(x, replace = 0)

Arguments

Value

A vector or magpie object, same as x.

Author(s)

Michaja Pehl

Examples

part  <- new.magpie(letters[1:3], years = 'y1995', names = 'foo')
total <- new.magpie(letters[1:3], years = 'y1995', names = 'foo')

part[,,]  <- c(0, 1, 2)
total[,,] <- c(0, 10, 10)

part / total

replace_non_finite(part / total)

Round-method for MAgPIE objects

Description

Round-method for MAgPIE-objects respectively. Works exactly as for arrays.

Usage

## S4 method for signature 'magpie'
round(x, digits = 0)

Arguments

Methods

x = "magpie"

works as round(x) for arrays.

~~ Methods for Function rowSums and rowMeans ~~

Description

~~ Methods for function rowSums and rowMeans~~

Usage

## S4 method for signature 'magpie'
rowSums(x, na.rm = FALSE, dims = 1, ...)

Arguments

Methods

list("signature(x = \"ANY\")")

normal rowSums and rowMeans method

list("signature(x = \"magpie\")")

classical method prepared to handle MAgPIE objects

Set Items

Description

Set items of a given (sub-)dimension of a MAgPIE-object

Usage

setItems(x, dim, value, maindim = NULL, raw = FALSE)

Arguments

Value

the manipulated MAgPIE object

Author(s)

Jan Philipp Dietrich

See Also

getItems

Examples

x <- maxample("pop")
setItems(x, "i", paste0("REG", 1:ncells(x)))

Get dataset names

Description

Extracts dataset names of a MAgPIE-object

Usage

## S4 method for signature 'magpie'
setNames(object = nm, nm)

Arguments

Details

setNames is a shortcut to use a MAgPIE object with manipulated data names. The setNames method uses the variable names "object" and "nm" in order to be consistent to the already existing function setNames.

Methods

list("signature(object = \"ANY\")")

normal setNames method

list("signature(object = \"magpie\")")

setNames for MAgPIE objects

See Also

getNames,

~~ Method for function show ~~

Description

Show a magpie object by calling the default show method, print, or str, depending on options("magclass_show_func").

Usage

## S4 method for signature 'magpie'
show(object)

Arguments

Author(s)

Pascal Sauer

sizeCheck

Description

Calculates expected magclass object length and checks that it stays below the limit defined with magclass_sizeLimit (default = 10^9). This is useful to prevent out of memory errors in case of unwanted object expansions Ignored if getOption("magclass_sizeLimit") is negative.

Usage

sizeCheck(dim)

Arguments

Author(s)

Jan Philipp Dietrich

Examples

pop <- maxample("pop")
magclass:::sizeCheck(dim(pop))

## Not run: 
magclass:::sizeCheck(c(6765L, 10946L, 17711L))

## End(Not run)

spatRasterToDataset

Description

Convert a SpatRaster to a SpatRasterDataset.

Usage

spatRasterToDataset(x)

Arguments

Value

SpatRasterDataset

Author(s)

Pascal Sauer

str

Description

str method for MAgPIE objects for conventient display of the structure of a magpie object.

Usage

## S3 method for class 'magpie'
str(object, ...)

Arguments

Details

In contrast to the default str this will not show the attributes of object@.Data as these contain only a duplicate of dimnames. Also, dim is not shown, because the information it provides is implicitly included in dimnames.

Author(s)

Pascal Sauer

See Also

str

Examples

str(maxample("pop"))

suppressSpecificWarnings

Description

Like suppressWarnings, but instead of suppressing all warnings this only suppresses warnings if they match the given pattern.

Usage

suppressSpecificWarnings(expr, regularExpr, fixed = FALSE)

Arguments

Value

The result of evaluating expr.

Author(s)

Pascal Sauer

See Also

suppressWarnings

time_interpolate

Description

Function to extrapolate missing years in MAgPIE objects.

Usage

time_interpolate(
  dataset,
  interpolated_year,
  integrate_interpolated_years = FALSE,
  extrapolation_type = "linear"
)

Arguments

Value

Uses linear extrapolation to estimate the values of the interpolated year, using the values of the two surrounding years. If the value is before or after the years in data, the two closest neighbours are used for extrapolation.

Author(s)

Benjamin Bodirsky, Jan Philipp Dietrich

See Also

convergence

Examples

p <- maxample("pop")
time_interpolate(p, "y2000", integrate = TRUE)
time_interpolate(p, c("y1980", "y2000"), integrate = TRUE, extrapolation_type = "constant")

splits a data.frame or vector of strings with form 'variable (unit)' into a data.frame with variable and unit separated

Description

splits a data.frame or vector of strings with form 'variable (unit)' into a data.frame with variable and unit separated

Usage

unitsplit(x, col = "variable")

Arguments

Author(s)

Jan Philipp Dietrich, Mika Pflüger, Oliver Richters

Unwrap

Description

Creates a higher dimensional array by separating all subdimensions in the third dimension of a MAgPIE object and returning them as separate dimension.

Usage

unwrap(x, sep = NULL)

Arguments

Value

An array with the full dimensionality of the original data

Author(s)

Jan Philipp Dietrich

See Also

wrap,fulldim

Examples

a <- as.magpie(array(1:6, c(3, 2), list(c("bla", "blub", "ble"), c("up", "down"))))
unwrap(a)

where

Description

Analysis function for magpie objects

Usage

where(x, plot = NULL)

Arguments

Value

A list of analysis parameters

Author(s)

Benjamin Leon Bodirsky, Jan Philipp Dietrich

See Also

whereplot in package luplot

Examples

p <- maxample("pop")
where(p > 500)

Wrap

Description

Reshape an array or a matrix by permuting and/or joining dimensions.

Usage

wrap(x, map = list(NA), sep = ".")

Arguments

Note

This function is extracted from the R.utils library which is licensed under LGPL>=2.1 and written by Henrik Bengtsson.

Author(s)

Henrik Bengtsson, Jan Philipp Dietrich

See Also

unwrap,fulldim

Write MAgPIE-object to file

Description

Writes a magpie object to a file. The file type is determined by the filename extension. The written file can be read again using read.magpie.

Usage

write.magpie(
  x,
  file_name,
  file_folder = "",
  file_type = NULL,
  append = FALSE,
  comment = NULL,
  comment.char = "*",
  mode = NULL,
  zname = "time",
  ...
)

Arguments

Details

This function supports writing the following file types:

• "cs2" is the new standard format for cellular data with or without header and the first columns (year,regiospatial) or only (regiospatial)

• "cs2b" is identical to "cs2" except that it will suppress the data name if it has only 1 element in the data dimension.

• "csv" is the standard format for regional data with or without header and the first columns (year,region,cellnumber) or only (region,cellnumber)

• "cs3" is another csv format which is specifically designed for multidimensional data for usage in GAMS.

• "cs4" alternative multidimensional format compatible to GAMS, in contrast to cs3 it can also handle sparse data

• "csvr", "cs2r", "cs3r", "cs4r" which are the same formats as the ones previously explained with the only difference that they have a REMIND compatible format

• "cs5" a more generalized version of cs4

• "rds" is an R-default format for storing R objects

• "m" (magpie) and "mz" (magpie zipped) are new formats developed to allow a less storage intensive management of MAgPIE-data. The only difference between both formats is that .mz is gzipped whereas .m is not compressed. So .mz needs less memory, whereas .m might have a higher compatibility to other languages

• "asc" is the ASCII grid format. It can only be applied for gridded data and writes one file per year per data column. In the case that more than one year and data column is supplied several files are written with the structure filename_year_datacolumn.asc

• "tif" is the GEOtiff format for gridded data.

• "grd" is the native raster format for gridded data.

• "nc" is the netCDF format for gridded data.

Note

The binary MAgPIE formats .m and .mz have the following content/structure (you only have to care for that if you want to implement read.magpie/write.magpie functions in other languages):

[ FileFormatVersion | Current file format version number (currently 6) | integer | 2 Byte ]
[ nchar_comment | Number of character bytes of the file comment | integer | 4 Byte ]
[ nbyte_metadata | Number of bytes of the serialized metadata (currently = 0) | integer | 4 Byte ]
[ nchar_sets | Number of characters bytes of all regionnames + 2 delimiter | integer | 2 Byte]
[ nyears | Number of years | integer | 2 Byte ]
[ yearList | All years of the dataset (0, if year is not present) | integer | 2*nyears Byte ]
[ ncells | Number of cells | integer | 4 Byte ]
[ nchar_cell | Number of characters bytes of all regionnames + (nreg-1) for delimiters | integer | 4 Byte ]
[ cells | Cell names saved as cell1\cell2 (\n is the delimiter) | character | 1*nchar_cell Byte ]
[ nelem | Total number of data elements | integer | 4 Byte ]
[ nchar_data | Number of char. bytes of all datanames + (ndata - 1) for delimiters | integer | 4 Byte ]
[ datanames | Names saved in the format data1\ndata2 (\n as del.) | character | 1*nchar_data Byte ]
[ data | Data of the MAgPIE array in vectorized form | numeric | 4*nelem Byte ]
[ comment | Comment with additional information about the data | character | 1*nchar_comment Byte ]
[ sets | Set names with \n as delimiter | character | 1*nchar_sets Byte]
[ metadata | serialized metadata information (currently not in use) | bytes | 1*nbyte_metadata Byte]

Author(s)

Jan Philipp Dietrich, Stephen Bi, Florian Humpenoeder, Pascal Sauer

See Also

"magpie", read.magpie,mbind

Examples

pop <- maxample("pop")
path <- tempfile(fileext = ".mz")
write.magpie(pop, path)
pop2 <- read.magpie(path)

Write file in report format

Description

This function writes the content of a MAgPIE object into a file or returns it directly using the reporting format as it is used for many model inter-comparisons.

Usage

write.report(
  x,
  file = NULL,
  model = NULL,
  scenario = NULL,
  unit = NULL,
  ndigit = 4,
  append = FALSE,
  skipempty = TRUE,
  extracols = NULL
)

Arguments

Author(s)

Jan Philipp Dietrich

See Also

read.report

Examples

write.report(maxample("pop"))

Write file in report format

Description

This function is deprecated, please use write.report instead.

Usage

write.report2(...)

Arguments

Author(s)

Jan Philipp Dietrich

See Also

write.report