Type: Package
Title: Download and Process GIMMS NDVI3g Data
Version: 1.2.2
URL: https://github.com/environmentalinformatics-marburg/gimms
BugReports: https://github.com/environmentalinformatics-marburg/gimms/issues
Maintainer: Florian Detsch <fdetsch@web.de>
Description: This is a set of functions to retrieve information about GIMMS NDVI3g files currently available online; download (and re-arrange, in the case of NDVI3g.v0) the half-monthly data sets; import downloaded files from ENVI binary (NDVI3g.v0) or NetCDF format (NDVI3g.v1) directly into R based on the widespread 'raster' package; conduct quality control; and generate monthly composites (e.g., maximum values) from the half-monthly input data. As a special gimmick, a method is included to conveniently apply the Mann-Kendall trend test upon 'Raster*' images, optionally featuring trend-free pre-whitening to account for lag-1 autocorrelation.
Depends: R (≥ 3.5), raster, methods
Imports: curl, parallel, Kendall, ncdf4, zyp
License: MIT + file LICENSE
LazyData: TRUE
RoxygenNote: 7.2.3
Encoding: UTF-8
Suggests: tinytest
NeedsCompilation: no
Packaged: 2023-08-09 18:38:06 UTC; Linda Schulte
Author: Florian Detsch [cre, aut]
Repository: CRAN
Date/Publication: 2023-08-09 19:00:02 UTC

Download and Process GIMMS NDVI3g Data

Description

Download and Process GIMMS NDVI3g Data

Details

This is a set of functions to retrieve information about GIMMS NDVI3g files currently available online; download (and re-arrange, in the case of NDVI3g.v0) the half-monthly data sets; import downloaded files from ENVI binary (NDVI3g.v0) or NetCDF format (NDVI3g.v1) directly into R based on the widespread raster package; conduct quality control; and generate monthly composites (e.g., maximum values) from the half-monthly input data. As a special gimmick, a method is included to conveniently apply the Mann-Kendall trend test upon Raster* images, optionally featuring trend-free pre-whitening to account for lag-1 autocorrelation.

Author(s)

Florian Detsch

References

Pinzon, JE & Tucker, CJ (2014). A Non-Stationary 1981-2012 AVHRR NDVI3g Time Series. Remote Sensing, 6(8), 6929-6960, doi:10.3390/rs6086929.

The National Center for Atmospheric Research (2018). A Big Earth Data Platform for Three Poles. Global GIMMS NDVI3g v1 dataset (1981-2015). Available online at http://poles.tpdc.ac.cn/en/data/9775f2b4-7370-4e5e-a537-3482c9a83d88/ (accessed on 2021-04-15).

Bale Mountains NDVI3g.v1

Description

Bale Mountains NDVI3g.v1.

Format

raster::RasterStack

Details

This dataset contains NDVI3g.v1 observations for the Bale Mountains National Park, southern Ethiopia (Jul 1981 to Dec 2015).

Download GIMMS NDVI3g Data

Description

Download GIMMS NDVI3g data, optionally for a given period of time. NDVI3g.v1 (NetCDF, until end 2015) is available from the "poles" and "ecocast" servers, whereas NDVI3g.v0 (ENVI binary, until end 2013) is "ecocast" and "nasanex" exclusive.

Usage

## S4 method for signature 'dateORmissing,dateORmissing'
downloadGimms(
  x,
  y,
  version = 1L,
  dsn = getwd(),
  overwrite = FALSE,
  quiet = TRUE,
  mode = "wb",
  cores = 1L,
  server = c("poles", "nasanex", "ecocast"),
  ...
)

## S4 method for signature 'numericORmissing,numericORmissing'
downloadGimms(
  x,
  y,
  version = 1L,
  dsn = getwd(),
  overwrite = FALSE,
  quiet = TRUE,
  mode = "wb",
  cores = 1L,
  server = c("poles", "nasanex", "ecocast"),
  ...
)

## S4 method for signature 'character,ANY'
downloadGimms(
  x,
  dsn = getwd(),
  overwrite = FALSE,
  quiet = TRUE,
  mode = "wb",
  cores = 1L,
  ...
)

## S4 method for signature 'missing,missing'
downloadGimms(
  version = 1L,
  dsn = getwd(),
  overwrite = FALSE,
  quiet = TRUE,
  mode = "wb",
  cores = 1L,
  server = c("poles", "nasanex", "ecocast"),
  ...
)

Arguments

x

Start time for data download as either Date object (e.g., as.Date("2000-01-01")) or numeric year (e.g., 2000). Alternatively, a character vector of online filepaths to download created from updateInventory. If missing, all files available online are being downloaded.

y

End time for data download as either Date object or numeric year. Ignored if 'x' is a character object or missing.

version

integer (or any other convertible class), defaults to 1L. Specifies desired GIMMS NDVI3g product version, see 'Details' in updateInventory. Ignored if 'x' is a character object.

dsn

character, defaults to the current working directory. Target folder for file download.

overwrite

logical, defaults to FALSE. If TRUE, identically named files in 'dsn' will be overwritten.

quiet

logical. If TRUE (default), console output is reduced.

mode

character. See download.file.

cores

integer, defaults to 1L. Number of cores used for parallel processing. Note that a fast internet connection is required in order for parallelization to take effect.

server

See updateInventory.

...

Further arguments passed to download.file, e.g. 'method'.

Value

A character vector of local filepaths.

See Also

updateInventory, download.file.

Examples

## Not run: 
tmp <- tempdir()

## 'Date' method
gimms_files_date <- downloadGimms(x = as.Date("2000-01-01"),
                                  y = as.Date("2000-12-31"),
                                  dsn = tmp)

## 'numeric' method (i.e., particular years)
gimms_files_year <- downloadGimms(x = 2000,
                                  y = 2002,
                                  dsn = tmp)

## 'character' method (i.e., particular files)
ecocast <- system.file("extdata", "inventory_ecv1.rds", package = "gimms")
gimms_files_char <- readRDS(ecocast)
gimms_files_char <- downloadGimms(x = gimms_files_char[1:6],
                                  dsn = tmp)

## 'missing' method (i.e., entire collection)
gimms_files_full <- downloadGimms(dsn = tmp)

## End(Not run)

Kilimanjaro NDVI3g.v0

Description

Kilimanjaro NDVI3g.v0.

Format

raster::RasterStack

Details

This dataset contains NDVI3g.v0 observations for the Kilimanjaro region, northern Tanzania (Jul 1981 to Dec 2013).

Calculate Monthly Composite Images

Description

Based on a user-defined function, e.g. max for maximum value composites (MVC), aggregate half-monthly GIMMS data sets to monthly composites.

Usage

## S4 method for signature 'RasterStackBrick'
monthlyComposite(x, indices, fun = max, cores = 1L, filename = "", ...)

## S4 method for signature 'character'
monthlyComposite(
  x,
  version = 1L,
  pos1 = ifelse(version == 1, 15L, 4L),
  pos2 = ifelse(version == 1, 23L, 8L),
  fun = max,
  cores = 1L,
  filename = "",
  ...
)

Arguments

x

Multi-layered Raster* object or character vector of filenames. If the latter applies and 'pos1', 'pos2' are not specified, the function will try to retrieve monthly indices from monthlyIndices. Note that the specification of NDVI3g.v0 ENVI binary files is hereby not allowed and in such a case, rasterizeGimms should be run beforehand.

indices

numeric. Indices to denote layers or files from identical months.

fun

function. Used to calculate monthly composite layers, defaults to max. Note that a separate 'na.rm' argument is passed down to stackApply via '...' and hence should not be included here.

cores

integer. Number of cores for parallel computing.

filename

character. Optional output filename passed to stackApply.

...

Further arguments passed to stackApply (i.e., 'na.rm') and its underlying writeRaster call.

version

integer (or any other class convertible to integer). Specifies GIMMS NDVI3g product version, see 'Details' in updateInventory.

pos1, pos2

integer. If 'x' is a vector of filenames, the first and last element of the date string to build monthly indices from. Defaults to the 'version'-specific NDVI3g naming convention, see monthlyIndices.

Value

If length(x) == 2, a single RasterLayer object, else a RasterStack object with monthly composite layers.

See Also

stackApply, monthlyIndices, writeRaster.

Examples

data("bale3g.v1")

## select layers from 1981 only
fls <- system.file("extdata/inventory_ecv1.rds", package = "gimms")
fls <- readRDS(fls)[1]
rst <- bale3g.v1[[1:12]]

## aggregate to monthly mvc layers
mvc <- monthlyComposite(rst, indices = monthlyIndices(fls))

Create Monthly Indices from NDVI3g Files

Description

Create numeric monthly indices from (local or online) GIMMS NDVI3g filenames as input for the monthlyComposite function.

Usage

monthlyIndices(
  x,
  version = 1L,
  pos1 = ifelse(version == 1, 15L, 4L),
  pos2 = ifelse(version == 1, 23L, 8L),
  timestamp = FALSE,
  ...
)

Arguments

x

character. Vector of (local or online) filenames.

version

integer (or any other class convertible to integer). Specifies GIMMS NDVI3g product version, see 'Details' in updateInventory.

pos1, pos2

numeric. The first and last element of the date string in 'x', defaults to the GIMMS naming convention of the specified product version.

timestamp

logical, defaults to FALSE. If TRUE, an actual Date object is returned rather than a numeric vector of indices.

...

Currently not used.

Value

A numeric vector with unique monthly indices or, if timestamp = TRUE, the actual timestamps as Date objects.

See Also

monthlyComposite.

Examples

## NDVI3g.v1
gimms_files_v1 <- readRDS(system.file("extdata", "inventory_ecv1.rds", package = "gimms"))

monthlyIndices(gimms_files_v1[1], version = 1)                   # indices
monthlyIndices(gimms_files_v1[1], version = 1, timestamp = TRUE) # dates

## Similarly, NDVI3g.v0
gimms_files_v0 <- readRDS(system.file("extdata", "inventory_ecv0.rds", package = "gimms"))
gimms_files_v0 <- rearrangeFiles(gimms_files_v0) # reorder by date

monthlyIndices(gimms_files_v0[1:12], version = 0)
monthlyIndices(gimms_files_v0[1:12], version = 0, timestamp = TRUE)

Create Traditional GIMMS NDVI3g File Names

Description

Reformat the names of local NDVI3g.v1 files according to "traditional" NDVI3g.v0 naming convention.

Usage

oldNaming(x, suffix = "")

Arguments

x

character. GIMMS NDVI3g.v1 file names to reformat, see 'Details'.

suffix

character. Optional suffix to append to the resulting file names, e.g. ".tif" for GeoTiff files (see also writeFormats).

Details

The satellite number, which was an essential constituent of the NDVI3g.v0 file naming convention (see 'References'), has been removed from NDVI3g.v1 file names. However, it can easily be accessed via ncvar_get given that the file has previously been downloaded. If an NDVI3g.v1 file specified to 'x' is not available locally, by contrast, the satellite number is retrieved from the built-in NDVI3g.v0 file inventory (until end 2013; see 'Examples') and newer files are tagged with "XX" indicating missing information.

Value

A character vector of file names following the "traditional" NDVI3g.v0 naming convention.

References

http://nasanex.s3.amazonaws.com/AVHRR/GIMMS/3G/00READMEgeo.txt (accessed on 2021-04-15).

See Also

nc_open, ncvar_get.

Examples

fls <- system.file("extdata", "inventory_ecv1.rds", package = "gimms")
fls <- basename(readRDS(fls))

oldNaming(fls[64:65], suffix = ".tif") # adds .tif extension
oldNaming(fls[65:66])                  # tags newer files with 'XX'

Perform Quality Control on GIMMS NDVI3g Data

Description

Perform quality control on GIMMS NDVI3g data based on the companion flag information. This is merely a wrapper around overlay and, since quality control is readily available through rasterizeGimms, it is strongly recommended to use the latter function for this purpose.

Usage

## S4 method for signature 'RasterStackBrick'
qualityControl(x, keep = NULL, filename = "", ...)

Arguments

x

A single 2-layered RasterStack object (NDVI and flags).

keep

integer. Accepted flag values (see 'Details').

filename

character. Optional output filename.

...

Further arguments passed to writeRaster.

Details

If 'keep' is missing, the function will automatically skip quality control and return the input object.

Value

A quality-controlled 'RasterLayer' object.

See Also

rasterizeGimms, overlay.

Examples

## Not run: 
tmp <- tempdir()

## Download NDVI3g.v1 sample data
ecocast <- system.file("extdata", "inventory_ecv1.rds", package = "gimms")
gimms_files <- downloadGimms(readRDS(ecocast)[1], dsn = tmp)

## Import data as 'Raster*' objects
ndvi <- raster::raster(gimms_files, varname = "ndvi")
ndvi[ndvi[] %in% c(-32768, -3000)] <- NA
ndvi <- ndvi / 1e4

flag <- floor(raster::raster(gimms_files, varname = "percentile") / 2e3)

## Perform quality control and visualize
to_check <- stack(ndvi[[1]], flag[[1]])
qcl <- qualityControl(to_check, keep = 0)

plot(qcl)

## End(Not run)

## see also 'Examples' section in ?rasterizeGimms for automated quality check

Rasterize GIMMS NDVI3g Data

Description

Import GIMMS NDVI3g (binary or NetCDF) data into R as Raster* objects.

Usage

rasterizeGimms(
  x,
  ext = NULL,
  snap = "out",
  keep = NULL,
  split = FALSE,
  cores = 1L,
  filename = "",
  ...
)

Arguments

x

character. Vector of local filepaths. Note that product versions must not be mixed, i.e. 'x' should represent files originating from either NDVI3g.v1 or NDVI3g.v0 only.

ext

Extent, or any object from which an Extent can be extracted, see crop.

snap

character, defaults to "out". Other available options are "in" and "near", see crop.

keep

integer. Flag values of NDVI3g pixels to spare during quality control. Pixels with non-included flag values are set to NA. If not specified (i.e., NULL; default), quality control is skipped.

split

logical, defaults to FALSE. If TRUE, a list of RasterStack objects (of length(x)) is returned rather than a single RasterStack.

cores

integer. Number of cores for parallel computing.

filename

character. Optional output filename. If specified, this must be of the same length as 'x'.

...

Further arguments passed to writeRaster.

Value

If split = TRUE, a list of NDVI3g RasterStack objects corresponding to the files specified in 'x'; else a single NDVI3g RasterStack object.

See Also

crop, qualityControl, writeRaster.

Examples

## Not run: 
tmp <- tempdir()

## Download NDVI3g.v1 sample data
gimms_files <- downloadGimms(x = as.Date("2000-01-01"),
                             y = as.Date("2000-12-31"),
                             dsn = tmp)

## Extent for clipping
shp <- getData("GADM", country = "DEU", level = 0, path = tmp)

## Rasterize without quality control
gimms_raster <- rasterizeGimms(x = gimms_files,
                               ext = shp) # clipping
plot(gimms_raster[[1]])
lines(shp)

## Rasterize with quality control
gimms_rasterq <- rasterizeGimms(x = gimms_files,
                                ext = shp, # clipping
                                keep = 0)  # quality control
plot(gimms_rasterq[[1]])
lines(shp)

## End(Not run)

Rearrange GIMMS NDVI3g.v0 Files

Description

Rearrange local GIMMS NDVI3g.v0 files in ascending order of time. Since the naming convention has significantly changed towards NDVI3g.v1, such a measure should only be relevant for older file formats.

Usage

rearrangeFiles(
  x,
  dsn = getwd(),
  pattern = "^geo.*.VI3g$",
  pos = c(4, 6, 11),
  ...
)

Arguments

x

character. Vector of local filepaths. If missing, 'dsn' will be searched for available files via pattern matching.

dsn

character, defaults to the current working directory. Path to look for GIMMS-related data if 'x' is missing.

pattern

character, defaults to "^geo.*.VI3g$" for standard NDVI3g.v0 files. A regular expression passed to list.files.

pos

integer, defaults to c(4, 6, 11) for standard NDVI3g.v0 files. The start positions of year, month and part of the month ('a' or 'b') in the target GIMMS files.

...

Further arguments passed to list.files.

Value

A character vector of filepaths arranged in ascending order of time.

See Also

list.files

Compute (Pre-Whitened) Kendall's \tau

Description

Apply the Mann-Kendall trend test (Mann, 1945) to a series of observations and return Kendall's \tau (Kendall, 1938) based on a predefined significance level. In contrast to other readily available implementations, it is up to the user to decide whether or not to apply pre-whitening as described in the zyp package vignette (Bronaugh and Werner, 2013).

Usage

## S4 method for signature 'numeric'
significantTau(
  x,
  p = 0.001,
  prewhitening = TRUE,
  method = c("yuepilon", "zhang"),
  df = FALSE
)

## S4 method for signature 'RasterStackBrick'
significantTau(
  x,
  p = 0.001,
  prewhitening = TRUE,
  method = c("yuepilon", "zhang"),
  filename = "",
  ...
)

Arguments

x

Either a Raster* object or a numeric vector.

p

numeric, defaults to 0.001. Significance level to be tested.

prewhitening

logical. If TRUE (default), pre-whitening is applied prior to the Mann-Kendall trend test.

method

character. The prewhitening method to apply, see zyp.trend.vector.

df

logical, defaults to FALSE. If TRUE, a data.frame holding the value of Kendall's \tau and the referring significance level.

filename

character. Optional output filename.

...

Further arguments passed to writeRaster.

Details

If available, the function will automatically use open multi-core clusters for parallel processing (see beginCluster and Examples).

Value

References

Kendall, M.G. (1938). A new measure of rank correlation. Biometrika 30(1/2), 81-93, doi: 10.2307/2332226.

Mann, H.B. (1945). Nonparametric tests against trend. Econometrica 13(3), 245-259, doi: 10.2307/1907187.

Zhang, X., Vincent, L.A., Hogg, W.D. and A. Niitsoo (2000). Temperature and Precipitation Trends in Canada during the 20th Century. Atmosphere-Ocean 38(3), 395-429, doi:10.1080/07055900.2000.9649654.

Yue, S., Pilon, P., Phinney, B. and G. Cavadias (2002). The influence of autocorrelation on the ability to detect trend in hydrological series. Hydrological Processes 16, 1807-1829,doi:10.1002/hyp.1095.

See Also

MannKendall, zyp.trend.vector.

Examples

## Example taken from ?Kendall::MannKendall
library(Kendall)
data(PrecipGL)
plot(PrecipGL)

## Mann-Kendall trend test without pre-whitening
x <- as.numeric(PrecipGL)
significantTau(x, p = 0.001, prewhitening = FALSE, df = TRUE)

## Mann-Kendall trend test with pre-whitening
significantTau(x, p = 0.001, prewhitening = TRUE, df = TRUE)

#############################################################################
### use case: significant mann-kendall trends in ndvi3g.v0          #########
#############################################################################

## Not run: 
## Sample data from 1982 to 2013
data("kili3g.v0")
rst <- kili3g.v0[[13:nlayers(kili3g.v0)]]

## Remove seasonal signal
library(remote)
dsn <- deseason(rst, cycle.window = 24)

## Apply trend-free pre-whitened Mann-Kendall test (note that
## non-significant pixels are set to NA)
trd1 <- significantTau(dsn, p = 0.01, prewhitening = TRUE)
plot(trd1)

## Or, alternatively, use multi-core functionality
cores <- parallel::detectCores() - 1
if (require(snow)) {
  beginCluster(cores)
  trd2 <- significantTau(dsn, p = 0.01, prewhitening = TRUE)
  endCluster()
}

## End(Not run)

Update GIMMS NDVI3g File Inventory

Description

Download the latest version of the GIMMS NDVI3g file inventory from the National Center for Atmospheric Research, NASA Earth Exchange (NEX) Amazon AWS or Ames Ecological Forecasting Lab (ECOCAST). If the specified endpoint is not reachable (e.g., if there is no active internet connection), the latest local version of the file inventory is used.

Usage

updateInventory(
  server = c("poles", "nasanex", "ecocast"),
  version = 1L,
  quiet = FALSE
)

Arguments

server

character. Specifies the remote server to use. Currently available options are "poles" (default), "nasanex" and "ecocast".

version

integer (or any other convertible class), defaults to 1L. Specifies desired GIMMS NDVI3g product version, see 'Details'. Currently ignored if server != "ecocast".

quiet

logical, defaults to FALSE. If TRUE, console output is disabled.

Details

GIMMS NDVI3g.v1 is currently available from ECOCAST and The National Center for Atmospheric Research until end 2015 and comes in NetCDF (.nc4) format. In contrast, NDVI3g.v0 is available as ENVI binary imagery and available from ECOCAST (NASANEX) until end 2013 (2012) only.

Value

A character vector of online filepaths.

References

The National Center for Atmospheric Research (2018). A Big Earth Data Platform for Three Poles. Global GIMMS NDVI3g v1 dataset (1981-2015). Available online at http://poles.tpdc.ac.cn/en/data/9775f2b4-7370-4e5e-a537-3482c9a83d88/ (accessed on 2021-04-15).

See Also

rearrangeFiles.

Examples

## Not run: 
updateInventory()
updateInventory(server = "nasanex", version = 0)

## End(Not run)

## note that local versions of the online file inventories are also available
ofl_ecv1 <- system.file("extdata", "inventory_ecv1.rds", package = "gimms")
readRDS(ofl_ecv1)

ofl_v0 <- system.file("extdata", "inventory_ecv0.rds", package = "gimms")
readRDS(ofl_v0)

ofl_plv1 <- system.file("extdata", "inventory_plv1.rds", package = "gimms")
readRDS(ofl_plv1)