Title: Data Source Catalogues Online for Southern Ocean Ecosystem Research
Version: 0.5.0
Description: Obtains lists of files of remote sensing collections for Southern Ocean surface properties. Commonly used data sources of sea surface temperature, sea ice concentration, and altimetry products such as sea surface height and sea surface currents are cached in object storage on the Pawsey Supercomputing Research Centre facility. Patterns of working to retrieve data from these object storage catalogues are described. The catalogues include complete collections of datasets Reynolds et al. (2008) "NOAA Optimum Interpolation Sea Surface Temperature (OISST) Analysis, Version 2.1" <doi:10.7289/V5SQ8XB5>, Spreen et al. (2008) "Artist Advanced Microwave Scanning Radiometer for Earth Observing System (AMSR-E) sea ice concentration" <doi:10.1029/2005JC003384>. In future releases helpers will be added to identify particular data collections and target specific dates for earth observation data for reading, as well as helpers to retrieve data set citation and provenance details. This work was supported by resources provided by the Pawsey Supercomputing Research Centre with funding from the Australian Government and the Government of Western Australia. This software was developed by the Integrated Digital East Antarctica program of the Australian Antarctic Division.
License: MIT + file LICENSE
Encoding: UTF-8
Language: en-US
RoxygenNote: 7.3.2
Imports: arrow, dplyr, S7, tibble
URL: https://github.com/mdsumner/sooty
BugReports: https://github.com/mdsumner/sooty/issues
Suggests: curl, spelling, testthat (≥ 3.0.0)
Config/testthat/edition: 3
NeedsCompilation: no
Packaged: 2025-05-22 09:10:58 UTC; mdsumner
Author: Michael D. Sumner [aut, cre], Aleks Terauds [cph, ctb] (Provided logo photo from p116 of 'Subantarctic wilderness: Macquarie Island, 2007(978-1741753028)')
Maintainer: Michael D. Sumner <michael.sumner@aad.gov.au>
Repository: CRAN
Date/Publication: 2025-05-22 09:30:02 UTC

List available datasets

Description

In sooty_files() the data source files are grouped by Dataset, this is the list of unique datasets, values that can be used in ⁠datasource(<name>)⁠.

Usage

available_datasets()

Value

character vector of available dataset ids for datasource()

Examples

available_datasets()

Create a datasource object. A data source provides a list of files that together comprise a dataset.

Description

Generates an object whose "@id" property may be set, which then communicates with a dataset of files/objects that sooty knows about.

Usage

datasource(id = NA_character_)

dataset(...)

Arguments

id

a dataset label, see 'datasource()@available_datasources' (get, and settable)

...

only used by deprecated function, will become defunct

Details

Compare 'curated' to 'sooty_files(curated = FALSE)', if it is curated sooty knows what dataset it belongs to, and otherwise it's just the huge list of files we're interested in for our work. All of the curation is done outside of sooty.

The following properties are available via the @ slot:

Note

This was originally called dataset() which usage has now been deprecated.

Examples

## available dataset names
if (interactive()) {
 available_datasets()
}
## set to one of those
ds  <- datasource("ghrsst-tif")
## access the 'ds@source' slot, files with 'date','source' (GDAL-readable)

Obtain object storage catalogues as a dataframe of file/object identifiers.

Description

The object (file) catalogue of available sources is stored in Parquet format on Pawsey object storage. This function retrieves the curated catalogue, or the raw catalogue.

Usage

sooty_files(curated = TRUE)

Arguments

curated

logical TRUE by default, set to FALSE to return raw object catalogue

Details

In the curated case, the returned data frame has columns 'date', 'source' which are the main useful fields, these describe the date of the data in the file, and its full URI (Uniform Resource Identifier) source on S3 object storage. There are also fields 'Bucket', 'Key', and 'protocol' from which 'source' is constructed.

The original publisher URI can be reconstructed by replacing the value of 'protocol' in 'source' with 'https://'.

The public object URI can be reconstructed by replacing the value of 'protocol' in 'source' with 'https://projects.pawsey.org.au'.

Value

a data frame, see details

Examples

if (interactive()) {
  sooty_files(FALSE)
}

sooty_files()