| Title: | Download Files from 'AWS S3' |
| Version: | 1.1.0 |
| Description: | Download files hosted on 'AWS S3' (Amazon Web Services Simple Storage Service; https://aws.amazon.com/s3/) to a local directory based on their URI. Avoid downloading files that are already present locally. Allow for customization of where to store downloaded files. |
| License: | MIT + file LICENSE |
| Encoding: | UTF-8 |
| RoxygenNote: | 7.3.2 |
| Imports: | cli, prettyunits, fs, rlang, httr, glue, purrr (≥ 0.3.4), dplyr, |
| Suggests: | aws.signature, testthat, digest, withr |
| URL: | https://github.com/brokamp-group/s3 |
| BugReports: | https://github.com/brokamp-group/s3/issues |
| NeedsCompilation: | no |
| Packaged: | 2024-09-04 12:26:10 UTC; cole |
| Author: | Cole Brokamp 
[aut, cre],
Erika Rasnick [aut],
Robrecht Cannoodt [aut] |
| Maintainer: | Cole Brokamp <cole.brokamp@gmail.com> |
| Repository: | CRAN |
| Date/Publication: | 2024-09-04 12:50:06 UTC |
s3: Download Files from 'AWS S3'
Description
Download files hosted on 'AWS S3' (Amazon Web Services Simple Storage Service; https://aws.amazon.com/s3/) to a local directory based on their URI. Avoid downloading files that are already present locally. Allow for customization of where to store downloaded files.
Author(s)
Maintainer: Cole Brokamp cole.brokamp@gmail.com (ORCID)
Authors:
Erika Rasnick erika.rasnick@cchmc.org
Robrecht Cannoodt
See Also
Useful links:
download s3 file
Description
Files are downloaded to the R user data directory (i.e., tools::R_user_dir("s3", "data")) so they
can be cached across all of an R user's sessions and projects.
Specify an alternative download location by setting the R_USER_DATA_DIR environment variable
(see ?tools::R_user_dir) or by using the data_dir argument.
Usage
s3_get(
s3_uri,
region = "us-east-2",
quiet = FALSE,
progress = FALSE,
force = FALSE,
public = FALSE,
data_dir = tools::R_user_dir("s3", "data")
)
Arguments
s3_uri |
URI for an S3 object |
region |
AWS region for bucket containing the file (defaults to "us-east-2", but only required for private files) |
quiet |
suppress messages? |
progress |
show download progress? (currently only for public objects) |
force |
force download to overwrite existing S3 object |
public |
defaults to FALSE; if TRUE, ignore any environment variables specifying AWS credentials and attempt to download the file as publicly available |
data_dir |
root directory for downloaded files (defaults to |
Value
a character string that is the file path to the downloaded file (invisibly)
Examples
Sys.setenv("R_USER_DATA_DIR" = tempdir())
the_file <- s3_get(s3_uri = "s3://geomarker/testing_downloads/mtcars.rds")
s3_get("s3://geomarker/testing_downloads/mtcars.rds") |>
readRDS()
unlink(the_file)
download several s3 files
Description
Progress messages for downloading several S3 objects at once cannot be silenced. Like s3_get, S3 objects that have already been downloaded will not be re downloaded
Usage
s3_get_files(
s3_uri,
region = "us-east-2",
progress = FALSE,
force = FALSE,
confirm = TRUE,
public = FALSE,
data_dir = tools::R_user_dir("s3", "data")
)
Arguments
s3_uri |
vector of S3 object URIs |
region |
AWS region for bucket containing the file (defaults to "us-east-2", but only required for private files) |
progress |
show download progress for each individual file? (currently only for public objects) |
force |
force download to overwrite existing S3 objects |
confirm |
ask user to interactively confirm downloads? (only possible when session is interactive) |
public |
defaults to FALSE; if TRUE, ignore any environment variables specifying AWS credentials and attempt to download the file as publicly available |
data_dir |
root directory for downloaded files (defaults to |
Value
data.frame (or tibble) with s3_uris and corresponding file paths to downloaded files (invisibly)
Examples
Sys.setenv("R_USER_DATA_DIR" = tempdir())
the_files <- s3_get_files(c(
"s3://geomarker/testing_downloads/mtcars.rds",
"s3://geomarker/testing_downloads/mtcars.fst"
))
unlink(the_files$file_path)