Put data places with {pins}

This page shows how to use pins to store data in various platforms as needed for the pipelines.

Posit Connect

This section goes through how to put data on Posit Connect using pins.

Connecting to Posit Connect

For the smoothest experience, we recommend that you authenticate using environment variables. The two variables you will need are CONNECT_SERVER and CONNECT_API_KEY.

Tip

The function usethis::edit_r_environ() can be very handy to open .Renviron file to specify your environment variables.

CONNECT_SERVER is the URL of the posit connect page. So if your connect server is accessed through https://example.com/connect/#/content/ then you can find CONNECT_SERVER by removing connect/ and everything that follows it, leaving you with https://example.com/.

CONNECT_API_KEY is created through your Connect server.

  1. Click on your name in the upper right.
  2. Click API keys.
  3. Click New API Key.
  4. Give your API key a name, click Create Key.

Once you have those two, you can add them to your .Renviron file in the following format:

CONNECT_SERVER=https://example.com/
CONNECT_API_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxx

Note that you don’t want to put quotes around the values.

Creating the board

library(pins)

board <- board_connect()

We load in the data we created in data-prep

flights <- readr::read_csv("data/laxflights2022_lite.csv")

Pinning data set to Connect

Now that we have the data set, we can send it to Posit Connect using the board we created, the data set and a name of our choosing.

board |>
  pin_write(flights, name = "laxflights2022_lite", type = "rds")

Amazon S3

This section goes through how to put data on Amazon S3 using pins.

Connecting to Amazon S3

For the smoothest experience, we recommend that you authenticate using environment variables. The two variables you will need are AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.

Warning

Depending on your S3 setup, you will need to use additional variables to connect. Please see https://github.com/paws-r/paws/blob/main/docs/credentials.md and this pins issue for help if the following paragraphs doesn’t work for you.

Tip

The function usethis::edit_r_environ() can be very handy to open .Renviron file to specify your environment variables.

You can find both of these keys in the same location.

  1. Open the AWS Console
  2. Click on your username near the top right and select Security Credentials
  3. Click on Users in the sidebar
  4. Click on your username
  5. Click on the Security Credentials tab
  6. Click Create Access Key
  7. Click Show User Security Credentials

Once you have those two, you can add them to your .Renviron file in the following format:

AWS_SECRET_ACCESS_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxx
AWS_ACCESS_KEY_ID=xxxxxxxxxxxxxxxxxxxxxxxxxxxx

Note that you don’t want to put quotes around the values.

Creating the board

library(pins)

board <- board_s3(
  "tidymodels-pipeline-example",
  region = "us-west-1"
)

We load in the data we created in data-prep

flights <- readr::read_csv("data/laxflights2022_lite.csv")

Pinning data set to Amazon S3

Now that we have the data set, we can send it to Amazon S3 using the board we created, the data set and a name of our choosing.

board |>
  pin_write(flights, name = "laxflights2022_lite", type = "rds")

Azure

This section goes through how to put data on Azure storage using pins.

Connecting to Azure

For the smoothest experience, we recommend that you authenticate using environment variables. The two variables you will need are CONNECT_SERVER and CONNECT_API_KEY.

Tip

The function usethis::edit_r_environ() can be very handy to open .Renviron file to specify your environment variables.

CONNECT_SERVER is the URL of the posit connect page. So if your connect server is accessed through https://example.com/connect/#/content/ then you can find CONNECT_SERVER by removing connect/ and everything that follows it, leaving you with https://example.com/.

CONNECT_API_KEY is created through your Connect server.

  1. Click on your name in the upper right.
  2. Click API keys.
  3. Click New API Key.
  4. Give your API key a name, click Create Key.

Once you have those two, you can add them to your .Renviron file in the following format:

CONNECT_SERVER=https://example.com/
CONNECT_API_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxx

Note that you don’t want to put quotes around the values.

Creating the board

library(pins)
library(AzureStor)

container <-
  storage_container(
    endpoint = Sys.getenv("AZURE_CONTAINER_ENDPOINT"),
    sas = Sys.getenv("AZURE_SAS_KEY")
  )

board <- board_azure(container)

We load in the data we created in data-prep

flights <- readr::read_csv("data/laxflights2022_lite.csv")

Pinning data set to Azure storage container

Now that we have the data set, we can send it to Azure using the board we created, the data set and a name of our choosing.

board |>
  pin_write(flights, name = "laxflights2022_lite", type = "rds")