Use Cloud Storage

On this page

CockroachDB constructs a secure API call to the cloud storage specified in a URL passed to one of the following statements:

• BACKUP
• RESTORE
• IMPORT
• EXPORT
• CREATE CHANGEFEED

URL format

URLs for the files you want to import must use the format shown below. For examples, see Example file URLs.

[scheme]://[host]/[path]?[parameters]

You can create an external connection to represent an external storage or sink URI. This allows you to specify the external connection's name in statements rather than the provider-specific URI. For detail on using external connections, see the CREATE EXTERNAL CONNECTION page.

The following table provides a list of the parameters supported by each storage scheme. For detail on authenticating to each cloud storage provider, see the Cloud Storage Authentication page.

¹ The file system backup location on the NFS drive is relative to the path specified by the --external-io-dir flag set while starting the node. If the flag is set to disabled, then imports from local directories and NFS drives are disabled.

² Using a nodeID is required and the data files will be in the extern directory of the specified node. In most cases (including single-node clusters), using nodelocal://1/<path> is sufficient. If every node has the --external-io-dir flag pointed to a common NFS mount, or other form of network-backed, shared, or synchronized storage, you can use the word self instead of a node ID to indicate that each node should write individual data files to its own extern directory.

³ The AWS_REGION parameter is optional since it is not a required parameter for most S3-compatible services. Specify the parameter only if your S3-compatible service requires it.

Example file URLs

Example URLs for BACKUP, RESTORE, changefeeds, or EXPORT given a bucket or container name of acme-co and an employees subdirectory:

For detail on forming the URLs and the different authentication methods, refer to the Cloud Storage Authentication page.

Example URLs for IMPORT and given a bucket or container name of acme-co and a filename of employees:

Example URLs for CREATE CHANGEFEED:

Encryption

Transport Layer Security (TLS) is used for encryption in transit when transmitting data to or from Amazon S3, Google Cloud Storage, and Azure.

For encryption at rest, if your cloud provider offers transparent data encryption, you can use that to ensure that your backups are not stored on disk in cleartext.

• Amazon S3
• Azure Blob Storage
• Google Cloud Storage

CockroachDB also provides client-side encryption of backup data, for more information, see Take and Restore Encrypted Backups.

Storage permissions

This section describes the minimum permissions required to run CockroachDB operations. While we provide the required permissions for Amazon S3 and Google Cloud Storage, the provider's documentation provides detail on the setup process and different options regarding access management.

Depending on the actions an operation performs, it will require different access permissions to a cloud storage bucket.

This table outlines the actions that each operation performs against the storage bucket:

{
    "Version": "2012-10-17",
    "Id": "Example_Policy",
    "Statement": [
        {
            "Sid": "ExampleStatement01",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::{ACCOUNT_ID}:user/{USER}"
            },
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::{BUCKET_NAME}",
                "arn:aws:s3:::{BUCKET_NAME}/*"
            ]
        }
    ]
}

Additional cloud storage feature support

Immutable storage

To complete a backup successfully, BACKUP requires read and write permissions to cloud storage buckets. Delete and overwrite permissions are not required. As a result, you can write backups to cloud storage buckets with object locking enabled. This allows you to store backup data using a write-once-read-many (WORM) model, which refers to storage that prevents any kind of deletion, encryption or modification to the objects once written.

For specific cloud-storage provider documentation, see the following:

• AWS S3 Object Lock
• Retention policies and Bucket Lock in Google Cloud Storage
• Immutable storage in Azure Storage

Amazon S3 storage classes

When storing objects in Amazon S3 buckets during backups, exports, and changefeeds, you can specify the S3_STORAGE_CLASS={class} parameter in the URI to configure a storage class type.

The following S3 connection URI uses the INTELLIGENT_TIERING storage class:

's3://{BUCKET NAME}?AWS_ACCESS_KEY_ID={KEY ID}&AWS_SECRET_ACCESS_KEY={SECRET ACCESS KEY}&S3_STORAGE_CLASS=INTELLIGENT_TIERING'

While Cockroach Labs supports configuring an AWS storage class, we only test against S3 Standard. We recommend implementing your own testing with other storage classes.

Incremental backups and archive storage classes

Incremental backups are not compatible with the S3 Glacier Flexible Retrieval or Glacier Deep Archive storage classes. Incremental backups require the reading of previous backups on an ad-hoc basis, which is not possible with backup files already in Glacier Flexible Retrieval or Glacier Deep Archive. This is because these storage classes do not allow immediate access to an S3 object without first restoring the archived object to its S3 bucket.

Refer to the AWS documentation on Restoring an archived object for steps.

When you are restoring archived backup files from Glacier Flexible Retrieval or Glacier Deep Archive back to an S3 bucket, you must restore both the full backup and incremental backup layers for that backup. By default, CockroachDB stores the incremental backup layers in a separate top-level directory at the backup's storage location. Refer to Backup collections for detail on the backup directory structure at its storage location.

Once you have restored all layers of a backup's archived files back to its S3 bucket, you can then restore the backup to your CockroachDB cluster.

Supported storage classes

This table lists the valid CockroachDB parameters that map to an S3 storage class:

You can view an object's storage class in the Amazon S3 Console from the object's Properties tab. Alternatively, use the AWS CLI to list objects in a bucket, which will also display the storage class:

aws s3api list-objects-v2 --bucket {bucket-name}

{
    "Key": "2022/05/02-180752.65/metadata.sst",
    "LastModified": "2022-05-02T18:07:54+00:00",
    "ETag": "\"c0f499f21d7886e4289d55ccface7527\"",
    "Size": 7865,
    "StorageClass": "STANDARD"
},
    ...

    "Key": "2022-05-06/202205061217256387084640000000000-1b4e610c63535061-1-2-00000000-
users-7.ndjson",
    "LastModified": "2022-05-06T12:17:26+00:00",
    "ETag": "\"c60a013619439bf83c505cb6958b55e2\"",
    "Size": 94596,
    "StorageClass": "INTELLIGENT_TIERING"
},

For a specific operation, see the following examples:

• Back up with an S3 storage class
• Create a changefeed with an S3 storage class
• Export tabular data with an S3 storage class

See also

• BACKUP
• RESTORE
• IMPORT
• EXPORT
• CREATE CHANGEFEED
• Cluster Settings

Was this helpful?