I am a big fan of Cloudflare R2, an object storage that provides egress-free downloads.

R2 is compatible with the AWS S3 API, so you can use the AWS CLI tool – with a few caveats. These include:

  • You need to add the --endpoint-url https://<account_id>.r2.cloudflarestorage.com argument for every call.
  • When copying to R2, you need to pass the --checksum-algorithm CRC32 argument.
  • I often store multiple AWS configurations, which requires passing an additional argument: --profile <your_r2_profile_name>.

To perform these steps automatically, I wrote a small Bash function. Add this to your ~/.bashrc or ~/.zshrc:

function r2() {
    if [ $# -eq 0 ]; then
        echo "No arguments provided"
        echo
        echo "Usage: r2 <command> <arguments>"
        return 1
    fi

    SUBCOMMAND=${1}
    shift 1
    ARGS=(${@})
    LAST_ARG=${ARGS: -1}

    if [[ ${SUBCOMMAND} == "cp" && "${LAST_ARG}" =~ "^s3://" ]]; then
        CHECKSUM_ARG=("--checksum-algorithm" "CRC32")
    else
        CHECKSUM_ARG=""
    fi

    aws s3 $SUBCOMMAND \
        --endpoint-url https://<account_id>.r2.cloudflarestorage.com \
        --profile <your_r2_profile_name> \
        ${CHECKSUM_ARG} \
        ${ARGS[@]}
}

This snippet defines the r2 command, which can used as follows:

r2 ls s3://your_bucket_name/some/path
r2 cp your_local_file s3://your_bucket_name/some/path
r2 cp s3://your_bucket_name/some/path/some_file .