Getting started with S3 Object Storage: buckets, access keys, AWS CLI, and rclone

AwanCerah Object Storage is S3-compatible, so it works with the same tools you use for Amazon S3. This guide starts from scratch: creating an S3 Account and bucket, setting up access keys, then using the AWS CLI, rclone, and restic for automatic backups.

All you need is an AwanCerah account with balance. We ran the commands in this guide against the AwanCerah endpoint.

Create an S3 Account

An S3 Account holds your buckets. One S3 Account can contain many buckets with a combined quota and a single bill.

1. Sign in to the console, open Object Storage, and click Create S3 Account.
2. Under Account type, choose Plans for a fixed quota and price, or Pay as you grow to pay for the data you store.
3. For Plans, pick a quota. Then choose a Payment option: Hourly (delete at any time) or Monthly (paid upfront and cheaper).
4. Enter an Account name, such as shop-media. It starts with a letter and uses lowercase letters, digits, and hyphens.
5. Click Create S3 Account and accept the Terms of Service.

The account is active right away. Plans start at Rp1.000 a month and there are no data transfer fees. See all prices on the pricing page.

Create a bucket

A bucket is where your files are stored. On the S3 Account page, enter a bucket name under Create bucket.

• Names are 3 to 63 characters: lowercase letters, digits, and hyphens.
• Bucket names are unique across AwanCerah and cannot be changed after creation.
• Split buckets by purpose, for example shop-backup for backups and shop-images for website images.

Set up access keys

S3 tools and apps sign in with an access key, not your account password. When the S3 Account is created, an access key named Utama is ready with read and write access to every bucket. Click Show secret to see its secret key.

For servers and apps, create separate access keys so their access can be limited:

1. In the Access key card, click Create access key.
2. Enter a Name, for example Office server backup.
3. Under Buckets, choose Specific buckets and grant access only to the buckets it needs. Read means listing and downloading files. Write means uploading, overwriting, and deleting files.
4. Set an Expiry if the key is only needed for a while.

The endpoint and region are shown in the S3 access card:

Use the AWS CLI

Save the endpoint in a profile so you do not have to type it in every command. Add this to ~/.aws/config:

[profile awancerah]
region = pwt1
endpoint_url = https://s3.awanbaru.my.id

Store the access key and secret key with aws configure --profile awancerah. Then:

# list files in a bucket
aws s3 ls s3://shop-backup --profile awancerah

# upload a file and a folder
aws s3 cp report.pdf s3://shop-backup/documents/ --profile awancerah
aws s3 sync ./photos s3://shop-images/products --profile awancerah

# temporary download link (1 hour) for a private file
aws s3 presign s3://shop-backup/documents/report.pdf --expires-in 3600 --profile awancerah

The endpoint_url profile setting needs AWS CLI 2.13 or later. On older versions, add --endpoint-url https://s3.awanbaru.my.id to every command.

Use rclone

rclone is great for copying and syncing large folders. Add a remote to ~/.config/rclone/rclone.conf:

[awancerah]
type = s3
provider = Other
endpoint = https://s3.awanbaru.my.id
region = pwt1
access_key_id = <access key>
secret_access_key = <secret key>

# copy a folder to a bucket (unchanged files are skipped)
rclone copy /var/www/uploads awancerah:shop-images/uploads --progress

# list bucket contents
rclone ls awancerah:shop-images

The console also has ready-to-copy AWS CLI and rclone examples in the How to use card, with a Fill in secret button.

Automatic server backups with restic

restic makes encrypted backups and only uploads the parts of files that changed, so daily backups stay small. Install restic on your server (apt install restic on Ubuntu and Debian), then create /root/backup-s3.sh:

#!/bin/bash
export AWS_ACCESS_KEY_ID=<access key>
export AWS_SECRET_ACCESS_KEY=<secret key>
export AWS_DEFAULT_REGION=pwt1
export RESTIC_REPOSITORY=s3:https://s3.awanbaru.my.id/shop-backup/web-server
export RESTIC_PASSWORD=<a long backup password>

# create the repository on the first run
restic cat config >/dev/null 2>&1 || restic init

# dump the database so it is included (MySQL/MariaDB example)
mysqldump --single-transaction --all-databases | gzip > /root/db.sql.gz

restic backup /var/www /etc /root/db.sql.gz
restic forget --keep-daily 7 --keep-weekly 4 --prune

Set the permissions, run it once, then schedule it every day at 02:00 with crontab -e:

chmod 700 /root/backup-s3.sh
/root/backup-s3.sh

# crontab line
0 2 * * * /root/backup-s3.sh >> /var/log/backup-s3.log 2>&1

To restore, run restic snapshots and then restic restore latest --target /root/restore with the same variables as the script. The access key used by restic needs read and write, because restic also reads the repository.

Upload from the console

For occasional uploads you do not need any tools. Open the bucket, go to the files tab, and click Upload or drag files onto the page. Browser uploads are limited to 5 GB per file. Larger files can be sent with the AWS CLI or rclone.

Click a file name to create a Temporary link that lasts 1 hour, 24 hours, or 7 days. You can share it without making the bucket public.

Next steps

• Connect your app to S3 in Store app files in S3 Object Storage.
• Compare plans and pay as you grow on the pricing page.