After comparing multiple backup solutions on Linux, I chose Restic as my main solution and I thought a small guide about how to configure Restic on Linux with S3 Storage using Docker.
Table of content
Context
Let’s start with some context. I want to back up my Nextcloud data (which is a Docker volume) daily, so I want to use an automated Docker backup system. I want it to be stored on a remote storage for an obvious reason, so I decided to use S3 for it’s simplicity of usage. Please note that I’m using an S3 compatible API with Backblaze and not Amazon S3 (but you can use any S3 compatible API).
Configure Restic options
You can use Restic by passing options to the CLI (like -r for repository) or with environment variables. I’m using a .env file to configure Restic options, as this is a good way to store secrets.
So let’s start configuring Restic password, I’ll use my password manager to generate a 64 characters password using letters (upper and lower cases), digits and symbols. You can use Bitwarden’s one if you don’t have any password manager.
I’m using the root user because I don’t want anybody else to access this information. You could use a dedicated user too.
Now let’s cd to your home directory and edit a restic.env file with your favorite editor:
# Set your random password here
RESTIC_PASSWORD=Now, let’s add the S3 storage :
RESTIC_REPOSITORY=s3:If you’re using a compatible S3 (not Amazon S3), add https:// in front of the hostname, if you’re using Amazon S3, do not put https:// in front of the hostname.
# Amazon S3
RESTIC_REPOSITORY=s3:s3.amazonaws.com/bucket_name
# Compatible S3
RESTIC_REPOSITORY=s3:https://s3.example.com/bucket_nameAnd you must use a path-style S3 URL, so you can’t use mybucket.s3.com, you must use s3.com/mybucket.
And now set the S3 credentials:
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=You must use these environment variables for both Amazon S3 or compatible S3 storage.
Now the complete file:
RESTIC_PASSWORD=
RESTIC_REPOSITORY=s3:
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=Here’s an example of what mine looks like:
RESTIC_PASSWORD=@7HvJZHxX9499#HaHXLrKo3kzyU&r9w8vvkEvjsktSk!@c@QqiCFq5cwWk*DCh6H
RESTIC_REPOSITORY=s3:https://s3.eu-central-003.backblazeb2.com/lenaics-nextcloud-backups
AWS_ACCESS_KEY_ID=003ae3cbc3f4edb000000000a
AWS_SECRET_ACCESS_KEY=K003yMMX3CszYQODHC+wHexfdeWwx2INow let’s start backing up!
Configuring backups
Now, let’s initialize the Restic repository. This is a one-time step that you have to do for every new repository.
docker container run --rm -it --env-file /root/restic.env restic/restic initOf course you need to change the /root/restic.env part to where your env file is on your server.
Here’s the command explaination:
docker container run: create and start a new container--rm: When the container stops, delete it. If you don’t set this parameter, every time you’ll use a Restic command with Docker, there will be a useless container using disk space, so we’ll better delete it after usage.-it: It tells Docker the session will be interactive and terminal like.--env-file /root/restic.env: We tell Docker to take everything in this file and set the content as environment variables at the container’s runtimerestic/restic: The official Restic image pulled from the Docker registryinit: The Restic command, this will be the only part changing from now on.
Then, you should have a message like this:
created restic repository b9bce4ee6c at s3:https://s3.eu-central-003.backblazeb2.com/lenaics-nextcloud-backups
Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is irrecoverably lost.Now we can start backing up. The first backup can take a few minutes or hours depending on your data size. The next ones are incremental so they’re pretty fast. To check that everything is going well, I’m going to use a screen (you can use Tmux as well) to start the backup in the background. If you don’t have screen installed, install the packages named screen on your system (using APT, DNF…)
screen -S backupNow that I’m in a screen, I can close my terminal without stopping the current process, which is useful for time-consuming processes like backups.
Now let’s start the first backup:
docker container run --rm -it --env-file /root/restic.env -h server.example.com -v my_nextcloud:/data restic/restic backup /dataI added two parameters:
-h server.example.com: you need to set the container hostname as the server’s one because Restic uses it in the index and Docker uses a new random hostname for every new container.-v my_nextcloud:/data: The volume I want to back up.
As you can see here, I’m mounting my Nextcloud data volume on /data with -v my_nextcloud:/data. Don’t forget to change the volume name (before the :). You can keep the mount point in the container. If you change it, don’t forget to change the /data at the end of the command, as this is the folder to back up.
You should get a message like this at the end:
repository b9bce4ee opened (version 2, compression level auto)
created new cache in /root/.cache/restic
no parent snapshot found, will read all files
Files: 23 new, 0 changed, 0 unmodified
Dirs: 2 new, 0 changed, 0 unmodified
Added to the repository: 24.712 MiB (10.083 MiB stored)
processed 23 files, 31.525 MiB in 0:11
snapshot fad934f1 savedHurray, It worked! Now let’s check if the last snapshot is in the S3 bucket:
root@cloud-sayna-io:~# docker container run --rm -it --env-file /root/restic.env restic/restic snapshots
repository b9bce4ee opened (version 2, compression level auto)
created new cache in /root/.cache/restic
ID Time Host Tags Paths
--------------------------------------------------------------------
7610b55b 2023-08-20 19:19:25 server.example.com /data
--------------------------------------------------------------------
1 snapshotsPerfect, I can see the backup of the path /data for the host server.example.com I just made.
Automate the backup
We’ve made our first backup. Now, we need to set up an automated task to do this everyday (or every time you want).
We’ll use the cron to automate this. Let’s enter the crontab:
crontab -eNow let’s add this line at the end:
0 2 * * * docker container run --rm -i -h server.example.com --env-file /root/restic.env -v meshcentral_data:/data restic/restic backup --no-scan /dataI’ve used the same backup command, with a few changes:
0 2 * * *: The system will run this command every day at 2:00 am. If you’re not familiar with crontab, you can generate your interval using a website like Crontab Generator.-i: I removed thetfrom-itbecause as cron is not a front end terminal, this option is useless and doesn’t work.--no-scan: This option tells Restic to not estimate the ETA by scanning the files’ number and size. This can make the backup slightly faster and as it’s an automated task, we don’t need the ETA.
Tomorrow, you’ll use the snapshot command we’ve used before to check the backup was correctly made at 2 am.
Debug
What if it doesn’t work? Restic should give you explicit error messages. Feel free to google it, search in the official documentation or ask in the comments below!
Debugging the crontab is not much harder. First of all, you can output the command messages and error in a file to read them. Just change your crontab like this:
0 2 * * * bash -c "docker container run --rm -i -h server.example.com --env-file /root/restic.env -v meshcentral_data:/data restic/restic backup --no-scan /data" > /root/restic.logs 2>&1So here, I’ve encapsulated the command in bash -c to make sure my output contract at the end is correctly taken by bash and not the Restic command inside the container. > /root/restic.logs 2>&1 tells the docker command to set stdout (the command standard output) in the /root/restic.logs file. 2>&1 tells the docker command to set stderr (the command error output) in the command standard output.
Now, if you don’t want to wait a full day that your crontab runs to see a new error, just change the execution time to your next minute:
# If it's 15:46
47 15 * * * ...And now just wait a minute (don’t forget to change the time back when you’re done). Then you can just display the errors with:
cat /root/restics.logsConclusion
Now we’re done! I hope this guide will help you secure your server for the better. If you see any mistakes or have any question, feel free to ask in the comments!