Managing SSH chroots to backup a lot of machines
Go to file
Jérémy Lecour 7d4de721ea new subcommands with backward compatibility 2020-10-11 23:41:38 +02:00
docs new subcommands with backward compatibility 2020-10-11 23:41:38 +02:00
lib new subcommands with backward compatibility 2020-10-11 23:41:38 +02:00
test new subcommands with backward compatibility 2020-10-11 23:41:38 +02:00
tpl improve documentation 2020-10-11 18:46:15 +02:00
.gitignore gitignore build directory 2020-04-18 10:26:08 +02:00
CHANGELOG.md bkctld-remove: confirmation before removal of jails if not in force mode 2020-10-11 15:19:00 +02:00
LICENSE fix file name 2017-08-05 17:33:00 -04:00
README.md improve documentation 2020-10-11 18:46:15 +02:00
Vagrantfile exclude build directory in vagrant rsync 2020-04-18 10:40:28 +02:00
bash_completion add shellcheck shell directive to bash_completion 2020-04-11 10:01:46 +02:00
bkctld new subcommands with backward compatibility 2020-10-11 23:41:38 +02:00
bkctld.8 new subcommands with backward compatibility 2020-10-11 23:41:38 +02:00
bkctld.conf bkctld.conf: Add NODE by default 2019-03-18 16:50:06 +01:00
bkctld.conf.5 improve documentation 2020-10-11 18:46:15 +02:00
bkctld.service Transform bkctld SysVinit script into systemd oneshot service 2019-01-08 11:26:42 +01:00
evobackup-incs-policy.5 new subcommands with backward compatibility 2020-10-11 23:41:38 +02:00
zzz_evobackup fix bad destination directory for mysql backups 2020-09-25 15:53:13 +02:00

README.md

Bkctld (aka server-side evobackup)

bkctld helps you manage the receiving side of a backup infrastructure. It is licensed under the AGPLv3.

With bkctld you create and manage "jails". They contain a chrooted and dedicated SSH server, with it's own TCP port and optionnaly it's own set of iptables rules.

With bkctld you can have hundreds of jails, one for each client to push its data (using Rsync/SFTP). Each client can only see its own data.

In addition to the traditional "ext4" filesystem, bkctld also supports the btrfs filesystem and manages subvolumes automatically.

With bkctld you can create "timestamped" copies of the data, to keep different versions of the same data at different points in time. If the filesystem is btrfs, it creates subvolumes snapshots, otherwise it creates copies with hard-links (for file-level deduplication).

With btrfs you can have a data retention policy to automatically destroy timestamped copies of your data. For example, keep a copy for the last 5 days and the first day of the last 3 months.

                                    Backup server
                                    ************
Client 1 ------ SSH/Rsync ------->  * tcp/2222 *
                                    ************
Client 2 ------ SSH/Rsync ------->  * tcp/2223 *
                                    ************

This method uses standard tools (ssh, rsync, cp -al, btrfs subvolume) and has been used for many years by Evolix to backup hundreds of servers, totaling many terabytes of data, each day. bkctld has been tested on Debian Jessie (8), Stretch (9) and Buster (10) and should be compatible with other Debian versions or derived distributions like Ubuntu.

A large enough volume must be mounted on /backup, we recommend the usage of BTRFS so you can use sub-volumes and snapshots. This volume can also be encrypted with LUKS.

Security considerations

The client obviously has access to its uploaded data (in the chroot), but the timestamped copies are outside the chroot, to reduce the risk or complete backup erasure from a compromised client.

Since the client connects to the backup server with root, it can mess with the jail and destroy the data. But the timestamped copies are out of reach because outside of the chroot.

It means that if the client server is compromised, an attacker can destroy the latest copy of the backed up data, but not the timestamped copies. And if the backup server is compromised an attacker has complete access to all the backup data (inside and outside the jails), but they don't have any access to the client.

This architecture is as secure as SSH, Rsync, chroot and iptables are.

Install

See the installation guide for instructions.

Server-side usage

See docs/usage.md.

The man(1) page, in troff(7) language, can be generated with pandoc:

pandoc -f markdown \
	-t man usage.md \
	--template default.man \
	-V title=bkctld \
	-V section=8 \
	-V date="$(date '+%d %b %Y')" \
	-V footer="$(git describe --tags)" \
	-V header="bkctld man page"
git clone https://gitea.evolix.org/evolix/evobackup.git
cd evobackup
man ./docs/bkctld.8

Client-side usage

As bkctld creates a dedicated SSH server, everything that connects over SSH can push files to the jail (Rsync, SFTP…).

Example with Rsync :

rsync -av -e "ssh -p SSH_PORT" /home/ root@SERVER_NAME:/var/backup/home/

An example synchronization script is present in zzz_evobackup, clone the evobackup repository and read the CLIENT CONFIGURATION section of the manual.

Testing

You can manage test environments with Vagrant.

There are 4 preconfigured environments :

  • Debian stetch with ext4
  • Debian stetch with btrfs
  • Debian buster with ext4
  • Debian buster with btrfs

You can start them all with :

vagrant up

… or just the one you want :

vagrant up buster-btrfs

Deployment

Run vagrant rsync-auto in a terminal for automatic synchronization of your local code with Vagrant environments :

vagrant rsync-auto

Bats

You can run bats tests with the test provisioner :

vagrant provision --provision-with test

You can also run the tests from inside the VM

localhost $ vagrant ssh buster-btrfs
vagrant@buster-btrfs $ sudo -i
root@buster-btrfs # bats /vagrant/test/*.bats

You should shellcheck your bats files, but with shellcheck > 0.4.6, because the 0.4.0 version doesn't support bats syntax.