Compare commits

...

33 Commits

Author SHA1 Message Date
Jérémy Lecour 859bb800d4 Merge pull request 'fix-tempfiles' (#72) from fix-tempfiles into master
gitea/evobackup/pipeline/head This commit looks good Details
Reviewed-on: #72
2024-03-22 11:27:34 +01:00
Brice Waegeneire 21533c6fb9 client: Write temporary files in /tmp
gitea/evobackup/pipeline/head This commit looks good Details
By default "mktemp" creates a temporary file in the current working directory,
which in the context of Debian's cron is /. To avoid littering /, we add the
option "--tmpdir" that use $TMPDIR or /tmp instead.
2024-03-22 10:05:10 +01:00
Brice Waegeneire 9510546d48 client: Correctly clean up temporary files
The function "build_rsync_main_cmd" is called in a subshell, so it can't
effectively modify it's parent variable "temp_files". To correctly cleanup
those temporary files, we do it when this specific function exits.
2024-03-22 09:56:03 +01:00
Mathieu Trossevin f1d4e6ed9d
fix(includes): Avoid breaking is_btrfs if path doesn't exists
gitea/evobackup/pipeline/head There was a failure building this commit Details
This should return with a err code of 1, not result in the entire
command breaking
2024-02-06 15:03:21 +01:00
Jérémy Lecour 491c839014
Add note for includes/excludes brace expansion
gitea/evobackup/pipeline/head This commit looks good Details
2023-12-28 15:29:39 +01:00
William Hirigoyen 4298da250b Add WARNING and CRITICAL values to default conf
gitea/evobackup/pipeline/head This commit looks good Details
2023-12-19 10:26:30 +01:00
William Hirigoyen d359883700 server: fix install.md paths
gitea/evobackup/pipeline/head This commit looks good Details
2023-12-19 09:45:25 +01:00
Bruno TATU 4cd1554780 Correction url dans README.md
gitea/evobackup/pipeline/head This commit looks good Details
2023-10-27 11:30:53 +02:00
William Hirigoyen 981f5118ce Ajout exclusion /var/lib/amavis/virusmails
gitea/evobackup/pipeline/head This commit looks good Details
2023-06-15 11:18:49 +02:00
Jérémy Lecour 5dac827bb5 invert dry-run logic
gitea/evobackup/pipeline/head This commit looks good Details
2023-05-03 22:08:17 +02:00
William Hirigoyen 4807dfbc99 Fix missing dump.rdb in Redis dump compression, which caused compression fail.
gitea/evobackup/pipeline/head This commit looks good Details
2023-04-19 17:46:34 +02:00
William Hirigoyen fd9bb57f8b #71538 : fix Redis dump list in case directory is a symlink
gitea/evobackup/pipeline/head This commit looks good Details
2023-04-17 17:13:23 +02:00
Alexis Ben Miloud--Josselin d62455cd1a Compress Redis dump
gitea/evobackup/pipeline/head This commit looks good Details
2023-04-04 15:05:30 +02:00
Bruno Tatu 72f5900cf3 On backup les données dans les bases
gitea/evobackup/pipeline/head This commit looks good Details
2023-03-30 16:37:36 +02:00
Jérémy Lecour 05a62e17b5 client: Release 22.12
gitea/evobackup/pipeline/head This commit looks good Details
2022-12-27 13:37:48 +01:00
Jérémy Lecour 8babc64e0d client: log line with more details
gitea/evobackup/pipeline/head This commit looks good Details
2022-12-27 13:36:24 +01:00
Jérémy Lecour aa7366ce2e client: separate Rsync for the canary file if the main Rsync has finished without errors
gitea/evobackup/pipeline/head This commit looks good Details
2022-12-27 11:56:07 +01:00
Jérémy Lecour ef744f77cf client: No more fallback if dump-server-state is missing
gitea/evobackup/pipeline/head This commit looks good Details
2022-12-27 11:11:02 +01:00
Jérémy Lecour 80426c9ba9 whitespaces 2022-12-27 11:09:37 +01:00
Jérémy Lecour b34ee2c3dc client: use long options for readability 2022-12-27 11:09:11 +01:00
Jérémy Lecour 94b470770f client: use sub shells instead of moving around
gitea/evobackup/pipeline/head This commit looks good Details
2022-12-27 10:46:53 +01:00
Jérémy Lecour ed0645c9d2 remodel how we build the rsync command (#63)
gitea/evobackup/pipeline/head This commit looks good Details
* use a log file for rsync
* build the command argument by argument, without backslashes
* move excludes into a file

Co-authored-by: Jeremy Lecour <jlecour@evolix.fr>
Reviewed-on: #63
2022-12-27 10:43:39 +01:00
Jérémy Lecour 140a498e28 client: Only one loop for all redis instances
gitea/evobackup/pipeline/head This commit looks good Details
2022-12-27 10:38:55 +01:00
David Prevot bdd3ef7350 CI: Push *.buildinfo too
gitea/evobackup/pipeline/head This commit looks good Details
2022-12-22 11:39:42 +01:00
Alexis Ben Miloud--Josselin 6e0ab85c6b client: ignore errors when listing instances
gitea/evobackup/pipeline/head This commit looks good Details
2022-12-13 09:53:10 +01:00
David Prevot ff681275e3 CI: Don’t use Drone CI anymore
gitea/evobackup/pipeline/head This commit looks good Details
2022-12-12 17:27:36 +01:00
David Prevot 1a892ba002 CI: Actually limit upload to .deb and .changes
continuous-integration/drone/push Build is passing Details
gitea/evobackup/pipeline/head This commit looks good Details
2022-12-12 17:13:28 +01:00
Alexis Ben Miloud--Josselin e3a969f3e2 client: replace rm ** by find -delete
continuous-integration/drone/push Build is passing Details
gitea/evobackup/pipeline/head This commit looks good Details
** has no particular meaning in bash unless shopt globstar is
set (it isn't by default).
2022-12-12 16:06:58 +01:00
David Prevot ca4fe4ffb5 CI: Limit upload to .deb and .changes
continuous-integration/drone/push Build is passing Details
gitea/evobackup/pipeline/head This commit looks good Details
2022-12-12 11:42:19 +01:00
David Prevot 2338c92fe3 CI: Upload to pub2
continuous-integration/drone/push Build is passing Details
gitea/evobackup/pipeline/head This commit looks good Details
2022-12-12 10:56:13 +01:00
David Prevot d2a76dce9c CI: use build-area instead of outside of scope subdirectory 2022-12-12 10:26:06 +01:00
David Prevot 4476802182 Use sbuild during CI
continuous-integration/drone/push Build is passing Details
gitea/evobackup/pipeline/head This commit looks good Details
2022-12-09 13:42:44 +01:00
Mathieu Trossevin 1e35aaa4db
Corrige messages d'erreurs intempestif
continuous-integration/drone/push Build is passing Details
gitea/evobackup/pipeline/head This commit looks good Details
2022-12-08 14:09:37 +01:00
9 changed files with 266 additions and 247 deletions

View File

@ -1,24 +1,15 @@
pipeline {
agent { label 'docker' }
agent { label 'sbuild' }
stages {
stage('Build Debian package') {
agent {
docker {
image 'evolix/gbp:bullseye'
args '-u root --privileged'
}
}
when {
branch 'debian'
}
steps {
script {
sh 'mk-build-deps --install --remove debian/control'
sh 'rm -rf {source,*.gz,*.bz2,*.xz,*.deb,*.dsc,*.changes,*.buildinfo,lintian.txt,.git}'
sh "gbp clone --debian-branch=$GIT_BRANCH $GIT_URL source"
sh 'cd source && git checkout $GIT_BRANCH && gbp buildpackage -us -uc'
sh 'gbp buildpackage'
}
archiveArtifacts allowEmptyArchive: true, artifacts: '*.gz,*.bz2,*.xz,*.deb,*.dsc,*.changes,*.buildinfo,lintian.txt'
archiveArtifacts allowEmptyArchive: true, artifacts: 'build-area/*.gz,build-area/*.bz2,build-area/*.xz,build-area/*.deb,build-area/*.dsc,build-area/*.changes,build-area/*.buildinfo,build-area/*.build,build-area/lintian.txt'
}
}
@ -28,10 +19,7 @@ pipeline {
}
steps {
script {
sh 'echo Dummy line to remove once something actually happens.'
/* No crendentials yet.
sh 'rsync -avP bkctld* droneci@pub.evolix.net:/home/droneci/bkctld/'
*/
sh 'rsync -avP build-area/bkctld*.deb build-area/bkctld*.changes build-area/bkctld*.buildinfo pub.evolix.org:/srv/upload/'
}
}
}

View File

@ -1,35 +0,0 @@
kind: pipeline
name: default
steps:
- name: fetch
image: alpine/git
commands:
- git fetch --tags
- name: build debian package
image: evolix/gbp:bullseye
branches:
- debian
commands:
- mk-build-deps --install --remove debian/control
- git clean --force
- gbp buildpackage -us -uc
when:
branch:
- debian
- name: upload debian package
image: drillster/drone-rsync
settings:
hosts: ["pub.evolix.net"]
port: 22
user: droneci
key:
from_secret: drone_private_key
target: /home/droneci/bkctld/
source: ../bkctld*
delete: true
when:
branch:
- debian

View File

@ -10,6 +10,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Changed
### Deprecated
### Removed
### Fixed
### Security
## [22.12]
### Changed
* Use --dump-dir instead of --backup-dir to suppress dump-server-state warning
* Do not use rsync compression
* Replace rsync option --verbose by --itemize-changes
@ -18,17 +29,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
* Add AGPL License and README
* Script now depends on Bash
* tolerate absence of mtr or traceroute
### Deprecated
* Only one loop for all Redis instances
* remodel how we build the rsync command
* use sub shells instead of moving around
* Separate Rsync for the canary file if the main Rsync has finished without errors
### Removed
* No more fallback if dump-server-state is missing
### Fixed
* Make start_time and stop_time compatible with OpenBSD
### Security
## [22.03]
Split client and server parts of the project

View File

@ -1,3 +1,3 @@
Pour l'installation de `zzz_evobackup`, voir <https://intra.evolix.net/Installation_jail_backup_Evolix#installation-du-client-evobackup>
Pour l'installation de `zzz_evobackup`, voir <https://intra.evolix.net/OutilsInternes/EvoBackupClient#installer-et-configurer-le-client-evobackup>
Pour `update-evobackup-canary`, voir <https://intra.evolix.net/OutilsInternes/update-evobackup-canary>
Pour `update-evobackup-canary`, voir <https://intra.evolix.net/OutilsInternes/update-evobackup-canary>

View File

@ -18,7 +18,7 @@
##### Configuration ###################################################
VERSION="22.05"
VERSION="22.12"
# email adress for notifications
MAIL=jdoe@example.com
@ -48,6 +48,9 @@ PIDFILE="/var/run/${PROGNAME}.pid"
# Customize the log path if you have multiple scripts and with separate logs
LOGFILE="/var/log/evobackup.log"
# Full Rsync log file, reset each time
RSYNC_LOGFILE="/var/log/${PROGNAME}.rsync.log"
HOSTNAME=$(hostname)
DATE_FORMAT="%Y-%m-%d %H:%M:%S"
@ -57,7 +60,77 @@ DATE_FORMAT="%Y-%m-%d %H:%M:%S"
# Enable/disable sync tasks (default: enabled)
: "${SYNC_TASKS:=1}"
##### SETUP AND FUNCTIONS #############################################
CANARY_FILE="/zzz_evobackup_canary"
# Source paths can be customized
# Empty lines, and lines containing # or ; are ignored
# NOTE: remember to single-quote paths if they contain globs (*)
# and you want to defer expansion
RSYNC_INCLUDES="
/etc
/root
/var
/home
"
# Excluded paths can be customized
# Empty lines, and lines beginning with # or ; are ignored
# NOTE: remember to single-quote paths if they contain globs (*)
# and you want to defer expansion
RSYNC_EXCLUDES="
/dev
/proc
/run
/sys
/tmp
/usr/doc
/usr/obj
/usr/share/doc
/usr/src
/var/apt
/var/cache
'/var/db/munin/*.tmp'
/var/lib/amavis/amavisd.sock
/var/lib/amavis/tmp
/var/lib/amavis/virusmails
'/var/lib/clamav/*.tmp'
/var/lib/elasticsearch
/var/lib/metche
/var/lib/mongodb
'/var/lib/munin/*tmp*'
/var/lib/mysql
/var/lib/php/sessions
/var/lib/php5
/var/lib/postgres
/var/lib/postgresql
/var/lib/sympa
/var/lock
/var/run
/var/spool/postfix
/var/spool/smtpd
/var/spool/squid
/var/state
/var/tmp
lost+found
'.nfs.*'
'lxc/*/rootfs/tmp'
'lxc/*/rootfs/usr/doc'
'lxc/*/rootfs/usr/obj'
'lxc/*/rootfs/usr/share/doc'
'lxc/*/rootfs/usr/src'
'lxc/*/rootfs/var/apt'
'lxc/*/rootfs/var/cache'
'lxc/*/rootfs/var/lib/php5'
'lxc/*/rootfs/var/lib/php/sessions'
'lxc/*/rootfs/var/lock'
'lxc/*/rootfs/var/run'
'lxc/*/rootfs/var/state'
'lxc/*/rootfs/var/tmp'
/home/mysqltmp
"
##### FUNCTIONS #######################################################
local_tasks() {
log "START LOCAL_TASKS"
@ -76,7 +149,7 @@ local_tasks() {
# rm -rf ${LOCAL_BACKUP_DIR}/mysql
# rm -rf ${LOCAL_BACKUP_DIR}/mysqlhotcopy
# rm -rf /home/mysqldump
# rm -f ${LOCAL_BACKUP_DIR}/*.err ${LOCAL_BACKUP_DIR}/**/*.err
# find ${LOCAL_BACKUP_DIR}/ -type f -name '*.err' -delete
## example with global and compressed mysqldump
# mysqldump --defaults-extra-file=/etc/mysql/debian.cnf -P 3306 \
@ -90,7 +163,7 @@ local_tasks() {
## example with compressed SQL dump (with data) for each databases
# mkdir -p -m 700 ${LOCAL_BACKUP_DIR}/mysql/
# for i in $(mysql --defaults-extra-file=/etc/mysql/debian.cnf -P 3306 -e 'show databases' -s --skip-column-names \
# | egrep -v "^(Database|information_schema|performance_schema|sys)"); do
# | grep --extended-regexp --invert-match "^(Database|information_schema|performance_schema|sys)"); do
# mysqldump --defaults-extra-file=/etc/mysql/debian.cnf --force -P 3306 --events --hex-blob $i 2> ${LOCAL_BACKUP_DIR}/${i}.err | gzip --best > ${LOCAL_BACKUP_DIR}/mysql/${i}.sql.gz
# last_rc=$?
# if [ ${last_rc} -ne 0 ]; then
@ -119,7 +192,7 @@ local_tasks() {
## example with SQL dump (schema only, no data) for each databases
# mkdir -p -m 700 ${LOCAL_BACKUP_DIR}/mysql/
# for i in $(mysql --defaults-extra-file=/etc/mysql/debian.cnf -P 3306 -e 'show databases' -s --skip-column-names \
# | egrep -v "^(Database|information_schema|performance_schema|sys)"); do
# | grep --extended-regexp --invert-match "^(Database|information_schema|performance_schema|sys)"); do
# mysqldump --defaults-extra-file=/etc/mysql/debian.cnf --force -P 3306 --no-data --databases $i 2> ${LOCAL_BACKUP_DIR}/${i}.schema.err > ${LOCAL_BACKUP_DIR}/mysql/${i}.schema.sql
# last_rc=$?
# if [ ${last_rc} -ne 0 ]; then
@ -141,7 +214,7 @@ local_tasks() {
## example with two dumps for each table (.sql/.txt) for all databases
# for i in $(echo SHOW DATABASES | mysql --defaults-extra-file=/etc/mysql/debian.cnf -P 3306 \
# | egrep -v "^(Database|information_schema|performance_schema|sys)" ); do
# | grep --extended-regexp --invert-match "^(Database|information_schema|performance_schema|sys)" ); do
# mkdir -p -m 700 /home/mysqldump/$i ; chown -RL mysql /home/mysqldump
# mysqldump --defaults-extra-file=/etc/mysql/debian.cnf --force -P 3306 -Q --opt --events --hex-blob --skip-comments \
# --fields-enclosed-by='\"' --fields-terminated-by=',' -T /home/mysqldump/$i $i 2> /home/mysqldump/$i.err"
@ -163,7 +236,7 @@ local_tasks() {
## example for multiples MySQL instances
# mysqladminpasswd=$(grep -m1 'password = .*' /root/.my.cnf|cut -d" " -f3)
# grep -E "^port\s*=\s*\d*" /etc/mysql/my.cnf |while read instance; do
# grep --extended-regexp "^port\s*=\s*\d*" /etc/mysql/my.cnf | while read instance; do
# instance=$(echo "$instance"|awk '{ print $3 }')
# if [ "$instance" != "3306" ]
# then
@ -188,9 +261,10 @@ local_tasks() {
# mv ~postgres/pg.dump.bak ${LOCAL_BACKUP_DIR}/
## another method with gzip directly piped
# cd /var/lib/postgresql
# sudo -u postgres pg_dumpall | gzip > ${LOCAL_BACKUP_DIR}/pg.dump.bak.gz
# cd - > /dev/null
# (
# cd /var/lib/postgresql;
# sudo -u postgres pg_dumpall | gzip > ${LOCAL_BACKUP_DIR}/pg.dump.bak.gz
# )
## example with all tables from MYBASE excepts TABLE1 and TABLE2
# pg_dump -p 5432 -h 127.0.0.1 -U USER --clean -F t --inserts -f ${LOCAL_BACKUP_DIR}/pg-backup.tar -t 'TABLE1' -t 'TABLE2' MYBASE
@ -201,10 +275,11 @@ local_tasks() {
## example with compressed PostgreSQL dump for each databases
# mkdir -p -m 700 ${LOCAL_BACKUP_DIR}/postgresql
# chown postgres:postgres ${LOCAL_BACKUP_DIR}/postgresql
# cd /var/lib/postgresql
# dbs=$(sudo -u postgres psql -U postgres -lt | awk -F\| '{print $1}' |grep -v template*)
# for databases in $dbs ; do sudo -u postgres /usr/bin/pg_dump --create -s -U postgres -d $databases | gzip --best -c > ${LOCAL_BACKUP_DIR}/postgresql/$databases.sql.gz ; done
# cd - > /dev/null
# (
# cd /var/lib/postgresql
# dbs=$(sudo -u postgres psql -U postgres -lt | awk -F\| '{print $1}' |grep -v template*)
# for databases in $dbs ; do sudo -u postgres /usr/bin/pg_dump --create -U postgres -d $databases | gzip --best -c > ${LOCAL_BACKUP_DIR}/postgresql/$databases.sql.gz ; done
# )
## MongoDB
@ -224,15 +299,14 @@ local_tasks() {
## Purge previous dumps
# rm -rf ${LOCAL_BACKUP_DIR}/redis/
# rm -rf ${LOCAL_BACKUP_DIR}/redis-*
## example with copy .rdb file
## for the default instance :
# mkdir -p -m 700 ${LOCAL_BACKUP_DIR}/redis/
# cp /var/lib/redis/dump.rdb ${LOCAL_BACKUP_DIR}/redis/
## for multiple instances :
# for instance in $(ls -d /var/lib/redis-*); do
# name=$(basename $instance)
# mkdir -p ${LOCAL_BACKUP_DIR}/${name}
# cp -a ${instance}/dump.rdb ${LOCAL_BACKUP_DIR}/${name}
## Copy dump.rdb file for each found instance
# for instance in $(find /var/lib/ -mindepth 1 -maxdepth 1 '(' -type d -o -type l ')' -name 'redis*'); do
# if [ -f "${instance}/dump.rdb" ]; then
# name=$(basename $instance)
# mkdir -p ${LOCAL_BACKUP_DIR}/${name}
# cp -a "${instance}/dump.rdb" "${LOCAL_BACKUP_DIR}/${name}"
# gzip "${LOCAL_BACKUP_DIR}/${name}/dump.rdb"
# fi
# done
## ElasticSearch
@ -251,7 +325,7 @@ local_tasks() {
# else
# echo 'Cannot make a snapshot of elasticsearch, at least one node is not mounting the repository.'
# fi
## If you need to keep older snapshot, for example the last 10 daily snapshots, replace the XDELETE and XPUT lines by :
## If you need to keep older snapshot, for example the last 10 daily snapshots, replace the XDELETE and XPUT lines by :
# for snapshot in $(curl -s -XGET "localhost:9200/_snapshot/snaprepo/_all?pretty=true" | grep -Eo 'snapshot_[0-9]{4}-[0-9]{2}-[0-9]{2}' | head -n -10); do
# curl -s -XDELETE "localhost:9200/_snapshot/snaprepo/${snapshot}" | grep -v -Fx '{"acknowledged":true}'
# done
@ -261,11 +335,11 @@ local_tasks() {
## RabbitMQ
## export config
#rabbitmqadmin export ${LOCAL_BACKUP_DIR}/rabbitmq.config >> "${LOGFILE}"
# rabbitmqadmin export ${LOCAL_BACKUP_DIR}/rabbitmq.config >> "${LOGFILE}"
## MegaCli config
#megacli -CfgSave -f ${LOCAL_BACKUP_DIR}/megacli_conf.dump -a0 >/dev/null
# megacli -CfgSave -f ${LOCAL_BACKUP_DIR}/megacli_conf.dump -a0 >/dev/null
## Dump network routes with mtr and traceroute (warning: could be long with aggressive firewalls)
network_targets="8.8.8.8 www.evolix.fr travaux.evolix.net"
@ -285,53 +359,11 @@ local_tasks() {
server_state_dir="${LOCAL_BACKUP_DIR}/server-state"
dump_server_state_bin=$(command -v dump-server-state)
if [ "${SYSTEM}" = "linux" ]; then
if [ -n "${dump_server_state_bin}" ]; then
${dump_server_state_bin} --all --force --dump-dir "${server_state_dir}"
last_rc=$?
if [ ${last_rc} -ne 0 ]; then
error "dump-server-state returned an error ${last_rc}, check ${server_state_dir}"
rc=1
fi
else
mkdir -p "${server_state_dir}"
## Dump system and kernel versions
uname -a > ${server_state_dir}/uname.txt
## Dump process with ps
ps auwwx > ${server_state_dir}/ps.txt
## Dump network connections with ss
ss -taupen > ${server_state_dir}/netstat.txt
## List Debian packages
dpkg -l > ${server_state_dir}/packages
dpkg --get-selections > ${server_state_dir}/packages.getselections
apt-cache dumpavail > ${server_state_dir}/packages.available
## Dump iptables
if [ -x /sbin/iptables ]; then
{ /sbin/iptables -L -n -v; /sbin/iptables -t filter -L -n -v; } > ${server_state_dir}/iptables.txt
fi
## Dump findmnt(8) output
FINDMNT_BIN=$(command -v findmnt)
if [ -x "${FINDMNT_BIN}" ]; then
${FINDMNT_BIN} > ${server_state_dir}/findmnt.txt
fi
## Dump MBR / table partitions
disks=$(lsblk -l | grep disk | grep -v -E '(drbd|fd[0-9]+)' | awk '{print $1}')
for disk in ${disks}; do
dd if="/dev/${disk}" of="${server_state_dir}/MBR-${disk}" bs=512 count=1 2>&1 | grep -Ev "(records in|records out|512 bytes)"
fdisk -l "/dev/${disk}" > "${server_state_dir}/partitions-${disk}" 2>&1
done
cat ${server_state_dir}/partitions-* > ${server_state_dir}/partitions
fi
if [ -z "${dump_server_state_bin}" ]; then
error "dump-server-state is missing"
rc=1
else
if [ -n "${dump_server_state_bin}" ]; then
if [ "${SYSTEM}" = "linux" ]; then
${dump_server_state_bin} --all --force --dump-dir "${server_state_dir}"
last_rc=$?
if [ ${last_rc} -ne 0 ]; then
@ -339,37 +371,93 @@ local_tasks() {
rc=1
fi
else
mkdir -p "${server_state_dir}"
## Dump system and kernel versions
uname -a > ${server_state_dir}/uname
## Dump process with ps
ps auwwx > ${server_state_dir}/ps.out
## Dump network connections with fstat
fstat | head -1 > ${server_state_dir}/netstat.out
fstat | grep internet >> ${server_state_dir}/netstat.out
## List OpenBSD packages
pkg_info -m > ${server_state_dir}/packages
## Dump MBR / table partitions
disklabel sd0 > ${server_state_dir}/partitions
## Dump pf infos
pfctl -sa > ${server_state_dir}/pfctl-sa.txt
${dump_server_state_bin} --all --force --dump-dir "${server_state_dir}"
last_rc=$?
if [ ${last_rc} -ne 0 ]; then
error "dump-server-state returned an error ${last_rc}, check ${server_state_dir}"
rc=1
fi
fi
fi
## Dump rights
#getfacl -R /var > ${server_state_dir}/rights-var.txt
#getfacl -R /etc > ${server_state_dir}/rights-etc.txt
#getfacl -R /usr > ${server_state_dir}/rights-usr.txt
#getfacl -R /home > ${server_state_dir}/rights-home.txt
# getfacl -R /var > ${server_state_dir}/rights-var.txt
# getfacl -R /etc > ${server_state_dir}/rights-etc.txt
# getfacl -R /usr > ${server_state_dir}/rights-usr.txt
# getfacl -R /home > ${server_state_dir}/rights-home.txt
log "STOP LOCAL_TASKS"
}
build_rsync_main_cmd() {
###################################################################
# /!\ WARNING /!\ WARNING /!\ WARNING /!\ WARNING /!\ WARNING /!\ #
###################################################################
# DO NOT USE COMMENTS in rsync lines #
# DO NOT ADD WHITESPACES AFTER \ in rsync lines #
# It breaks the command and destroys data #
# You should not modify this, unless you are really REALLY sure #
###################################################################
# Create a temp file for excludes and includes
includes_file="$(mktemp --tmpdir "${PROGNAME}.includes.XXXXXX")"
excludes_file="$(mktemp --tmpdir "${PROGNAME}.excludes.XXXXXX")"
# … and add them to the list of files to delete at exit
temp_files="${includes_file} ${excludes_file}"
trap "rm -f ${temp_files}" EXIT
# Store includes/excludes in files
# without blank lines of comments (# or ;)
echo "${RSYNC_INCLUDES}" | sed -e 's/\s*\(#\|;\).*//; /^\s*$/d' > "${includes_file}"
echo "${RSYNC_EXCLUDES}" | sed -e 's/\s*\(#\|;\).*//; /^\s*$/d' > "${excludes_file}"
# Rsync command
cmd="$(command -v rsync)"
# Rsync main options
cmd="${cmd} --archive"
cmd="${cmd} --itemize-changes"
cmd="${cmd} --quiet"
cmd="${cmd} --stats"
cmd="${cmd} --human-readable"
cmd="${cmd} --relative"
cmd="${cmd} --partial"
cmd="${cmd} --delete"
cmd="${cmd} --delete-excluded"
cmd="${cmd} --force"
cmd="${cmd} --ignore-errors"
cmd="${cmd} --log-file=${RSYNC_LOGFILE}"
cmd="${cmd} --rsh='ssh -p ${SSH_PORT} -o \"ConnectTimeout ${SSH_CONNECT_TIMEOUT}\"'"
# Rsync excludes
while read line ; do
cmd="${cmd} --exclude ${line}"
done < "${excludes_file}"
# Rsync local sources
cmd="${cmd} ${default_includes}"
while read line ; do
cmd="${cmd} ${line}"
done < "${includes_file}"
# Rsync remote destination
cmd="${cmd} root@${SSH_SERVER}:/var/backup/"
# output final command
echo "${cmd}"
}
build_rsync_canary_cmd() {
# Rsync command
cmd="$(command -v rsync)"
# Rsync options
cmd="${cmd} --rsh='ssh -p ${SSH_PORT} -o \"ConnectTimeout ${SSH_CONNECT_TIMEOUT}\"'"
# Rsync local source
cmd="${cmd} ${CANARY_FILE}"
# Rsync remote destination
cmd="${cmd} root@${SSH_SERVER}:/var/backup/"
# output final command
echo "${cmd}"
}
sync_tasks() {
n=0
server=""
@ -394,93 +482,48 @@ sync_tasks() {
SSH_SERVER=$(echo "${server}" | cut -d':' -f1)
SSH_PORT=$(echo "${server}" | cut -d':' -f2)
if [ "${SYSTEM}" = "linux" ]; then
rep="/bin /boot /lib /opt /sbin /usr"
else
rep="/bsd /bin /sbin /usr"
fi
log "START SYNC_TASKS - server=${server}"
update-evobackup-canary --who "${PROGNAME}"
# default paths, depending on system
if [ "${SYSTEM}" = "linux" ]; then
default_includes="/bin /boot /lib /opt /sbin /usr"
else
default_includes="/bsd /bin /sbin /usr"
fi
# Remote shell command
RSH_COMMAND="ssh -p ${SSH_PORT} -o 'ConnectTimeout ${SSH_CONNECT_TIMEOUT}'"
# reset Rsync log file
if [ -n "$(command -v truncate)" ]; then
truncate -s 0 "${RSYNC_LOGFILE}"
else
printf "" > "${RSYNC_LOGFILE}"
fi
###################################################################
# /!\ WARNING /!\ WARNING /!\ WARNING /!\ WARNING /!\ WARNING /!\ #
###################################################################
# DO NOT USE COMMENTS in rsync lines #
# DO NOT ADD WHITESPACES AFTER \ in rsync lines #
# It breaks the command and destroys data #
# => Only remove (or add) lines. #
###################################################################
# Build the final Rsync command
rsync_main_cmd=$(build_rsync_main_cmd)
# ignore check because we want it to split the different arguments to $rep
# shellcheck disable=SC2086
rsync --archive \
--itemize-changes --stats --human-readable \
--relative --partial \
--delete --delete-excluded --force --ignore-errors \
--exclude "dev" \
--exclude "lost+found" \
--exclude ".nfs.*" \
--exclude "/usr/doc" \
--exclude "/usr/obj" \
--exclude "/usr/share/doc" \
--exclude "/usr/src" \
--exclude "/var/apt" \
--exclude "/var/cache" \
--exclude "/var/lib/amavis/amavisd.sock" \
--exclude "/var/lib/amavis/tmp" \
--exclude "/var/lib/clamav/*.tmp" \
--exclude "/var/lib/elasticsearch" \
--exclude "/var/lib/metche" \
--exclude "/var/lib/munin/*tmp*" \
--exclude "/var/db/munin/*.tmp" \
--exclude "/var/lib/mongodb" \
--exclude "/var/lib/mysql" \
--exclude "/var/lib/php5" \
--exclude "/var/lib/php/sessions" \
--exclude "/var/lib/postgres" \
--exclude "/var/lib/postgresql" \
--exclude "/var/lib/sympa" \
--exclude "/var/lock" \
--exclude "/var/run" \
--exclude "/var/spool/postfix" \
--exclude "/var/spool/smtpd" \
--exclude "/var/spool/squid" \
--exclude "/var/state" \
--exclude "/var/tmp" \
--exclude "lxc/*/rootfs/tmp" \
--exclude "lxc/*/rootfs/usr/doc" \
--exclude "lxc/*/rootfs/usr/obj" \
--exclude "lxc/*/rootfs/usr/share/doc" \
--exclude "lxc/*/rootfs/usr/src" \
--exclude "lxc/*/rootfs/var/apt" \
--exclude "lxc/*/rootfs/var/cache" \
--exclude "lxc/*/rootfs/var/lib/php5" \
--exclude "lxc/*/rootfs/var/lib/php/sessions" \
--exclude "lxc/*/rootfs/var/lock" \
--exclude "lxc/*/rootfs/var/log" \
--exclude "lxc/*/rootfs/var/run" \
--exclude "lxc/*/rootfs/var/state" \
--exclude "lxc/*/rootfs/var/tmp" \
--exclude "/home/mysqltmp" \
${rep} \
/etc \
/root \
/var \
/home \
/zzz_evobackup_canary \
-e "${RSH_COMMAND}" \
"root@${SSH_SERVER}:/var/backup/" \
| tail -30 >> "${LOGFILE}"
# … log it
log "SYNC_TASKS - Rsync main command : ${rsync_main_cmd}"
rsync_rc=$?
if [ ${rsync_rc} -ne 0 ]; then
error "rsync returned an error ${rsync_rc}, check ${LOGFILE}"
# … execute it
eval "${rsync_main_cmd}"
rsync_main_rc=$?
# Copy last lines of rsync log to the main log
tail -n 30 "${RSYNC_LOGFILE}" >> "${LOGFILE}"
if [ ${rsync_main_rc} -ne 0 ]; then
error "rsync returned an error ${rsync_main_rc}, check ${LOGFILE}"
rc=201
else
# Build the canary Rsync command
rsync_canary_cmd=$(build_rsync_canary_cmd)
# … log it
log "SYNC_TASKS - Rsync canary command : ${rsync_canary_cmd}"
# … execute it
eval "${rsync_canary_cmd}"
fi
log "STOP SYNC_TASKS - server=${server}"
@ -584,8 +627,16 @@ main() {
fi
fi
echo "$$" > "${PIDFILE}"
# Initialize a list of files to delete at exit
# Any file added to the list will also be deleted at exit
temp_files="${PIDFILE}"
# shellcheck disable=SC2064
trap "rm -f ${PIDFILE}" EXIT
trap "rm -f ${temp_files}" EXIT
# Update canary to keep track of each run
update-evobackup-canary --who "${PROGNAME}"
if [ "${LOCAL_TASKS}" = "1" ]; then
local_tasks

View File

@ -15,3 +15,5 @@
#LOGLEVEL=6
#NODE=''
#ARCHIVESDIR='/backup/archives'
#WARNING=48
#CRITICAL=72

View File

@ -11,17 +11,17 @@ Warning: `cp`-ing the files without `-n` or `-i` will replace existing files !
~~~
# git clone https://gitea.evolix.org/evolix/evobackup.git
# cd evobackup
# cp bkctld /usr/local/sbin/
# cp server/bkctld /usr/local/sbin/
# mkdir -p /usr/local/lib/bkctld
# cp lib/* /usr/local/lib/bkctld/
# cp server/lib/* /usr/local/lib/bkctld/
# mkdir -p /usr/local/share/bkctld
# cp tpl/* /usr/local/share/bkctld/
# cp bkctld.service /lib/systemd/system/
# cp server/tpl/* /usr/local/share/bkctld/
# cp server/bkctld.service /lib/systemd/system/
# mkdir -p /usr/local/share/doc/bkctld
# cp zzz_evobackup /usr/local/share/doc/bkctld/
# cp client/zzz_evobackup /usr/local/share/doc/bkctld/
# mkdir -p /usr/local/share/bash_completion/
# cp bash_completion /usr/local/share/bash_completion/bkctld
# cp bkctld.conf /etc/default/bkctld
# cp server/bash_completion /usr/local/share/bash_completion/bkctld
# cp server/bkctld.conf /etc/default/bkctld
~~~
## Chroot dependencies

View File

@ -51,15 +51,15 @@ if dry_run; then
else
mv "${jail_path}" "${new_jail_path}"
fi
if dry_run; then
if [ -d "${incs_path}" ]; then
if [ -d "${incs_path}" ]; then
if dry_run; then
echo "[dry-run] rename ${incs_path} to ${new_incs_path}"
fi
else
if [ -d "${incs_path}" ]; then
else
mv "${incs_path}" "${new_incs_path}"
fi
fi
if [ -d "${jail_config_dir}" ]; then
if dry_run; then
echo "[dry-run] rename ${jail_config_dir} to ${new_jail_config_dir}"

View File

@ -136,7 +136,7 @@ is_btrfs() {
inode=$(stat --format=%i "${path}")
test $inode -eq 256
test "$inode" -eq 256
}
# Returns the list of jails found in the "jails" directory (default)