canary to validate inc backup everyday #47

New Issue

Open

opened 2021-05-06 10:32:42 +02:00 by gcolpart · 5 comments

gcolpart commented

2021-05-06 10:32:42 +02:00

Owner

we can :

add /zzz_evobackup_canary.txt who changes everytime : it needs to be the last file transferred and we can check if transfer is really finished
capture rsync stats (volume, number of files transferred) and compare everyday with last days to assure ratio is coherent
...add your ideas here... :)

we can : - add /zzz_evobackup_canary.txt who changes everytime : it needs to be the last file transferred and we can check if transfer is really finished - capture rsync stats (volume, number of files transferred) and compare everyday with last days to assure ratio is coherent - ...add your ideas here... :)

bwaegeneire commented

2021-05-06 10:41:41 +02:00

Owner

zzz_evobackup_canary.txt could simply contains the epoch

For the statistics there is the --stats flag¹

¹ https://www.mankier.com/1/rsync#--stats

`zzz_evobackup_canary.txt` could simply contains the epoch For the statistics there is the `--stats` flag¹ ¹ https://www.mankier.com/1/rsync#--stats

jlecour commented

2021-05-06 12:07:54 +02:00

Owner

We could even have someting like this (in the canary file) :

2021-05-06T08:00:01+02:00 Updated by crontab
2021-05-06T12:04:16+02:00 Updated by zzz_evobackup

It allows to keep track of many ways to update the file.
By reading the last line before the first space we have the last update date.
But this gives more information.

Even with a full year of daily updates the file won't ever grow big.

We could even have someting like this (in the canary file) : ``` 2021-05-06T08:00:01+02:00 Updated by crontab 2021-05-06T12:04:16+02:00 Updated by zzz_evobackup ``` It allows to keep track of many ways to update the file. By reading the last line before the first space we have the last update date. But this gives more information. Even with a full year of daily updates the file won't ever grow big.

jlecour commented

2021-05-06 12:12:53 +02:00

Owner

more clarification for my previous comment :

The "touch" by crontab ensures that even if the backup script fails, the file is updated
The "touch" by the backup script(s) logs that the backup script has been run.

By looking at the canary file on the backup server we can see if the backup script has "touched" the canary file just before sending it.

Multiple wins with a simple text file \o/

more clarification for my previous comment : The "touch" by crontab ensures that even if the backup script fails, the file is updated The "touch" by the backup script(s) logs that the backup script has been run. By looking at the canary file on the backup server we can see if the backup script has "touched" the canary file just before sending it. Multiple wins with a simple text file \o/

👍 1

jlecour commented

2022-05-04 07:46:15 +02:00

Owner

For the statistics there is the --stats flag¹

¹ https://www.mankier.com/1/rsync#--stats

I've checked and we already use the --stats flag

> For the statistics there is the `--stats` flag¹ > > ¹ https://www.mankier.com/1/rsync#--stats I've checked and we already use the `--stats` flag

jlecour commented

2022-05-04 08:24:26 +02:00

Owner

I've made a quick script to easily update the canary file : update-evobackup-canary.

# update-evobackup-canary --help
update-evobackup-canary is updating a canary file for evobackup.

Usage: update-evobackup-canary [OPTIONS]

Main options
 -w, --who      who has updated the file (default: logname())
 -f, --file     path of the canary file (default: /zzz_evobackup_canary)
 -V, --version  print version and exit
 -h, --help     print this message and exit

It works well!

But…

Our current Rsync strategy doesn't include /, which implies that the canary file is not sent to the backup server 🤦

I've made a quick script to easily update the canary file : `update-evobackup-canary`. ``` # update-evobackup-canary --help update-evobackup-canary is updating a canary file for evobackup. Usage: update-evobackup-canary [OPTIONS] Main options -w, --who who has updated the file (default: logname()) -f, --file path of the canary file (default: /zzz_evobackup_canary) -V, --version print version and exit -h, --help print this message and exit ``` It works well! **But**… Our current Rsync strategy doesn't include `/`, which implies that the canary file is not sent to the backup server 🤦