Compare commits

..

1 commit

Author SHA1 Message Date
Brice Waegeneire 497dc26742 nagios-nrpe: check_phpfpm_multi use new locations of pools
All checks were successful
Ansible Lint |Total|New|Outstanding|Fixed|Trend |:-:|:-:|:-:|:-:|:-: |2671|0|2671|0|:zzz:
gitea/ansible-roles/pipeline/head This commit looks good
2024-01-12 13:51:38 +01:00
208 changed files with 2989 additions and 8675 deletions

View file

@ -6,131 +6,25 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
This project does not follow semantic versioning.
The **major** part of the version is the year
The **minor** part is the month
The **patch** part is incremented if multiple releases happen the same month
The **minor** part changes is the month
The **patch** part changes is incremented if multiple releases happen the same month
## [Unreleased]
### Added
### Changed
### Fixed
### Removed
### Security
## [24.05] 2024-05-15
### Added
* apt: add list-upgradable-held-packages.sh
### Changed
* evobackup-client: upstream release 24.05.1
* evolinux-base: improve adding the current user to SSH AllowGroups of AllowUsers
* evolinux-users: improve SSH configuration
* evomaintenance: upstream release 24.05
* evomaintenance: move upstream files into upstream folder
### Fixed
* apt: use archive.debian.org with Buster
## [24.04] 2024-04-30
### Added
* proftpd: optional configuration of IP whitelists per groups of users
### Changed
* autosysadmin-agent: upstream release 24.03.2
* evobackup-client: replace non-functional role with install tasks
* evobackup-client: upstream release 24.04.1
* evolinux-base: Add new variable to disable global customisation of bash config
* evolinux-base: Disable logcheck monitoring of journald only if journald.logfiles exists
* evolinux-users: Add sudo mvcli for nagios user
* haproxy: support bookworm for backport packages
* nrpe: !disk1 exclude filesystem type overlay
* postfix/amavis: max servers is now 3 (previously 2)
* roundcube: Use /var/log/roundcube directly
* vrrpd: configure and restart minifirewall before starting VRRP
* vrrpd: configure minifirewall with blocks instead of lines
### Fixed
* certbot: Fix HAPEE renewal hook
* certbot: Fix HAProxy renewal hook
* evolinux-base/logcheck: fix conf patch, journal check was not disabled when asked
* fail2ban: SQLite purge script didn't vacuum as expected + error when vacuum cannot be done
* keepalived: Fix tasks that use file instead of copy
* memcached: Fix conditions not properly writen (installation was always in multi-instance mode)
* nagios-nrpe: create /etc/bash_completion.d if missing
* openvpn: install packages manually, because openbsd_pkg module is broken since OpenBSD 7.4 with the version of Ansible we currently use
* packweb: fix old bug (2017!) .orig file created by module patch and taken in account by ProFTPd
* redis: replace inline argument with environment variable for the password
### Removed
* docker-host: Removed `docker_conf_use_iptables` variable (iptable usage forced to true)
## [24.03] 2024-03-01
### Added
* autosysadmin-agent: upstream release 24.03
* autosysadmin-restart_nrpe: add role
* certbot: Renewal hook for NRPE
* kvm-host: add minifirewall rules if DRBD interface is configured
* proftpd: add whitelist ip
### Changed
* apt: add ftp.evolix.org as recognized system source
* autosysadmin-agent: logs clearing is done weekly
* autosysadmin-agent: rename /usr/share/scripts/autosysadmin/{auto,restart}
* certbot: use pkey to test the key
* evolinux-base: execute autosysadmin-agent and autosysadmin-restart_nrpe roles
* lxc-php, php: Update sury PGP key
* openvpn: earlier alert for CA expiration
* redis: create sysfs config file if missing
* nextcloud: use latest version by default
### Removed
* autosysadmin: replaced by autosysadmin-agent
## [24.02.1] 2024-02-08
### Fixed
* fail2ban: fix Ansible syntax
## [24.02] 2024-02-08
### Added
* Support for PHP 8.3 with bookworm LXC containers
* Preliminary work for php83
* apt: add task file to install ELTS repository (default: False)
* autosysadmin: Add a role to automatically deploy autosysadmin on evolixisation
* check_free_space: added role
* etc-git: add /var/chroot-bind/etc/bind repo
* fail2ban: add script unban_ip
* generateldif: new Services for check_pressure_{cpu,io,mem}
* kvm-host: Automatically add an LVM filter when LVM is present
* lxc-php: Allow one to install php83 on Bookworm container
* minifirewall: Fix nagios check for old versions of minifirewall
* mongodb: add gpg key for 7.0
* nagios-nrpe: add check_sentinel for monitoring Redis Sentinel
* nagios-nrpe: new check_pressure_{cpu,io,mem}
* remount-usr: do not try to remount /usr RW if /usr is not a mounted partition
* vrrpd: configure minifirewall
* vrrpd: test if interface exists before deleting it
* webapps/evoadmin-mail: package installed via public.evolix.org/evolix repo starting with Bookworm
* webapps/nextcloud: Add condition for archive tasks
* webapps/nextcloud: Add condition for config tasks
* webapps/nextcloud: Added var nextcloud_user_uid to enforce uid for nextcloud user
* webapps/nextcloud: Set ownership and permissions of data directory
@ -138,20 +32,17 @@ The **patch** part is incremented if multiple releases happen the same month
### Changed
* add-vm.sh: allow VM name max length > 20
* amavis: make ldap_suffix mandatory
* apache : fix goaway pattern for bad bots
* apache : rename MaxRequestsPerChild to MaxConnectionsPerChild (new name)
* apache: use backward compatible Redirect directive
* apt: Disable archive repository for Debian 8
* apt: Use the GPG version of the key for Debian 8-9
* bind: Update role for Buster, Bullseye and Bookworm support
* dovecot: add variables for LDAP
* dovecot: Munin plugin conf path is now `/etc/munin/plugin-conf.d/zzz-dovecot` (instead of `z-evolinux-dovecot`)
* evocheck: upstream release 24.01
* evolinux-base: dump-server-state upstream release 23.11
* evolinux-base: use separate default config file for rsyslog
* kvmstats: use .capacity instead of .physical for disk size
* ldap: make ldap_suffix mandatory
* listupgrade : old-kernel-removal.sh upstream release 24.01
* log2mail: move custom config in separate file
* lxc: init /etc git repository in lxc container
@ -159,62 +50,49 @@ The **patch** part is incremented if multiple releases happen the same month
* nagios: add dockerd check in nrpe check template
* nagios: cleaning nrpe check template
* nagios: rename var `nagios_nrpe_process_processes` into `nagios_nrpe_processes` and check systemd-timesyncd instead of ntpd in Debian 12
* nagios: add option --full to check pressure IO and mem to avoid flaps
* proftpd: in SFTP vhost, enable SSH keys login, enable ed25549 host key for Debian >= 11
* redis: manage config template inside a block, to allow custom modifications outside
* spamassassin: Use spamd starting with Bookworm
* squid: config directory seems to have changed from /etc/squid3 to /etc/squid in Debian 8
* spamassassin: Use spamd starting with Bookworm
* unbound: Add config file to allow configuration reload on Debian 11 and lower
* unbound: Add munin configuration & setup plugin
* unbound: Big cleanup
* unbound: Move generated config file to `/etc/unbound/unbound.conf.d/evolinux.conf`
* unbound: Use root hints provided by debian package dns-root-data instead of downloading them
* vrrpd: replace switch script with custom one (fix MAC issue, use `ip(8)`, shell cleanup…)
* vrrpd: variable to force update the switch script (default: false)
* webapps/nextcloud: Add Ceph volume to fstab
* webapps/nextcloud: Set home directory's mode
### Fixed
* Add php-fpm82 to LDAP when relevant
* Check stat.exists before stat.isdir
* check stat.exists before stat.isdir
* apache: fix MaxRequestsPerChild value to be sync with wiki.e.o
* apt: use archive.debian.org with Stretch
* certbot: fix hook for dovecot when more than one certificate is used (eg. different certificates for POP3 and IMAP)
* dovecot: add missing LDAP conf iterate_filter to exclude disabled accounts in users list (caused « User no longer exists » errors in commands listing users like « doveadm user -u '*' » or « doveadm expunge -u "*" mailbox INBOX savedbefore 7d »).
* dovecot: fix missing default mails
* dovecot: fix plugin dovecot1
* evoadmin-web: Fix PHP version for Bookworm
* evolinux-base: fix hardware.yml (wrong repo, missing update cache)
* evolinux-base: start to install linux-image-cloud-amd64 with Buster
* fail2ban: fix template marker
* minifirewall: ports 25, 53, 443, 993, 995 not opened publicly by default anymore, ports 20, 21, 110, 143 not opened semi-publicly by default anymore.
* nagios: fix default file to monitor for check_clamav_db
* nginx: add "when: not ansible_check_mode" in various tasks to prevent fail in check mode
* nginx: fix mistake between "check_mode: no" and "when: not ansible_check_mode" (fail in check mode)
* nginx: fix mistake between "check_mode: no" and "when: not ansible_check_mode" (fail in check mode)
* nginx: keep indentation
* nginx: take care of « already defined » and « not yet defined » server status suffix in check mode
* php: Bullseye/Sury > Honor the php_version asked in the pub.evolix.org repository
* php: drop apt_preferences(5) file for sury
* postfix: remove dependency on evolinux_fqdn var
* proftpd: set missing default listen IP for SFTP
* roundcube: set default SMTP port to 25 instead of 587, which failed because of missing SSL conf (local connexion does not need SSL)
* ssl: no not execute haproxy tasks and reload if haproxy is disabled
* unbound: Add a apt cache validity to enforce an apt update if needed
* webapps/nextcloud: added check that nextcloud uid is over 3000
* webapps/nextcloud: fix Add Ceph volume to fstab : missing UUID= in src
* webapps/nextcloud: fix misplaced gid attribute
* webapps/nextcloud: fix missing gid
* webapps/roundcube & evoadminmail: make roles more idempotent (were failing when played twice)
* amavis: Add variables for generate "ldap_suffix"
* proftpd: fix error when no SSH key is provided
### Removed
* evolinux-base: no need to remove update-evobackup-canary from sbin anymore
* evolinux-base: no need to symlink backup-server-state to dump-server-state anymore
### Security
## [23.10] 2023-10-14
### Added

View file

@ -1,5 +0,0 @@
---
ldap_hostname: "{{ ansible_hostname }}"
ldap_domain: "{{ ansible_domain }}"
ldap_suffix: "dc={{ ldap_hostname }},dc={{ ldap_domain.split('.')[-2] }},dc={{ ldap_domain.split('.')[-1] }}"

View file

@ -6,7 +6,7 @@
- amavisd-new
state: present
tags:
- amavis
- amavis
- name: configure Amavis
ansible.builtin.template:
@ -15,7 +15,7 @@
mode: "0644"
notify: restart amavis
tags:
- amavis
- amavis
- name: Install purge custom cron
ansible.builtin.copy:
@ -23,5 +23,5 @@
dest: /etc/cron.daily/amavis_purge_virusmails
mode: "0755"
tags:
- amavis
- amavis_purge_cron
- amavis
- amavis_purge_cron

View file

@ -39,12 +39,12 @@ $sa_spam_subject_tag = '[SPAM]';
$log_level = 2;
# En fonction besoin/ressources, on a juste le nbre de process
$max_servers = 3;
$max_servers = 2;
$enable_ldap = 1;
$default_ldap = {
hostname => '127.0.0.1', tls => 0,
base => '{{ ldap_suffix | mandatory }}', scope => 'sub',
base => '{{ ldap_suffix }}', scope => 'sub',
query_filter => '(&(mailacceptinggeneralid=%m)(isActive=TRUE))'
};

View file

@ -5,7 +5,6 @@
dest: /etc/apache2/ipaddr_whitelist.conf
line: "Require ip {{ item }}"
state: present
create: yes
loop: "{{ apache_ipaddr_whitelist_present }}"
notify: reload apache
tags:

View file

@ -29,7 +29,4 @@ apt_check_hold_cron_weekday: "*"
apt_check_hold_cron_day: "*"
apt_check_hold_cron_month: "*"
apt_list_upgradable_held_enabled: False
apt_list_upgradable_held_special_time: "weekly"
apt_keyring_dir: "{{ ansible_distribution_major_version is version('12', '<') | ternary('/etc/apt/trusted.gpg.d', '/etc/apt/keyrings') }}"

View file

@ -0,0 +1,3 @@
Package: *
Pin: release a=bookworm-backports
Pin-Priority: 50

View file

@ -0,0 +1,3 @@
Package: *
Pin: release a=bullseye-backports
Pin-Priority: 50

View file

@ -0,0 +1,3 @@
Package: *
Pin: release a=buster-backports
Pin-Priority: 50

View file

@ -1,11 +1,5 @@
#!/usr/bin/env python3
##########
# This script takes a multi-lines input of "oneliner-style" APT sources definitions.
# It converts them into "deb822-style" sources.
# Each generated file will have only one stanza, possibly with multiple Types/Suites/Components
##########
import re
import sys
import os
@ -16,16 +10,11 @@ import apt_pkg
# Order matters !
destinations = {
"debian-security": "security.sources",
".*-backports": "backports.sources",
".debian.org": "system.sources",
"mirror.evolix.org": "system.sources",
"ftp.evolix.org": "system.sources",
"pub.evolix.net": "evolix_public_old.sources.bak",
"pub.evolix.org": "evolix_public.sources",
"artifacts.elastic.co": "elastic.sources",
"download.docker.com": "docker.sources",
"downloads.linux.hpe.com": "hp.sources",
@ -87,11 +76,6 @@ def prepare_sources(lines):
key, value = option.split("=")
options[key] = value
### WARNING ###
# if there are multiple lines with different URIS for a given destination (eg. "system")
# each one will overwrite the previous one
# and the last evaluated will be what remains.
if dest in sources:
sources[dest]["Types"].add(matches["type"])
sources[dest]["URIs"] = matches["uri"]

View file

@ -1,11 +1,5 @@
#!/bin/sh
##########
# This script changes all "one-line" APT sources into "deb822" sources.
# It is responsible for searching and processing the files.
# The actual format migration is done by a python script.
##########
deb822_migrate_script=$(command -v deb822-migration.py)
if [ -z "${deb822_migrate_script}" ]; then
@ -52,4 +46,4 @@ for file in $(find /etc/apt/sources.list.d -mindepth 1 -maxdepth 1 -type f -name
done
echo "${count} file(s) migrated"
exit ${rc}
exit ${rc}

View file

@ -0,0 +1,3 @@
Package: *
Pin: release a=jessie-backports
Pin-Priority: 50

View file

@ -1,127 +0,0 @@
#!/bin/bash
#####
# This script will send an email if some packages are on hold
# but have available updates.
#####
readonly VERSION="24.05"
# set all programs to C language (english)
export LC_ALL=C
# If expansion is attempted on an unset variable or parameter, the shell prints an
# error message, and, if not interactive, exits with a non-zero status.
set -o nounset
# The pipeline's return status is the value of the last (rightmost) command
# to exit with a non-zero status, or zero if all commands exit successfully.
set -o pipefail
# Enable trace mode if called with environment variable TRACE=1
if [[ "${TRACE-0}" == "1" ]]; then
set -o xtrace
fi
# shellcheck disable=SC2155
readonly PROGPATH=$(readlink -m "${0}")
# readonly PROGNAME=$(basename "${PROGPATH}")
# # shellcheck disable=SC2124
# readonly ARGS=$@
# Fetch values from evomaintenance configuration
get_evomaintenance_mail() {
grep "EVOMAINTMAIL=" /etc/evomaintenance.cf | cut -d '=' -f2
}
get_fqdn() {
hostname --fqdn
}
get_complete_hostname() {
REAL_HOSTNAME="$(get_fqdn)"
if [ "${HOSTNAME}" = "${REAL_HOSTNAME}" ]; then
echo "${HOSTNAME}"
else
echo "${HOSTNAME} (${REAL_HOSTNAME})"
fi
}
format_mail() {
cat <<EOTEMPLATE
From: Evolix <${EMAIL_FROM}>
Content-Type: text/plain; charset=UTF-8
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Script: ${PROGPATH}
X-Script-Version: ${VERSION}
To: ${EMAIL_CLIENT:-alert5@evolix.fr}
Subject: Mise a jour manuelle disponible
Bonjour,
Un ou plusieurs paquets dont la mise à jour automatique a été
explicitement bloquée ont une nouvelle version disponible.
Nom du serveur :
${HOSTNAME_TEXT}
Liste des paquets :
${upgradable_held_packages}
Pour que nous appliquions ces mises Ă  jour vous devez
nous contacter explicitement, de préférence par ticket,
en mentionnant le serveur et les paquets concernés,
ainsi que les modalités de mise à jour (créneau horaire,
procédure technique…).
Cordialement
--
Evolix
EOTEMPLATE
}
main() {
held_packages=$(apt-mark showhold)
upgradable_packages=$(apt list --upgradable 2> /dev/null)
if [ -z "${held_packages}" ]; then
# No packages are on hold
exit 0
elif [ -z "${upgradable_packages}" ]; then
# No packages are upgradable
exit 0
fi
kept_back_output=$(LC_ALL=C apt-get upgrade --dry-run | grep -A 1 'The following packages have been kept back:')
if [ -z "${kept_back_output}" ]; then
# No packages are kept back
exit 0
fi
upgradable_held_packages=$(apt list --upgradable 2> /dev/null | grep -f <(echo "${kept_back_output}" | tail -1 | tr ' ' '\n' | sed -e '/^$/d'))
if [ -z "${upgradable_held_packages}" ]; then
# No held packages are upgradable
exit 0
fi
HOSTNAME="$(get_fqdn)"
HOSTNAME_TEXT="$(get_complete_hostname)"
EMAIL_CLIENT="$(get_evomaintenance_mail)"
EMAIL_FROM="equipe@evolix.fr"
MAIL_CONTENT="$(format_mail)"
SENDMAIL_BIN="$(command -v sendmail)"
if [ -z "${SENDMAIL_BIN}" ]; then
>&2 echo "ERROR: No \`sendmail' command has been found, can't send mail."
exit 1
fi
if [ ! -x "${SENDMAIL_BIN}" ]; then
>&2 echo "ERROR: \`${SENDMAIL_BIN}' is not executable, can't send mail."
exit 1
fi
echo "${MAIL_CONTENT}" | "${SENDMAIL_BIN}" -oi -t -f "equipe@evolix.fr"
exit 0
}
main

View file

@ -0,0 +1,3 @@
Package: *
Pin: release a=stretch-backports
Pin-Priority: 50

View file

@ -10,9 +10,19 @@
tags:
- apt
- name: Backports configuration
ansible.builtin.copy:
src: '{{ ansible_distribution_release }}_backports_preferences'
dest: /etc/apt/preferences.d/0-backports-defaults
force: true
mode: "0640"
register: apt_backports_config
tags:
- apt
- name: Apt update
ansible.builtin.apt:
update_cache: yes
when: apt_backports_sources is changed
when: apt_backports_sources is changed or apt_backports_config is changed
tags:
- apt

View file

@ -17,6 +17,16 @@
tags:
- apt
- name: Backports configuration
ansible.builtin.copy:
src: '{{ ansible_distribution_release }}_backports_preferences'
dest: /etc/apt/preferences.d/0-backports-defaults
force: true
mode: "0640"
register: apt_backports_config
tags:
- apt
- name: Archived backport are accepted (jessie)
ansible.builtin.lineinfile:
dest: '/etc/apt/apt.conf.d/99no-check-valid-until'
@ -32,4 +42,4 @@
update_cache: yes
tags:
- apt
when: apt_backports_list is changed
when: apt_backports_list is changed or apt_backports_config is changed

View file

@ -76,16 +76,7 @@
tags:
- apt
- name: List scripts is installed
ansible.builtin.copy:
src: list-upgradable-held-packages.sh
dest: /usr/share/scripts/list-upgradable-held-packages.sh
force: true
mode: "0755"
tags:
- apt
- name: Check if cron is installed
- name: Check if Cron is installed
ansible.builtin.shell:
cmd: "dpkg --list 'cron' 2>/dev/null | grep -q -E '^(i|h)i'"
register: is_cron
@ -110,15 +101,3 @@
tags:
- apt
when: is_cron.rc == 0
- name: List upgradable held packages (script)
ansible.builtin.cron:
cron_file: apt-hold-packages
name: list-upgradable-held-packages
job: "/usr/share/scripts/list-upgradable-held-packages.sh"
user: root
special_time: "{{ apt_list_upgradable_held_special_time | mandatory }}"
state: "{{ apt_list_upgradable_held_enabled | bool | ternary('present', 'absent') }}"
tags:
- apt
when: is_cron.rc == 0

View file

@ -81,7 +81,7 @@
- ansible_distribution_major_version is version('12', '>=')
- name: Install Extended-LTS repositories (Debian < 10)
ansible.builtin.import_tasks: extended-lts.oneline.yml
ansible.builtin.import_tasks: extended-lts.oneline.yml.yml
tags:
- apt
when:

View file

@ -31,11 +31,6 @@
tags:
- apt
- name: Is system.sources present?
ansible.builtin.stat:
path: /etc/apt/sources.list.d/system.sources
register: _system_sources
- name: Add signed-by when relevant for bookworm
ansible.builtin.lineinfile:
dest: /etc/apt/sources.list.d/system.sources
@ -44,12 +39,6 @@
state: present
tags:
- apt
when: _system_sources.stat.exists or not ansible_check_mode
- name: Is security.sources present?
ansible.builtin.stat:
path: /etc/apt/sources.list.d/security.sources
register: _security_sources
- name: Add signed-by when relevant for bookworm-security
ansible.builtin.lineinfile:
@ -59,4 +48,3 @@
state: present
tags:
- apt
when: _security_sources.stat.exists or not ansible_check_mode

View file

@ -1,3 +1,3 @@
# {{ ansible_managed }}
deb http://archive.debian.org/debian buster-backports {{ apt_backports_components | mandatory }}
deb http://mirror.evolix.org/debian buster-backports {{ apt_backports_components | mandatory }}

View file

@ -1,4 +1,5 @@
# {{ ansible_managed }}
deb http://archive.debian.org/debian buster {{ apt_basics_components | mandatory }}
deb http://mirror.evolix.org/debian buster {{ apt_basics_components | mandatory }}
deb http://mirror.evolix.org/debian buster-updates {{ apt_basics_components | mandatory }}
deb http://security.debian.org/debian-security buster/updates {{ apt_basics_components | mandatory }}

View file

@ -1,17 +0,0 @@
---
general_scripts_dir: "/usr/share/scripts"
autosysadmin_agent_bin_dir: "/usr/local/bin/autosysadmin"
autosysadmin_agent_lib_dir: "/usr/local/lib/autosysadmin"
autosysadmin_agent_auto_dir: "{{ general_scripts_dir }}/autosysadmin/restart"
autosysadmin_agent_crontab_enabled: true
autosysadmin_agent_log_retention_days: 365
autosysadmin_config: []
### All repair are disabled if set to 'off'
### even if a specific repair value is 'on'
# repair_all: 'on'
### Default values for checks
# repair_foo: 'off'

View file

@ -1,25 +0,0 @@
#!/bin/bash
days=${1:-365}
log_dir="/var/log/autosysadmin/"
if [ -d "${log_dir}" ]; then
find_run_dirs() {
find "${log_dir}" \
-mindepth 1 \
-maxdepth 1 \
-type d \
-ctime "+${days}" \
-print0
}
log() {
/usr/bin/logger -p local0.notice -t autosysadmin "${1}"
}
while IFS= read -r -d '' run_dir; do
rm --recursive --force "${run_dir}"
log "Delete ${run_dir} (older than ${days} days)"
done < <(find_run_dirs)
fi
exit 0

View file

@ -1,907 +0,0 @@
#!/bin/bash
VERSION="24.03.2"
# Common functions for "repair" and "restart" scripts
set -u
# Initializes the program, context, configuration…
initialize() {
PATH="${PATH}":/usr/sbin:/sbin
# Used in many places to refer to the program name.
# Examples: repair_mysql, restart_nrpe…
PROGNAME=$(basename "${0}")
# find out if running in interactive mode, or not
if [ -t 0 ]; then
INTERACTIVE=1
else
INTERACTIVE=0
fi
readonly INTERACTIVE
# Default empty value for Debug mode
DEBUG="${DEBUG:-""}"
# Repair scripts obey to the value of a variable named after the script
# You can set the value ("on" or "off") in /etc/evolinux/autosysadmin
# Here we set the default value to "on".
declare -g "${PROGNAME}"=on # dynamic variable assignment ($PROGNAME == repair_*)
PID=$$
readonly PID
# Each execution (run) gets a unique ID
RUN_ID="$(date +"%Y-%m-%d_%H-%M")_${PROGNAME}_${PID}"
readonly RUN_ID
# Main log directory
MAIN_LOG_DIR="/var/log/autosysadmin"
readonly MAIN_LOG_DIR
# shellcheck disable=SC2174
mkdir --mode=750 --parents "${MAIN_LOG_DIR}"
chgrp adm "${MAIN_LOG_DIR}"
# Each execution store some information
# in a unique directory based on the RUN_ID
RUN_LOG_DIR="${MAIN_LOG_DIR}/${RUN_ID}"
readonly RUN_LOG_DIR
# shellcheck disable=SC2174
mkdir --mode=750 --parents "${RUN_LOG_DIR}"
chgrp adm "${RUN_LOG_DIR}"
# This log file contains all events
RUN_LOG_FILE="${RUN_LOG_DIR}/autosysadmin.log"
readonly RUN_LOG_FILE
# This log file contains notable actions
ACTIONS_FILE="${RUN_LOG_DIR}/actions.log"
readonly ACTIONS_FILE
touch "${ACTIONS_FILE}"
# This log file contains abort reasons (if any)
ABORT_FILE="${RUN_LOG_DIR}/abort.log"
readonly ABORT_FILE
# touch "${ABORT_FILE}"
# Date format for log messages
DATE_FORMAT="%Y-%m-%d %H:%M:%S"
# This will contain lock, last-run markers…
# It's ok to lose the content after a reboot
RUN_DIR="/run/autosysadmin"
readonly RUN_DIR
mkdir -p "${RUN_DIR}"
# Only a singe instace of each script can run simultaneously
# We use a customizable lock name for this.
# By default it's the script's name
LOCK_NAME=${LOCK_NAME:-${PROGNAME}}
# If a lock is found, we can wait for it to disappear.
# The value must be understood by sleep(1)
LOCK_WAIT="0"
# Default values for email headers
EMAIL_FROM="equipe+autosysadmin@evolix.net"
EMAIL_INTERNAL="autosysadmin@evolix.fr"
LOCK_FILE="${RUN_DIR}/${LOCK_NAME}.lock"
readonly LOCK_FILE
# Remove lock file at exit
cleanup() {
# shellcheck disable=SC2317
rm -f "${LOCK_FILE}"
}
trap 'cleanup' 0
# Load configuration
# shellcheck disable=SC1091
test -f /etc/evolinux/autosysadmin && source /etc/evolinux/autosysadmin
log_all "Begin ${PROGNAME} RUN_ID: ${RUN_ID}"
log_all "Log directory is ${RUN_LOG_DIR}"
}
# Executes a list of tasks before exiting:
# * prepare a summary of actions and possible abort reasons
# * send emails
# * do some cleanup
quit() {
log_all "End ${PROGNAME} RUN_ID: ${RUN_ID}"
summary="RUN_ID: ${RUN_ID}"
if [ -s "${ABORT_FILE}" ]; then
# Add abort reasons to summary
summary="${summary}\n$(print_abort_reasons)"
hook_mail "abort"
return_code=1
else
if [ -s "${ACTIONS_FILE}" ]; then
# Add notable actions to summary
summary="${summary}\n$(print_actions "Aucune action")"
hook_mail "success"
fi
return_code=0
fi
hook_mail "internal"
if is_interactive; then
# shellcheck disable=SC2001
echo "${summary}" | sed -e 's/\\n/\n/g'
else
/usr/share/scripts/evomaintenance.sh --auto --user autosysadmin --message "${summary}" --no-commit --no-mail
fi
teardown
# shellcheck disable=SC2086
exit ${return_code}
}
teardown() {
:
}
# Return true/false
is_interactive() {
test "${INTERACTIVE}" -eq "1"
}
save_server_state() {
DUMP_SERVER_STATE_BIN="$(command -v dump-server-state || command -v backup-server-state)"
if [ -z "${DUMP_SERVER_STATE_BIN}" ]; then
log_all "Warning: dump-server-state is not present. No server state recorded."
fi
if [ -x "${DUMP_SERVER_STATE_BIN}" ]; then
DUMP_DIR=$(file_path_in_log_dir "server-state")
# We don't want the logging to take too much time,
# so we kill it if it takes more than 20 seconds.
timeout --signal 9 20 \
"${DUMP_SERVER_STATE_BIN}" \
--dump-dir="${DUMP_DIR}" \
--df \
--dmesg \
--iptables \
--lxc \
--netcfg \
--netstat \
--uname \
--processes \
--systemctl \
--uptime \
--virsh \
--disks \
--mysql-processes \
--no-apt-states \
--no-apt-config \
--no-dpkg-full \
--no-dpkg-status \
--no-mount \
--no-packages \
--no-sysctl \
--no-etc
log_run "Server state saved in \`server-state' directory."
fi
}
is_debug() {
# first time: do the check…
# other times: pass
if [ -z "${DEBUG:-""}" ]; then
debug_file="/etc/evolinux/autosysadmin.debug"
if [ -e "${debug_file}" ]; then
last_change=$(stat -c %Z "${debug_file}")
limit_date=$(date --date "14400 seconds ago" +"%s")
if [ $(( last_change - limit_date )) -le "0" ]; then
log_run "Debug mode disabled; file is too old (%{last_change} seconds)."
rm "${debug_file}"
# Debug mode disabled
DEBUG="0"
else
log_run "Debug mode enabled."
# Debug mode enabled
DEBUG="1"
fi
else
# log_run "Debug mode disabled; file is absent."
# Debug mode disabled
DEBUG="0"
fi
fi
# return the value
test "${DEBUG}" -eq "1"
}
# Uses the who(1) definition of "active"
currently_active_users() {
LC_ALL=C who --users | grep --extended-regexp "\s+\.\s+" | awk '{print $1}' | sort --human-numeric-sort | uniq
}
# Users active in the last 29 minutes
recently_active_users() {
LC_ALL=C who --users | grep --extended-regexp "\s+00:(0|1|2)[0-9]\s+" | awk --field-separator ' ' '{print $1,$6}'
}
# Save the list of users to a file in the log directory
save_active_users() {
LC_ALL=C who --users | save_in_log_dir "who-users"
}
# An autosysadmin must not perform actions if a user is active or was active recently.
#
# This can by bypassed in interactive mode.
# It's OK to lose this data after a reboot.
ensure_no_active_users_or_exit() {
# Save all active users
save_active_users
if is_debug; then
log_run "Debug mode enabled: continue without checking active users."
return 0;
fi
# Is there any currently active user?
currently_active_users=$(currently_active_users)
if [ -n "${currently_active_users}" ]; then
# shellcheck disable=SC2001
users_oneliner=$(echo "${currently_active_users}" | sed -e 's/\n/ /')
log_run "Currently active users: ${users_oneliner}"
if is_interactive; then
echo "Some users are currently active:"
# shellcheck disable=SC2001
echo "${currently_active_users}" | sed -e 's/\(.\+\)/* \1/'
answer=""
while :; do
printf "> Continue? [Y,n,?] "
read -r answer
case ${answer} in
[Yy]|"" )
log_run "Active users check bypassed manually in interactive mode."
return
;;
[Nn] )
log_run "Active users check confirmed manually in interactive mode."
log_abort_and_quit "Active users detected: ${users_oneliner}"
;;
* )
printf "y - yes, continue\n"
printf "n - no, exit\n"
printf "? - print this help\n"
;;
esac
done
else
log_abort_and_quit "Currently active users detected: ${users_oneliner}."
fi
else
# or recently (the last 30 minutes) active user?
recently_active_users=$(recently_active_users)
if [ -n "${recently_active_users}" ]; then
# shellcheck disable=SC2001
users_oneliner=$(echo "${recently_active_users}" | sed -e 's/\n/ /')
log_run "Recently active users: ${users_oneliner}"
if is_interactive; then
echo "Some users were recently active:"
# shellcheck disable=SC2001
echo "${recently_active_users}" | sed -e 's/\(.\+\)/* \1/'
answer=""
while :; do
printf "> Continue? [Y,n,?] "
read -r answer
case ${answer} in
[Yy]|"" )
log_run "Active users check bypassed manually in interactive mode."
return
;;
[Nn] )
log_run "Active users check confirmed manually in interactive mode."
log_abort_and_quit "Recently active users detected: ${users_oneliner}."
;;
* )
printf "y - yes, continue\n"
printf "n - no, exit\n"
printf "? - print this help\n"
;;
esac
done
else
log_abort_and_quit "Recently active users detected: ${users_oneliner}."
fi
fi
fi
}
# Takes an NRPE command name as 1st parameter,
# and executes the full command if found in the configuration.
# Return the result and the return code of the command.
check_nrpe() {
check="$1"
nrpe_files=""
# Check if NRPE config is found
if [ -f "/etc/nagios/nrpe.cfg" ]; then
nrpe_files="${nrpe_files} /etc/nagios/nrpe.cfg"
else
msg="NRPE configuration not found: /etc/nagios/nrpe.cfg"
log_run "${msg}"
echo "${msg}"
return 3
fi
# Search for included files
# shellcheck disable=SC2086
while IFS= read -r include_file; do
nrpe_files="${nrpe_files} ${include_file}"
done < <(grep --extended-regexp '^\s*include=.+' ${nrpe_files} | cut -d = -f 2)
# Search for files in included directories
# shellcheck disable=SC2086
while IFS= read -r include_dir; do
nrpe_files="${nrpe_files} ${include_dir}/*.cfg"
done < <(grep --extended-regexp '^\s*include_dir=.+' ${nrpe_files} | cut -d = -f 2)
# Fetch uncommented commands in (sorted) config files
# shellcheck disable=SC2086
nrpe_commands=$(grep --no-filename --exclude=*~ --fixed-strings "[${check}]" ${nrpe_files} | grep --invert-match --extended-regexp '^\s*#\s*command' | cut -d = -f 2)
nrpe_commands_count=$(echo "${nrpe_commands}" | wc -l)
if is_debian_version "9" "<=" && [ "${nrpe_commands_count}" -gt "1" ]; then
# On Debian <= 9, NRPE loading was not sorted
# we need to raise an error if we have multiple defined commands
msg="Unable to determine which NRPE command to run"
log_run "${msg}"
echo "${msg}"
return 3
else
# On Debian > 9, use the last command
nrpe_command=$(echo "${nrpe_commands}" | tail -n 1)
nrpe_result=$(${nrpe_command})
nrpe_rc=$?
log_run "NRPE command (exited with ${nrpe_rc}): ${nrpe_command}"
log_run "${nrpe_result}"
echo "${nrpe_result}"
return "${nrpe_rc}"
fi
}
# An autosysadmin script must not run twice (or more) simultaneously.
# We use a customizable (with LOCK_NAME) lock file to keep track of this.
# A wait time can be configured.
#
# This can by bypassed in interactive mode.
# It's OK to lose this data after a reboot.
acquire_lock_or_exit() {
lock_file="${1:-${LOCK_FILE}}"
lock_wait="${2:-${LOCK_WAIT}}"
# lock_wait must be compatible with sleep(1), otherwise fallback to 0
if ! echo "${lock_wait}" | grep -Eq '^[0-9]+[smhd]?$'; then
log_run "Lock wait: incorrect value '${lock_wait}', fallback to 0."
lock_wait=0
fi
if [ "${lock_wait}" != "0" ] && [ -f "${lock_file}" ]; then
log_run "Lock file present. Let's wait ${lock_wait} and check again."
sleep "${lock_wait}"
fi
if [ -f "${lock_file}" ]; then
log_abort_and_quit "Lock file still present."
else
log_run "Lock file absent. Let's put one."
touch "${lock_file}"
fi
}
# If a script has been run in the ast 30 minutes, running it again won't fix the issue.
# We use a /run/ausosysadmin/${PROGNAME}_lastrun file to keep track of this.
#
# This can by bypassed in interactive mode.
# This is bypassed in debug mode.
# It's OK to lose this data after a reboot.
ensure_not_too_soon_or_exit() {
if is_debug; then
log_run "Debug mode enabled: continue without checking when was the last run."
return 0;
fi
lastrun_file="${RUN_DIR}/${PROGNAME}_lastrun"
if [ -f "${lastrun_file}" ]; then
lastrun_age="$(($(date +%s)-$(stat -c "%Y" "${lastrun_file}")))"
log_run "Last run was ${lastrun_age} seconds ago."
if [ "${lastrun_age}" -lt 1800 ]; then
if is_interactive; then
echo "${PROGNAME} was run ${lastrun_age} seconds ago."
answer=""
while :; do
printf "> Continue? [Y,n,?] "
read -r answer
case ${answer} in
[Yy]|"" )
log_run "Last run check bypassed manually in interactive mode."
break
;;
[Nn] )
log_run "Last run check confirmed manually in interactive mode."
log_abort_and_quit 'Last run too recent.'
;;
* )
printf "y - yes, continue\n"
printf "n - no, exit\n"
printf "? - print this help\n"
;;
esac
done
else
log_abort_and_quit "Last run too recent."
fi
fi
fi
touch "${lastrun_file}"
}
# Populate DEBIAN_VERSION and DEBIAN_RELEASE variables
# based on gathered information about the operating system
detect_os() {
DEBIAN_RELEASE="unknown"
DEBIAN_VERSION="unknown"
LSB_RELEASE_BIN="$(command -v lsb_release)"
if [ -e /etc/debian_version ]; then
DEBIAN_VERSION="$(cut -d "." -f 1 < /etc/debian_version)"
if [ -x "${LSB_RELEASE_BIN}" ]; then
DEBIAN_RELEASE="$("${LSB_RELEASE_BIN}" --codename --short)"
else
case "${DEBIAN_VERSION}" in
7) DEBIAN_RELEASE="wheezy" ;;
8) DEBIAN_RELEASE="jessie" ;;
9) DEBIAN_RELEASE="stretch" ;;
10) DEBIAN_RELEASE="buster" ;;
11) DEBIAN_RELEASE="bullseye" ;;
12) DEBIAN_RELEASE="bookworm" ;;
13) DEBIAN_RELEASE="trixie" ;;
esac
fi
# log_run "Detected OS: Debian version=${DEBIAN_VERSION} release=${DEBIAN_RELEASE}"
# else
# log_run "Detected OS: unknown (missing /etc/debian_version)"
fi
}
is_debian_wheezy() {
test "${DEBIAN_RELEASE}" = "wheezy"
}
is_debian_jessie() {
test "${DEBIAN_RELEASE}" = "jessie"
}
is_debian_stretch() {
test "${DEBIAN_RELEASE}" = "stretch"
}
is_debian_buster() {
test "${DEBIAN_RELEASE}" = "buster"
}
is_debian_bullseye() {
test "${DEBIAN_RELEASE}" = "bullseye"
}
is_debian_bookworm() {
test "${DEBIAN_RELEASE}" = "bookworm"
}
is_debian_trixie() {
test "${DEBIAN_RELEASE}" = "trixie"
}
is_debian_version() {
local version=$1
local relation=${2:-"eq"}
if [ -z "${DEBIAN_VERSION:-""}" ]; then
detect_os
fi
dpkg --compare-versions "${DEBIAN_VERSION}" "${relation}" "${version}"
}
# List systemd services (only names), even if stopped
systemd_list_services() {
pattern=$1
systemctl list-units --all --no-legend --type=service "${pattern}" | grep --only-matching --extended-regexp '\S+\.service'
}
is_systemd_enabled() {
systemctl --quiet is-enabled "$1" 2> /dev/null
}
is_systemd_active() {
systemctl --quiet is-active "$1" 2> /dev/null
}
is_sysvinit_enabled() {
find /etc/rc2.d/ -name "$1" > /dev/null
}
get_fqdn() {
# shellcheck disable=SC2155
local system=$(uname -s)
if [ "${system}" = "Linux" ]; then
hostname --fqdn
elif [ "${system}" = "OpenBSD" ]; then
hostname
else
log_abort_and_quit "System '${system}' not recognized."
fi
}
get_complete_hostname() {
REAL_HOSTNAME="$(get_fqdn)"
if [ "${HOSTNAME}" = "${REAL_HOSTNAME}" ]; then
echo "${HOSTNAME}"
else
echo "${HOSTNAME} (${REAL_HOSTNAME})"
fi
}
# Fetch values from evomaintenance configuration
get_evomaintenance_mail() {
grep "EVOMAINTMAIL=" /etc/evomaintenance.cf | cut -d '=' -f2
}
get_evomaintenance_emergency_mail() {
grep "URGENCYFROM=" /etc/evomaintenance.cf | cut -d '=' -f2
}
get_evomaintenance_emergency_tel() {
grep "URGENCYTEL=" /etc/evomaintenance.cf | cut -d '=' -f2
}
# Log a message to the log file in the log directory
log_run() {
local msg="${1:-$(cat /dev/stdin)}"
# shellcheck disable=SC2155
local date=$(/bin/date +"${DATE_FORMAT}")
printf "[%s] %s[%s]: %s\\n" \
"${date}" "${PROGNAME}" "${PID}" "${msg}" \
>> "${RUN_LOG_FILE}"
}
# Log a message in the system log file (syslog or journald)
log_global() {
local msg="${1:-$(cat /dev/stdin)}"
echo "${msg}" \
| /usr/bin/logger -p local0.notice -t autosysadmin
}
# Log a message in both places
log_all() {
local msg="${1:-$(cat /dev/stdin)}"
log_global "${msg}"
log_run "${msg}"
}
# Log a notable action in regular places
# and append it to the dedicated list
log_action() {
log_all "$*"
append_action "$*"
}
# Append a line in the actions.log file in the log directory
append_action() {
echo "$*" >> "${ACTIONS_FILE}"
}
# Print the content of the actions.log file
# or a fallback content (1st parameter) if empty
# shellcheck disable=SC2120
print_actions() {
local fallback=${1:-""}
if [ -s "${ACTIONS_FILE}" ]; then
cat "${ACTIONS_FILE}"
elif [ -n "${fallback}" ]; then
echo "${fallback}"
fi
}
# Log a an abort reason in regular places
# and append it to the dedicated list
log_abort() {
log_all "$*"
append_abort_reason "$*"
}
# Append a line in the abort.log file in the log directory
append_abort_reason() {
echo "$*" >> "${ABORT_FILE}"
}
# Print the content of the abort.log file
# or a fallback content (1st parameter) if empty
# shellcheck disable=SC2120
print_abort_reasons() {
local fallback=${1:-""}
if [ -s "${ABORT_FILE}" ]; then
cat "${ABORT_FILE}"
elif [ -n "${fallback}" ]; then
echo "${fallback}"
fi
}
# Print the content of the main log from the log directory
print_main_log() {
cat "${RUN_LOG_FILE}"
}
# Log an abort reason and quit the script
log_abort_and_quit() {
log_abort "$*"
quit
}
# Store the content from standard inpu
# into a file in the log directory named after the 1st parameter
save_in_log_dir() {
local file_name=$1
local file_path="${RUN_LOG_DIR}/${file_name}"
cat /dev/stdin > "${file_path}"
log_run "Saved \`${file_name}' file."
}
# Return the full path of the file in log directory
# based on the name in the 1st parameter
file_path_in_log_dir() {
echo "${RUN_LOG_DIR}/${1}"
}
format_mail_success() {
cat <<EOTEMPLATE
From: AutoSysadmin Evolix <${EMAIL_FROM}>
Content-Type: text/plain; charset=UTF-8
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Script: ${PROGNAME}
X-RunId: ${RUN_ID}
To: ${EMAIL_CLIENT:-alert5@evolix.fr}
Cc: ${EMAIL_INTERNAL}
Subject: [autosysadmin] Intervention automatisée sur ${HOSTNAME_TEXT}
Bonjour,
Une intervention automatisée vient de se terminer.
Nom du serveur : ${HOSTNAME_TEXT}
Heure d'intervention : $(LC_ALL=fr_FR.utf8 date)
Script déclenché : ${PROGNAME}
### Actions réalisées
$(print_actions "Aucune")
### RĂ©agir Ă  cette intervention
Vous pouvez répondre à ce message (${EMAIL_FROM}).
En cas d'urgence, utilisez l'adresse ${EMERGENCY_MAIL}
ou notre ligne d'astreinte (${EMERGENCY_TEL})
--
Votre AutoSysadmin
EOTEMPLATE
}
format_mail_abort() {
cat <<EOTEMPLATE
From: AutoSysadmin Evolix <${EMAIL_FROM}>
Content-Type: text/plain; charset=UTF-8
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Script: ${PROGNAME}
X-RunId: ${RUN_ID}
To: ${EMAIL_CLIENT:-alert5@evolix.fr}
Cc: ${EMAIL_INTERNAL}
Subject: [autosysadmin] Intervention automatisée interrompue sur ${HOSTNAME_TEXT}
Bonjour,
Une intervention automatisée a été déclenchée mais s'est interrompue.
Nom du serveur : ${HOSTNAME_TEXT}
Heure d'intervention : $(LC_ALL=fr_FR.utf8 date)
Script déclenché : ${PROGNAME}
### Actions réalisées
$(print_actions "Aucune")
### Raison(s) de l'interruption
$(print_abort_reasons "Inconnue")
### RĂ©agir Ă  cette intervention
Vous pouvez répondre à ce message (${EMAIL_FROM}).
En cas d'urgence, utilisez l'adresse ${EMERGENCY_MAIL}
ou notre ligne d'astreinte (${EMERGENCY_TEL})
--
Votre AutoSysadmin
EOTEMPLATE
}
# shellcheck disable=SC2028
print_report_information() {
echo "**Uptime**"
echo ""
uptime
echo ""
echo "**Utilisateurs récents**"
echo ""
who_file=$(file_path_in_log_dir "who-users")
if [ -s "${who_file}" ]; then
cat "${who_file}"
else
who --users
fi
echo ""
echo "**Espace disque**"
echo ""
df_file=$(file_path_in_log_dir "server-state/df.txt")
if [ -s "${df_file}" ]; then
cat "${df_file}"
else
df -h
fi
echo ""
echo "**Dmesg**"
echo ""
dmesg_file=$(file_path_in_log_dir "server-state/dmesg.txt")
if [ -s "${dmesg_file}" ]; then
tail -n 5 "${dmesg_file}"
else
dmesg | tail -n 5
fi
echo ""
echo "**systemd failed services**"
echo ""
failed_services_file=$(file_path_in_log_dir "server-state/systemctl-failed-services.txt")
if [ -s "${failed_services_file}" ]; then
cat "${failed_services_file}"
else
systemctl --no-legend --state=failed --type=service
fi
if command -v lxc-ls > /dev/null 2>&1; then
echo ""
echo "**LXC containers**"
echo ""
lxc_ls_file=$(file_path_in_log_dir "server-state/lxc-list.txt")
if [ -s "${lxc_ls_file}" ]; then
cat "${lxc_ls_file}"
else
lxc-ls --fancy
fi
fi
apache_errors_file=$(file_path_in_log_dir "apache-errors.log")
if [ -f "${apache_errors_file}" ]; then
echo ""
echo "**Apache errors**"
echo ""
cat "${apache_errors_file}"
fi
nginx_errors_file=$(file_path_in_log_dir "nginx-errors.log")
if [ -f "${nginx_errors_file}" ]; then
echo ""
echo "**Nginx errors**"
echo ""
cat "${nginx_errors_file}"
fi
}
format_mail_internal() {
cat <<EOTEMPLATE
From: AutoSysadmin Evolix <${EMAIL_FROM}>
Content-Type: text/plain; charset=UTF-8
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Script: ${PROGNAME}
X-RunId: ${RUN_ID}
To: ${EMAIL_INTERNAL}
Subject: [autosysadmin] Rapport interne d'intervention sur ${HOSTNAME_TEXT}
Bonjour,
Une intervention automatique vient de se terminer.
Nom du serveur : ${HOSTNAME_TEXT}
Heure d'intervention : $(LC_ALL=fr_FR.utf8 date)
Script déclenché : ${PROGNAME}
### Actions réalisées
$(print_actions "Aucune")
### Raison(s) de l'interruption
$(print_abort_reasons "Aucune")
### Log autosysadmin
$(print_main_log)
### Informations additionnelles
$(print_report_information)
--
Votre AutoSysadmin
EOTEMPLATE
}
# Generic function to send emails at the end of the script.
# Takes a template as 1st parameter
hook_mail() {
if is_debug; then
log_run "Debug mode enabled: continue without sending mail."
return 0;
fi
HOSTNAME="${HOSTNAME:-"$(get_fqdn)"}"
HOSTNAME_TEXT="$(get_complete_hostname)"
EMAIL_CLIENT="$(get_evomaintenance_mail)"
EMERGENCY_MAIL="$(get_evomaintenance_emergency_mail)"
EMERGENCY_TEL="$(get_evomaintenance_emergency_tel)"
MAIL_CONTENT="$(format_mail_"$1")"
SENDMAIL_BIN="$(command -v sendmail)"
if [ -z "${SENDMAIL_BIN}" ]; then
log_global "ERROR: No \`sendmail' command has been found, can't send mail."
fi
if [ -x "${SENDMAIL_BIN}" ]; then
echo "${MAIL_CONTENT}" | "${SENDMAIL_BIN}" -oi -t -f "equipe@evolix.fr"
log_global "Sent '$1' mail for RUN_ID: ${RUN_ID}"
fi
}
is_holiday() {
# gcal mark today as a holiday by surrounding with < and > the day
# of the month of that holiday line. For example if today is 2022-05-01 we'll
# get among other lines:
# FĂŞte du Travail (FR) + Di, < 1>Mai 2022
# Jour de la Victoire (FR) + Di, : 8:Mai 2022 = +7 jours
LANGUAGE=fr_FR.UTF-8 TZ=Europe/Paris gcal --cc-holidays=fr --holiday-list=short | grep -E '<[0-9 ]{2}>' --quiet
}
is_weekend() {
day_of_week=$(date +%u)
if [ "${day_of_week}" != 6 ] && [ "${day_of_week}" != 7 ]; then
return 1
fi
}
is_workday() {
if is_holiday || is_weekend; then
return 1
fi
}
is_worktime() {
if ! is_workday; then
return 1
fi
hour=$(date +%H)
if [ "${hour}" -lt 9 ] || { [ "${hour}" -ge 12 ] && [ "${hour}" -lt 14 ] ; } || [ "${hour}" -ge 18 ]; then
return 1
fi
}

View file

@ -1,112 +0,0 @@
#!/bin/bash
# Specific functions for "repair" scripts
is_all_repair_disabled() {
# Fetch values from the config
# and if it is not defined or has no value, then assign "on"
local status=${repair_all:=on}
test "${status}" = "off" || test "${status}" = "0"
}
is_current_repair_disabled() {
# Fetch values from the config
# and if it is not defined or has no value, then assign "on"
local status=${!PROGNAME:=on}
test "${status}" = "off" || test "${status}" = "0"
}
ensure_not_disabled_or_exit() {
if is_all_repair_disabled; then
log_global 'All repair scripts are disabled.'
exit 0
fi
if is_current_repair_disabled; then
log_global "Current repair script (${PROGNAME}) is disabled."
exit 0
fi
}
# Set of actions to do at the begining of a "repair" script
pre_repair() {
initialize
# Are we supposed to run?
ensure_not_disabled_or_exit
# Has it recently been run?
ensure_not_too_soon_or_exit
# Can we acquire a lock?
acquire_lock_or_exit
# Is there any active user?
ensure_no_active_users_or_exit
# Save important information
save_server_state
}
# Set of actions to do at the end of a "repair" script
post_repair() {
quit
}
repair_lxc_php() {
container_name=$1
if is_systemd_enabled 'lxc.service'; then
lxc_path=$(lxc-config lxc.lxcpath)
if lxc-info --name "${container_name}" > /dev/null; then
rootfs="${lxc_path}/${container_name}/rootfs"
case "${container_name}" in
php56) fpm_log_file="${rootfs}/var/log/php5-fpm.log" ;;
php70) fpm_log_file="${rootfs}/var/log/php7.0-fpm.log" ;;
php73) fpm_log_file="${rootfs}/var/log/php7.3-fpm.log" ;;
php74) fpm_log_file="${rootfs}/var/log/php7.4-fpm.log" ;;
php80) fpm_log_file="${rootfs}/var/log/php8.0-fpm.log" ;;
php81) fpm_log_file="${rootfs}/var/log/php8.1-fpm.log" ;;
php82) fpm_log_file="${rootfs}/var/log/php8.2-fpm.log" ;;
php83) fpm_log_file="${rootfs}/var/log/php8.3-fpm.log" ;;
*)
log_abort_and_quit "Unknown container '${container_name}'"
;;
esac
# Determine FPM Pool path
php_path_pool=$(find "${lxc_path}/${container_name}/" -type d -name "pool.d")
# Save LXC info (before restart)
lxc-info --name "${container_name}" | save_in_log_dir "lxc-${container_name}.before.status"
# Save last lines of FPM log (before restart)
tail "${fpm_log_file}" | save_in_log_dir "$(basename "${fpm_log_file}" | sed -e 's/.log/.before.log/')"
# Save NRPE check (before restart)
/usr/local/lib/nagios/plugins/check_phpfpm_multi "${php_path_pool}" | save_in_log_dir "check_fpm_${container_name}.before.out"
lxc-stop --timeout 20 --name "${container_name}"
lxc-start --daemon --name "${container_name}"
rc=$?
if [ "${rc}" -eq "0" ]; then
log_action "Restart LXC container '${container_name}: OK"
else
log_action "Restart LXC container '${container_name}: failed"
fi
# Save LXC info (after restart)
lxc-info --name "${container_name}" | save_in_log_dir "lxc-${container_name}.after.status"
# Save last lines of FPM log (after restart)
tail "${fpm_log_file}" | save_in_log_dir "$(basename "${fpm_log_file}" | sed -e 's/.log/.after.log/')"
# Save NRPE check (after restart)
/usr/local/lib/nagios/plugins/check_phpfpm_multi "${php_path_pool}" | save_in_log_dir "check_fpm_${container_name}.after.out"
else
log_abort_and_quit "LXC container '${container_name}' doesn't exist."
fi
else
log_abort_and_quit 'LXC not found.'
fi
}

View file

@ -1,76 +0,0 @@
#!/bin/bash
# Specific functions for "restart" scripts
running_custom() {
# Placeholder that returns 1, to prevent running if not redefined
log_global "running_custom() function has not been redefined! Let's quit."
return 1
}
# Examine RUNNING variable and decide if the script should run or not
is_supposed_to_run() {
if is_debug; then return 0; fi
case ${RUNNING} in
never)
# log_global "is_supposed_to_run: no (never)"
return 1
;;
always)
# log_global "is_supposed_to_run: yes (always)"
return 0
;;
nwh-fr)
! is_worktime
rc=$?
# if [ ${rc} -eq 0 ]; then
# log_global "is_supposed_to_run: yes (nwh-fr returned ${rc})"
# else
# log_global "is_supposed_to_run: no (nwh-fr returned ${rc})"
# fi
return ${rc}
;;
nwh-ca)
# Not implemented yet
return 0
;;
custom)
running_custom
rc=$?
# if [ ${rc} -eq 0 ]; then
# log_global "is_supposed_to_run: yes (custom returned ${rc})"
# else
# log_global "is_supposed_to_run: no (custom returned ${rc})"
# fi
return ${rc}
;;
esac
}
ensure_supposed_to_run_or_exit() {
if ! is_supposed_to_run; then
# simply quit (no logging, no notifications…)
# log_global "${PROGNAME} is not supposed to run (RUNNING=${RUNNING})."
exit 0
fi
}
# Set of actions to do at the begining of a "restart" script
pre_restart() {
initialize
# Has it recently been run?
ensure_not_too_soon_or_exit
# Can we acquire a lock?
acquire_lock_or_exit
# Save important information
save_server_state
}
# Set of actions to do at the end of a "restart" script
post_restart() {
quit
}

View file

@ -1,157 +0,0 @@
#!/bin/bash
: "${AUTOSYSADMIN_LIB:=/usr/local/lib/autosysadmin}"
source "${AUTOSYSADMIN_LIB}/common.sh" || exit 1
source "${AUTOSYSADMIN_LIB}/repair.sh" || exit 1
pre_repair
# We always keep some reserved blocks to avoid missing some logs
# https://gitea.evolix.org/evolix/autosysadmin/issues/22
RESERVED_BLOCKS_MIN=1
get_mountpoints() {
# the $(...) get the check_disk1 command
# the cut command selects the critical part of the check_disk1 output
# the grep command extracts the mountpoints and available disk space
# the last cut command selects the mountpoints
check_disk1_command=$(grep check_disk1 /etc/nagios/nrpe.d/evolix.cfg | cut -d'=' -f2-)
${check_disk1_command} -e | cut -d'|' -f1 | grep --extended-regexp --only-matching '/[[:graph:]]* [0-9]+ [A-Z][A-Z]' | cut -d' ' -f1
}
is_reserved_blocks_nominal() {
partition=${1}
fs_type="$(findmnt -n --output=fstype "${partition}")"
if [ "${fs_type}" = "ext4" ]; then
device="$(findmnt -n --output=source "${partition}")"
reserved_block_count="$(tune2fs -l "${device}" | grep 'Reserved block count' | awk -F':' '{ gsub (" ", "", $0); print $2}')"
block_count="$(tune2fs -l "${device}" | grep 'Block count' | awk -F':' '{ gsub (" ", "", $0); print $2}')"
percentage=$(awk "BEGIN { pc=100*${reserved_block_count}/${block_count}; i=int(pc); print (pc-i<0.5)?i:i+1 }")
log_run "Reserved blocks for ${partition} is currently at ${percentage}%"
if [ "${percentage}" -gt "${RESERVED_BLOCKS_MIN}" ]; then
log_run "Allowing tune2fs action to reduce the number of reserved blocks"
return 0
else
log_run "Reserved blocks already at or bellow ${RESERVED_BLOCKS_MIN}%, no automatic action possible"
return 1
fi
else
log_run "Filesystem for ${partition} (${fs_type}) is incompatible with reserved block reduction."
return 1
fi
}
reduce_reserved_blocks() {
partition=${1}
device=$(findmnt -n --output=source "${partition}")
tune2fs -m "${RESERVED_BLOCKS_MIN}" "${device}"
log_action "Reserved blocks for ${partition} changed to ${RESERVED_BLOCKS_MIN} percent"
}
is_tmp_to_delete() {
size="$(find /var/log/ -type f -ctime +1 -exec du {} \+ | awk '{s+=$1}END{print s / 1024}')"
if [ -n "${size}" ]; then
return 0
else
return 1
fi
}
is_log_to_delete() {
size="$(find /var/log/ -type f -mtime +365 -exec du {} \+ | awk '{s+=$1}END{print s / 1024}')"
if [ -n "${size}" ]; then
return 0
else
return 1
fi
}
clean_apt_cache() {
for container in $(lxc-ls -1); do
if [ -e "$(lxc-config lxc.lxcpath)/${container}/rootfs/var/cache" ]; then
lxc-attach --name "${container}" -- apt-get clean
log_action "Clean apt cache in LXC container ${container}";
fi
done
# NOTE: "head -n 1" might be superfluous, but let's be sure to have only the first returned value
biggest_subdir=$(du --summarize --one-file-system "/var/*" | sort --numeric-sort --reverse | sed 's/^[0-9]\+[[:space:]]\+//;q' | head -n 1)
case "${biggest_subdir}" in
'/var/cache')
apt-get clean
log_action 'Clean apt cache'
;;
esac
}
clean_amavis_virusmails() {
if du --inodes /var/lib/* | sort --numeric-sort | tail -n 3 | grep --quiet 'virusmails$'; then
find /var/lib/amavis/virusmails/ -type f -atime +30 -delete
log_action 'Clean amavis infected mails'
fi
}
critical_mountpoints=$(get_mountpoints)
if [ -z "${critical_mountpoints}" ]; then
log_abort_and_quit "No partition is in critical state, nothing left to do."
else
for mountpoint in ${critical_mountpoints}; do
case "${mountpoint}" in
/var)
#if is_log_to_delete
#then
# find /var/log/ -type f -mtime +365 -delete
# log_action "$size Mo of disk space freed in /var"
#fi
if is_reserved_blocks_nominal /var; then
reduce_reserved_blocks /var
clean_apt_cache
clean_amavis_virusmails
fi
;;
/tmp)
#if is_tmp_to_delete
#then
# find /tmp/ -type f -ctime +1 -delete
# log_action "$size Mo of disk space freed in /tmp"
#fi
if is_reserved_blocks_nominal /tmp; then
reduce_reserved_blocks /tmp
fi
;;
/home)
if is_reserved_blocks_nominal /home; then
reduce_reserved_blocks /home
fi
;;
/srv)
if is_reserved_blocks_nominal /srv; then
reduce_reserved_blocks /srv
fi
;;
/filer)
if is_reserved_blocks_nominal /filer; then
reduce_reserved_blocks /filer
fi
;;
/)
if is_reserved_blocks_nominal /; then
reduce_reserved_blocks /
# Suggest remove old kernel ?
fi
;;
*)
# unknown
log_run 'Unknown partition (or weird case) or nothing to do'
;;
esac
done
fi
post_repair

View file

@ -1,35 +0,0 @@
#!/bin/bash
: "${AUTOSYSADMIN_LIB:=/usr/local/lib/autosysadmin}"
source "${AUTOSYSADMIN_LIB}/common.sh" || exit 1
source "${AUTOSYSADMIN_LIB}/repair.sh" || exit 1
pre_repair
service="elasticsearch.service"
service_name="elasticsearch"
if is_systemd_enabled "${service}"; then
if is_systemd_active "${service}"; then
log_abort_and_quit "${service} is active, nothing left to do."
else
# Save service status before restart
systemctl status "${service}" | save_in_log_dir "${service_name}.before.status"
# Try to restart
timeout 20 systemctl restart "${service}" > /dev/null
rc=$?
if [ "${rc}" -eq "0" ]; then
log_action "Restart ${service_name}: OK"
else
log_action "Restart ${service_name}: failed"
fi
# Save service status after restart
systemctl status "${service}" | save_in_log_dir "${service_name}.after.status"
fi
else
log_abort_and_quit "${service} is disabled (or missing), nothing left to do."
fi
post_repair

View file

@ -1,131 +0,0 @@
#!/bin/bash
: "${AUTOSYSADMIN_LIB:=/usr/local/lib/autosysadmin}"
source "${AUTOSYSADMIN_LIB}/common.sh" || exit 1
source "${AUTOSYSADMIN_LIB}/repair.sh" || exit 1
pre_repair
## Apache
service="apache2.service"
service_name="apache2"
if is_systemd_enabled "${service}"; then
if is_systemd_active "${service}"; then
log_all "${service} is active. Skip."
else
# Save service status before restart
systemctl status "${service}" | save_in_log_dir "${service_name}.before.status"
# check syntax
if apache2ctl -t > /dev/null 2>&1; then
# Try to restart
timeout 20 systemctl restart "${service}" > /dev/null
rc=$?
if [ "${rc}" -eq "0" ]; then
log_action "Restart ${service_name}: OK"
else
log_action "Restart ${service_name}: failed"
fi
# Save service status after restart
systemctl status "${service}" | save_in_log_dir "${service_name}.after.status"
# Save error logs
date=$(LANG=en_US.UTF-8 date '+%b %d')
grep "${date}" /home/*/log/error.log /var/log/apache2/*error.log \
| grep -v \
-e "Got error 'PHP message:" \
-e "No matching DirectoryIndex" \
-e "client denied by server configuration" \
-e "server certificate does NOT include an ID which matches the server name" \
| save_in_log_dir "apache-errors.log"
else
log_action "Restart ${service_name}: skip (invalid configuration)"
fi
fi
else
log_all "${service} is disabled (or missing). Skip."
fi
## Nginx
service="nginx.service"
service_name="nginx"
if is_systemd_enabled "${service}"; then
if is_systemd_active "${service}"; then
log_all "${service} is active. Skip."
else
# Save service status before restart
systemctl status "${service}" | save_in_log_dir "${service_name}.before.status"
# check syntax
if nginx -t > /dev/null 2>&1; then
# Try to restart
timeout 20 systemctl restart "${service}" > /dev/null
rc=$?
if [ "${rc}" -eq "0" ]; then
log_action "Restart ${service_name}: OK"
else
log_action "Restart ${service_name}: failed"
fi
# Save service status after restart
systemctl status "${service}" | save_in_log_dir "${service_name}.after.status"
# Save error logs
### Consider doing for Nginx the same as Apache
else
log_action "Restart ${service_name}: skip (invalid configuration)"
fi
fi
else
log_all "${service} is disabled (or missing). Skip."
fi
## LXC
if is_systemd_enabled 'lxc.service'; then
for container in $(lxc-ls -1 | grep --fixed-strings 'php' | grep --extended-regexp --invert-match --regexp '\bold\b' --regexp '\bdisabled\b'); do
repair_lxc_php "${container}"
done
else
log_all "LXC is disabled (or missing). Skip."
fi
## FPM
fpm_services=$(systemd_list_services 'php*-fpm*')
if [ -n "${fpm_services}" ]; then
for service in ${fpm_services}; do
service_name="${service//.service/}"
if is_systemd_enabled "${service}"; then
if is_systemd_active "${service}"; then
log_all "${service} is active. Skip."
else
# Save service status before restart
systemctl status "${service}" | save_in_log_dir "${service_name}.before.status"
# Try to restart
timeout 20 systemctl restart "${service}" > /dev/null
rc=$?
if [ "${rc}" -eq "0" ]; then
log_action "Restart ${service_name}: OK"
else
log_action "Restart ${service_name}: failed"
fi
# Save service status after restart
systemctl status "${service}" | save_in_log_dir "${service_name}.after.status"
fi
else
log_all "${service} is disabled (or missing). Skip."
fi
done
else
log_all "PHP FPM not found. Skip."
fi
post_repair

View file

@ -1,69 +0,0 @@
#!/bin/bash
: "${AUTOSYSADMIN_LIB:=/usr/local/lib/autosysadmin}"
source "${AUTOSYSADMIN_LIB}/common.sh" || exit 1
source "${AUTOSYSADMIN_LIB}/repair.sh" || exit 1
pre_repair
if is_debian_version "8" "<="; then
if is_sysvinit_enabled '*mysql*'; then
if ! pgrep -u mysql mysqld > /dev/null; then
# Save service status before restart
timeout 2 mysqladmin status 2>&1 | save_in_log_dir "mysql.before.status"
timeout 20 /etc/init.d/mysql restart > /dev/null
rc=$?
if [ "${rc}" -eq "0" ]; then
log_action "Restart mysql: OK"
else
log_action "Restart mysql: failed"
fi
# Save service status after restart
timeout 2 mysqladmin status 2>&1 | save_in_log_dir "mysql.after.status"
else
log_abort_and_quit "mysqld process alive. Aborting"
fi
else
log_abort_and_quit "MySQL not enabled. Aborting"
fi
else
if is_debian_version "12" ">="; then
service="mariadb.service"
service_name="mariadb"
else
service="mysql.service"
service_name="mysql"
fi
if is_systemd_enabled "${service}"; then
if is_systemd_active "${service}"; then
log_abort_and_quit "${service} is active, nothing left to do."
else
# Save service status before restart
systemctl status "${service}" | save_in_log_dir "${service_name}.before.status"
# Try to restart
timeout 20 systemctl restart "${service}" > /dev/null
rc=$?
if [ "${rc}" -eq "0" ]; then
log_action "Restart ${service_name}: OK"
else
log_action "Restart ${service_name}: failed"
fi
# Save service status after restart
systemctl status "${service}" | save_in_log_dir "${service_name}.after.status"
fi
else
log_abort_and_quit "${service} is disabled (or missing), nothing left to do."
fi
fi
post_repair

View file

@ -1,35 +0,0 @@
#!/bin/bash
: "${AUTOSYSADMIN_LIB:=/usr/local/lib/autosysadmin}"
source "${AUTOSYSADMIN_LIB}/common.sh" || exit 1
source "${AUTOSYSADMIN_LIB}/repair.sh" || exit 1
pre_repair
service="opendkim.service"
service_name="opendkim"
if is_systemd_enabled "${service}"; then
if is_systemd_active "${service}"; then
log_abort_and_quit "${service} is active, nothing left to do."
else
# Save service status before restart
systemctl status "${service}" | save_in_log_dir "${service_name}.before.status"
# Try to restart
timeout 20 systemctl restart "${service}" > /dev/null
rc=$?
if [ "${rc}" -eq "0" ]; then
log_action "Restart ${service_name}: OK"
else
log_action "Restart ${service_name}: failed"
fi
# Save service status after restart
systemctl status "${service}" | save_in_log_dir "${service_name}.after.status"
fi
else
log_abort_and_quit "${service} is disabled (or missing). Abort."
fi
post_repair

View file

@ -1,14 +0,0 @@
#!/bin/bash
: "${AUTOSYSADMIN_LIB:=/usr/local/lib/autosysadmin}"
source "${AUTOSYSADMIN_LIB}/common.sh" || exit 1
source "${AUTOSYSADMIN_LIB}/repair.sh" || exit 1
LOCK_WAIT="15s"
LOCK_NAME="repair_http"
pre_repair
repair_lxc_php php56
post_repair

View file

@ -1,14 +0,0 @@
#!/bin/bash
: "${AUTOSYSADMIN_LIB:=/usr/local/lib/autosysadmin}"
source "${AUTOSYSADMIN_LIB}/common.sh" || exit 1
source "${AUTOSYSADMIN_LIB}/repair.sh" || exit 1
LOCK_WAIT="15s"
LOCK_NAME="repair_http"
pre_repair
repair_lxc_php php70
post_repair

View file

@ -1,14 +0,0 @@
#!/bin/bash
: "${AUTOSYSADMIN_LIB:=/usr/local/lib/autosysadmin}"
source "${AUTOSYSADMIN_LIB}/common.sh" || exit 1
source "${AUTOSYSADMIN_LIB}/repair.sh" || exit 1
LOCK_WAIT="15s"
LOCK_NAME="repair_http"
pre_repair
repair_lxc_php php73
post_repair

View file

@ -1,14 +0,0 @@
#!/bin/bash
: "${AUTOSYSADMIN_LIB:=/usr/local/lib/autosysadmin}"
source "${AUTOSYSADMIN_LIB}/common.sh" || exit 1
source "${AUTOSYSADMIN_LIB}/repair.sh" || exit 1
LOCK_WAIT="15s"
LOCK_NAME="repair_http"
pre_repair
repair_lxc_php php74
post_repair

View file

@ -1,14 +0,0 @@
#!/bin/bash
: "${AUTOSYSADMIN_LIB:=/usr/local/lib/autosysadmin}"
source "${AUTOSYSADMIN_LIB}/common.sh" || exit 1
source "${AUTOSYSADMIN_LIB}/repair.sh" || exit 1
LOCK_WAIT="15s"
LOCK_NAME="repair_http"
pre_repair
repair_lxc_php php80
post_repair

View file

@ -1,14 +0,0 @@
#!/bin/bash
: "${AUTOSYSADMIN_LIB:=/usr/local/lib/autosysadmin}"
source "${AUTOSYSADMIN_LIB}/common.sh" || exit 1
source "${AUTOSYSADMIN_LIB}/repair.sh" || exit 1
LOCK_WAIT="15s"
LOCK_NAME="repair_http"
pre_repair
repair_lxc_php php81
post_repair

View file

@ -1,14 +0,0 @@
#!/bin/bash
: "${AUTOSYSADMIN_LIB:=/usr/local/lib/autosysadmin}"
source "${AUTOSYSADMIN_LIB}/common.sh" || exit 1
source "${AUTOSYSADMIN_LIB}/repair.sh" || exit 1
LOCK_WAIT="15s"
LOCK_NAME="repair_http"
pre_repair
repair_lxc_php php82
post_repair

View file

@ -1,14 +0,0 @@
#!/bin/bash
: "${AUTOSYSADMIN_LIB:=/usr/local/lib/autosysadmin}"
source "${AUTOSYSADMIN_LIB}/common.sh" || exit 1
source "${AUTOSYSADMIN_LIB}/repair.sh" || exit 1
LOCK_WAIT="15s"
LOCK_NAME="repair_http"
pre_repair
repair_lxc_php php83
post_repair

View file

@ -1,32 +0,0 @@
#!/bin/bash
: "${AUTOSYSADMIN_LIB:=/usr/local/lib/autosysadmin}"
source "${AUTOSYSADMIN_LIB}/common.sh" || exit 1
source "${AUTOSYSADMIN_LIB}/repair.sh" || exit 1
pre_repair
for service in $(systemd_list_services 'redis-server*'); do
service_name="${service//.service/}"
if is_systemd_active "${service}"; then
log_all "${service} is active. Skip."
else
# Save service status before restart
systemctl status "${service}" | save_in_log_dir "${service_name}.before.status"
# Try to restart
timeout 20 systemctl restart "${service}" > /dev/null
rc=$?
if [ "${rc}" -eq "0" ]; then
log_action "Restart ${service_name}: OK."
else
log_action "Restart ${service_name}: failed."
fi
# Save service status after restart
systemctl status "${service}" | save_in_log_dir "${service_name}.after.status"
fi
done
post_repair

View file

@ -1,41 +0,0 @@
#!/bin/bash
: "${AUTOSYSADMIN_LIB:=/usr/local/lib/autosysadmin}"
source "${AUTOSYSADMIN_LIB}/common.sh" || exit 1
source "${AUTOSYSADMIN_LIB}/repair.sh" || exit 1
## Custom lock wait and/or lock name
# LOCK_WAIT="15s"
# LOCK_NAME="repair_http"
pre_repair
## The name of the service, mainly for logging
service_name="example"
## The systemd service name
systemd_service="${service_name}.service"
if is_systemd_enabled "${systemd_service}"; then
if is_systemd_active "${systemd_service}"; then
log_abort_and_quit "${systemd_service} is active, nothing left to do."
else
# Save service status before restart
systemctl status "${systemd_service}" | save_in_log_dir "${service_name}.before.status"
# Try to restart
timeout 20 systemctl restart "${systemd_service}" > /dev/null
rc=$?
if [ "${rc}" -eq "0" ]; then
log_action "Restart ${service_name}: OK"
else
log_action "Restart ${service_name}: failed"
fi
# Save service status after restart
systemctl status "${systemd_service}" | save_in_log_dir "${service_name}.after.status"
fi
else
log_abort_and_quit "${service_name} is disabled (or missing), nothing left to do."
fi
post_repair

View file

@ -1,19 +0,0 @@
Autosysadmin "restart auto" scripts
===================================
In this directory you can place scripts that will be executed automatically by a cron job (stored in `/etc/cron.d/autosysadmin`).
They must satisfy the default `run-parts(8)` constraints :
* be "executable"
* belong to the Debian cron script namespace (`^[a-zA-Z0-9_-]+$`), example: `restart_amavis`
Warning: scripts that do not satisfy those criteria will NOT be run (silently)!
You can print the names of the scripts which would be run, without actually running them, with this command :
```
$ run-parts --test /usr/share/scripts/autosysadmin/restart
```
You can use `zzz-restart_example.template` as boilerplate code to make your own "restart" script.

View file

@ -1,120 +0,0 @@
#!/bin/bash
: "${AUTOSYSADMIN_LIB:=/usr/local/lib/autosysadmin}"
source "${AUTOSYSADMIN_LIB}/common.sh" || exit 1
source "${AUTOSYSADMIN_LIB}/restart.sh" || exit 1
# shellcheck disable=SC2034
RUNNING="nwh-fr"
## Possible values for RUNNING :
## never => disabled
## always => enabled
## nwh-fr => enabled during non-working-hours in France
## nwh-ca => enabled during non-working-hours in Canada (not supported yet)
## custom => enabled if `running_custom()` function returns 0, otherwise disabled.
## Uncomment and customize this method if you want to have a special logic :
##
## return 1 if we should not run
## return 0 if we should run
##
## Some available functions :
## is_weekend() : Saturday or Sunday
## is_holiday() : holiday in France (based on `gcal(1)`)
## is_workday() : not weekend and not holiday
## is_worktime() : work day between 9-12h and 14-18h
#
# running_custom() {
# # implement your own custom method to decide if we should run or not
# }
## The name of the service, mainly for logging
service_name="example"
## The SysVinit script name
sysvinit_script="${service_name}"
## The systemd service name
systemd_service="${service_name}.service"
is_service_alive() {
## this must return 0 if the service is alive, otherwise return 1
## Example:
pgrep -u USER PROCESS_NAME > /dev/null
}
## Action for SysVinit system
sysvinit_action() {
# Save service status before restart
timeout 2 "/etc/init.d/${sysvinit_script}" status | save_in_log_dir "${service_name}.before.status"
# Try to restart
timeout 20 "/etc/init.d/${sysvinit_script}" restart > /dev/null
rc=$?
if [ "${rc}" -eq "0" ]; then
log_action "Restart ${service_name}: OK"
else
log_action "Restart ${service_name}: failed"
fi
# Save service status after restart
timeout 2 "/etc/init.d/${sysvinit_script}" status | save_in_log_dir "${service_name}.after.status"
}
## Action for systemd system
systemd_action() {
# Save service status before restart
systemctl status "${systemd_service}" | save_in_log_dir "${service_name}.before.status"
# Try to restart
# systemctl (only for NRPE ?) sometimes returns 0 even if the service has failed to start
# so we check the status explicitly
timeout 20 systemctl restart "${systemd_service}" > /dev/null \
&& sleep 1 \
&& systemctl status "${systemd_service}" > /dev/null
rc=$?
if [ "${rc}" -eq "0" ]; then
log_action "Restart ${service_name}: OK"
else
log_action "Restart ${service_name}: failed"
fi
# Save service status after restart
systemctl status "${systemd_service}" | save_in_log_dir "${service_name}.after.status"
}
# Should we run?
if ! is_supposed_to_run; then
# log_global "${PROGNAME} is not supposed to run (RUNNING=${RUNNING})."
exit 0
fi
if is_service_alive; then
# log_global "${service_name} process alive. Aborting"
exit 0
fi
# Yes we do, so check for sysvinit or systemd
if is_debian_version "8" "<="; then
if ! is_sysvinit_enabled "*${sysvinit_script}*"; then
# log_global "${service_name} not enabled. Aborting"
exit 0
fi
# Let's finally do the action
pre_restart
sysvinit_action
post_restart
else
if ! is_systemd_enabled "${systemd_service}"; then
# log_global "${service_name} is disabled (or missing), nothing left to do."
exit 0
fi
if is_systemd_active "${systemd_service}"; then
# log_global "${service_name} is active, nothing left to do."
exit 0
fi
# Let's finally do the action
pre_restart
systemd_action
post_restart
fi

View file

@ -1,16 +0,0 @@
---
- name: restart nagios-nrpe-server
service:
name: nagios-nrpe-server
state: restarted
- name: restart nrpe
service:
name: nrpe
state: restarted
- name: restart rsyslog
service:
name: rsyslog
state: restarted

View file

@ -1,25 +0,0 @@
---
- name: "Add begin marker if missing"
ansible.builtin.lineinfile:
path: "/etc/cron.d/autosysadmin"
line: "# BEGIN ANSIBLE MANAGED SECTION FOR AUTOSYSADMIN"
insertbefore: BOF
create: yes
- name: "Add end marker if missing"
ansible.builtin.lineinfile:
path: "/etc/cron.d/autosysadmin"
line: "# END ANSIBLE MANAGED SECTION FOR AUTOSYSADMIN"
insertbefore: "EOF"
create: yes
- name: "Create config if missing"
ansible.builtin.blockinfile:
path: "/etc/cron.d/autosysadmin"
marker: "# {mark} ANSIBLE MANAGED SECTION FOR AUTOSYSADMIN"
block: "{{ lookup('ansible.builtin.template', '../templates/autosysadmin.cron.j2') }}"
owner: root
group: root
mode: "0750"
create: yes

View file

@ -1,114 +0,0 @@
---
- name: "Remount /usr if needed"
ansible.builtin.include_role:
name: remount-usr
- name: Previous autosysadmin restart directory is renamed
command:
cmd: mv "/usr/share/scripts/autosysadmin/auto" "{{ autosysadmin_agent_auto_dir }}"
removes: "/usr/share/scripts/autosysadmin/auto"
creates: "{{ autosysadmin_agent_auto_dir }}"
- name: Create autosysadmin directories
ansible.builtin.file:
path: "{{ item }}"
state: directory
owner: "root"
group: "root"
mode: "0750"
loop:
- "{{ autosysadmin_agent_bin_dir }}"
- "{{ autosysadmin_agent_lib_dir }}"
- "{{ autosysadmin_agent_auto_dir }}"
- name: Copy libraries
ansible.builtin.copy:
src: "upstream/lib/"
dest: "{{ autosysadmin_agent_lib_dir }}/"
owner: root
group: root
mode: "0750"
- name: Copy repair scripts
ansible.builtin.copy:
src: "upstream/repair/"
dest: "{{ autosysadmin_agent_bin_dir }}/"
owner: root
group: root
mode: "0750"
- name: Copy other utilities
ansible.builtin.copy:
src: "upstream/bin/"
dest: "{{ autosysadmin_agent_bin_dir }}/"
owner: root
group: root
mode: "0750"
### WARNING: thos files are explicitly marked as non-executable
### to prevent them from being run automatically by run-parts
- name: Copy restart scripts
ansible.builtin.copy:
src: "upstream/restart/"
dest: "{{ autosysadmin_agent_auto_dir }}/"
owner: root
group: root
mode: "0640"
- name: Ensure /etc/evolinux folder exists
ansible.builtin.file:
path: "/etc/evolinux"
state: directory
owner: "root"
group: "root"
mode: "0700"
- name: Copy the configuration file if missing
ansible.builtin.template:
src: "autosysadmin.cf.j2"
dest: "/etc/evolinux/autosysadmin"
owner: root
group: root
mode: "0640"
force: no
# Repair scripts are supposed to be 'on' by default
# A line "repair_XXX=off" is added to the file only if the script is to be disabled.
# That's why all the ternary logic for the state is reversed.
- name: Update value per variable
ansible.builtin.lineinfile:
dest: "/etc/evolinux/autosysadmin"
line: "{{ item }}={{ autosysadmin_config[item] | default(true) | bool | ternary('on', 'off') }}"
regexp: '^(#\s*)?{{ item }}=.*'
state: "{{ autosysadmin_config[item] | default(true) | bool | ternary('absent', 'present') }}"
register: _line
loop: "{{ autosysadmin_repair_scripts | union(['repair_all']) }}"
- name: Ensure restart folder exists
ansible.builtin.file:
path: "auto"
state: directory
owner: "root"
group: "root"
mode: "0700"
- name: Legacy scripts are removed
ansible.builtin.file:
path: "{{ general_scripts_dir }}/autosysadmin/{{ item }}"
state: absent
loop:
- repair_amavis.sh
- repair_disk.sh
- repair_elasticsearch.sh
- repair_http.sh
- repair_mysql.sh
- repair_opendkim.sh
- repair_php_fpm56.sh
- repair_php_fpm70.sh
- repair_php_fpm73.sh
- repair_php_fpm74.sh
- repair_php_fpm80.sh
- repair_php_fpm81.sh
- repair_redis.sh
- repair_tomcat_instance.sh

View file

@ -1,31 +0,0 @@
---
- name: The list of all repair scripts is composed.
set_fact:
autosysadmin_repair_scripts: "{{ lookup('ansible.builtin.fileglob', '../../../autosysadmin/agent/repair/repair_*', wantlist=True) | map('basename') | sort }}"
- name: Install dependencies
ansible.builtin.include_tasks: dependencies.yml
- name: Install autosysadmin
ansible.builtin.include_tasks: install.yml
- name: Crontab configuration
ansible.builtin.include_tasks: crontab.yml
- name: NRPE configuration
ansible.builtin.include_tasks: nrpe.yml
- name: sudo configuration
ansible.builtin.include_tasks: sudo.yml
- name: rsyslog configuration
ansible.builtin.include_tasks: rsyslog.yml
- name: logrotate configuration
ansible.builtin.include_tasks: logrotate.yml
- name: Install latest version of dump-server-state
ansible.builtin.include_role:
name: evolinux-base
tasks_from: dump-server-state.yml

View file

@ -1,9 +0,0 @@
---
- name: custom configuration is present
ansible.builtin.template:
src: autosysadmin.nrpe.cfg.j2
dest: /etc/nagios/nrpe.d/autosysadmin.cfg
group: nagios
mode: "0640"
force: yes
notify: restart nagios-nrpe-server

View file

@ -1,12 +0,0 @@
# This configuration is partially managed by Ansible
# You can change specific values manually, but they may be overridden by Ansible
#
# To be safe, update the hosts_vars/group_vars in the autosysadmin project
# https://gitea.evolix.org/evolix/autosysadmin/src/branch/master
# then use the "agent" playbook to deploy.
#
# Configuration for autosysadmin
# Use this file to change configuration values defined in repair scripts
# To disable all repair scripts : repair_all=off
# To disable "repair_http" : repair_http=off
#

View file

@ -1,7 +0,0 @@
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
# Run each enabled script
*/5 * * * * root run-parts /usr/share/scripts/autosysadmin/restart
# Clean run log files
@weekly root {{ autosysadmin_agent_bin_dir | mandatory }}/delete_old_logs.sh {{ autosysadmin_agent_log_retention_days | default('365') }}

View file

@ -1,8 +0,0 @@
#
# Ansible managed - DO NOT MODIFY, your changes will be overwritten !
#
# Autosysadmin repair commands
{% for script in lookup('ansible.builtin.fileglob', '../../../autosysadmin/agent/repair/repair_*', wantlist=True) | map("basename") | sort %}
command[{{ script }}]=sudo {{ autosysadmin_agent_bin_dir }}/{{ script }}
{% endfor %}

View file

@ -1,7 +0,0 @@
#
# Ansible managed - DO NOT MODIFY, your changes will be overwritten !
#
{% for script in lookup('ansible.builtin.fileglob', '../../../autosysadmin/agent/repair/repair_*', wantlist=True) | map("basename") | sort %}
nagios ALL = NOPASSWD: {{ autosysadmin_agent_bin_dir }}/{{ script }}
{% endfor %}

View file

@ -1,8 +0,0 @@
---
general_scripts_dir: "/usr/share/scripts"
restart_nrpe_path: "{{ general_scripts_dir }}/autosysadmin/restart/restart_nrpe"
# Change this to customize the RUNNING value in the script
restart_nrpe_running: Null

View file

@ -1,105 +0,0 @@
#!/bin/bash
: "${AUTOSYSADMIN_LIB:=/usr/local/lib/autosysadmin}"
source "${AUTOSYSADMIN_LIB}/common.sh" || exit 1
source "${AUTOSYSADMIN_LIB}/restart.sh" || exit 1
## Possible values for RUNNING :
## never => disabled
## always => enabled
## nwh-fr => enabled during non-working-hours in France
## nwh-ca => enabled during non-working-hours in Canada (not supported yet)
## custom => enabled if `running_custom()` function return 0, otherwise disabled.
# shellcheck disable=SC2034
RUNNING="nwh-fr"
## The name of the service, mainly for logging
service_name="nagios-nrpe-server"
## The SysVinit script name
sysvinit_script="${service_name}"
## The systemd service name
systemd_service="${service_name}.service"
is_service_alive() {
## this must return 0 if the service is alive, otherwise return 1
## Example:
pgrep -u nagios nrpe > /dev/null
}
## Action for SysVinit system
sysvinit_action() {
# Save service status before restart
timeout 2 "/etc/init.d/${sysvinit_script}" status | save_in_log_dir "${service_name}.before.status"
# Try to restart
timeout 20 "/etc/init.d/${sysvinit_script}" restart > /dev/null
rc=$?
if [ "${rc}" -eq "0" ]; then
log_action "Restart ${service_name}: OK"
else
log_action "Restart ${service_name}: failed"
fi
# Save service status after restart
timeout 2 "/etc/init.d/${sysvinit_script}" status | save_in_log_dir "${service_name}.after.status"
}
## Action for systemd system
systemd_action() {
# Save service status before restart
systemctl status "${systemd_service}" | save_in_log_dir "${service_name}.before.status"
# Try to restart
# systemctl (only for NRPE ?) sometimes returns 0 even if the service has failed to start
# so we check the status explicitly
timeout 20 systemctl restart "${systemd_service}" > /dev/null \
&& sleep 1 \
&& systemctl status "${systemd_service}" > /dev/null
rc=$?
if [ "${rc}" -eq "0" ]; then
log_action "Restart ${service_name}: OK"
else
log_action "Restart ${service_name}: failed"
fi
# Save service status after restart
systemctl status "${systemd_service}" | save_in_log_dir "${service_name}.after.status"
}
# Should we run?
if ! is_supposed_to_run; then
# log_global "${PROGNAME} is not supposed to run (RUNNING=${RUNNING})."
exit 0
fi
if is_service_alive; then
# log_global "${service_name} process alive. Aborting"
exit 0
fi
# Yes we do, so check for sysvinit or systemd
if is_debian_version "8" "<="; then
if ! is_sysvinit_enabled "*${sysvinit_script}*"; then
# log_global "${service_name} not enabled. Aborting"
exit 0
fi
# Let's finally do the action
pre_restart
sysvinit_action
post_restart
else
if ! is_systemd_enabled "${systemd_service}"; then
# log_global "${service_name} is disabled (or missing), nothing left to do."
exit 0
fi
if is_systemd_active "${systemd_service}"; then
# log_global "${service_name} is active, nothing left to do."
exit 0
fi
# Let's finally do the action
pre_restart
systemd_action
post_restart
fi

View file

@ -1,24 +0,0 @@
---
- name: "Remount /usr if needed"
ansible.builtin.include_role:
name: remount-usr
- name: "Copy restart_nrpe"
ansible.builtin.copy:
src: upstream/restart_nrpe
dest: "{{ restart_nrpe_path }}"
owner: "root"
group: "root"
mode: "0750"
- name: "Customize RUNNING value"
ansible.builtin.lineinfile:
path: "{{ restart_nrpe_path }}"
line: "RUNNING=\"{{ restart_nrpe_running }}\""
regexp: "^ *RUNNING="
create: False
when:
- restart_nrpe_running is defined
- restart_nrpe_running != None
- restart_nrpe_running | length > 0

View file

@ -0,0 +1,22 @@
---
general_scripts_dir: "/usr/share/scripts"
autosysadmin_dir: "{{ general_scripts_dir }}/autosysadmin"
# Default values for enabled checks
repair_amavis: 'on'
repair_disk: 'on'
repair_elasticsearch: 'on'
repair_http: 'on'
repair_mysql: 'on'
repair_opendkim: 'off'
repair_php_fpm56: 'off'
repair_php_fpm70: 'off'
repair_php_fpm73: 'off'
repair_php_fpm74: 'off'
repair_php_fpm80: 'off'
repair_php_fpm81: 'off'
repair_php_fpm82: 'off'
repair_php_fpm83: 'off'
repair_redis: 'off'
repair_tomcat_instance: 'off'

View file

@ -0,0 +1,478 @@
#!/bin/bash
get_system() {
uname -s
}
get_fqdn() {
if [ "$(get_system)" = "Linux" ]; then
hostname --fqdn
elif [ "$(get_system)" = "OpenBSD" ]; then
hostname
else
log_error_exit "OS not detected!"
fi
}
get_complete_hostname() {
REAL_HOSTNAME="$(get_fqdn)"
if [ "${HOSTNAME}" = "${REAL_HOSTNAME}" ]; then
echo "${HOSTNAME}"
else
echo "${HOSTNAME} (${REAL_HOSTNAME})"
fi
}
get_evomaintenance_mail() {
email="$(grep "EVOMAINTMAIL=" /etc/evomaintenance.cf | cut -d '=' -f2)"
if [[ -z "$email" ]]; then
email='alert5@evolix.fr'
fi
echo "${email}"
}
arguments="${*}"
get_argument() {
no_found=1
for argument in ${arguments} ; do
if [ "${argument}" = "${1}" ] ;
then
no_found=0
fi
done
return ${no_found}
}
internal_info() {
INTERNAL_INFO="$(printf '%b\n%s' "${INTERNAL_INFO}" "$*")"
}
log_action() {
log "Action : $*"
ACTIONS="$(printf '%s\n%s' "${ACTIONS}" "$*")"
}
log() {
INTERNAL_LOG="$(printf '%s\n%s %s %s %s' "${INTERNAL_LOG}" "$(date -Isec)" "$(hostname)" "$(basename "$0")" "$*")"
printf '%s %s %s %s\n' "$(date -Isec)" "$(hostname)" "$(basename "$0")" "$*" | tee -a "${LOG_DIR}/autosysadmin.log"
echo "$*" | /usr/bin/logger -p local0.notice -t autosysadmin."$0"
}
log_error_exit() {
log "ERROR : $*"
AUTOSYSADMIN=1 /usr/share/scripts/evomaintenance.sh -m "$0: $*" --no-commit --no-mail
exit 1
}
log_check_php_fpm() {
# Extraire seulement les chiffres du nom du script exécuté
# ./repair_php_fpm81.sh ==> 81
PHP_VERSION="${0//[^0-9]/}"
PHP_PATH_POOL=$(find /var/lib/lxc/php"${PHP_VERSION}"/ -type d -name "pool.d")
/usr/local/lib/nagios/plugins/check_phpfpm_multi "${PHP_PATH_POOL}" > "${LOG_DIR}/nrpe.txt"
}
log_system_status() {
DUMP_SERVER_STATE_BIN="$(command -v dump-server-state || command -v backup-server-state)"
if [ -z "${DUMP_SERVER_STATE_BIN}" ]; then
log "Warning: dump-server-state is not present. No server state recorded...."
fi
if [ -x "${DUMP_SERVER_STATE_BIN}" ]; then
# NOTE We don't want the logging to take too much time, so we kill it
# if it take more than 20 seconds.
timeout --signal 9 20 \
"${DUMP_SERVER_STATE_BIN}" \
--dump-dir="$LOG_DIR" \
--df \
--dmesg \
--iptables \
--lxc \
--netcfg \
--netstat \
--uname \
--processes \
--systemctl \
--uptime \
--virsh \
--disks \
--mysql-processes \
--no-apt-states \
--no-apt-config \
--no-dpkg-full \
--no-dpkg-status \
--no-mount \
--no-packages \
--no-sysctl \
--no-etc
log "System status logged in ${LOG_DIR}"
fi
}
read_log_system_status(){
files="df.txt dmesg.txt lxc-list.txt netstat-legacy.txt netstat-ss.txt pstree.txt ps.txt systemctl-failed-services.txt"
echo -e "\n\n#### DĂ©tails de dump-server-state"
for file in ${files} ; do
echo -e "\n### cat ${LOG_DIR}/${file} :"
tail -n 1000 "${LOG_DIR}"/"${file}"
done
}
ensure_no_active_users_or_exit() {
if is_debug; then return; fi
# Is there any active user ?
for user in $(LC_ALL=C who --users|awk '{print $1}'); do
idle_time="$(LC_ALL=C who --users | grep "${user}" | awk '{ print $6}')"
for sameusertime in $(LC_ALL=C who --users | grep "${user}" | awk '{ print $6}'); do
if is_active_user "$sameusertime"; then
hook_mail abort_active_users
log_error_exit 'At least one user was recently active. That requires human intervention. Nothing to do here!'
fi
done
done
}
is_active_user() {
# Check if a user was active in the last 30 minutes
idle_time="$1"
if [ "${idle_time}" = "old" ];
then
return 1
elif [ "${idle_time}" = "." ];
then
return 0
else
hh="$(echo "${idle_time}" | awk -F':' '{print $1}')"
mm="$(echo "${idle_time}" | awk -F':' '{print $2}')"
idle_minutes="$(( 60 * "${hh}" + "${mm}" ))"
if [ "${idle_minutes}" -ge 30 ];
then
return 1
else
return 0
fi
fi
}
is_debug() {
debug_file="/etc/evolinux/autosysadmin.debug"
if [ -e "${debug_file}" ]; then
last_change=$(stat -c %Z "${debug_file}")
limit_date=$(date --date "14400 seconds ago" +"%s")
if [ $(( last_change - limit_date )) -le "0" ]; then
rm "${debug_file}"
else
return 0
fi
fi
return 1
}
check_nrpe() {
check="$1"
list_command_nrpe=$( grep --exclude=*~ -E "\[${check}\]" -r /etc/nagios/ | grep -v '#command' )
command_nrpe_primary=$( echo "${list_command_nrpe}" | grep "/etc/nagios/nrpe.d/evolix.cfg" | cut -d'=' -f2- )
command_nrpe_secondary=$( echo "${list_command_nrpe}" | head -n1 | cut -d'=' -f2- )
if [ -z "${command_nrpe_primary}" ] && [ -z "${command_nrpe_secondary}" ]
then
return 1
else
if [ -n "${command_nrpe_primary}" ]
then
${command_nrpe_primary}
else
${command_nrpe_secondary}
fi
fi
}
acquire_lock_or_exit() {
lockfile="$1"
waittime="$2"
# si le temps d’attente n’est pas compréhensible par sleep(1), il vaut 0
if ! echo "${waittime}" | grep -Eq '^[0-9]+[smhd]?$'
then
waittime=0
fi
# si le temps d’attente est supérieur à 0 et si le lock existe, on attend
if test "${waittime}" -gt 0 && test -f "${lockfile}"
then
sleep "${waittime}"
fi
# si le lock existe, on s’arrête
if test -f "${lockfile}"
then
log_error_exit "lock file ${lockfile} exists"
fi
touch "${lockfile}"
}
is_too_soon() {
if is_debug; then return; fi
witness="/tmp/autosysadmin_witness_$(basename "$0")"
if test -f "${witness}"
then
compare="$(($(date +%s)-$(stat -c "%Y" "${witness}")))"
if [ "${compare}" -lt 1800 ];
then
log_error_exit 'already executed less than 30 minutes ago'
fi
rm "${witness}"
fi
touch "${witness}"
}
init_autosysadmin() {
PATH="${PATH}":/usr/sbin:/sbin↩
unset ACTIONS
SCRIPTNAME=$(basename "$0")
PROGNAME=${SCRIPTNAME%.sh}
RUN_ID="$(date +"%Y-%m-%d_%H-%M")_${SCRIPTNAME}_$(openssl rand -hex 6)"
LOG_DIR="/var/log/autosysadmin/${RUN_ID}"
mkdir -p "${LOG_DIR}"
log "Autosysadmin : Script ${SCRIPTNAME} triggered"
# Detect operating system name, version and release↩
detect_os
}
load_conf() {
# Load conf and enable script by default.
# To disable script locally, set "$PROGNAME"=off in /etc/evolinux/autosysadmin.
# To disable script globally, set "$PROGNAME"=off in the script, after load_conf() call.
declare -g "$PROGNAME"=on # dynamic variable assignment ($PROGNAME == repair_*)
# Source configuration file
# shellcheck source=../roles/deploy_autosysadmin/templates/autosysadmin.cfg.j2
test -f /etc/evolinux/autosysadmin && source /etc/evolinux/autosysadmin
}
detect_os() {
# OS detection
DEBIAN_RELEASE=""
LSB_RELEASE_BIN="$(command -v lsb_release)"
if [ -e /etc/debian_version ]; then
DEBIAN_VERSION="$(cut -d "." -f 1 < /etc/debian_version)"
if [ -x "${LSB_RELEASE_BIN}" ]; then
DEBIAN_RELEASE="$("${LSB_RELEASE_BIN}" --codename --short)"
else
case "${DEBIAN_VERSION}" in
8) DEBIAN_RELEASE="jessie";;
9) DEBIAN_RELEASE="stretch";;
10) DEBIAN_RELEASE="buster";;
11) DEBIAN_RELEASE="bullseye";;
esac
fi
fi
}
is_debian_jessie() {
test "${DEBIAN_RELEASE}" = "jessie"
}
is_debian_stretch() {
test "${DEBIAN_RELEASE}" = "stretch"
}
is_debian_buster() {
test "${DEBIAN_RELEASE}" = "buster"
}
is_debian_bullseye() {
test "${DEBIAN_RELEASE}" = "bullseye"
}
systemd_list_service_failed() {
systemctl list-units --failed --no-legend --full --type=service "$1" |
awk '{print $1}'
}
systemd_list_units_enabled() {
list_units_enabled=$(systemctl list-unit-files --state=enabled --no-legend | awk "/$1/{print \$1}")
if [ -z "${list_units_enabled}" ]
then
return 1
else
echo "${list_units_enabled}"
fi
}
format_mail_success() {
cat <<EOTEMPLATE
From: AutoSysadmin Evolix <equipe+autosysadmin@evolix.net>
Content-Type: text/plain; charset=UTF-8
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Script: $(basename "$0")
X-RunId: ${RUN_ID}
To: ${EMAIL_CLIENT:-alert5@evolix.fr}
Cc: autosysadmin@evolix.fr
Subject: [autosysadmin] Intervention sur ${HOSTNAME_TEXT}
Bonjour,
Une intervention automatique vient de se terminer.
Nom du serveur : ${HOSTNAME_TEXT}
Heure d'intervention : $(LC_ALL=fr_FR.utf8 date)
### Renseignements sur l'intervention
${ACTIONS}
### RĂ©agir Ă  cette intervention
Vous pouvez répondre à ce message (sur l'adresse mail equipe@evolix.net).
En cas d'urgence, utilisez l'adresse maintenance@evolix.fr ou
notre téléphone portable d'astreinte (04.26.99.99.26)
--
Votre AutoSysadmin
EOTEMPLATE
}
format_mail_abort_active_users() {
cat <<EOTEMPLATE
From: AutoSysadmin Evolix <equipe+autosysadmin@evolix.net>
Content-Type: text/plain; charset=UTF-8
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Script: $(basename "$0")
X-RunId: ${RUN_ID}
To: ${EMAIL_CLIENT:-alert5@evolix.fr}
Cc: autosysadmin@evolix.fr
Subject: [autosysadmin] Intervention interrompue sur ${HOSTNAME_TEXT}
Bonjour,
Une intervention automatique a été interrompue en raison
d'un utilisateur actuellement actif sur le serveur.
Nom du serveur : ${HOSTNAME_TEXT}
Heure d'intervention : $(LC_ALL=fr_FR.utf8 date)
### Utilisateur(s) connecté(s)
$(w)
--
Votre AutoSysadmin
EOTEMPLATE
}
format_mail_internal_info() {
cat <<EOTEMPLATE
From: AutoSysadmin Evolix <equipe+autosysadmin@evolix.net>
Content-Type: text/plain; charset=UTF-8
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Script: $(basename "$0")
X-RunId: ${RUN_ID}
To: autosysadmin@evolix.fr
Subject: [autosysadmin] Complements (interne) - Intervention sur ${HOSTNAME_TEXT}
Bonjour,
Une intervention automatique vient de se terminer.
Nom du serveur : ${HOSTNAME_TEXT}
Heure d'intervention : $(LC_ALL=fr_FR.utf8 date)
Script déclenché : $(basename "$0")
### Actions effectuées
${ACTIONS}
### Logs autosysadmin
${INTERNAL_LOG}
### Utilisateur(s) connecté(s)
$(w)
### Informations additionnelles données par le script $(basename "$0")
${INTERNAL_INFO}
--
Votre AutoSysadmin
EOTEMPLATE
}
hook_mail() {
if is_debug; then return; fi
HOSTNAME="${HOSTNAME:-"$(get_fqdn)"}"
HOSTNAME_TEXT="$(get_complete_hostname)"
EMAIL_CLIENT="$(get_evomaintenance_mail)"
MAIL_CONTENT="$(format_mail_"$1")"
SENDMAIL_BIN="$(command -v sendmail)"
if [ -z "${SENDMAIL_BIN}" ]; then
log "No \`sendmail' command has been found, can't send mail."
fi
if [ -x "${SENDMAIL_BIN}" ]; then
echo "${MAIL_CONTENT}" | "${SENDMAIL_BIN}" -oi -t -f "equipe@evolix.net"
fi
}
# We need stable output for gcal, so we force some language environment variables
export TZ=Europe/Paris
export LANGUAGE=fr_FR.UTF-8
is_holiday() {
# gcal mark today as a holiday by surrounding with < and > the day
# of the month of that holiday line. For exemple if today is 2022-05-01 we'll
# get among other lines:
# FĂŞte du Travail (FR) + Di, < 1>Mai 2022
# Jour de la Victoire (FR) + Di, : 8:Mai 2022 = +7 jours
gcal --cc-holidays=fr --holiday-list=short | grep -E '<[0-9 ]{2}>' --quiet
}
is_weekend() {
day_of_week=$(date +%u)
if [ "$day_of_week" != 6 ] && [ "$day_of_week" != 7 ]; then
return 1
fi
}
is_workday() {
if is_holiday || is_weekend; then
return 1
fi
}
is_worktime() {
if ! is_workday; then
return 1
fi
hour=$(date +%H)
if [ "${hour}" -lt 9 ] || { [ "${hour}" -ge 12 ] && [ "${hour}" -lt 14 ] ; } || [ "${hour}" -ge 18 ]; then
return 1
fi
}

View file

@ -0,0 +1,33 @@
#!/bin/bash
# Source functions file
# shellcheck source=./functions.sh
source /usr/share/scripts/autosysadmin/functions.sh
# shellcheck source=./restart_amavis.sh
source /usr/share/scripts/autosysadmin/restart_amavis.sh
init_autosysadmin
load_conf
test "${repair_amavis:=off}" = off && log_error_exit 'Script disabled, nothing to do here!'
# Verify if check_nrpe are not OK
check_nrpe "check_amavis" && log_error_exit 'check_amavis is OK, nothing to do here!'
# Has it recently been run?
get_argument "--no-delay" || is_too_soon
lockfile="/run/lock/repair_amavis"
cleanup() {
rm -f "${lockfile}"
}
trap 'cleanup' 0
acquire_lock_or_exit "${lockfile}"
ensure_no_active_users_or_exit
# The actual work starts below !
restart_amavis
hook_mail success
AUTOSYSADMIN=1 /usr/share/scripts/evomaintenance.sh -m "$0: done" --no-commit --no-mail

View file

@ -0,0 +1,173 @@
#!/bin/bash
# Source functions file
# shellcheck source=./functions.sh
source /usr/share/scripts/autosysadmin/functions.sh
init_autosysadmin
load_conf
test "${repair_disk:=off}" = off && log_error_exit 'Script disabled, nothing to do here!'
# Has it recently been run?
is_too_soon
lockfile="/run/lock/repair_disk"
cleanup() {
rm -f "${lockfile}"
}
trap 'cleanup' 0
acquire_lock_or_exit "${lockfile}"
ensure_no_active_users_or_exit
# The actual work starts below !
get_mountpoints() {
# the $(...) get the check_disk1 command
# the cut command selects the critical part of the check_disk1 output
# the grep command extracts the mountpoints and available disk space
# the last cut command selects the mountpoints
$(grep check_disk1 /etc/nagios/nrpe.d/evolix.cfg | cut -d'=' -f2-) -e | cut -d'|' -f1 | grep -Eo '/[[:graph:]]* [0-9]+ [A-Z][A-Z]' | cut -f1 -d' '
}
is_reserved-blocks() {
fs_type="$(findmnt -n --output=fstype "$1")"
if [ "${fs_type}" = "ext4" ];
then
device="$(findmnt -n --output=source "$1")"
reserved_block_count="$(tune2fs -l "${device}" | grep 'Reserved block count' | awk -F':' '{ gsub (" ", "", $0); print $2}')"
block_count="$(tune2fs -l "${device}" | grep 'Block count' | awk -F':' '{ gsub (" ", "", $0); print $2}')"
percentage=$(awk "BEGIN { pc=100*${reserved_block_count}/${block_count}; i=int(pc); print (pc-i<0.5)?i:i+1 }")
log "Reserved blocks for $1 is curently at $percentage%"
if [ "${percentage}" -gt "1" ]
then
log "Allowing tune2fs action to reduce the number of reserved blocks"
return 0
else
log "Reserved blocks already at or bellow 1%, no automatic action possible"
return 1
fi
else
log "Filesystem for $1 partition is not ext4"
return 1
fi
}
change_reserved-blocks() {
# We alwasy keep some reserved blocks to avoid missing some logs
# https://gitea.evolix.org/evolix/autosysadmin/issues/22
tune2fs -m 1 "$(findmnt -n --output=source "$1")"
log_action "Reserved blocks for $1 changed to 1 percent"
}
is_tmp_to_delete() {
size="$(find /var/log/ -type f -ctime +1 -exec du {} \+ | awk '{s+=$1}END{print s / 1024}')"
if [ -n "${size}" ]
then
return 0
else
return 1
fi
}
is_log_to_delete() {
size="$(find /var/log/ -type f -mtime +365 -exec du {} \+ | awk '{s+=$1}END{print s / 1024}')"
if [ -n "${size}" ]
then
return 0
else
return 1
fi
}
clean_apt_cache() {
for lxc in $(du -ax /var | sort -nr | head -n10 | grep -E '/var/lib/lxc/php[0-9]+/rootfs/var/cache$' | grep -Eo 'php[0-9]+')
do
lxc-attach --name "${lxc}" -- apt-get clean
log_action '[lxc/'"${lxc}"'] Clean apt cache'
done
case "$(du -sx /var/* | sort -rn | sed 's/^[0-9]\+[[:space:]]\+//;q')" in
'/var/cache')
apt-get clean
log_action 'Clean apt cache'
;;
esac
}
clean_amavis_virusmails() {
if du --inodes /var/lib/* | sort -n | tail -n3 | grep -q 'virusmails$'
then
find /var/lib/amavis/virusmails/ -type f -atime +30 -delete
log_action 'Clean /var/lib/amavis/virusmails'
fi
}
for mountpoint in $(get_mountpoints)
do
case "${mountpoint}" in
/var)
#if is_log_to_delete
#then
# find /var/log/ -type f -mtime +365 -delete
# log_action "$size Mo of disk space freed in /var"
#fi
if is_reserved-blocks /var
then
change_reserved-blocks /var
clean_apt_cache
clean_amavis_virusmails
hook_mail success
fi
;;
/tmp)
#if is_tmp_to_delete
#then
# find /tmp/ -type f -ctime +1 -delete
# log_action "$size Mo of disk space freed in /tmp"
#fi
if is_reserved-blocks /tmp
then
change_reserved-blocks /tmp
hook_mail success
fi
;;
/home)
if is_reserved-blocks /home
then
change_reserved-blocks /home
hook_mail success
fi
;;
/srv)
if is_reserved-blocks /srv
then
change_reserved-blocks /srv
hook_mail success
fi
;;
/filer)
if is_reserved-blocks /filer
then
change_reserved-blocks /filer
hook_mail success
fi
;;
/)
if is_reserved-blocks /
then
change_reserved-blocks /
hook_mail success
# Suggest remove old kernel ?
fi
;;
*)
# unknown
log 'Unknown partition (or weird case) or nothing to do'
;;
esac
done
AUTOSYSADMIN=1 /usr/share/scripts/evomaintenance.sh -m "$0: done" --no-commit --no-mail

View file

@ -0,0 +1,57 @@
#!/bin/bash
# Source functions file
# shellcheck source=./functions.sh
source /usr/share/scripts/autosysadmin/functions.sh
init_autosysadmin
load_conf
test "${repair_elasticsearch:=off}" = off && log_error_exit 'Script disabled, nothing to do here!'
# Has it recently been run?
is_too_soon
lockfile="/run/lock/repair_elasticsearch"
cleanup() {
rm -f "${lockfile}"
}
trap 'cleanup' 0
acquire_lock_or_exit "${lockfile}"
ensure_no_active_users_or_exit
# The actual work starts below !
elasticsearch_is_enabled() {
systemd_list_units_enabled "elasticsearch.service"
}
elasticsearch_restart() {
if ! timeout 60 systemctl restart elasticsearch.service > /dev/null
then
log_error_exit 'failed to restart elasticsearch'
fi
}
# Test functions
test_elasticsearch_process_present() {
pgrep -u elasticsearch > /dev/null
}
if elasticsearch_is_enabled
then
if ! test_elasticsearch_process_present
then
log_action "Redémarrage de elasticsearch"
elasticsearch_restart
hook_mail success
else
log_error_exit "Elasticsearch process alive. Aborting"
fi
else
log_error_exit "Elasticsearch is not enabled. Aborting"
fi
AUTOSYSADMIN=1 /usr/share/scripts/evomaintenance.sh -m "$0: done" --no-commit --no-mail

View file

@ -0,0 +1,141 @@
#!/bin/bash
# Source functions file
# shellcheck source=./functions.sh
source /usr/share/scripts/autosysadmin/functions.sh
init_autosysadmin
load_conf
test "${repair_http:=off}" = off && log_error_exit 'Script disabled, nothing to do here!'
# Has it recently been run?
is_too_soon
lockfile="/run/lock/repair_http"
cleanup() {
rm -f "${lockfile}"
}
trap 'cleanup' 0
acquire_lock_or_exit "${lockfile}"
ensure_no_active_users_or_exit
# The actual work starts below !
log_system_status
http_detect_service() {
# check whether nginx, apache or both are supposed to be running
if is_debian_jessie; then
find /etc/rc2.d/
else
systemctl list-unit-files --state=enabled
fi | awk '/nginx/ { nginx = 1 } /apache2/ { apache2 = 1 } END { if (nginx && apache2) { print "both" } else if (nginx) { print "nginx" } else if (apache2) { print "apache2" } }'
# The previous awk command looks for two patterns: "nginx"
# and "apache2". If a line matches the patterns, a variable
# "nginx" or "apache2" is set to 1 (true). The "END" checks
# if one or both patterns has been found.
}
http_handle_apache() {
# check syntax
if ! apache2ctl -t > /dev/null 2> /dev/null
then
log_error_exit 'apache2 configuration syntax is not valid'
fi
# try restart
if ! timeout 20 systemctl restart apache2.service > /dev/null 2> /dev/null
then
log_error_exit 'failed to restart apache2'
fi
log_action "Redémarrage de Apache"
internal_info "#### grep $(LANG=en_US.UTF-8 date '+%b %d') /home/*/log/error.log /var/log/apache2/*error.log (avec filtrage)"
ERROR_LOG=$(grep "$(LANG=en_US.UTF-8 date '+%b %d')" /home/*/log/error.log /var/log/apache2/*error.log | grep -v -e "Got error 'PHP message:" -e "No matching DirectoryIndex" -e "client denied by server configuration" -e "server certificate does NOT include an ID which matches the server name" )
internal_info "$ERROR_LOG"
}
http_handle_nginx() {
# check syntax
if ! nginx -t > /dev/null 2> /dev/null
then
log_error_exit 'nginx configuration syntax is not valid'
fi
# try restart
if ! timeout 20 systemctl restart nginx.service > /dev/null 2> /dev/null
then
log_error_exit 'failed to restart nginx'
fi
log_action "Redémarrage de Nginx"
}
http_handle_lxc_php() {
# check whether containers are used for PHP and reboot them if so
if systemd_list_units_enabled 'lxc'
then
for php in $(lxc-ls | grep 'php'); do
lxc-stop -n "$php"
lxc-start --daemon -n "$php"
log_action "lxc-fpm - Redémarrage container ${php}"
done
fi
}
http_handle_fpm_php() {
# check whether php-fpm is installed and restart it if so
if enabled_units="$(systemd_list_units_enabled "php.*-fpm")"
then
systemctl restart "${enabled_units}"
log_action 'php-fpm - Redémarrage de php-fpm'
fi
}
case "$(http_detect_service)" in
nginx)
http_handle_nginx
http_handle_lxc_php
http_handle_fpm_php
hook_mail success
hook_mail internal_info
;;
apache2)
http_handle_apache
http_handle_lxc_php
http_handle_fpm_php
hook_mail success
hook_mail internal_info
;;
both)
http_handle_nginx
http_handle_apache
http_handle_lxc_php
http_handle_fpm_php
hook_mail success
hook_mail internal_info
;;
*)
# unknown
log 'nothing to do'
;;
esac
AUTOSYSADMIN=1 /usr/share/scripts/evomaintenance.sh -m "$0: done" --no-commit --no-mail

View file

@ -0,0 +1,71 @@
#!/bin/bash
# Source functions file
# shellcheck source=./functions.sh
source /usr/share/scripts/autosysadmin/functions.sh
init_autosysadmin
load_conf
test "${repair_mysql:=off}" = off && log_error_exit 'Script disabled, nothing to do here!'
# Has it recently been run?
is_too_soon
lockfile="/run/lock/repair_mysql"
cleanup() {
rm -f "${lockfile}"
}
trap 'cleanup' 0
acquire_lock_or_exit "${lockfile}"
ensure_no_active_users_or_exit
# The actual work starts below !
log_system_status
mysql_is_enabled() {
if is_debian_jessie
then
find /etc/rc2.d/ -name '*mysql*' > /dev/null
else
systemd_list_units_enabled "mysql.service"
fi
}
mysql_restart() {
if is_debian_jessie
then
if ! timeout 60 /etc/init.d/mysql restart > /dev/null
then
log_error_exit 'failed to restart mysql'
fi
else
if ! timeout 60 systemctl restart mysql.service > /dev/null
then
log_error_exit 'failed to restart mysql'
fi
fi
}
# Test functions
test_mysql_process_present() {
pgrep -u mysql mysqld > /dev/null
}
if mysql_is_enabled
then
if ! test_mysql_process_present
then
log_action "Redémarrage de MySQL"
mysql_restart
hook_mail success
else
log_error_exit "mysqld process alive. Aborting"
fi
else
log_error_exit "MySQL/MariaDB not enabled. Aborting"
fi
AUTOSYSADMIN=1 /usr/share/scripts/evomaintenance.sh -m "$0: done" --no-commit --no-mail

View file

@ -0,0 +1,61 @@
#!/bin/bash
# Source functions file
# shellcheck source=./functions.sh
source /usr/share/scripts/autosysadmin/functions.sh
init_autosysadmin
load_conf
test "${repair_opendkim:=off}" = off && log_error_exit 'Script disabled, nothing to do here!'
# Has it recently been run?
is_too_soon
lockfile="/run/lock/repair_opendkim"
cleanup() {
rm -f "${lockfile}"
}
trap 'cleanup' 0
acquire_lock_or_exit "${lockfile}"
ensure_no_active_users_or_exit
log_system_status
# Functions dedicated to this repair script
opendkim_is_enabled() {
systemd_list_units_enabled "opendkim.service"
}
opendkim_restart() {
if ! timeout 60 systemctl restart opendkim.service > /dev/null
then
log_error_exit 'failed to restart opendkim'
fi
}
opendkim_test_process_present() {
pgrep -u opendkim > /dev/null
}
# Main logic
if opendkim_is_enabled
then
if ! opendkim_test_process_present
then
log_action "Redémarrage de opendkim"
opendkim_restart
hook_mail success
else
log_error_exit "opendkim process alive. Aborting"
fi
else
log_error_exit "opendkim is not enabled. Aborting"
fi
AUTOSYSADMIN=1 /usr/share/scripts/evomaintenance.sh -m "$0: done" --no-commit --no-mail

View file

@ -0,0 +1,53 @@
#!/bin/bash
# Source functions file
# shellcheck source=./functions.sh
source /usr/share/scripts/autosysadmin/functions.sh
init_autosysadmin
load_conf
test "${repair_php_fpm56:=off}" = off && log_error_exit 'Script disabled, nothing to do here!'
# Has it recently been run?
is_too_soon
lockfile="/run/lock/repair_http"
cleanup() {
rm -f "${lockfile}"
}
trap 'cleanup' 0
acquire_lock_or_exit "${lockfile}" 15s
ensure_no_active_users_or_exit
# The actual work starts below !
log_system_status
log_check_php_fpm
if systemd_list_units_enabled 'lxc'
then
if lxc-ls | grep -q php56
then
lxc-stop -n php56
lxc-start --daemon -n php56
log_action "lxc-fpm - Redémarrage container php56"
internal_info "#### tail /var/lib/lxc/php56/rootfs/var/log/php5-fpm.log"
FPM_LOG=$(tail /var/lib/lxc/php56/rootfs/var/log/php5-fpm.log)
internal_info "$FPM_LOG" "$(read_log_system_status)"
hook_mail success
hook_mail internal_info
else
log 'Not possible :v'
fi
else
log 'Error, not a multi-php install'
fi
AUTOSYSADMIN=1 /usr/share/scripts/evomaintenance.sh -m "$0: done" --no-commit --no-mail

View file

@ -0,0 +1,53 @@
#!/bin/bash
# Source functions file
# shellcheck source=./functions.sh
source /usr/share/scripts/autosysadmin/functions.sh
init_autosysadmin
load_conf
test "${repair_php_fpm70:=off}" = off && log_error_exit 'Script disabled, nothing to do here!'
# Has it recently been run?
is_too_soon
lockfile="/run/lock/repair_http"
cleanup() {
rm -f "${lockfile}"
}
trap 'cleanup' 0
acquire_lock_or_exit "${lockfile}" 15s
ensure_no_active_users_or_exit
# The actual work starts below !
log_system_status
log_check_php_fpm
if systemd_list_units_enabled 'lxc'
then
if lxc-ls | grep -q php70
then
lxc-stop -n php70
lxc-start --daemon -n php70
log_action "lxc-fpm - Redémarrage container php70"
internal_info "#### tail /var/lib/lxc/php70/rootfs/var/log/php7.0-fpm.log"
FPM_LOG=$(tail /var/lib/lxc/php70/rootfs/var/log/php7.0-fpm.log)
internal_info "$FPM_LOG" "$(read_log_system_status)"
hook_mail success
hook_mail internal_info
else
log 'Not possible :v'
fi
else
log 'Error, not a multi-php install'
fi
AUTOSYSADMIN=1 /usr/share/scripts/evomaintenance.sh -m "$0: done" --no-commit --no-mail

View file

@ -0,0 +1,53 @@
#!/bin/bash
# Source functions file
# shellcheck source=./functions.sh
source /usr/share/scripts/autosysadmin/functions.sh
init_autosysadmin
load_conf
test "${repair_php_fpm73:=off}" = off && log_error_exit 'Script disabled, nothing to do here!'
# Has it recently been run?
is_too_soon
lockfile="/run/lock/repair_http"
cleanup() {
rm -f "${lockfile}"
}
trap 'cleanup' 0
acquire_lock_or_exit "${lockfile}" 15s
ensure_no_active_users_or_exit
# The actual work starts below !
log_system_status
log_check_php_fpm
if systemd_list_units_enabled 'lxc'
then
if lxc-ls | grep -q php73
then
lxc-stop -n php73
lxc-start --daemon -n php73
log_action "lxc-fpm - Redémarrage container php73"
internal_info "#### tail /var/lib/lxc/php73/rootfs/var/log/php7.3-fpm.log"
FPM_LOG=$(tail /var/lib/lxc/php73/rootfs/var/log/php7.3-fpm.log)
internal_info "$FPM_LOG" "$(read_log_system_status)"
hook_mail success
hook_mail internal_info
else
log 'Not possible :v'
fi
else
log 'Error, not a multi-php install'
fi
AUTOSYSADMIN=1 /usr/share/scripts/evomaintenance.sh -m "$0: done" --no-commit --no-mail

View file

@ -0,0 +1,53 @@
#!/bin/bash
# Source functions file
# shellcheck source=./functions.sh
source /usr/share/scripts/autosysadmin/functions.sh
init_autosysadmin
load_conf
test "${repair_php_fpm74:=off}" = off && log_error_exit 'Script disabled, nothing to do here!'
# Has it recently been run?
is_too_soon
lockfile="/run/lock/repair_http"
cleanup() {
rm -f "${lockfile}"
}
trap 'cleanup' 0
acquire_lock_or_exit "${lockfile}" 15s
ensure_no_active_users_or_exit
# The actual work starts below !
log_system_status
log_check_php_fpm
if systemd_list_units_enabled 'lxc'
then
if lxc-ls | grep -q php74
then
lxc-stop -n php74
lxc-start --daemon -n php74
log_action "lxc-fpm - Redémarrage container php74"
internal_info "#### tail /var/lib/lxc/php74/rootfs/var/log/php7.4-fpm.log"
FPM_LOG=$(tail /var/lib/lxc/php74/rootfs/var/log/php7.4-fpm.log)
internal_info "$FPM_LOG" "$(read_log_system_status)"
hook_mail success
hook_mail internal_info
else
log 'Not possible :v'
fi
else
log 'Error, not a multi-php install'
fi
AUTOSYSADMIN=1 /usr/share/scripts/evomaintenance.sh -m "$0: done" --no-commit --no-mail

View file

@ -0,0 +1,53 @@
#!/bin/bash
# Source functions file
# shellcheck source=./functions.sh
source /usr/share/scripts/autosysadmin/functions.sh
init_autosysadmin
load_conf
test "${repair_php_fpm80:=off}" = off && log_error_exit 'Script disabled, nothing to do here!'
# Has it recently been run?
is_too_soon
lockfile="/run/lock/repair_http"
cleanup() {
rm -f "${lockfile}"
}
trap 'cleanup' 0
acquire_lock_or_exit "${lockfile}" 15s
ensure_no_active_users_or_exit
# The actual work starts below !
log_system_status
log_check_php_fpm
if systemd_list_units_enabled 'lxc'
then
if lxc-ls | grep -q php80
then
lxc-stop -n php80
lxc-start --daemon -n php80
log_action "lxc-fpm - Redémarrage container php80"
internal_info "#### tail /var/lib/lxc/php80/rootfs/var/log/php8.0-fpm.log"
FPM_LOG=$(tail /var/lib/lxc/php80/rootfs/var/log/php8.0-fpm.log)
internal_info "$FPM_LOG" "$(read_log_system_status)"
hook_mail success
hook_mail internal_info
else
log 'Not possible :v'
fi
else
log 'Error, not a multi-php install'
fi
AUTOSYSADMIN=1 /usr/share/scripts/evomaintenance.sh -m "$0: done" --no-commit --no-mail

View file

@ -0,0 +1,53 @@
#!/bin/bash
# Source functions file
# shellcheck source=./functions.sh
source /usr/share/scripts/autosysadmin/functions.sh
init_autosysadmin
load_conf
test "${repair_php_fpm81:=off}" = off && log_error_exit 'Script disabled, nothing to do here!'
# Has it recently been run?
is_too_soon
lockfile="/run/lock/repair_http"
cleanup() {
rm -f "${lockfile}"
}
trap 'cleanup' 0
acquire_lock_or_exit "${lockfile}" 15s
ensure_no_active_users_or_exit
# The actual work starts below !
log_system_status
log_check_php_fpm
if systemd_list_units_enabled 'lxc'
then
if lxc-ls | grep -q php81
then
lxc-stop -n php81
lxc-start --daemon -n php81
log_action "lxc-fpm - Redémarrage container php81"
internal_info "#### tail /var/lib/lxc/php81/rootfs/var/log/php8.1-fpm.log"
FPM_LOG=$(tail /var/lib/lxc/php81/rootfs/var/log/php8.1-fpm.log)
internal_info "$FPM_LOG" "$(read_log_system_status)"
hook_mail success
hook_mail internal_info
else
log 'Not possible :v'
fi
else
log 'Error, not a multi-php install'
fi
AUTOSYSADMIN=1 /usr/share/scripts/evomaintenance.sh -m "$0: done" --no-commit --no-mail

View file

@ -0,0 +1,53 @@
#!/bin/bash
# Source functions file
# shellcheck source=./functions.sh
source /usr/share/scripts/autosysadmin/functions.sh
init_autosysadmin
load_conf
test "${repair_php_fpm82:=off}" = off && log_error_exit 'Script disabled, nothing to do here!'
# Has it recently been run?
is_too_soon
lockfile="/run/lock/repair_http"
cleanup() {
rm -f "${lockfile}"
}
trap 'cleanup' 0
acquire_lock_or_exit "${lockfile}" 15s
ensure_no_active_users_or_exit
# The actual work starts below !
log_system_status
log_check_php_fpm
if systemd_list_units_enabled 'lxc'
then
if lxc-ls | grep -q php82
then
lxc-stop -n php82
lxc-start --daemon -n php82
log_action "lxc-fpm - Redémarrage container php82"
internal_info "#### tail /var/lib/lxc/php82/rootfs/var/log/php8.2-fpm.log"
FPM_LOG=$(tail /var/lib/lxc/php82/rootfs/var/log/php8.2-fpm.log)
internal_info "$FPM_LOG" "$(read_log_system_status)"
hook_mail success
hook_mail internal_info
else
log 'Not possible :v'
fi
else
log 'Error, not a multi-php install'
fi
AUTOSYSADMIN=1 /usr/share/scripts/evomaintenance.sh -m "$0: done" --no-commit --no-mail

View file

@ -0,0 +1,53 @@
#!/bin/bash
# Source functions file
# shellcheck source=./functions.sh
source /usr/share/scripts/autosysadmin/functions.sh
init_autosysadmin
load_conf
test "${repair_php_fpm83:=off}" = off && log_error_exit 'Script disabled, nothing to do here!'
# Has it recently been run?
is_too_soon
lockfile="/run/lock/repair_http"
cleanup() {
rm -f "${lockfile}"
}
trap 'cleanup' 0
acquire_lock_or_exit "${lockfile}" 15s
ensure_no_active_users_or_exit
# The actual work starts below !
log_system_status
log_check_php_fpm
if systemd_list_units_enabled 'lxc'
then
if lxc-ls | grep -q php83
then
lxc-stop -n php83
lxc-start --daemon -n php83
log_action "lxc-fpm - Redémarrage container php83"
internal_info "#### tail /var/lib/lxc/php83/rootfs/var/log/php8.3-fpm.log"
FPM_LOG=$(tail /var/lib/lxc/php83/rootfs/var/log/php8.3-fpm.log)
internal_info "$FPM_LOG" "$(read_log_system_status)"
hook_mail success
hook_mail internal_info
else
log 'Not possible :v'
fi
else
log 'Error, not a multi-php install'
fi
AUTOSYSADMIN=1 /usr/share/scripts/evomaintenance.sh -m "$0: done" --no-commit --no-mail

View file

@ -0,0 +1,58 @@
#!/bin/bash
# Source functions file
# shellcheck source=./functions.sh
source /usr/share/scripts/autosysadmin/functions.sh
init_autosysadmin
load_conf
test "${repair_redis:=off}" = off && log_error_exit 'Script disabled, nothing to do here!'
# Has it recently been run?
is_too_soon
lockfile="/run/lock/repair_redis"
cleanup() {
rm -f "${lockfile}"
}
trap 'cleanup' 0
acquire_lock_or_exit "${lockfile}"
ensure_no_active_users_or_exit
# The actual work starts below !
handle_redis() {
for service in $(systemd_list_service_failed redis*)
do
# ne rien faire si le service est désactivé
if ! systemctl is-enabled --quiet "${service}"
then
continue
fi
# ne rien faire si le service est actif
if systemctl is-active --quiet "${service}"
then
continue
fi
if ! timeout 20 systemctl restart redis.service > /dev/null 2> /dev/null
then
log_error_exit "failed to restart redis ${service}"
fi
log_action "Redémarrer service ${service}"
done
}
if ( systemd_list_units_enabled 'redis.*\.service$' ) > /dev/null
then
handle_redis
hook_mail success
else
log 'Error: redis service is not enabled'
fi
AUTOSYSADMIN=1 /usr/share/scripts/evomaintenance.sh -m "$0: done" --no-commit --no-mail

View file

@ -0,0 +1,63 @@
#!/bin/bash
# Source functions file
# shellcheck source=./functions.sh
source /usr/share/scripts/autosysadmin/functions.sh
init_autosysadmin
load_conf
# Comment this line to enable
repair_template=off
test "${repair_template:=off}" = off && log_error_exit 'Script disabled, nothing to do here!'
# Has it recently been run?
is_too_soon
lockfile="/run/lock/repair_template"
cleanup() {
rm -f "${lockfile}"
}
trap 'cleanup' 0
acquire_lock_or_exit "${lockfile}"
ensure_no_active_users_or_exit
log_system_status
# Functions dedicated to this repair script
template_is_enabled() {
systemd_list_units_enabled "template.service"
}
template_restart() {
if ! timeout 60 systemctl restart template.service > /dev/null
then
log_error_exit 'failed to restart template'
fi
}
template_test_process_present() {
pgrep -u template > /dev/null
}
# Main logic
if template_is_enabled
then
if ! template_test_process_present
then
log_action "Redémarrage de template"
template_restart
hook_mail success
else
log_error_exit "template process alive. Aborting"
fi
else
log_error_exit "template is not enabled. Aborting"
fi
AUTOSYSADMIN=1 /usr/share/scripts/evomaintenance.sh -m "$0: done" --no-commit --no-mail

View file

@ -1,24 +1,43 @@
#!/bin/bash
: "${AUTOSYSADMIN_LIB:=/usr/local/lib/autosysadmin}"
source "${AUTOSYSADMIN_LIB}/common.sh" || exit 1
source "${AUTOSYSADMIN_LIB}/repair.sh" || exit 1
# Source functions file
# shellcheck source=./functions.sh
source /usr/share/scripts/autosysadmin/functions.sh
pre_repair
init_autosysadmin
load_conf
test "${repair_tomcat_instance:=off}" = off && log_error_exit 'Script disabled, nothing to do here!'
# Has it recently been run?
is_too_soon
lockfile="/run/lock/repair_tomcat_instance"
cleanup() {
rm -f "${lockfile}"
}
trap 'cleanup' 0
acquire_lock_or_exit "${lockfile}"
ensure_no_active_users_or_exit
# The actual work starts below !
log_system_status
repair_tomcat_instance_handle_tomcat() {
if /bin/su - "${1}" -c "/bin/systemctl --quiet --user is-active tomcat.service" ; then
if ! /bin/su - "${1}" -c "/usr/bin/timeout 20 /bin/systemctl --quiet --user restart tomcat.service"
then
log_abort_and_quit "Echec de redémarrage instance tomcat utilisateur ${1}"
log_error_exit "Echec de redémarrage instance tomcat utilisateur ${1}"
else
log_action "Redémarrage instance tomcat utilisateur ${1}"
fi
elif /bin/systemctl --quiet is-active "${1}".service ; then
if ! /usr/bin/timeout 20 systemctl --quiet restart "${1}".service
then
log_abort_and_quit "Echec de redémarrage instance tomcat ${1}"
log_error_exit "Echec de redémarrage instance tomcat ${1}"
else
log_action "Redémarrage instance tomcat ${1}"
fi
@ -31,4 +50,4 @@ do
repair_tomcat_instance_handle_tomcat "${instance}"
done
post_repair
AUTOSYSADMIN=1 /usr/share/scripts/evomaintenance.sh -m "$0: done" --no-commit --no-mail

View file

@ -0,0 +1,35 @@
#!/bin/bash
restart_amavis() {
/etc/init.d/amavis stop 2>/dev/null
/etc/init.d/clamav-freshclam stop 2>/dev/null
/etc/init.d/clamav-daemon stop 2>/dev/null
if systemctl is-enabled --quiet 'clamav-freshclam.service'
then
freshclam
log_action "Mise à jour des définitions antivirus"
fi
if systemctl is-enabled --quiet 'clamav-daemon.service'
then
/etc/init.d/clamav-daemon start
log_action "Redémarrage de clamav-daemon"
else
log 'Error, clamav not installed'
fi
if systemctl is-enabled --quiet 'clamav-freshclam.service'
then
/etc/init.d/clamav-freshclam start
log_action "Redémarrage de clamav-freshclam"
fi
if systemctl is-enabled --quiet 'amavis.service'
then
/etc/init.d/amavis start
log_action "Redémarrage de amavis"
else
log 'Error, amavis not installed'
fi
}

View file

@ -0,0 +1,16 @@
---
- name: Restart nagios-nrpe-server
ansible.builtin.service:
name: nagios-nrpe-server
state: restarted
- name: Restart nrpe
ansible.builtin.service:
name: nrpe
state: restarted
- name: Restart rsyslog
ansible.builtin.service:
name: rsyslog
state: restarted

View file

@ -0,0 +1,61 @@
---
- name: "Remount /usr if needed"
ansible.builtin.import_role:
name: remount-usr
- name: Create autosysadmin directory
ansible.builtin.file:
path: "{{ autosysadmin_dir }}"
state: directory
owner: "root"
group: "root"
mode: "0750"
tags:
- autosysadmin
- name: Copy scripts
ansible.builtin.copy:
src: "files/scripts/{{ item }}"
dest: "{{ autosysadmin_dir }}/{{ item }}"
owner: root
group: root
mode: "0750"
loop:
- "functions.sh"
- "restart_amavis.sh"
- "repair_amavis.sh"
- "repair_disk.sh"
- "repair_elasticsearch.sh"
- "repair_http.sh"
- "repair_mysql.sh"
- "repair_php_fpm56.sh"
- "repair_php_fpm70.sh"
- "repair_php_fpm73.sh"
- "repair_php_fpm74.sh"
- "repair_php_fpm80.sh"
- "repair_php_fpm81.sh"
- "repair_php_fpm82.sh"
- "repair_php_fpm83.sh"
- "repair_tomcat_instance.sh"
tags:
- autosysadmin
- name: Ensure /etc/evolinux folder exists
ansible.builtin.file:
path: "/etc/evolinux"
state: directory
owner: "root"
group: "root"
mode: "0700"
tags:
- autosysadmin
- name: Copy the configuration file
ansible.builtin.template:
src: "autosysadmin.cf.j2"
dest: "/etc/evolinux/autosysadmin"
owner: root
group: root
mode: "0640"
tags:
- autosysadmin

View file

@ -1,8 +1,10 @@
---
- name: Copy logrotate configuration for autosysadmin
ansible.builtin.copy:
src: "files/autosysadmin.logrotate.conf"
src: "files/logrotate_autosysadmin.conf"
dest: "/etc/logrotate.d/autosysadmin"
owner: root
group: root
mode: "0644"
tags:
- autosysadmin

View file

@ -0,0 +1,37 @@
---
- name: Install dependencies
ansible.builtin.import_tasks: dependencies.yml
tags:
- autosysadmin
- name: Install autosysadmin scripts
ansible.builtin.import_tasks: autosysadmin_scripts.yml
tags:
- autosysadmin
- name: Amend NRPE configuration
ansible.builtin.import_tasks: nrpe.yml
tags:
- autosysadmin
- name: Amend sudo configuration
ansible.builtin.import_tasks: sudo.yml
tags:
- autosysadmin
- name: Amend rsyslog configuration
ansible.builtin.import_tasks: rsyslog.yml
tags:
- autosysadmin
- name: Amend logrotate configuration
ansible.builtin.import_tasks: logrotate.yml
tags:
- autosysadmin
- name: Install last version of dump-server-state
ansible.builtin.import_role:
name: evolinux-base
tasks_from: dump-server-state.yml
tags:
- autosysadmin

View file

@ -0,0 +1,11 @@
---
- name: Custom configuration is present
ansible.builtin.template:
src: autosysadmin.cfg.j2
dest: /etc/nagios/nrpe.d/autosysadmin.cfg
group: nagios
mode: "0640"
force: true
notify: Restart nagios-nrpe-server
tags:
- autosysadmin

View file

@ -1,9 +1,11 @@
---
- name: Copy rsyslog configuration for autosysadmin
ansible.builtin.copy:
src: "files/autosysadmin.rsyslog.conf"
src: "files/rsyslog_autosysadmin.conf"
dest: "/etc/rsyslog.d/autosysadmin.conf"
owner: root
group: root
mode: "0644"
notify: restart rsyslog
notify: Restart rsyslog
tags:
- autosysadmin

View file

@ -1,7 +1,9 @@
---
- name: Add autosysadmin sudoers file
ansible.builtin.template:
src: autosysadmin.sudoers.j2
src: sudoers.j2
dest: /etc/sudoers.d/autosysadmin
mode: "0600"
validate: "visudo -cf %s"
tags:
- autosysadmin

View file

@ -0,0 +1,74 @@
#
# Ansible managed - DO NOT MODIFY, your changes will be **overwritten** !
#
# Update the hosts_vars/group_vars on the autosysadmin project
# https://gitea.evolix.org/evolix/autosysadmin/src/branch/master
#
# Configuration for autosysadmin
# Use this file to change configuration values defined in repair scripts
# Ex : repair_http=off
{% if repair_amavis == "off" %}
repair_amavis=off
{% endif %}
{% if repair_disk == "off" %}
repair_disk=off
{% endif %}
{% if repair_elasticsearch == "off" %}
repair_elasticsearch=off
{% endif %}
{% if repair_http == "off" %}
repair_http=off
{% endif %}
{% if repair_mysql == "off" %}
repair_mysql=off
{% endif %}
{% if repair_opendkim == "off" %}
repair_opendkim=off
{% endif %}
{% if repair_php_fpm56 == "off" %}
repair_php_fpm56=off
{% endif %}
{% if repair_php_fpm70 == "off" %}
repair_php_fpm70=off
{% endif %}
{% if repair_php_fpm73 == "off" %}
repair_php_fpm73=off
{% endif %}
{% if repair_php_fpm74 == "off" %}
repair_php_fpm74=off
{% endif %}
{% if repair_php_fpm80 == "off" %}
repair_php_fpm80=off
{% endif %}
{% if repair_php_fpm81 == "off" %}
repair_php_fpm81=off
{% endif %}
{% if repair_php_fpm82 == "off" %}
repair_php_fpm82=off
{% endif %}
{% if repair_php_fpm83 == "off" %}
repair_php_fpm83=off
{% endif %}
{% if repair_redis == "off" %}
repair_redis=off
{% endif %}
{% if repair_tomcat_instance == "off" %}
repair_tomcat_instance=off
{% endif %}

View file

@ -0,0 +1,22 @@
#
# Ansible managed - DO NOT MODIFY, your changes will be overwritten !
#
# Autosysadmin repair commands
command[repair_amavis]=sudo {{ autosysadmin_dir }}/repair_amavis.sh
command[repair_disk]=sudo {{ autosysadmin_dir }}/repair_disk.sh
command[repair_elasticsearch]=sudo {{ autosysadmin_dir }}/repair_elasticsearch.sh
command[repair_http]=sudo {{ autosysadmin_dir }}/repair_http.sh
command[repair_mysql]=sudo {{ autosysadmin_dir }}/repair_mysql.sh
command[repair_opendkim]=sudo {{ autosysadmin_dir }}/repair_opendkim.sh
command[repair_php_fpm56]=sudo {{ autosysadmin_dir }}/repair_php_fpm56.sh
command[repair_php_fpm70]=sudo {{ autosysadmin_dir }}/repair_php_fpm70.sh
command[repair_php_fpm73]=sudo {{ autosysadmin_dir }}/repair_php_fpm73.sh
command[repair_php_fpm74]=sudo {{ autosysadmin_dir }}/repair_php_fpm74.sh
command[repair_php_fpm80]=sudo {{ autosysadmin_dir }}/repair_php_fpm80.sh
command[repair_php_fpm81]=sudo {{ autosysadmin_dir }}/repair_php_fpm81.sh
command[repair_php_fpm82]=sudo {{ autosysadmin_dir }}/repair_php_fpm82.sh
command[repair_php_fpm83]=sudo {{ autosysadmin_dir }}/repair_php_fpm83.sh
command[repair_redis]=sudo {{ autosysadmin_dir }}/repair_redis.sh
command[repair_tomcat_instance]=sudo {{ autosysadmin_dir }}/repair_tomcat_instance.sh

View file

@ -0,0 +1,21 @@
#
# Ansible managed - DO NOT MODIFY, your changes will be overwritten !
#
nagios ALL = NOPASSWD: {{ autosysadmin_dir }}/repair_amavis.sh
nagios ALL = NOPASSWD: {{ autosysadmin_dir }}/repair_disk.sh
nagios ALL = NOPASSWD: {{ autosysadmin_dir }}/repair_elasticsearch.sh
nagios ALL = NOPASSWD: {{ autosysadmin_dir }}/repair_http.sh
nagios ALL = NOPASSWD: {{ autosysadmin_dir }}/repair_mysql.sh
nagios ALL = NOPASSWD: {{ autosysadmin_dir }}/repair_opendkim.sh
nagios ALL = NOPASSWD: {{ autosysadmin_dir }}/repair_php_fpm56.sh
nagios ALL = NOPASSWD: {{ autosysadmin_dir }}/repair_php_fpm70.sh
nagios ALL = NOPASSWD: {{ autosysadmin_dir }}/repair_php_fpm73.sh
nagios ALL = NOPASSWD: {{ autosysadmin_dir }}/repair_php_fpm74.sh
nagios ALL = NOPASSWD: {{ autosysadmin_dir }}/repair_php_fpm80.sh
nagios ALL = NOPASSWD: {{ autosysadmin_dir }}/repair_php_fpm81.sh
nagios ALL = NOPASSWD: {{ autosysadmin_dir }}/repair_php_fpm82.sh
nagios ALL = NOPASSWD: {{ autosysadmin_dir }}/repair_php_fpm83.sh
nagios ALL = NOPASSWD: {{ autosysadmin_dir }}/repair_redis.sh
nagios ALL = NOPASSWD: {{ autosysadmin_dir }}/repair_tomcat_instance.sh

View file

@ -39,8 +39,8 @@ concat_files() {
chown root: "${hapee_cert_file}"
}
cert_and_key_mismatch() {
hapee_cert_md5=$(openssl x509 -noout -pubkey -in "${hapee_cert_file}" | openssl md5)
hapee_key_md5=$(openssl pkey -pubout -in "${hapee_cert_file}" | openssl md5)
hapee_cert_md5=$(openssl x509 -noout -modulus -in "${hapee_cert_file}" | openssl md5)
hapee_key_md5=$(openssl rsa -noout -modulus -in "${hapee_cert_file}" | openssl md5)
test "${hapee_cert_md5}" != "${hapee_key_md5}"
}

View file

@ -29,8 +29,8 @@ concat_files() {
chown root: "${haproxy_cert_file}"
}
cert_and_key_mismatch() {
haproxy_cert_md5=$(openssl x509 -noout -pubkey -in "${haproxy_cert_file}" | openssl md5)
haproxy_key_md5=$(openssl pkey -pubout -in "${haproxy_cert_file}" | openssl md5)
haproxy_cert_md5=$(openssl x509 -noout -modulus -in "${haproxy_cert_file}" | openssl md5)
haproxy_key_md5=$(openssl rsa -noout -modulus -in "${haproxy_cert_file}" | openssl md5)
test "${haproxy_cert_md5}" != "${haproxy_key_md5}"
}

View file

@ -1,44 +0,0 @@
#!/bin/sh
error() {
>&2 echo "${PROGNAME}: $1"
exit 1
}
debug() {
if [ "${VERBOSE}" = "1" ] && [ "${QUIET}" != "1" ]; then
>&2 echo "${PROGNAME}: $1"
fi
}
daemon_found_and_running() {
test -n "$(pidof nrpe)"
}
letsencrypt_lineaged_used() {
grep -r "^ssl_cert_file" /etc/nagios/ | grep "letsencrypt" | grep -q "$(basename "${RENEWED_LINEAGE}")"
}
copy_letsencrypt_cert() {
DEST_CERTIFICATE=$(grep -r "^ssl_cert_file" /etc/nagios/ | awk -F'=' '{print $2}')
DEST_PRIVATE_KEY=$(grep -r "^ssl_privatekey_file" /etc/nagios/ | awk -F'=' '{print $2}')
install --mode 440 --group nagios ${RENEWED_LINEAGE}/fullchain.pem ${DEST_CERTIFICATE}
install --mode 440 --group nagios ${RENEWED_LINEAGE}/privkey.pem ${DEST_PRIVATE_KEY}
}
main() {
if daemon_found_and_running; then
if letsencrypt_lineaged_used; then
debug "NRPE detected... Copying certificates to the right place & permissions"
copy_letsencrypt_cert
debug "Restarting NRPE"
systemctl restart nagios-nrpe-server
else
debug "NRPE doesn't use the given Let's Encrypt certificate. Skip."
fi
else
debug "NRPE is not running or missing. Skip."
fi
}
readonly PROGNAME=$(basename "$0")
readonly VERBOSE=${VERBOSE:-"0"}
readonly QUIET=${QUIET:-"0"}
main

View file

@ -1,6 +0,0 @@
---
check_free_space_partitions:
- "/home"
- "/srv"
check_free_space_max_percent: 70
check_free_space_mailto: Null

View file

@ -1,166 +0,0 @@
#!/bin/sh
# This script verifies if the specified partitions on a machine are filled
# at more than x%.
#
# If so, it sends a mail to the admin of that machine, warning him/her
# that mesures should be taken.
#
# Two outputs are provided to the recipient of the mail:
# * some general infos with `df`
# * a more indepth inspection with `duc`
#
# This script takes 3 (mandatory) arguments:
# * a list of the partitions to check (space separated)
# * the maximum allowed percentage
# * the email template to use
#
# This script should be ran by cron @daily.
#
#
# Copyright (C) 2016 Louis-Philippe VĂ©ronneau <lpveronneau@evolix.ca, Evolix <info@evolix.fr>
#
# This program is licensed under GPLv3 +
# Check argument sanity
PID_FILE='/var/run/check_free_space.pid'
if test -f "$PID_FILE"
then
pid=$(cat "$PID_FILE")
ps -p "$pid" > /dev/null
if test $? -eq 0
then
echo "$0 already run !" >&2
exit 1
else
rm $PID_FILE
fi
fi
echo $$ > $PID_FILE
if test -z "$1" || test -z "$2" || test -z "$3" # is non null
then
echo "Some arguments are missing. Please issue a partition list, a" \
"maximum percentage and an email template."
exit 1
elif ! [ "$2" -le 100 -a "$2" -ge 0 ] # is a percentage
then
echo "Please enter a maximum percentage value between 0 and 100."
exit 1
fi
# Argument processing
partition_list=$1
max_percentage=$((100-$2))
email_template=$3
HOSTNAME=$(hostname)
debian_version=$(lsb_release -c)
check_disk='/usr/lib/nagios/plugins/check_disk'
test -f /etc/evomaintenance.cf && . /etc/evomaintenance.cf
# Test what version of df we have
old_df=false
case "$debian_version" in
*squeeze* ) old_df=true ;;
*wheezy* ) old_df=true ;;
esac
# Check disk space
df_options="size,avail,pcent,itotal,iavail,ipcent"
for partition in $partition_list
do
if ! $check_disk -w $max_percentage% -W $max_percentage% $partition > /dev/null
then
# the 'newline' is a hack to make sed behave
PARTITION_DATA="$PARTITION_DATA newline $partition newline"
if [ $old_df ]
then
PARTITION_DATA="$PARTITION_DATA $(/bin/df -h $partition) newline"
PARTITION_DATA="$PARTITION_DATA newline $(df -ih $partition) newlinenewline"
else
PARTITION_DATA="$PARTITION_DATA $(/bin/df -h --output=$df_options $partition) newline"
fi
full_partitions="$full_partitions $partition"
partname=$(echo $partition|tr -s '/' '-')
graph_list="$graph_list -a /home/duc${partname}.png"
fi
done
# Exit if everything is OK
if test -z "$PARTITION_DATA"
then
exit 0
fi
# If there is indeed a problem, get more infos with duc
/usr/bin/ionice -c3 /usr/bin/duc index -H -d /home/duc.idx -x $full_partitions -q
for partition in $full_partitions
do
duc_temp=$(/usr/bin/duc ls -d /home/duc.idx -Fg $partition)
duc_temp=$(printf "$duc_temp" | sed -e "s@]@]newline@" | grep -v "lost+found")
DUC_OUTPUT="$DUC_OUTPUT newline$partition newline$duc_temp"
partname=$(echo $partition|tr -s '/' '-')
duc graph -d /home/duc.idx -o /home/duc${partname}.png -l8 -s 1024 $partition
done
# Replace placeholders & send the mail !
PARTITION_DATA="$(echo "$PARTITION_DATA"|tr -d $'\n')" # make sed accept the input
DUC_OUTPUT="$(echo "$DUC_OUTPUT"|tr -d $'\n')"
if [ $old_df ]
then
sed -e "s/__TO__/$EVOMAINTMAIL/" \
-e "s/__HOSTNAME__/$HOSTNAME/" \
-e "s@__PARTITION_DATA__@$PARTITION_DATA@" \
-e "s@__DUC_OUTPUT__@$DUC_OUTPUT@" \
-e "s/newline/\n/g" \
-e "s/IUse%/IUse%\n/g" \
-e "s/ Use%/ Use%\n/g" \
-e "s@Filesystem \{12\}@@g" \
-e "s@Mounted on\/dev\/[a-z]\{3\}[0-9]\+ \{13\}@@g" \
-e "s@% \/[a-z]\+@%@g" \
-e "s/__MAX_PERCENTAGE__/$max_percentage/" \
-e "s/__FULLFROM__/$FULLFROM/" \
-e "s/__FROM__/$FROM/" \
-e "s/__URGENCYFROM__/$URGENCYFROM/" \
-e "s/__URGENCYTEL__/$URGENCYTEL/" \
$email_template | \
/usr/bin/mutt -H - $graph_list
else
sed -e "s/__TO__/$EVOMAINTMAIL/" \
-e "s/__HOSTNAME__/$HOSTNAME/" \
-e "s@__PARTITION_DATA__@$PARTITION_DATA@" \
-e "s@__DUC_OUTPUT__@$DUC_OUTPUT@" \
-e "s/newline/\n/g" \
-e "s/IUse%/IUse%\n/g" \
-e "s/__MAX_PERCENTAGE__/$max_percentage/" \
-e "s/__FULLFROM__/$FULLFROM/" \
-e "s/__FROM__/$FROM/" \
-e "s/__URGENCYFROM__/$URGENCYFROM/" \
-e "s/__URGENCYTEL__/$URGENCYTEL/" \
$email_template | \
/usr/bin/mutt -H - $graph_list
fi
rm -f $PID_FILE

View file

@ -1,24 +0,0 @@
From: __FULLFROM__
Content-Type: text/plain; charset=UTF-8
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
To: __TO__
Subject: [WARNING] Espace disque faible sur __HOSTNAME__
Bonjour,
Ceci est un message automatique pour vous informer qu'il y a un
souci d'espace disque sur votre serveur __HOSTNAME__
Voici les informations sur l'espace disque qui pose problème :
__PARTITION_DATA__
Détails sur les partitions problématiques :
__DUC_OUTPUT__
Un graphe par partition problématique est disponible en pièce jointe.
Nous vous recommandons d'effectuer du ménage pour maintenir
chaque partition avec un minimum de __MAX_PERCENTAGE__% d'espace disque libre.
Cordialement,
--
__FULLFROM__

View file

@ -1,37 +0,0 @@
---
- ansible.builtin.include_role:
name: evolix/remount-usr
- name: Copy check_free_space.sh script
ansible.builtin.copy:
src: files/check_free_space.sh
dest: /usr/share/scripts/check_free_space
owner: root
group: root
mode: "0750"
- name: Copy email template
ansible.builtin.copy:
src: files/check_free_space.tpl
dest: /usr/share/scripts/check_free_space.tpl
owner: root
group: root
mode: "0644"
# not using the cron_module for this since it is buggy
- name: check_free_space.sh is run by cron
ansible.builtin.template:
src: templates/cron_check_free_space.j2
dest: /etc/cron.d/check_free_space
owner: root
group: root
mode: "0644"
force: false
- name: Duc and Mutt are installed
ansible.builtin.apt:
pkg:
- mutt
- duc
state: present

View file

@ -1,30 +0,0 @@
---
- include_role:
name: evolix/remount-usr
- name: shell script
copy:
src: files/check_free_space.sh
dest: /usr/share/scripts/check_free_space
owner: root
group: root
mode: "0750"
- name: email template
copy:
src: files/check_free_space.tpl
dest: /usr/share/scripts/check_free_space.tpl
owner: root
group: root
mode: "0644"
# not using the cron_module for this since it is buggy
- name: cron
template:
src: templates/cron_check_free_space.j2
dest: /etc/cron.d/check_free_space
owner: root
group: root
mode: "0644"
force: false

View file

@ -1,4 +0,0 @@
{% if check_free_space_mailto and check_free_space_mailto != "" %}
MAILTO={{ check_free_space_mailto }}
{% endif %}
30 4 * * 1 root /usr/share/scripts/check_free_space "{{ check_free_space_partitions | join(' ') }}" {{ check_free_space_max_percent }} /usr/share/scripts/check_free_space.tpl

View file

@ -3,25 +3,28 @@
docker_home: /var/lib/docker
docker_tmpdir: "{{ docker_home }}/tmp"
# Chose to use iptables instead of docker-proxy userland process
docker_conf_use_iptables: False
# Disable the possibility for containers processes to gain new privileges
docker_conf_no_newprivileges: false
docker_conf_no_newprivileges: False
# Toggle live restore (need to be disabled in swarm mode)
docker_conf_live_restore: true
docker_conf_live_restore: True
# Toggle user namespace
docker_conf_user_namespace: true
docker_conf_user_namespace: True
# Disable all default network connectivity
docker_conf_disable_default_networking: false
docker_conf_disable_default_networking: False
# Remote access
docker_remote_access_enabled: false
docker_remote_access_enabled: False
docker_daemon_port: 2376
docker_daemon_listening_ip: 0.0.0.0
# TLS
docker_tls_enabled: false
docker_tls_enabled: False
docker_tls_path: "{{ docker_home }}/tls"
docker_tls_ca: ca/ca.pem
docker_tls_ca_key: ca/ca-key.pem
@ -29,4 +32,4 @@ docker_tls_cert: server/cert.pem
docker_tls_key: server/key.pem
docker_tls_csr: server/server.csr
apt_keyring_dir: "{{ ansible_distribution_major_version is version('12', '<') | ternary('/etc/apt/trusted.gpg.d', '/etc/apt/keyrings') }}"
apt_keyring_dir: "{{ ansible_distribution_major_version is version('12', '<') | ternary('/etc/apt/trusted.gpg.d', '/etc/apt/keyrings') }}"

View file

@ -1,14 +1,5 @@
# This role installs the docker daemon
---
- name: Fail if docker_conf_use_iptables is defined
ansible.builtin.fail:
msg: "Variable docker_conf_use_iptables is deprecated and not configurable anymore. Please remove it from your variables. Also double-check the daemon.json config for docker"
when:
- docker_conf_use_iptables is defined
tags:
- always
- name: Remove older docker packages
ansible.builtin.apt:
name:
@ -32,7 +23,7 @@
when: ansible_distribution_major_version is version('10', '<')
- name: "Ensure {{ apt_keyring_dir }} directory exists"
ansible.builtin.file:
file:
path: "{{ apt_keyring_dir }}"
state: directory
mode: "755"
@ -53,34 +44,35 @@
repo: 'deb [signed-by={{ apt_keyring_dir }}/docker-debian.asc] https://download.docker.com/linux/debian {{ ansible_distribution_release }} stable'
filename: docker
state: present
update_cache: true
update_cache: yes
when: ansible_distribution_major_version is version('12', '<')
- name: Add Docker repository (Debian >=12)
ansible.builtin.template:
src: docker.sources.j2
dest: /etc/apt/sources.list.d/docker.sources
owner: root
group: root
mode: "0644"
register: docker_sources
when: ansible_distribution_major_version is version('12', '>=')
- name: Update APT cache
ansible.builtin.apt:
update_cache: yes
when: docker_sources is changed
- name: Install Docker
ansible.builtin.apt:
name:
- docker-ce
- docker-ce-cli
- containerd.io
update_cache: true
cache_valid_time: 3600
- name: Package python-docker is installed
- name: python-docker is installed
ansible.builtin.apt:
name: python-docker
state: present
when: ansible_python_version is version('3', '<')
- name: Package python3-docker is installed
- name: python3-docker is installed
ansible.builtin.apt:
name: python3-docker
state: present
@ -90,9 +82,6 @@
ansible.builtin.template:
src: daemon.json.j2
dest: /etc/docker/daemon.json
owner: root
group: root
mode: "0644"
notify: restart docker
- name: Creating Docker tmp directory

Some files were not shown because too many files have changed in this diff Show more