270 lines
8.8 KiB
Markdown
270 lines
8.8 KiB
Markdown
Integrating Patroni with Debian
|
|
===============================
|
|
|
|
Introduction
|
|
------------
|
|
|
|
Patroni manages PostgreSQL instances and mostly expects a blank sheet, i.e
|
|
prefers to initialize and insists to start and stop the database itself. Debian
|
|
on the other hand includes the `postgresql-common` (called pg-common in the
|
|
following) framework that manages concurrent major versions of PostgreSQL and
|
|
possibly database instances for each of them. This document details a design on
|
|
how to integrate Patroni (in form of the Debian `patroni` package, preferable
|
|
provided by PGDG's apt repository) with Debian's pg-common framework and
|
|
policies.
|
|
|
|
pg-common nomenclature and directory layout
|
|
-------------------------------------------
|
|
|
|
pg-common enhances the usual `libpq` environment variables like `PGHOST` and
|
|
`PGDATABASE` with `PGCLUSTER`, that is the Debian standard way of addressing a
|
|
specific version/instance combination. The `PGCLUSTER` environment variable
|
|
consists of two parts, the PostgreSQL major version and the instance name
|
|
(`main` being the default instance name), serarated by a '/', e.g. `'10/main'`.
|
|
The two components are usually abbreviated or referred to as '%v' and '%c',
|
|
respectively.
|
|
|
|
The versioned binaries and libraries are located in
|
|
`/usr/lib/postgresql/%v/{bin,lib}`. The configuration files usually reside in
|
|
`/etc/postgresql/%v/%c/` and the default log file (set via the `pg_ctlcluster`
|
|
wrapper) is `/var/log/postgresql/postgresql-%v-%c.log`. The default data
|
|
directory is `/var/lib/postgresql/%v/%c` and the listening sockets are located
|
|
in `/var/run/postgresql/`.
|
|
|
|
Patroni configuration
|
|
---------------------
|
|
|
|
The Patroni configuration is in YAML format. Different Patroni instances or
|
|
installations are identified by the `scope` configuration option, which mostly
|
|
maps to an instance name.
|
|
|
|
As Patroni is usually started via Docker or another container runtime, there is
|
|
no opinioated default file system layout or even configuratin file location.
|
|
|
|
It would be desirable to create/maintain a Patroni configuration for each
|
|
pg-common instance as `/etc/patroni/%v-%c.yml`, i.e. e.g.
|
|
`/etc/patroni/10-main.yml`. Another possibility would be to prefix the
|
|
configuration file name with `patroni-`.
|
|
|
|
Automatic generation of Patroni configuration
|
|
---------------------------------------------
|
|
|
|
Currently, the pg-common instance specific configuration file
|
|
`/etc/patroni/%v-%c.yml` needs to be deployed/adopted manually as the pg-common
|
|
framework and `pg_createcluster` have no possibilty to run external programs as
|
|
hooks after instance creation.
|
|
|
|
Another possibilty is to create the configuration file via an external config
|
|
management program like Ansible or Puppet.
|
|
|
|
Regardless of this, a simple program that creates the instance-specific
|
|
configuration from a template (e.g. `/etc/patroni/patroni.yml.in`) would be
|
|
desirable and could be shipped in the `patroni` Debian package.
|
|
|
|
Debian-adopted Patroni configuration
|
|
------------------------------------
|
|
|
|
it is possble to mimick the default Debian layout via the following
|
|
configuration parameters:
|
|
|
|
```
|
|
data_dir = "/var/lib/postgresql/%v/%c"
|
|
bin_dir = "/usr/lib/postgresql/%v/bin"
|
|
config_dir: "/etc/postgresql/%v/%c"
|
|
```
|
|
|
|
The logfile location and filename as well as the socket directory have to be
|
|
explicitly set via the configuration file:
|
|
|
|
```
|
|
unix_socket_directories = '/var/run/postgresql/'
|
|
log_directory = '/var/log/postgresql'
|
|
log_filename = 'postgresql-%v-%c.log'
|
|
```
|
|
|
|
The `postgresql.conf` configuration file
|
|
----------------------------------------
|
|
|
|
Patroni expects to deploy a configuration file and to be able to change it and
|
|
keep it in sync across nodes. If a configuration file `postgresql.conf` is
|
|
already present in the `config_dir`, it renames it to `postgresql.base.conf` and
|
|
includes it at the top of the `postgresql.conf` it writes instead.
|
|
|
|
Initialization of the first and standby instances
|
|
-------------------------------------------------
|
|
|
|
Patroni by default runs `initdb` on the data directory during bootrap while
|
|
pg-common provides the `pg_createcluster` command for this purpose.
|
|
|
|
It is possible to tell Patroni to run an external bootstrap command which is
|
|
passed the `--scope` and `--datadir` command-line options. This makes it
|
|
possible to have a small wrapper script like the following that runs
|
|
`pg_createcluster` instead:
|
|
|
|
```
|
|
#!/bin/sh
|
|
|
|
for i in "$@"
|
|
do
|
|
case $i in
|
|
--scope=*)
|
|
SCOPE="${i#*=}"
|
|
shift # past argument=value
|
|
;;
|
|
--datadir=*)
|
|
DATADIR="${i#*=}"
|
|
shift # past argument=value
|
|
;;
|
|
*)
|
|
# unknown option
|
|
;;
|
|
esac
|
|
done
|
|
|
|
VERSION=$(echo $SCOPE | sed -e 's/\/.*//')
|
|
CLUSTER=$(echo $SCOPE | sed -e 's/.*\///')
|
|
|
|
pg_createcluster $VERSION $CLUSTER
|
|
exit $?
|
|
```
|
|
|
|
This requires the following in the Patroni YAML configuratio:
|
|
|
|
```
|
|
bootstrap:
|
|
# Custom bootstrap method
|
|
method: pg_createcluster
|
|
pg_createcluster:
|
|
command: <custom script above>
|
|
```
|
|
|
|
Subsequent nodes are boostrapped from the primary via base backups. Just running
|
|
`pg_createcluster` on them is not possible as then the cluster IDs (the
|
|
'Database system identifier' in the `pg_controldata` output) would differ.
|
|
|
|
Again, it is possible to provide an external clone program, which can run
|
|
`pg_basebackup` itself. The pg-common framework currently does not provide for
|
|
this, but it is possible to run `pg_basebackup` after `pg_createcluster` (which
|
|
creates the configuration directory) and purging the data directory:
|
|
|
|
```
|
|
#!/bin/sh
|
|
|
|
for i in "$@"
|
|
do
|
|
case $i in
|
|
--scope=*)
|
|
SCOPE="${i#*=}"
|
|
shift # past argument=value
|
|
;;
|
|
--role=*)
|
|
ROLE="${i#*=}"
|
|
shift # past argument=value
|
|
;;
|
|
--datadir=*)
|
|
DATADIR="${i#*=}"
|
|
shift # past argument=value
|
|
;;
|
|
--connstring=*)
|
|
CONNSTR="${i#*=}"
|
|
shift # past argument=value
|
|
;;
|
|
*)
|
|
# unknown option
|
|
;;
|
|
esac
|
|
done
|
|
|
|
VERSION=$(echo $SCOPE | sed -e 's/\/.*//')
|
|
CLUSTER=$(echo $SCOPE | sed -e 's/.*\///')
|
|
|
|
if [ -f /etc/postgresql/$VERSION/$CLUSTER/postgresql.conf ]
|
|
then
|
|
pg_dropcluster $VERSION $CLUSTER
|
|
fi
|
|
|
|
pg_createcluster $VERSION $CLUSTER && rm -rf $DATADIR && pg_basebackup --pgdata $DATADIR -X stream --dbname=$CONNSTR
|
|
exit $?
|
|
```
|
|
|
|
Those two scripts could be shipped in the `patroni` Debian package e.g. as
|
|
`/usr/share/patroni/pg_createcluster_patroni` and
|
|
`/usr/share/patroni/pg_clonecluster_patroni`.
|
|
|
|
Systemd services
|
|
----------------
|
|
|
|
The Patroni daemon/agent needs to be started for each configuration file in
|
|
`/etc/patroni`, i.e. a systemd service unit is required for each. This can be
|
|
facilitated via the `patroni@.service` that acts as a wild card and could have
|
|
the following pg-common specific content:
|
|
|
|
```
|
|
[Unit]
|
|
ConditionPathExists=/etc/patroni/%i.yml
|
|
|
|
[Service]
|
|
ExecStart=/usr/bin/patroni /etc/patroni/%i.yml
|
|
```
|
|
|
|
This makes it possible to e.g. start Patroni for the '10/main' instance with
|
|
`systemctl start patroni@10/main`.
|
|
|
|
DCS Debian availability
|
|
-----------------------
|
|
|
|
The only DCS server available in Debian 9 (stretch) is zookeeper. Both etcd and
|
|
consul were blacklisted before release by the Debian security team due to being
|
|
written in Go.
|
|
|
|
Current testing (buster) also includes etcd and consul.
|
|
|
|
The latest Patroni Debian package is available for both Debian stable and
|
|
testing/unstable via the PGDG APT repository.
|
|
|
|
DCS setup
|
|
---------
|
|
|
|
One outstanding issue with providing a general Patroni configuration template
|
|
and possibly a program that creates a Patroni configuration file per pg-common
|
|
instance is the distributed consensus store, which also needs to be configured
|
|
in the Patroni YAML file.
|
|
|
|
One possibility would be to introduce a file `/etc/default/patroni` where the
|
|
DCS and it's ip-address(es) could be configured which could be sourced by the
|
|
configuration file generator.
|
|
|
|
Security
|
|
--------
|
|
|
|
The proposed/current setup should be considered a Proof of Concept. In
|
|
particular, care must be taken that the postgres db/system user password is not
|
|
leaked and that the DCS access is secured/hardened.
|
|
|
|
Making `patronictl` pg-common aware
|
|
-----------------------------------
|
|
|
|
The `patronictl` CLI currently looks for a configuration file
|
|
`~/.config/patronictl.yml`. This could potentially be a symlink to the
|
|
configuration under `/etc/patroni/`. In the pg-common scope, it could
|
|
addtionally either display all pg-common instances and/or accept the '%v/%c'
|
|
instance notation in order to select a specific instance to display or act on.
|
|
|
|
Current status
|
|
--------------
|
|
|
|
As of the postgresql-common_194 and patroni_1.5.0-9 packages, the following
|
|
steps are still needed for each pg-common instance %v/%c:
|
|
|
|
1. Setup DCS and possibly install its client python packages
|
|
2. Install postgresql-common package
|
|
3. Disable automatic instance creation in
|
|
/etc/postgresql-common/createcluster.conf
|
|
4. Install postgresql-%v package
|
|
5. Setup authentication via pgpass or otherwise
|
|
6. Create a /etc/patroni/%v-%c.yml configuration file
|
|
7. Start the patroni systemd service for the instance
|
|
|
|
In particular, no changes are required to the Patroni or postgresql-common
|
|
upstream codebases.
|