Setting distribution to unreleased

the citus unit test for now as it fails on some configurations.

debian/README.Debian

@@ -96,7 +96,10 @@ pg_rewind requires remote superuser access) and Patroni will re-clone a former
 primary if its timeline has diverged from the cluster.
 
 If pg_rewind is to be used, then the `bootstrap.use_pg_rewind' parameter in
-/etc/patroni/config.yml.in needs to be set to `true' and the
-`postgresql.superuser.password' parameter needs to be set to the desired
-postgres password.  Finally, password (md5) connections need to be allowed on
-the local network.
+/etc/patroni/config.yml.in needs to be set to `true' and either the
+`postgresql.authentication.superuser.password' parameter needs to be set to the
+desired postgres password or (for PostgreSQL versions 11 and up) the
+`postgresql.authenticaion.rewind.{user,password}` needs to be set.  If the
+`rewind` user is set, Patroni will grant it the necessary rights to run
+pg_rewind without superuser privileges. Finally, password (md5) connections
+need to be allowed on the local network.

debian/changelog

@@ -1,3 +1,268 @@
+patroni (3.0.1-2) UNRELEASED; urgency=medium
+
+
+ -- Debian PostgreSQL Maintainers <team+postgresql@tracker.debian.org>  Fri, 17 Feb 2023 09:34:22 +0100
+
+patroni (3.0.1-1) unstable; urgency=medium
+
+  * New upstream release.
+  * debian/control (Uploaders): Updated.
+
+ -- Michael Banck <mbanck@debian.org>  Fri, 17 Feb 2023 09:33:48 +0100
+
+patroni (3.0.0-1) unstable; urgency=medium
+
+  * New major upstream release.
+  * debian/patches/acceptance_tests_system_patroni.patch: Refreshed.
+  * debian/patches/avoid_overwriting_configuration_during_boostrap.patch:
+    Likewise.
+  * debian/tests/control: Add procps as dependency and skip DCS failsafe mode
+    acceptance tests on zookeeper.
+  * debian/patches/regression_tests_skip_citus_test.patch: New patch, skips
+    the citus unit test for now as it fails on some configurations.
+  * debian/control (Depends): Removed obsolete lsb-base. 
+
+ -- Michael Banck <michael.banck@credativ.de>  Tue, 31 Jan 2023 22:27:24 +0100
+
+patroni (2.1.7-1) unstable; urgency=medium
+
+  * New upstream release.
+  * debian/patches/acceptance_tests_reenable_etcdv2.patch: Removed, no longer
+    needed. 
+  * debian/patches/compatibility_with_old_modules.patch: Likewise.
+
+ -- Michael Banck <michael.banck@credativ.de>  Mon, 23 Jan 2023 21:24:46 +0100
+
+patroni (2.1.6-2) unstable; urgency=medium
+
+  [ Shengjing Zhu ]
+  * debian/tests/acceptance: Ensure no etcd server is running before acceptance
+    tests. Otherwise, acceptance tests can't spin up a new etcd server which
+    uses the same port (Closes: #1027707).
+
+ -- Michael Banck <michael.banck@credativ.de>  Sun, 08 Jan 2023 22:06:02 +0100
+
+patroni (2.1.6-1) unstable; urgency=medium
+
+  * New upstream release.
+  * debian/patches/acceptance_tests_system_patroni.patch: Refreshed.
+  * debian/patches/acceptance_tests_reenable_etcdv2.patch: New patch,
+    re-enables V2 API on etcd-3.4 and later. 
+  * debian/tests/control: Add etcd3 behave test. 
+  * debian/patches/compatibility_with_old_modules.patch: New patch, fixes
+    unit test failures with older modules, taken from upstream commit 442bd3f.
+
+ -- Michael Banck <michael.banck@credativ.de>  Wed, 04 Jan 2023 10:51:02 +0100
+
+patroni (2.1.5-1) unstable; urgency=medium
+
+  * Team upload.
+  * New upstream release.
+  * Mark Consul test as skip-not-installable (not in testing, see #1017982).
+
+ -- Christoph Berg <myon@debian.org>  Mon, 28 Nov 2022 13:07:28 +0100
+
+patroni (2.1.4-3) unstable; urgency=medium
+
+  * Enable RAFT support and add python3-pysyncobj to dep alternatives.
+
+ -- Christoph Berg <myon@debian.org>  Thu, 17 Nov 2022 16:14:33 +0100
+
+patroni (2.1.4-2) unstable; urgency=medium
+
+  * Team upload.
+
+  [ Michael Banck ]
+  * debian/patches/avoid_overwriting_configuration_during_boostrap.patch: New
+    patch, avoids streaming a backup of the primary's postgresql.conf file
+    during bootstrap, overwriting the standby's postgresql.base.conf and
+    possible preventing PostgreSQL to start up.
+  * debian/patches/acceptance_tests_speedup.patch: New patch, reduces loop_wait
+    to 2 thus speeding up the behave tests, taken from upstream commit ead798.
+
+  [ Christoph Berg ]
+  * Add a test using etcd running basic_replication.feature only, and mark the
+    full tests as flaky. (Closes: #1017527)
+
+ -- Christoph Berg <myon@debian.org>  Thu, 08 Sep 2022 17:04:35 +0200
+
+patroni (2.1.4-1) unstable; urgency=medium
+
+  * New upstream release.
+  * debian/patches/pg_rewind_fix_get_guc_value.patch: Removed, no longer
+    needed.
+  * debian/control, debian/tests/control: Replaced python3-boto with
+    python3-boto3.
+
+ -- Michael Banck <michael.banck@credativ.de>  Thu, 02 Jun 2022 16:58:10 +0200
+
+patroni (2.1.3-3) unstable; urgency=medium
+
+  * debian/patches/regression_tests_disable_raft_tests.py: Drop unrelated
+    etcd3 validator removal, pointed out by Kirill Petrov.
+
+ -- Michael Banck <michael.banck@credativ.de>  Fri, 29 Apr 2022 19:17:12 +0200
+
+patroni (2.1.3-2) unstable; urgency=medium
+
+  * debian/patches/pg_rewind_fix_get_guc_value.patch: New patch, changes the
+    pg_rewind calling behaviour so that it can deal with postgresql.conf not
+    being in the data directory; taken from upstream commit 7626b5f by Nick
+    Bluth.
+
+ -- Michael Banck <michael.banck@credativ.de>  Thu, 24 Mar 2022 22:22:16 +0100
+
+patroni (2.1.3-1) unstable; urgency=medium
+
+  * New upstream release.
+  * debian/patches/regression_tests_disable_requirement_download.patch:
+    Removed, no longer needed.
+  * debian/patches/regression_tests_psutil_compat.patch: Likewise.
+  * debian/config.yml.in: Update configuration:
+    - Set use_pg_rewind, use_slots and check_timeline to true.
+    - Update comments and formatting.
+
+ -- Michael Banck <michael.banck@credativ.de>  Fri, 18 Feb 2022 22:16:13 +0100
+
+patroni (2.1.2-3) unstable; urgency=medium
+
+  * debian/patches/regression_tests_psutil_compat.patch: New patch, add
+    compatibility for newer psutil in the regression tests (Closes: #1003562).
+  * debian/changelog: Fix whitespace.
+  * debian/tests/acceptance: Skip standby_cluster/logical_slots_in_sync
+    scenario on v10, as it is not supported there.
+
+ -- Michael Banck <michael.banck@credativ.de>  Tue, 15 Feb 2022 23:06:15 +0100
+
+patroni (2.1.2-2) unstable; urgency=medium
+
+  * Team upload.
+  * debian/clean: Remove patroni.egg-info/.
+  * debian/py3dist-overrides: Tell dh_python3 to find "ipaddress" in python3
+    >= 3.3.
+  * debian/tests/control: Mark zookeeper test as skip-not-installable.
+  * debian/tests/acceptance: `set -o pipefail` so failures aren't masked by
+    `ts`.
+
+ -- Christoph Berg <myon@debian.org>  Wed, 22 Dec 2021 15:22:09 +0100
+
+patroni (2.1.2-1) unstable; urgency=medium
+
+  * New upstream release.
+
+  [ Christoph Berg ]
+  * patroni@.service: Remove redundant (and broken) WorkingDirectory.
+    Thanks Ludovic Gasc!
+  * debian/tests/control: Give Test-Commands meaningful names.
+
+  [ Michael Banck ]
+  * debian/patches/acceptance_tests_system_patroni.patch: Refreshed.
+
+ -- Michael Banck <michael.banck@credativ.de>  Fri, 03 Dec 2021 17:13:13 +0100
+
+patroni (2.1.1-1) unstable; urgency=medium
+
+  * New upstream release.
+  * debian/patches/regression_tests_disable_raft_tests.py: Refreshed.
+
+ -- Michael Banck <michael.banck@credativ.de>  Fri, 20 Aug 2021 10:53:23 +0200
+
+patroni (2.1.0-1) experimental; urgency=medium
+
+  * New upstream major release. New features:
+   + Compatibility with PostgreSQL v14
+   + Failover logical slots
+   + Allowlist for Patroni REST API
+   + Support of replication connections via unix socket
+   + Health check on user-defined tags
+   + Prometheus /metrics endpoint
+   + Reduced chattiness of Patroni logs
+  * debian/patches/regression_tests_disable_requirement_download.patch:
+    Refreshed.
+  * debian/patches/requirements_cdiff.patch: Likewise.
+  * debian/patches/requirements_setuptools.patch: Likewise.
+
+ -- Michael Banck <michael.banck@credativ.de>  Fri, 09 Jul 2021 16:58:39 +0200
+
+patroni (2.0.2-1) unstable; urgency=medium
+
+  * New upstream release.
+  * debian/patches/acceptance_tests_system_patroni.patch: Refreshed.
+  * debian/patches/regression_tests_disable_requirement_download.patch:
+    Likewise.
+  * debian/patches/regression_tests_disable_raft_tests.py: New patch, reverts
+    upstream commit 3a87d0e.
+
+ -- Michael Banck <michael.banck@credativ.de>  Thu, 04 Mar 2021 08:59:40 +0100
+
+patroni (2.0.1-3) unstable; urgency=medium
+
+  * debian/pg_clonecluster_patroni: Supress pg_createcluster output as it can
+    be misleading.
+  * debian/pg_clonecluster_patroni: Remove postgresql.base.conf before running
+    pg_dropcluster.
+  * debian/config.yml.in: Make base data directory a template variable
+    @DATADIR_BASE@.
+  * debian/pg_createconfig_patroni: Use data_directory setting from
+    /etc/postgresql-common/createcluster.conf as DATADIR_BASE if set.
+
+ -- Michael Banck <michael.banck@credativ.de>  Fri, 22 Jan 2021 12:45:15 +0100
+
+patroni (2.0.1-2) unstable; urgency=medium
+
+  * debian/tests/control: Added moreutils to acceptance-test Depends.
+  * debian/tests/acceptance: Pipe test outputs to ts in oder to
+    display timestamps.
+
+ -- Michael Banck <michael.banck@credativ.de>  Thu, 10 Dec 2020 13:33:18 +0100
+
+patroni (2.0.1-1) unstable; urgency=medium
+
+  * New upstream major release.
+  * debian/README.Debian: Add note about rewind user.
+  * debian/patches/offline_intersphinx.patch: Refreshed.
+  * debian/patches/requirements_setuptools.patch: Refreshed.
+  * debian/patches/startup_scripts.patch: Refreshed.
+  * debian/control (patroni/Depends): Added python3-cdiff.
+  * debian/patches/regression_tests_disable_requirement_download.patch: New
+    patch, disables downloading python packages during regression tests.
+  * debian/tests/test: Remove Raft tests for now (not yet packaged pysyncobj
+    module).
+  * debian/patches/requirements_cdiff.patch: New patches, changes ydiff (not
+    yet packaged python module) back to cdiff in requirements.
+
+ -- Michael Banck <michael.banck@credativ.de>  Tue, 06 Oct 2020 18:10:10 +0200
+
+patroni (1.6.5-5) unstable; urgency=medium
+
+  [ Dominik George ]
+  * debian/pg_createconfig_patroni: Correctly determine prefix length
+    in $NETWORK.
+
+  [ Michael Banck ]
+  * debian/config.yml.in: Listen to all interfaces by default.
+
+ -- Michael Banck <michael.banck@credativ.de>  Wed, 26 Aug 2020 15:33:04 +0200
+
+patroni (1.6.5-4) unstable; urgency=medium
+
+  * debian/config.yml, debian/dcs.yml: Update example consul configuration,
+    addresses upstream issue #1574.
+
+ -- Michael Banck <michael.banck@credativ.de>  Wed, 03 Jun 2020 16:10:59 +0200
+
+patroni (1.6.5-3) unstable; urgency=medium
+
+  * debian/pg_clonecluster_patroni: Always use explicit pg_basebackup version,
+    in case more than one major version of PostgreSQL is installed.
+  * debian/config.yml.in: Add commented out section on a rewind user, that can
+    be used on PostgreSQL 11+ to run pg_rewind as a non-superuser.
+  * debian/pg_createconfig_patroni: Parse multiple DCS endpoints correctly,
+    patch by Tomas Pospisek (Closes: #953197).
+  * debian/tests/acceptance: Start zookeeper manually.
+
+ -- Michael Banck <michael.banck@credativ.de>  Fri, 08 May 2020 21:06:58 +0200
+
 patroni (1.6.5-2) unstable; urgency=medium
 
   * debian/pg_createconfig_patroni: Allow re-creating configuration file for
@@ -14,7 +279,7 @@ patroni (1.6.5-2) unstable; urgency=medium
 patroni (1.6.5-1) unstable; urgency=medium
 
   * New upstream release.
-  * debian/patches/fix_failing_tests.patch: Removed, no longer needed. 
+  * debian/patches/fix_failing_tests.patch: Removed, no longer needed.
 
  -- Michael Banck <michael.banck@credativ.de>  Mon, 27 Apr 2020 18:24:43 +0200
 
@@ -36,7 +301,7 @@ patroni (1.6.4-2) unstable; urgency=medium
 patroni (1.6.4-1) unstable; urgency=medium
 
   * New upstream release.
-  * debian/patches/check_postmaster.patch: Removed. 
+  * debian/patches/check_postmaster.patch: Removed.
   * debian/patches/acceptance_tests_system_patroni.patch: Refreshed.
   * debian/patches/v12_fix_recovery.conf_parameters.patch: Removed, no longer
     needed.
@@ -107,7 +372,7 @@ patroni (1.6.0-4) unstable; urgency=medium
 
 patroni (1.6.0-3) unstable; urgency=medium
 
-  * debian/control (patroni/Depends): Added python3-distutils. 
+  * debian/control (patroni/Depends): Added python3-distutils.
   * debian/patches/acceptance_tests_timeouts.patch: New patch, increases a
     timeout of an often failing autopkgtest test.
 
@@ -163,13 +428,13 @@ patroni (1.5.5-1) unstable; urgency=medium
   * debian/patches/consul_import.patch: Refreshed.
   * debian/patches/reclone-unable-pg_rewind.patch: Removed, no longer needed.
   * debian/config.yml.in: Set use_pg_rewind to false again.
- 
+
  -- Michael Banck <michael.banck@credativ.de>  Sat, 16 Feb 2019 07:30:02 +0100
 
 patroni (1.5.4-8) unstable; urgency=medium
 
   * debian/pg_createconfig_patroni: Remove debugging output and fix premature
-    exit. 
+    exit.
   * debian/patches/acceptance_tests_coverage_binary.patch: Updated according to
     upstream feedback.
   * debian/patches/behave_failure_logging.patch: Likewise.
@@ -340,7 +605,7 @@ patroni (1.5.1-1) unstable; urgency=medium
   * debian/patches/consul_import.patch: Likewise.
   * debian/patches/coverage_binary.patch: Likewise.
   * debian/patches/startup_scripts.patch: Likewise.
-  * debian/pg_createconfig_patroni: New script. 
+  * debian/pg_createconfig_patroni: New script.
   * debian/dcs.yml, debian/config.yml.in: New configuration templates.
   * debian/patroni.install: Install new script to /usr/bin and configuration
     templates to /etc/patroni.
@@ -349,7 +614,7 @@ patroni (1.5.1-1) unstable; urgency=medium
 
 patroni (1.5.0-12) unstable; urgency=medium
 
-  * debian/patroni.install: Install binaries as well. 
+  * debian/patroni.install: Install binaries as well.
   * debian/patches/acceptance_tests_system_patroni.patch: New patch, use the
     system installed patroni for acceptance tests.
 
@@ -369,7 +634,7 @@ patroni (1.5.0-10) unstable; urgency=medium
   * debian/patroni.install: Move installation of HTML documentation to ...
   * debian/patroni-doc.install: ... this new file.
   * debian/rules (override_dh_installinit): Install init scripts in
-    patroni package only. 
+    patroni package only.
 
  -- Michael Banck <michael.banck@credativ.de>  Fri, 26 Oct 2018 12:43:07 +0100
 
@@ -415,7 +680,7 @@ patroni (1.5.0-6) unstable; urgency=medium
 
 patroni (1.5.0-5) unstable; urgency=medium
 
-  * debian/patches/check_postmaster.patch: Update unit tests as well. 
+  * debian/patches/check_postmaster.patch: Update unit tests as well.
 
  -- Michael Banck <michael.banck@credativ.de>  Wed, 26 Sep 2018 16:08:50 +0200
 
@@ -430,7 +695,7 @@ patroni (1.5.0-4) unstable; urgency=medium
 
 patroni (1.5.0-3) unstable; urgency=medium
 
-  * debian/gitlab-ci.yml: New file. 
+  * debian/gitlab-ci.yml: New file.
   * debian/patches/behave_failure_logging.patch: New patch, copy over output
     directory of a failed feature with `_failed' at the end.
   * debian/tests/acceptance: Dump logs for failed features.

debian/clean

@@ -0,0 +1 @@
+patroni.egg-info/

debian/config.yml

@@ -20,8 +20,7 @@ restapi:
 #  host: 127.0.0.1:2379
 
 #consul:
-#  host: http://127.0.0.1:8500
-#  host: https://127.0.0.1:8500
+#  host: 127.0.0.1:8500
 
 zookeeper:
   hosts: 127.0.0.1:2181

debian/config.yml.in

@@ -26,13 +26,15 @@ bootstrap:
   pg_createcluster:
     command: /usr/share/patroni/pg_createcluster_patroni
 
-  # this section will be written into Etcd:/<namespace>/<scope>/config after initializing new cluster
-  # and all other cluster members will use it as a `global configuration`
+  # This section will be written into /<namespace>/<scope>/config after
+  # initializing a new cluster and all other cluster members will use it as a
+  # `global configuration`
   dcs:
     ttl: 30
     loop_wait: 10
     retry_timeout: 10
     maximum_lag_on_failover: 1048576
+    check_timeline: true
 #    master_start_timeout: 300
 #    synchronous_mode: false
 #    standby_cluster:
@@ -40,10 +42,10 @@ bootstrap:
 #      port: 1111
 #      primary_slot_name: patroni
     postgresql:
-      use_pg_rewind: false
+      use_pg_rewind: true
       remove_data_directory_on_rewind_failure: true
       remove_data_directory_on_diverged_timelines: true
-#      use_slots: true
+      use_slots: true
       # The following parameters are given as command line options
       # overriding the settings in postgresql.conf.
       parameters:
@@ -69,10 +71,16 @@ bootstrap:
       - host    replication     all             127.0.0.1/32            md5
       - host    replication     all             ::1/128                 md5
       - host    replication     all             @NETWORK@               md5
-#  # Some possibly desired options for 'initdb'
-#  initdb:  # Note: It needs to be a list (some options need values, others are switches)
+#  # Some possibly desired options for 'initdb'. Note: It needs to be a list
+#  # (some options need values, others are # switches)
+#  initdb:
 #  - encoding: UTF8
 #  - data-checksums
+
+#  # Additional script to be launched after initial cluster creation (will be
+#  # passed the connection URL as parameter)
+#  post_init: /usr/local/bin/setup_cluster.sh
+
 #  # Additional users to be created after initializing the cluster
 #  users:
 #    foo:
@@ -90,26 +98,40 @@ postgresql:
   pg_clonecluster:
     command: /usr/share/patroni/pg_clonecluster_patroni
 
-  listen: "@HOSTIP@@LISTEN_VIP@,127.0.0.1:@PORT@"
+  # Listen to all interfaces by default, this makes vip-manager work
+  # out-of-the-box without having to set net.ipv4.ip_nonlocal_bind or similar.
+  # If you prefer to only listen on some interfaces, edit the below:
+#  listen: "@HOSTIP@@LISTEN_VIP@,127.0.0.1:@PORT@"
+  listen: "*:@PORT@"
   connect_address: @HOSTIP@:@PORT@
   use_unix_socket: true
-  data_dir: /var/lib/postgresql/@VERSION@/@CLUSTER@
+  # Default Debian/Ubuntu directory layout
+  data_dir: @DATADIR_BASE@/@VERSION@/@CLUSTER@
   bin_dir: /usr/lib/postgresql/@VERSION@/bin
   config_dir: /etc/postgresql/@VERSION@/@CLUSTER@
   pgpass: /var/lib/postgresql/@VERSION@-@CLUSTER@.pgpass
   authentication:
     replication:
-      username: replicator
-      password: rep-pass
-# A superuser role is required in order for Patroni to manage the local
-# Postgres instance.  If the option `use_unix_socket' is set to `true', then
-# specifying an empty password results in no md5 password for the superuser
-# being set and sockets being used for authentication. The `password:' line is
-# nevertheless required.  Note that pg_rewind will not work if no md5 password
-# is set.
+      username: "replicator"
+      password: "rep-pass"
+    # A superuser role is required in order for Patroni to manage the local
+    # Postgres instance.  If the option `use_unix_socket' is set to `true',
+    # then specifying an empty password results in no md5 password for the
+    # superuser being set and sockets being used for authentication. The
+    # `password:' line is nevertheless required.  Note that pg_rewind will not
+    # work if no md5 password is set unless a rewind user is configured, see
+    # below.
     superuser:
-      username: postgres
+      username: "postgres"
       password:
+    # A rewind role can be specified in order for Patroni to use on PostgreSQL
+    # 11 or later for pg_rewind, i.e. rewinding a former primary after failover
+    # without having to re-clone it. Patroni will assign this user the
+    # necessary permissions (that only exist from PostgreSQL)
+#    rewind:
+#      username: "rewind"
+#      password: "rewind-pass"
+
   parameters:
     unix_socket_directories: '/var/run/postgresql/'
     # Emulate default Debian/Ubuntu logging

debian/control

@@ -2,7 +2,7 @@ Source: patroni
 Section: database
 Priority: optional
 Maintainer: Debian PostgreSQL Maintainers <team+postgresql@tracker.debian.org>
-Uploaders: Michael Banck <michael.banck@credativ.de>,
+Uploaders: Michael Banck <mbanck@debian.org>,
  Adrian Vondendriesch <adrian.vondendriesch@credativ.de>,
 Build-Depends:
  debhelper (>= 10),
@@ -12,7 +12,7 @@ Build-Depends:
  libjs-underscore,
  pyflakes3,
  python3,
- python3-boto,
+ python3-boto3,
  python3-cdiff,
  python3-click (>= 4.1),
  python3-consul (>= 0.7.0),
@@ -46,8 +46,9 @@ Homepage: https://github.com/zalando/patroni
 
 Package: patroni
 Architecture: all
-Depends: ${misc:Depends}, ${python3:Depends}, lsb-base (>= 3.0-6), python3-psycopg2,
- python3-etcd (>= 0.4.3) | python3-consul (>= 0.7.0) | python3-kazoo | python3-kubernetes
+Depends: ${misc:Depends}, ${python3:Depends}, python3-psycopg2,
+ python3-etcd (>= 0.4.3) | python3-consul (>= 0.7.0) | python3-kazoo |
+ python3-kubernetes | python3-pysyncobj, python3-cdiff
 Recommends: iproute2
 Suggests: postgresql, etcd-server | consul | zookeeperd, vip-manager, haproxy, patroni-doc
 Description: PostgreSQL High Availability with ZooKeeper, etcd, Consul, or Kubernetes

debian/dcs.yml

@@ -2,8 +2,7 @@
 #  host: 127.0.0.1:2379
 
 #consul:
-#  host: http://127.0.0.1:8500
-#  host: https://127.0.0.1:8500
+#  host: 127.0.0.1:8500
 
 zookeeper:
   hosts: 127.0.0.1:2181

debian/patches/acceptance_tests_system_patroni.patch

@@ -2,13 +2,12 @@ Index: patroni/features/environment.py
 ===================================================================
 --- patroni.orig/features/environment.py
 +++ patroni/features/environment.py
-@@ -138,8 +138,7 @@ class PatroniController(AbstractControll
-         if isinstance(self._context.dcs_ctl, KubernetesController):
-             self._context.dcs_ctl.create_pod(self._name[8:], self._scope)
-             os.environ['PATRONI_KUBERNETES_POD_IP'] = '10.0.0.' + self._name[-1]
--        return subprocess.Popen([sys.executable, '-m', 'coverage', 'run',
--                                '--source=patroni', '-p', 'patroni.py', self._config],
-+        return subprocess.Popen([sys.executable, '-m', 'coverage', 'run', '-p', '/usr/bin/patroni', self._config],
-                                 stdout=self._log, stderr=subprocess.STDOUT, cwd=self._work_directory)
- 
-     def stop(self, kill=False, timeout=15, postgres=False):
+@@ -150,7 +150,7 @@ class PatroniController(AbstractControll
+         if os.name == 'nt':
+             env['BEHAVE_DEBUG'] = 'true'
+         patroni = subprocess.Popen([sys.executable, '-m', 'coverage', 'run',
+-                                   '--source=patroni', '-p', 'patroni.py', self._config], env=env,
++                                   '-p', '/usr/bin/patroni', self._config], env=env,
+                                    stdout=self._log, stderr=subprocess.STDOUT, cwd=self._work_directory)
+         if os.name == 'nt':
+             patroni.terminate = self.terminate

debian/patches/avoid_overwriting_configuration_during_boostrap.patch

@@ -0,0 +1,41 @@
+commit a3776e0dfb2f0c5a9e5174f948122b8fb33ceebc
+Author: Michael Banck <michael.banck@credativ.de>
+Date:   Fri Jul 29 14:55:15 2022 +0200
+
+    Make backup copies of configuration files in config_dir.
+    
+    Before, the .backup files were always done in the data directory. However, if
+    there is a distinct configuration directory, it seems logical to put them
+    there. If config_dir is not configured, it defaults back to the data directory,
+    so this will not change things for setups where postgresql.conf is in the data
+    directory.
+    
+    On the other hand, it will help with setups where postgresql.conf is outside
+    the data directory and has local changes. In this case, those no longer get
+    overwritten on boostrap/clone from the primary's backup configuration file that
+    is streamed to the boostrapped node.
+    
+    Close #2370
+
+Index: patroni/patroni/postgresql/config.py
+===================================================================
+--- patroni.orig/patroni/postgresql/config.py
++++ patroni/patroni/postgresql/config.py
+@@ -355,7 +355,7 @@ class ConfigHandler(object):
+             try:
+                 for f in self._configuration_to_save:
+                     config_file = os.path.join(self._config_dir, f)
+-                    backup_file = os.path.join(self._postgresql.data_dir, f + '.backup')
++                    backup_file = os.path.join(self._config_dir, f + '.backup')
+                     if os.path.isfile(config_file):
+                         shutil.copy(config_file, backup_file)
+             except IOError:
+@@ -367,7 +367,7 @@ class ConfigHandler(object):
+         try:
+             for f in self._configuration_to_save:
+                 config_file = os.path.join(self._config_dir, f)
+-                backup_file = os.path.join(self._postgresql.data_dir, f + '.backup')
++                backup_file = os.path.join(self._config_dir, f + '.backup')
+                 if not os.path.isfile(config_file):
+                     if os.path.isfile(backup_file):
+                         shutil.copy(backup_file, config_file)

debian/patches/offline_intersphinx.patch

@@ -1,6 +1,8 @@
---- ./docs/conf.py.orig	2020-01-23 18:12:00.026645214 +0100
-+++ ./docs/conf.py	2020-01-23 18:13:43.635004952 +0100
-@@ -189,7 +189,17 @@
+Index: patroni/docs/conf.py
+===================================================================
+--- patroni.orig/docs/conf.py
++++ patroni/docs/conf.py
+@@ -189,7 +189,17 @@ epub_exclude_files = ['search.html']
  
  
  # Example configuration for intersphinx: refer to the Python standard library.

debian/patches/regression_tests_skip_citus_test.patch

@@ -0,0 +1,13 @@
+--- ./tests/test_citus.py.orig	2023-01-31 09:22:20.849730475 +0100
++++ ./tests/test_citus.py	2023-01-31 09:22:46.673632646 +0100
+@@ -4,7 +4,10 @@
+ from . import BaseTestPostgresql, MockCursor, psycopg_connect, SleepException
+ from .test_ha import get_cluster_initialized_with_leader
+ 
++import unittest
+ 
++
++@unittest.skipIf(True, "Citus not tested")
+ @patch('patroni.postgresql.citus.Thread', Mock())
+ @patch('patroni.psycopg.connect', psycopg_connect)
+ class TestCitus(BaseTestPostgresql):

debian/patches/requirements_cdiff.patch

@@ -0,0 +1,11 @@
+Index: patroni/requirements.txt
+===================================================================
+--- patroni.orig/requirements.txt
++++ patroni/requirements.txt
+@@ -12,5 +12,5 @@ python-dateutil
+ pysyncobj>=0.3.8
+ cryptography>=1.4
+ psutil>=2.0.0
+-ydiff>=1.2.0
++cdiff
+ setuptools

debian/patches/requirements_setuptools.patch

@@ -1,7 +1,9 @@
---- ./requirements.txt.orig	2020-01-23 15:23:54.943628527 +0100
-+++ ./requirements.txt	2020-01-23 15:24:12.763690401 +0100
-@@ -12,3 +12,4 @@
+Index: patroni/requirements.txt
+===================================================================
+--- patroni.orig/requirements.txt
++++ patroni/requirements.txt
+@@ -13,3 +13,4 @@ pysyncobj>=0.3.8
+ cryptography>=1.4
  psutil>=2.0.0
- cdiff
- kubernetes>=2.0.0,<=10.0.1,!=4.0.*,!=5.0.*
+ ydiff>=1.2.0
 +setuptools

debian/patches/series

@@ -2,3 +2,6 @@ startup_scripts.patch
 acceptance_tests_system_patroni.patch
 requirements_setuptools.patch
 offline_intersphinx.patch
+requirements_cdiff.patch
+avoid_overwriting_configuration_during_boostrap.patch
+regression_tests_skip_citus_test.patch

debian/patches/startup_scripts.patch

@@ -1,7 +1,5 @@
-Index: patroni/extras/startup-scripts/patroni
-===================================================================
---- patroni.orig/extras/startup-scripts/patroni
-+++ patroni/extras/startup-scripts/patroni
+--- a/extras/startup-scripts/patroni
++++ b/extras/startup-scripts/patroni
 @@ -18,7 +18,7 @@ USER="postgres"
  GROUP="postgres"
  
@@ -33,10 +31,8 @@ Index: patroni/extras/startup-scripts/patroni
      exit 1
      ;;
  esac
-Index: patroni/extras/startup-scripts/patroni.service
-===================================================================
---- patroni.orig/extras/startup-scripts/patroni.service
-+++ patroni/extras/startup-scripts/patroni.service
+--- a/extras/startup-scripts/patroni.service
++++ b/extras/startup-scripts/patroni.service
 @@ -1,9 +1,7 @@
 -# This is an example systemd config file for Patroni
 -# You can copy it to "/etc/systemd/system/patroni.service",
@@ -49,9 +45,9 @@ Index: patroni/extras/startup-scripts/patroni.service
  
  [Service]
  Type=simple
-@@ -16,17 +14,13 @@ EnvironmentFile=-/etc/patroni_env.conf
- 
- WorkingDirectory=~
+@@ -17,17 +15,13 @@ EnvironmentFile=-/etc/patroni_env.conf
+ # The default is the user's home directory, and if you want to change it, you must provide an absolute path.
+ # WorkingDirectory=/home/sameuser
  
 -# Where to send early-startup messages from the server
 -# This is normally controlled by the global default set by systemd

debian/patroni@.service

@@ -17,7 +17,6 @@ Group=postgres
 EnvironmentFile=-/etc/patroni/env.conf
 
 PermissionsStartOnly=True
-WorkingDirectory=~
 RuntimeDirectory=postgresql/%i.pg_stat_tmp
 RuntimeDirectoryMode=2775
 SyslogIdentifier=patroni@%i

debian/pg_clonecluster_patroni

@@ -37,8 +37,9 @@ fi
 
 if [ -f /etc/postgresql/$VERSION/$CLUSTER/postgresql.conf ]
 then
+    rm -f etc/postgresql/$VERSION/$CLUSTER/postgresql.base.conf
     pg_dropcluster $VERSION $CLUSTER
 fi
 
-pg_createcluster --start-conf=manual --datadir=$DATADIR $VERSION $CLUSTER && rm -rf $DATADIR && pg_basebackup --pgdata $DATADIR -X stream --dbname="$CONNSTR"
+pg_createcluster --start-conf=manual --datadir=$DATADIR $VERSION $CLUSTER > /dev/null && rm -rf $DATADIR && /usr/lib/postgresql/$VERSION/bin/pg_basebackup --pgdata $DATADIR -X stream --dbname="$CONNSTR"
 exit $?

debian/pg_createconfig_patroni

@@ -24,6 +24,7 @@ case $i in
     shift # past argument=value
     ;;
     --endpoint=*)
+    # one or more endpoints joined by commas are allowed
     DCS_ENDPOINT="${i#*=}"
     shift # past argument=value
     ;;
@@ -90,6 +91,14 @@ fi
 # Postgres port.  2576 is 8008 - 5432.
 API_PORT=$((2576+$PORT))
 
+# set base data directory
+if [ -f /etc/postgresql-common/createcluster.conf ]; then
+    DATADIR_BASE=$(grep ^data_directory /etc/postgresql-common/createcluster.conf | sed -e 's/^[^\/]*\//\//g' -e 's/\%.*//' -e 's/\/$//')
+fi
+if [ -z "${DATADIR_BASE}" ]; then
+    DATADIR_BASE=/var/lib/postgresql
+fi
+
 # check DCS configuration
 if [ ! -f /etc/patroni/dcs.yml ]; then
     echo "DCS not configured yet, edit /etc/patroni/dcs.yml"
@@ -138,18 +147,20 @@ if [ -n "$VIP_IP" ]; then
 	exit 1
     fi
     if [ -z "$DCS_ENDPOINT" ]; then
-        DCS_ENDPOINT="$(egrep -v '^[[:space:]]*$|^ *#' /etc/patroni/dcs.yml | egrep '(host|-)' | egrep '[[:digit:]]+\.[[:digit:]]+\.[[:digit:]]+\.[[:digit:]]+' | sed -r -e s/.*host:// -e s/-// -e 's/ //g' -e 's/^([0-9])/http:\/\/\1/')"
+        DCS_ENDPOINT="$(egrep -v '^[[:space:]]*$|^ *#' /etc/patroni/dcs.yml | egrep '(host|-)' | egrep -v '^[[:space:]]*$' | egrep '[[:digit:]]+\.[[:digit:]]+\.[[:digit:]]+\.[[:digit:]]+' | sed -r -e s/.*host:// -e s/-// -e 's/ //g' -e 's/^([0-9])/http:\/\/\1/')"
         if [ -z "$DCS_ENDPOINT" ]; then
             echo "DCS endpoint URL could not be determined from /etc/patroni/dcs.yml and --endpoint not provided, cannot write VIP file"
 	    exit 1
         fi
         if [ $(echo "$DCS_ENDPOINT" | wc -l) != 1 ]; then
-            echo "DCS endpoint URL could not be determined from /etc/patroni/dcs.yml and --endpoint not provided, cannot write VIP file"
-            exit 1
+	    # join multiple lines with commas
+	    # sed -> remove trailing comma
+	    DCS_ENDPOINT=$( echo "$DCS_ENDPOINT" | tr '\n' ',' | sed 's/,$//' )
         fi
     fi
-    if ! $(echo "$DCS_ENDPOINT" | egrep -q '^http.*://[[:digit:]]+\.[[:digit:]]+\.[[:digit:]]+\.[[:digit:]]+:[[:digit:]]+$'); then
-        echo "DCS endpoint URL not in 'http://1.2.3.4:1234' format, cannot write VIP file"
+    if ! $(echo "$DCS_ENDPOINT" | egrep -q '^(http.*://[[:digit:]]+\.[[:digit:]]+\.[[:digit:]]+\.[[:digit:]]+:[[:digit:]]+){1}(,http.*://[[:digit:]]+\.[[:digit:]]+\.[[:digit:]]+\.[[:digit:]]+:[[:digit:]]+)*$'); then
+        echo "DCS_ENDPOINT: $DCS_ENDPOINT"
+        echo "DCS endpoint URL not in 'http://1.2.3.4:1234[,http://2.3.4.5:1234]*' format, cannot write VIP file"
         exit 1
     fi
     LISTEN_VIP=",$VIP_IP"
@@ -182,7 +193,11 @@ fi
 
 if [ -z "$NETWORK" ]; then
     if [ -x /bin/ip ]; then
-        NETWORK=$(/bin/ip -4 route get 8.8.8.8 | grep ^8.8.8.8 | sed -e s/.*src.// -e s/\ .*//g -e s/\.[0-9]*$/.0/)/24
+	ROUTE=$(/bin/ip -4 route get 8.8.8.8 | grep ^8.8.8.8)
+	DEV=$(echo "$ROUTE" | grep -P -o "(?<=dev )[^ ]+")
+	ADDRESS=$(echo "$ROUTE" | grep -P -o "(?<=src )[^ ]+")
+	PREFIX=$(ip -4 address show $DEV | grep -P -o "(?<=inet $ADDRESS/)[0-9]+")
+        NETWORK="$ADDRESS/$PREFIX"
     else
         echo "iproute2 package missing, cannot determine network and --network is not set"
         rm -f $CONFIG_FILE
@@ -197,6 +212,7 @@ cat /etc/patroni/config.yml.in |		\
         -e "s/@HOSTNAME@/${HOSTNAME}/g"		\
         -e "s/@HOSTIP@/${HOSTIP}/g"		\
         -e "s/@LISTEN_VIP@/${LISTEN_VIP}/g"	\
+        -e "s#@DATADIR_BASE@#${DATADIR_BASE}#g"	\
         -e "s#@NETWORK@#${NETWORK}#g"		\
         -e "s/@API_PORT@/${API_PORT}/g"		\
         -e "s/@PORT@/${PORT}/g"			\

debian/py3dist-overrides

@@ -0,0 +1 @@
+ipaddress python3:any (>= 3.3)

debian/tests/acceptance

@@ -1,11 +1,12 @@
 #!/bin/bash
 
-set -e
+set -eu
 
 export LC_ALL=C.UTF-8
 export LANG=C.UTF-8
 export TZ=UTC # suppress UnknownTimeZoneError('Etc/UTC',)
 DCS=$1
+shift
 
 if [ $(id -u) -eq 0 ]
 then
@@ -16,9 +17,7 @@ then
 
         # zookeeper must be started manually (as root)
         if [ "$DCS" = "zookeeper" ]; then
-                # JAVA_OPTS need to be set, otherwise zookeeper is not reachable via IPV4
-                sed -i -e 's/#JAVA_OPTS=""/JAVA_OPTS="-Djava.net.preferIPv4Stack=true"/' /etc/default/zookeeper
-                /etc/init.d/zookeeper start
+		JAVA_OPTS="-Djava.net.preferIPv4Stack=true" /usr/bin/java -cp /etc/zookeeper/conf:/usr/share/java/jline.jar:/usr/share/java/log4j-1.2.jar:/usr/share/java/xercesImpl.jar:/usr/share/java/xmlParserAPIs.jar:/usr/share/java/netty.jar:/usr/share/java/slf4j-api.jar:/usr/share/java/slf4j-log4j12.jar:/usr/share/java/zookeeper.jar -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.local.only=true -Dzookeeper.log.dir=/var/log/zookeeper -Dzookeeper.root.logger=INFO,ROLLINGFILE org.apache.zookeeper.server.quorum.QuorumPeerMain /etc/zookeeper/conf/zoo.cfg &
 		# let Java have some more time to startup
 		sleep 5
         fi
@@ -27,7 +26,7 @@ else
 fi
 
 # clean up afterwards
-trap 'rm -f /tmp/pgpass?; if [ $(id -u) -eq 0 ] && [ -x /etc/init.d/zookeeper ]; then /etc/init.d/zookeeper stop; fi' 0 2 3 15
+trap 'rm -f /tmp/pgpass?; if [ $(id -u) -eq 0 ] && [ -x /etc/init.d/zookeeper ]; then /etc/init.d/zookeeper stop; fi' EXIT
 
 # set ETCD_ARCH for ETCD_UNSUPPORTED_ARCH
 # see https://github.com/etcd-io/etcd/blob/master/Documentation/op-guide/supported-platform.md#current-support
@@ -37,10 +36,7 @@ case $DEB_HOST_ARCH in
 	arm64)
 		ETCD_ARCH=arm64
 		;;
-	armel)
-		ETCD_ARCH=arm
-		;;
-	armhf)
+	armel|armhf)
 		ETCD_ARCH=arm
 		;;
 	i386)
@@ -48,9 +44,33 @@ case $DEB_HOST_ARCH in
 		;;
 esac
 
+# ensure no etcd server is running.
+if [ $(id -u) -eq 0 ]
+then
+	if [ "$DCS" = "etcd" -o "$DCS" = "etcd3" ]
+	then
+		service etcd stop
+		service etcd status || true
+	fi
+fi
+
 set -x
-for PG_VERSION in /usr/lib/postgresql/*; do
-	$SU "ETCD_UNSUPPORTED_ARCH=$ETCD_ARCH DCS=$DCS PATH=/usr/lib/postgresql/$(basename \
-		${PG_VERSION})/bin:$PATH behave" || \
-		(for file in $(ls features/output/*_failed/*); do echo "$file:"; cat $file; done && exit 1)
+for PG_VERSION in $(ls -1r /usr/lib/postgresql/); do
+	echo "### PostgreSQL $PG_VERSION acceptance-$DCS $@ ###"
+	if [ "${PG_VERSION}" == "10" ]; then
+		SKIP=--tags=-skip
+	else
+		SKIP=
+	fi
+	if ! $SU "set -o pipefail; ETCD_UNSUPPORTED_ARCH=$ETCD_ARCH DCS=$DCS \
+		PATH=/usr/lib/postgresql/${PG_VERSION}/bin:$PATH \
+		behave ${SKIP} $@ | ts"; then
+		for file in features/output/*_failed/*; do
+			case $file in *.journal*) continue ;; esac # skip RAFT journal files
+			echo "$file:"
+			cat $file
+		done
+		exit 1
+	fi
+	echo "### End $PG_VERSION acceptance-$DCS $@ ###"
 done

debian/tests/control

@@ -1,45 +1,90 @@
-# consul
-Depends:
- consul,
- postgresql,
- python3-behave,
- python3-consul (>= 0.7.0),
- python3-coverage,
- @,
-Test-Command: debian/tests/acceptance consul
-Restrictions: allow-stderr
-
-# etcd
+# etc3
+Features: test-name=acceptance-etcd3
 Depends:
  etcd-server,
+ moreutils,
+ postgresql,
+ python3-behave,
+ python3-coverage,
+ python3-etcd,
+ @,
+Test-Command: debian/tests/acceptance etcd3
+Restrictions: allow-stderr, flaky
+
+# etcd
+Features: test-name=acceptance-etcd-basic
+Depends:
+ etcd-server,
+ moreutils,
+ postgresql,
+ python3-behave,
+ python3-coverage,
+ python3-etcd (>= 0.4.3),
+ @,
+Test-Command: debian/tests/acceptance etcd features/basic_replication.feature
+Restrictions: needs-root, allow-stderr
+
+Features: test-name=acceptance-etcd
+Depends:
+ etcd-server,
+ moreutils,
  postgresql,
  python3-behave,
  python3-coverage,
  python3-etcd (>= 0.4.3),
  @,
 Test-Command: debian/tests/acceptance etcd
-Restrictions: allow-stderr
+Restrictions: needs-root, allow-stderr, flaky
 
-# zookeper
+# consul
+Features: test-name=acceptance-consul
+Depends:
+ consul,
+ moreutils,
+ postgresql,
+ python3-behave,
+ python3-consul (>= 0.7.0),
+ python3-coverage,
+ @,
+Test-Command: debian/tests/acceptance consul
+Restrictions: allow-stderr, flaky, skip-not-installable
+
+# zookeeper
+Features: test-name=acceptance-zookeeper
 Depends:
  zookeeperd,
+ moreutils,
  postgresql,
  python3-behave,
  python3-coverage,
  python3-kazoo,
+ procps,
  @,
-Test-Command: debian/tests/acceptance zookeeper
-Restrictions: needs-root, allow-stderr
+Test-Command: debian/tests/acceptance zookeeper "-e dcs_failsafe_mode"
+Restrictions: needs-root, allow-stderr, flaky, skip-not-installable
+
+# raft
+Features: test-name=acceptance-raft
+Depends:
+ moreutils,
+ postgresql,
+ python3-behave,
+ python3-coverage,
+ python3-pysyncobj,
+ @,
+Test-Command: debian/tests/acceptance raft
+Restrictions: allow-stderr, flaky
 
 # regression tests
 Depends:
- python3-boto,
+ python3-boto3,
  python3-consul (>= 0.7.0),
  python3-etcd (>= 0.4.3),
  python3-flake8,
  python3-kazoo,
  python3-kubernetes,
  python3-mock,
+ python3-pysyncobj,
  python3-pytest-cov,
  @,
 Tests: test