Commit graph

27 commits

Author SHA1 Message Date
benoit 78ef0f6ada Fix cluster_node_count's management of replication states
The service now supports the `streaming` state.

Since we dont check for lag or timeline in this service, a healthy node
is :

* leader : in a running state
* standby_leader : running (pre Patroni 3.0.4), streaming otherwise
* standby & sync_standby : running (pre Patroni 3.0.4), streaming otherwise

Updated the tests for this service.
2024-01-09 06:50:00 +01:00
benoit 46db3e2d15 Fix the cluster_has_leader service for standby clusters
Before this patch we checked the expected standby leader state
was `running` for all versions of Patroni.

With this patch, for:
* Patroni < 3.0.4, standby leaders are in `running` state.
* Patroni >= 3.0.4, standby leaders can be in `streaming` or `in
archive recovey` state. We will raise a warning for the latter.

The tests where modified to account for this.

Co-authored-by: Denis Laxalde <denis@laxalde.org>
2023-12-18 13:17:37 +01:00
benoit 8d6b8502b6 cluster_has_replica: fix the way a healthy replica is detected
For patroni >= version 3.0.4:
* the role is `replica` or `sync_standby`
* the state is `streaming` or `in archive recovery`
* the timeline is the same as the leader
* the lag is lower or equal to `max_lag`

For prio versions of patroni:
* the role is `replica` or `sync_standby`
* the state is `running`
* the timeline is the same as the leader
* the lag is lower or equal to `max_lag`

Additionnally, we now display the timeline in the perfstats. We also try
to display the perf stats of unhealthy replica as much as possible.

Update tests for cluster_has_replica:
* Fix the tests to make them work with the new algotithm
* Add a specific test for tl divergences
2023-11-11 10:50:35 +01:00
Denis Laxalde 95f21a133d Drop superfluous type annotation of 'self'
See https://mypy.readthedocs.io/en/stable/cheat_sheet_py3.html#classes
> For instance methods, omit type for "self"
2023-10-03 09:39:40 +02:00
benoit ee3837fab1 Add info and options about sync standby to cluster_has_replica
* Add `--sync-warning` and `--sync-critical`
* Add `sync_replica` to track the number of sync replica in the perf data
* Add `MEMBER-sync` to track if a member is a sync replica in the perf data
2023-08-24 17:34:08 +02:00
benoit 8883d6bdc4 Add standby-leader as a valid leader type for cluster_has_leader 2023-08-23 18:22:49 +02:00
benoit 99bf1c5bb5 Add new service cluster_has_scheduled_action 2023-08-23 12:08:19 +02:00
benoit d99faeba15 Add tests for the output of the script and support pre/post 3.0.4
* Change all replica status from `running` to `streaming`
* Add an option to pytest to change the state back to `running`
* Also tests the output of the script
* Add a quick test script for live clusters
2023-08-23 10:53:09 +02:00
benoit bb68d937e6 Fix a mistake in the vvv output of ClusterConfigHasChanged 2023-08-22 11:15:44 +02:00
benoit a01a535680 Add sync_standby as an acceptable state for a replica 2023-08-21 13:11:08 +02:00
benoit 021b572e53 Redefining cluster_node_count using Patroni 3.0.4's new status indicators
Previously, replica nodes were labeled with a `running` state. As a
result, our checks were based on nodes marked as `running` through
the `--running-[warning|critical]` options.

However, with the recent changes in Patroni 3.0.4, replica nodes now
carry a `streaming` state. This shift in terminology calls for an
adjustment in our approach. A new state, `healthy_member`, has been
introduced to encompass both `running` and `streaming` nodes.

Key Modifications:

* The existing `--running-[warning|critical]` option is now designated
  as `--healthy-[warning|critical]`.
* Introduction of the `healthy_member` perfdata, which serves as the
  reference point for the aforementioned options.
* Updates to documentation, help messages, and tests.
2023-08-21 11:59:55 +02:00
benoit 2f250e846e State change in patroni 3.0.4
Since patroni 3.0.4, standby node nominal state is "streaming" instead
of "running". Some services need to be changed to account for that.

Reported in issue #28
2023-08-21 11:59:55 +02:00
benoit df744bf7dc Use isort to automatically sort imports 2023-03-20 14:56:11 +01:00
benoit dff95eae2f Fixing logging issues in the previous modifications 2023-03-20 12:25:32 +01:00
benoit 2443505ad6 Change logging
We now use our own logger.
When debug is set (-vvv), we also display urllib3's debug info.
2023-03-20 12:25:32 +01:00
benoit 9cd80f5af8 Move from urllib3 to requests 2023-03-20 12:25:32 +01:00
benoit 93b196fb77 Fix a format string for cluster_config_has_changed 2023-03-02 17:32:18 +01:00
benoit 908669f073 Add a --save option when state files are used
The checks `cluster_config_has_changed` and `node_tl_has_changed` use a
state file to store the previous value of the config hash and the
timeline.

Previously the check would fail if something changed, but the new value
would be saved directly. This behavious has changed. The new value
is saved only if `--save` is passed to the check.

The mimics the way [check_pgactivity] manages this kind of checks.

[check_pgactivity]: https://github.com/OPMDG/check_pgactivity
2023-03-02 17:32:18 +01:00
benoit 64d87d505e Black run 2022-07-11 15:16:19 +02:00
benoit aa1de928d3 An attempt at correctly ordering imports 2022-02-07 15:11:05 +01:00
benoit 9ed9b6466d Read state file in cli to pass the info to the checks and summaries 2022-02-07 15:01:50 +01:00
benoit 4de20fefdc Node and Cluster services reviews 2022-02-07 14:18:14 +01:00
benoit d4e974da51 --max-lag additions and fixups
* add --max-lag to cluster_has_replica
* change --lag to --max-lag in node_is_replica
* update tests
* update README.md
2021-08-13 11:00:43 +02:00
benoit 6fbe96cc63 Update .gitignore + black run 2021-08-12 13:15:37 +02:00
benoit dd8130a459 New service cluster_is_in_maintenance 2021-08-12 13:15:37 +02:00
benoit db3f008b81 More Typing 2021-08-12 13:07:28 +02:00
benoit 1e6adc6a1a First working version 2021-08-12 13:07:10 +02:00