Denis Laxalde 903b83e211 Use fake HTTP server for the Patroni API in tests

We introduce a patroni_api fixture, defined in tests/conftest.py, which
sets up an HTTP server serving files in a temporary directory. The
server is itself defined by the PatroniAPI class; it has a 'routes()'
context manager method to be used in actual tests to setup expected
responses based on specified JSON files.

We set up some logging in order to improve debugging.

The direct advantage of this is that PatroniResource.rest_api() method
is now covered by the test suite.

Coverage before this commit:

  Name                        Stmts   Miss  Cover
  -----------------------------------------------
  check_patroni/__init__.py       3      0   100%
  check_patroni/cli.py          193     18    91%
  check_patroni/cluster.py      113      0   100%
  check_patroni/convert.py       23      5    78%
  check_patroni/node.py         146      1    99%
  check_patroni/types.py         50     23    54%
  -----------------------------------------------
  TOTAL                         528     47    91%

and after this commit:

  Name                        Stmts   Miss  Cover
  -----------------------------------------------
  check_patroni/__init__.py       3      0   100%
  check_patroni/cli.py          193     18    91%
  check_patroni/cluster.py      113      0   100%
  check_patroni/convert.py       23      5    78%
  check_patroni/node.py         146      1    99%
  check_patroni/types.py         50      9    82%
  -----------------------------------------------
  TOTAL                         528     33    94%

In actual test functions, we either invoke patroni_api.routes() to
configure which JSON file(s) should be served for each endpoint, or we
define dedicated fixtures (e.g. cluster_config_has_changed()) to
configure this for several test functions or the whole module.

The 'old_replica_state' parametrized fixture is used when needed to
adjust such fixtures, e.g. in cluster_has_replica_ok(), to modify the
JSON content using cluster_api_set_replica_running() (previously in
tests/tools.py, now in tests/__init__.py).

The dependency on pytest-mock is no longer needed.

2023-10-06 10:40:29 +02:00

2.7 KiB

Raw Permalink Blame History

Contributing to check_patroni

Thanks for your interest in contributing to check_patroni.

Clone Git Repository

Installation from the git repository:

$ git clone https://github.com/dalibo/check_patroni.git
$ cd check_patroni

Change the branch if necessary.

Create Python Virtual Environment

You need a dedicated environment, install dependencies and then check_patroni from the repo:

$ python3 -m venv .venv
$ . .venv/bin/activate
(.venv) $ pip3 install .[test]
(.venv) $ pip3 install -r requirements-dev.txt
(.venv) $ check_patroni

To quit this env and destroy it:

$ deactivate
$ rm -r .venv

Development Environment

A vagrant file is available to create a icinga / opm / grafana stack and install check_patroni. You can then add a server to the supervision and watch the graphs in grafana. It's in the vagrant directory.

A vagrant file can be found in this repository to generate a patroni/etcd setup.

The README.md can be generated with ./docs/make_readme.sh.

Executing Tests

Crafting repeatable tests using a live Patroni cluster can be intricate. To simplify the development process, a fake HTTP server is set up as a test fixture and serves static files (either from tests/json directory or from in-memory data).

An important consideration is that there is a potential drawback: if the JSON data is incorrect or if modifications have been made to Patroni without corresponding updates to the tests documented here, the tests might still pass erroneously.

The tests are executed automatically for each PR using the ci (see .github/workflow/lint.yml and .github/workflow/tests.yml).

Running the tests,

manually:
```
pytest --cov tests
```

or using tox:

tox -e lint    # mypy + flake8 + black + isort ° codespell
tox            # pytests and "lint" tests for all supported version of python
tox -e py      # pytests and "lint" tests for the default version of python

Please note that when dealing with any service that checks the state of a node, the related tests must use the old_replica_state fixture to test with both old (pre 3.0.4) and new replica states.

A bash script, check_patroni.sh, is provided to facilitate testing all services on a Patroni endpoint (./vagrant/check_patroni.sh). It requires one parameter: the endpoint URL that will be used as the argument for the -e/--endpoints option of check_patroni. This script essentially compiles a list of service calls and executes them sequentially in a bash script. It creates a state file in the directory from which you run the script.

Here's an example usage:

./vagrant/check_patroni.sh http://10.20.30.51:8008