check-patroni/tests/__init__.py

66 lines
2.3 KiB
Python
Raw Normal View History

Use fake HTTP server for the Patroni API in tests We introduce a patroni_api fixture, defined in tests/conftest.py, which sets up an HTTP server serving files in a temporary directory. The server is itself defined by the PatroniAPI class; it has a 'routes()' context manager method to be used in actual tests to setup expected responses based on specified JSON files. We set up some logging in order to improve debugging. The direct advantage of this is that PatroniResource.rest_api() method is now covered by the test suite. Coverage before this commit: Name Stmts Miss Cover ----------------------------------------------- check_patroni/__init__.py 3 0 100% check_patroni/cli.py 193 18 91% check_patroni/cluster.py 113 0 100% check_patroni/convert.py 23 5 78% check_patroni/node.py 146 1 99% check_patroni/types.py 50 23 54% ----------------------------------------------- TOTAL 528 47 91% and after this commit: Name Stmts Miss Cover ----------------------------------------------- check_patroni/__init__.py 3 0 100% check_patroni/cli.py 193 18 91% check_patroni/cluster.py 113 0 100% check_patroni/convert.py 23 5 78% check_patroni/node.py 146 1 99% check_patroni/types.py 50 9 82% ----------------------------------------------- TOTAL 528 33 94% In actual test functions, we either invoke patroni_api.routes() to configure which JSON file(s) should be served for each endpoint, or we define dedicated fixtures (e.g. cluster_config_has_changed()) to configure this for several test functions or the whole module. The 'old_replica_state' parametrized fixture is used when needed to adjust such fixtures, e.g. in cluster_has_replica_ok(), to modify the JSON content using cluster_api_set_replica_running() (previously in tests/tools.py, now in tests/__init__.py). The dependency on pytest-mock is no longer needed.
2023-09-28 09:41:33 +02:00
import json
import logging
import shutil
from contextlib import contextmanager
from functools import partial
from http.server import HTTPServer, SimpleHTTPRequestHandler
from pathlib import Path
from typing import Any, Iterator, Mapping, Union
logger = logging.getLogger(__name__)
class PatroniAPI(HTTPServer):
def __init__(self, directory: Path, *, datadir: Path) -> None:
self.directory = directory
self.datadir = datadir
handler_cls = partial(SimpleHTTPRequestHandler, directory=str(directory))
super().__init__(("", 0), handler_cls)
def serve_forever(self, *args: Any) -> None:
logger.info(
"starting fake Patroni API at %s (directory=%s)",
self.endpoint,
self.directory,
)
return super().serve_forever(*args)
@property
def endpoint(self) -> str:
return f"http://{self.server_name}:{self.server_port}"
@contextmanager
def routes(self, mapping: Mapping[str, Union[Path, str]]) -> Iterator[None]:
"""Temporarily install specified files in served directory, thus
building "routes" from given mapping.
The 'mapping' defines target route paths as keys and files to be
installed in served directory as values. Mapping values of type 'str'
are assumed be relative file path to the 'datadir'.
"""
for route_path, fpath in mapping.items():
if isinstance(fpath, str):
fpath = self.datadir / fpath
shutil.copy(fpath, self.directory / route_path)
try:
yield None
finally:
for fname in mapping:
(self.directory / fname).unlink()
def cluster_api_set_replica_running(in_json: Path, target_dir: Path) -> Path:
# starting from 3.0.4 the state of replicas is streaming or in archive recovery
# instead of running
Use fake HTTP server for the Patroni API in tests We introduce a patroni_api fixture, defined in tests/conftest.py, which sets up an HTTP server serving files in a temporary directory. The server is itself defined by the PatroniAPI class; it has a 'routes()' context manager method to be used in actual tests to setup expected responses based on specified JSON files. We set up some logging in order to improve debugging. The direct advantage of this is that PatroniResource.rest_api() method is now covered by the test suite. Coverage before this commit: Name Stmts Miss Cover ----------------------------------------------- check_patroni/__init__.py 3 0 100% check_patroni/cli.py 193 18 91% check_patroni/cluster.py 113 0 100% check_patroni/convert.py 23 5 78% check_patroni/node.py 146 1 99% check_patroni/types.py 50 23 54% ----------------------------------------------- TOTAL 528 47 91% and after this commit: Name Stmts Miss Cover ----------------------------------------------- check_patroni/__init__.py 3 0 100% check_patroni/cli.py 193 18 91% check_patroni/cluster.py 113 0 100% check_patroni/convert.py 23 5 78% check_patroni/node.py 146 1 99% check_patroni/types.py 50 9 82% ----------------------------------------------- TOTAL 528 33 94% In actual test functions, we either invoke patroni_api.routes() to configure which JSON file(s) should be served for each endpoint, or we define dedicated fixtures (e.g. cluster_config_has_changed()) to configure this for several test functions or the whole module. The 'old_replica_state' parametrized fixture is used when needed to adjust such fixtures, e.g. in cluster_has_replica_ok(), to modify the JSON content using cluster_api_set_replica_running() (previously in tests/tools.py, now in tests/__init__.py). The dependency on pytest-mock is no longer needed.
2023-09-28 09:41:33 +02:00
with in_json.open() as f:
js = json.load(f)
for node in js["members"]:
if node["role"] in ["replica", "sync_standby", "standby_leader"]:
if node["state"] in ["streaming", "in archive recovery"]:
Use fake HTTP server for the Patroni API in tests We introduce a patroni_api fixture, defined in tests/conftest.py, which sets up an HTTP server serving files in a temporary directory. The server is itself defined by the PatroniAPI class; it has a 'routes()' context manager method to be used in actual tests to setup expected responses based on specified JSON files. We set up some logging in order to improve debugging. The direct advantage of this is that PatroniResource.rest_api() method is now covered by the test suite. Coverage before this commit: Name Stmts Miss Cover ----------------------------------------------- check_patroni/__init__.py 3 0 100% check_patroni/cli.py 193 18 91% check_patroni/cluster.py 113 0 100% check_patroni/convert.py 23 5 78% check_patroni/node.py 146 1 99% check_patroni/types.py 50 23 54% ----------------------------------------------- TOTAL 528 47 91% and after this commit: Name Stmts Miss Cover ----------------------------------------------- check_patroni/__init__.py 3 0 100% check_patroni/cli.py 193 18 91% check_patroni/cluster.py 113 0 100% check_patroni/convert.py 23 5 78% check_patroni/node.py 146 1 99% check_patroni/types.py 50 9 82% ----------------------------------------------- TOTAL 528 33 94% In actual test functions, we either invoke patroni_api.routes() to configure which JSON file(s) should be served for each endpoint, or we define dedicated fixtures (e.g. cluster_config_has_changed()) to configure this for several test functions or the whole module. The 'old_replica_state' parametrized fixture is used when needed to adjust such fixtures, e.g. in cluster_has_replica_ok(), to modify the JSON content using cluster_api_set_replica_running() (previously in tests/tools.py, now in tests/__init__.py). The dependency on pytest-mock is no longer needed.
2023-09-28 09:41:33 +02:00
node["state"] = "running"
assert target_dir.is_dir()
out_json = target_dir / in_json.name
with out_json.open("w") as f:
json.dump(js, f)
return out_json