From b2c626039dba517f8c3fabea410cf713966796c4 Mon Sep 17 00:00:00 2001 From: Ludovic Poujol Date: Wed, 23 Feb 2022 11:44:47 +0100 Subject: [PATCH] Suite MaJ HowtoElasticsearch.md --- HowtoElasticsearch.md | 293 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 286 insertions(+), 7 deletions(-) diff --git a/HowtoElasticsearch.md b/HowtoElasticsearch.md index 381a2237..455cc504 100644 --- a/HowtoElasticsearch.md +++ b/HowtoElasticsearch.md @@ -218,6 +218,16 @@ On check sur la page `/_cat/health` si le status n'est pas en **red**. /usr/lib/nagios/plugins/check_http -I 127.0.0.1 -u /_cat/health?h=st -p 9200 -r 'red' --invert-regex ~~~ +**Attention** : Il faut compléter cette commande avec les options suivantes si les options de sécurisation d'elasticsearch son activées ! (ie: Authentification HTTP & Chiffrement avec TLS) : +* `--authorization 'remote_monitoring_user:xxxx'` - Pour l'authentification HTTP +* `-S` - Utilisation de TLS + +On obtient ainsi : + +~~~ +/usr/lib/nagios/plugins/check_http -I 127.0.0.1 -S -u /_cat/health?h=st -p 9200 -r 'red' --invert-regex --authorization 'remote_monitoring_user:xxxx' +~~~ + ### Munin Des plugins pour Munin sont disponible sur ce dépot Github : https://github.com/y-ken/munin-plugin-elasticsearch/ @@ -270,7 +280,7 @@ Il faut d'abord activer les fonctions de sécurité dans `/etc/elasticsearch/ela xpack.security.enabled: true ~~~ -Après un redémarrage d'elasticsearch pour la prise en compte de l'activation de xpack, on utilisera la commande `/usr/share/elasticsearch/bin/elasticsearch-setup-passwords auto` pour créer un mot de passe pour tous les comptes par défaut. +Après un redémarrage d'Elasticsearch pour la prise en compte de l'activation de xpack, on utilisera la commande `/usr/share/elasticsearch/bin/elasticsearch-setup-passwords auto` pour créer un mot de passe pour tous les comptes par défaut. ~~~ # /usr/share/elasticsearch/bin/elasticsearch-setup-passwords auto @@ -316,7 +326,7 @@ On pourra ainsi créer des comptes applicatifs, avec #### Ajustement configuration monitoring -On ajoutera l'argument suivant à la commande de check `--authorization 'remote_monitoring_user:PASSWORD'` +On ajoutera l'argument suivant à la commande de check_http pour s'authentifier `--authorization 'remote_monitoring_user:PASSWORD'` #### Ajustement configuration Kibana @@ -383,12 +393,12 @@ Comme ce n'est pas un dossier conventionnel, on peut choisir de le déplacer dan #### Communication inter-nœuds -Pour chaque nœud elastic, il faut générer un certificat avec l'outil `/usr/share/elasticsearch/bin/elasticsearch-certutil` +Pour chaque nœud Elasticsearch, il faut générer un certificat avec l'outil `/usr/share/elasticsearch/bin/elasticsearch-certutil` On peut essayer, pour garder les choses au clair, d'avoir la nomenclature suivante pour les noms de certificats `$(hostname).$(service).p12` ~~~ -# /usr/share/elasticsearch/bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12 --days 7300 +# /usr/share/elasticsearch/bin/elasticsearch-certutil cert --ca /etc/elasticsearch/elastic-stack-ca.p12 --days 7300 This tool assists you in the generation of X.509 certificates and certificate signing requests for use with SSL/TLS in the Elastic stack. @@ -458,7 +468,7 @@ Par la suite on va `mv` le certificat généré vers `/etc/elasticsearch` (ou le # chmod g+r /etc/elasticsearch/HOSTNAME.elasticsearch.p12 ~~~ -Puis on ajout la configuration de elasticsearch +Puis on ajout la configuration de Elasticsearch : ~~~yaml @@ -468,12 +478,281 @@ xpack.security.transport.ssl.keystore.path: HOSTNAME.elasticsearch.p12 xpack.security.transport.ssl.truststore.path: HOSTNAME.elasticsearch.p12 ~~~ -Après, un redémarrage avec `systemctl restart elasticsearch` +Après, redémarrage avec `systemctl restart elasticsearch` #### Communication client-serveur -**TODO** +Pour chaque nœud elastic, il faut générer un certificat avec l'outil `/usr/share/elasticsearch/bin/elasticsearch-certutil` + +On peut essayer, pour garder les choses au clair, d'avoir la nomenclature suivante pour les noms de certificats `$(hostname).http.p12` +Dans le cas d'un déploiement de nombreux noeuds, on peut avoir recours a un certificat *wildcard* qu'on déploiera partout. + +Ici exemple pour créer un seul certificat pour notre instance : + + +~~~ +# /usr/share/elasticsearch/bin/elasticsearch-certutil http + +## Elasticsearch HTTP Certificate Utility + +The 'http' command guides you through the process of generating certificates +for use on the HTTP (Rest) interface for Elasticsearch. + +This tool will ask you a number of questions in order to generate the right +set of files for your needs. + +## Do you wish to generate a Certificate Signing Request (CSR)? + +A CSR is used when you want your certificate to be created by an existing +Certificate Authority (CA) that you do not control (that is, you don't have +access to the keys for that CA). + +If you are in a corporate environment with a central security team, then you +may have an existing Corporate CA that can generate your certificate for you. +Infrastructure within your organisation may already be configured to trust this +CA, so it may be easier for clients to connect to Elasticsearch if you use a +CSR and send that request to the team that controls your CA. + +If you choose not to generate a CSR, this tool will generate a new certificate +for you. That certificate will be signed by a CA under your control. This is a +quick and easy way to secure your cluster with TLS, but you will need to +configure all your clients to trust that custom CA. + +Generate a CSR? [y/N]n + +## Do you have an existing Certificate Authority (CA) key-pair that you wish to use to sign your certificate? + +If you have an existing CA certificate and key, then you can use that CA to +sign your new http certificate. This allows you to use the same CA across +multiple Elasticsearch clusters which can make it easier to configure clients, +and may be easier for you to manage. + +If you do not have an existing CA, one will be generated for you. + +Use an existing CA? [y/N]y + +## What is the path to your CA? + +Please enter the full pathname to the Certificate Authority that you wish to +use for signing your new http certificate. This can be in PKCS#12 (.p12), JKS +(.jks) or PEM (.crt, .key, .pem) format. +CA Path: /etc/elasticsearch/elastic-stack-ca.p12 +Reading a PKCS12 keystore requires a password. +It is possible for the keystore's password to be blank, +in which case you can simply press at the prompt +Password for elastic-stack-ca.p12: + +## How long should your certificates be valid? + +Every certificate has an expiry date. When the expiry date is reached clients +will stop trusting your certificate and TLS connections will fail. + +Best practice suggests that you should either: +(a) set this to a short duration (90 - 120 days) and have automatic processes +to generate a new certificate before the old one expires, or +(b) set it to a longer duration (3 - 5 years) and then perform a manual update +a few months before it expires. + +You may enter the validity period in years (e.g. 3Y), months (e.g. 18M), or days (e.g. 90D) + +For how long should your certificate be valid? [5y] + +## Do you wish to generate one certificate per node? + +If you have multiple nodes in your cluster, then you may choose to generate a +separate certificate for each of these nodes. Each certificate will have its +own private key, and will be issued for a specific hostname or IP address. + +Alternatively, you may wish to generate a single certificate that is valid +across all the hostnames or addresses in your cluster. + +If all of your nodes will be accessed through a single domain +(e.g. node01.es.example.com, node02.es.example.com, etc) then you may find it +simpler to generate one certificate with a wildcard hostname (*.es.example.com) +and use that across all of your nodes. + +However, if you do not have a common domain name, and you expect to add +additional nodes to your cluster in the future, then you should generate a +certificate per node so that you can more easily generate new certificates when +you provision new nodes. + +Generate a certificate per node? [y/N]y + +## What is the name of node #1? + +This name will be used as part of the certificate file name, and as a +descriptive name within the certificate. + +You can use any descriptive name that you like, but we recommend using the name +of the Elasticsearch node. + +node #1 name: HOSTNAME.example.net + +## Which hostnames will be used to connect to HOSTNAME.example.net? + +These hostnames will be added as "DNS" names in the "Subject Alternative Name" +(SAN) field in your certificate. + +You should list every hostname and variant that people will use to connect to +your cluster over http. +Do not list IP addresses here, you will be asked to enter them later. + +If you wish to use a wildcard certificate (for example *.es.example.com) you +can enter that here. + +Enter all the hostnames that you need, one per line. +When you are done, press once more to move on to the next step. + +HOSTNAME.example.net +localhost + +You entered the following hostnames. + + - HOSTNAME.example.net + - localhost + +Is this correct [Y/n] + +## Which IP addresses will be used to connect to HOSTNAME.example.net? + +If your clients will ever connect to your nodes by numeric IP address, then you +can list these as valid IP "Subject Alternative Name" (SAN) fields in your +certificate. + +If you do not have fixed IP addresses, or not wish to support direct IP access +to your cluster then you can just press to skip this step. + +Enter all the IP addresses that you need, one per line. +When you are done, press once more to move on to the next step. + +192.0.2.10 +2001:DB8::10 +127.0.0.1 +::1 + +You entered the following IP addresses. + + - 192.0.2.10 + - 2001:DB8::10 + - 127.0.0.1 + - ::1 + +Is this correct [Y/n]y + +## Other certificate options + +The generated certificate will have the following additional configuration +values. These values have been selected based on a combination of the +information you have provided above and secure defaults. You should not need to +change these values unless you have specific requirements. + +Key Name: HOSTNAME.example.net +Subject DN: CN=HOSTNAME, DC=example, DC=net +Key Size: 2048 + +Do you wish to change any of these options? [y/N] +Generate additional certificates? [Y/n]n + +## What password do you want for your private key(s)? + +Your private key(s) will be stored in a PKCS#12 keystore file named "http.p12". +This type of keystore is always password protected, but it is possible to use a +blank password. + +If you wish to use a blank password, simply press at the prompt below. +Provide a password for the "http.p12" file: [ for none] + +## Where should we save the generated files? + +A number of files will be generated including your private key(s), +public certificate(s), and sample configuration options for Elastic Stack products. + +These files will be included in a single zip archive. + +What filename should be used for the output zip file? [/usr/share/elasticsearch/elasticsearch-ssl-http.zip] + +Zip file written to /usr/share/elasticsearch/elasticsearch-ssl-http.zip +~~~ + +On récupère ainsi une archive zip contenant tous les fichiers nécessaires : + +~~~ +~/tmp# unzip elasticsearch-ssl-http.zip +~/tmp# tree . +. +├── elasticsearch +│   ├── http.p12 +│   ├── README.txt +│   └── sample-elasticsearch.yml +├── elasticsearch-ssl-http.zip +└── kibana + ├── elasticsearch-ca.pem + ├── README.txt + └── sample-kibana.yml + +2 directories, 7 files +~~~ + +Le dossier `elasticsearch` contient le fichier certificat + clé au format *p12* pour elasticsearch ainsi qu'un extrait de configuration d'exemple pour configurer le HTTPS client. Le dossier `kibana` contient le certificat de la CA (pour pouvoir valider le certificat donné par le serveur elastic) ainsi qu'un extrait de configuration d'exemple pour activer le tls client. + + +> **Note** On pourra ré-utiliser le certificat CA `kibana/elasticsearch-ca.pem` avec tous les services/logiciels clients du cluster elastic pour pouvoir vérifier la validité du certificat donné par le serveur. + + +On déplace les fichiers aux bons endroits +~~~ +~/tmp# mv elasticsearch/http.p12 /etc/elasticsearch/HOSTNAME.http.p12 +~/tmp# chgrp elasticsearch /etc/elasticsearch/HOSTNAME.http.p12 +~/tmp# chmod g+r /etc/elasticsearch/HOSTNAME.http.p12 +~/tmp# +~/tmp# mv kibana/elasticsearch-ca.pem /etc/kibana/elasticsearch-ca.pem +~/tmp# chgrp kibana /etc/kibana/elasticsearch-ca.pem +~/tmp# chmod g+r /etc/kibana/elasticsearch-ca.pem +~~~ + +Puis on ajoute l'extrait de configuration suivant pour elasticsearch : + +~~~yaml +xpack.security.http.ssl.enabled: true +xpack.security.http.ssl.keystore.path: HOSTNAME.http.p12 +~~~ + +Après, redémarrage avec `systemctl restart elasticsearch` + + +#### Ajustement configuration monitoring + +On ajoutera l'argument suivant à la commande de check_http pour passer en HTTPS `-S` + +#### Ajustement configuration Kibana + +Dans `/etc/kibana/kibana.yml`, on doit faire les changelents suivants : + +~~~diff +--- a/kibana/kibana.yml ++++ b/kibana/kibana.yml +@@ -31,7 +31,7 @@ server.basePath: "/kibana" + #server.name: "your-hostname" + + # The URLs of the Elasticsearch instances to use for all your queries. +-#elasticsearch.hosts: ["http://localhost:9200"] ++elasticsearch.hosts: ["https://localhost:9200"] + + # Kibana uses an index in Elasticsearch to store saved searches, visualizations and + # dashboards. Kibana creates a new index if the index doesn't already exist. +@@ -65,7 +65,7 @@ server.basePath: "/kibana" + + # Optional setting that enables you to specify a path to the PEM file for the certificate + # authority for your Elasticsearch instance. +-#elasticsearch.ssl.certificateAuthorities: [ "/path/to/your/CA.pem" ] ++elasticsearch.ssl.certificateAuthorities: [ "/etc/kibana/elasticsearch-ca.pem" ] + + # To disregard the validity of SSL certificates, change this setting's value to 'none'. + #elasticsearch.ssl.verificationMode: full +~~~ + +Puis redémarrer Kibana `systemctl restart kibana` ## Snapshots et sauvegardes