evolix
/
ansible-roles
16
2
Fork 2
Code Issues 60 Pull Requests 18 Releases 27 Wiki Activity

Add mysql-oracle role

Browse Source 
Install and configure MySQL 5.7 with packages from Oracle
evoadmin-mail
Jérémy Lecour 5 years ago committed by Jérémy Lecour
parent ef3287f7a2
commit 738d56db68
28 changed files with 3460 additions and 0 deletions
Show Stats Download Patch File Download Diff File

1
CHANGELOG.md
Unescape View File

@ -14,6 +14,7 @@ The **patch** part changes incrmentally at each release.
* postfix: add lines in /etc/.gitignore
* nagios-nrpe: add "check_open_files" plugin
* nagios-nrpe: mark plugins as executable
* mysql-oracle: new role to install MySQL 5.7 with Oracle packages
### Changed
* elasticsearch: use ES_TMPDIR variable for custom tmpdir, (from `/etc/default/elasticsearch` instead of changing `/etc/elesticsearch/jvm.options`).

28
mysql-oracle/.kitchen.yml
Unescape View File

@ -0,0 +1,28 @@
---
driver:
name: docker
privileged: true
use_sudo: false
provisioner:
name: ansible_playbook
hosts: test-kitchen
roles_path: ../
ansible_verbose: true
require_ansible_source: false
require_chef_for_busser: false
idempotency_test: true
platforms:
- name: debian
driver_config:
image: evolix/ansible:2.2.1
suites:
- name: default
provisioner:
name: ansible_playbook
playbook: ./tests/test.yml
transport:
max_ssh_sessions: 6

40
mysql-oracle/README.md
Unescape View File

@ -0,0 +1,40 @@
# mysql
Install MySQL (from Oracle)
## Tasks
Tasks are extracted in several files, included in `tasks/main.yml` :
* `packages.yml` : packages installation ;
* `users.yml` : replacement of `root` user by `mysqladmin` user ;
* `config.yml` : configurations ;
* `datadir.yml` : data directory customization ;
* `tmpdir.yml` : temporary directory customization ;
* `nrpe.yml` : `nrpe` user for Nagios checks ;
* `munin.yml` : Munin plugins ;
* `log2mail.yml` : log2mail patterns ;
* `utils.yml` : useful tools.
## Available variables
* `mysql_replace_root_with_mysqladmin`: switch from `root` to `mysqladmin` user or not ;
* `mysql_thread_cache_size`: number of threads for the cache ;
* `mysql_innodb_buffer_pool_size`: amount of RAM dedicated to InnoDB ;
* `mysql_custom_datadir`: custom datadir
* `mysql_custom_tmpdir`: custom tmpdir.
* `general_alert_email`: email address to send various alert messages (default: `root@localhost`).
* `log2mail_alert_email`: email address to send Log2mail messages to (default: `general_alert_email`).
* `general_scripts_dir`: general directory for scripts installation (default: `/usr/local/bin`).
* `mysql_scripts_dir`: email address to send Log2mail messages to (default: `general_scripts_dir`).
* `mysql_force_new_nrpe_password` : change the password for NRPE even if it exists already (default: `False`).
NB : changing the _datadir_ location can be done multiple times, as long as it is not restored to the default initial location, (because a symlink is created and can't be switched back, yet).
## Misc
We use the `mysql-apt-config` package from Oracle to configure APT sources. It is used right from the role since there is no apparent stable URL to download it from Oracle servers. We should verify from time to time if a new version is available to update the .deb in the role.
The MySQL debian package made by Oracle doesn't work with systemd out of the box. We've used the `mysql-systemd-start` script and the systemd unit made by Debian for the "mysql-5.7" package (currently only available for Sid).
On Stretch, "mytop" is not packages anymore (only provided by "mariadb-client-10.1"). We're using the version from the "mytop" package (currently only available for Sid).

22
mysql-oracle/defaults/main.yml
Unescape View File

@ -0,0 +1,22 @@
---
general_alert_email: "root@localhost"
log2mail_alert_email: Null
general_scripts_dir: "/usr/share/scripts"
mysql_scripts_dir: Null
mysql_replace_root_with_mysqladmin: True
mysql_custom_datadir: ''
mysql_custom_tmpdir: ''
mysql_thread_cache_size: '{{ ansible_processor_cores }}'
mysql_innodb_buffer_pool_size: '{{ (ansible_memtotal_mb * 0.3) | int }}M'
mysql_cron_optimize: True
mysql_cron_optimize_frequency: weekly
mysql_cron_mysqltuner: True
mysql_cron_mysqltuner_frequency: monthly
mysql_force_new_nrpe_password: False

22
mysql-oracle/files/debian/mysql-server-5.7.mysql.service
Unescape View File

@ -0,0 +1,22 @@
# MySQL systemd service file
[Unit]
Description=MySQL Community Server
After=network.target
[Install]
WantedBy=multi-user.target
[Service]
Type=forking
User=mysql
Group=mysql
PIDFile=/run/mysqld/mysqld.pid
PermissionsStartOnly=true
ExecStartPre=/usr/share/mysql/mysql-systemd-start pre
ExecStart=/usr/sbin/mysqld --daemonize --pid-file=/run/mysqld/mysqld.pid
TimeoutSec=600
Restart=on-failure
RuntimeDirectory=mysqld
RuntimeDirectoryMode=755
LimitNOFILE=5000

56
mysql-oracle/files/debian/mysql-systemd-start
Unescape View File

@ -0,0 +1,56 @@
#!/bin/bash
#
# Scripts to run by MySQL systemd service
#
# Needed argument: pre
#
# pre mode : try to perform sanity check for configuration, log, data
# Read a config option from mysql. Note that this will only work if a config
# file actually specifies the option, so it's important to specify a default
# $1 is application (e.g. mysqld for server)
# $2 is option
# $3 is default value used if no config value is found
get_mysql_option() {
result=$(my_print_defaults "$1" | sed -n "s/^--$2=//p" | tail -n 1)
if [ -z "$result" ];
then
result="$3"
fi
echo "$result"
}
sanity () {
if [ ! -r /etc/mysql/my.cnf ]; then
echo "MySQL configuration not found at /etc/mysql/my.cnf. Please create one."
exit 1
fi
datadir=$(get_mysql_option mysqld datadir "/var/lib/mysql")
if [ ! -d "${datadir}" ] && [ ! -L "${datadir}" ]; then
echo "MySQL data dir not found at ${datadir}. Please create one."
exit 1
fi
if [ ! -d "${datadir}/mysql" ] && [ ! -L "${datadir}/mysql" ]; then
echo "MySQL system database not found in ${datadir}. Please run mysqld --initialize."
exit 1
fi
# Do a test start to make sure there are no configuration issues or similar
# mysqld --verbose --help will output a full listing of settings and plugins.
# To do so it needs to initialize the database, so it can be used as a test
# for whether or not the server can start. We redirect stdout to /dev/null so
# only the error messages are left.
result=0
output=$(mysqld --verbose --help --innodb-read-only 2>&1 > /dev/null) || result=$?
if [ ! "$result" = "0" ]; then
echo "ERROR: Unable to start MySQL server:" >&2
echo "$output" >&2
echo "Please take a look at https://wiki.debian.org/Teams/MySQL/FAQ for tips on fixing common upgrade issues." >&2
echo "Once the problem is resolved, restart the service." >&2
exit 1
fi
}
case $1 in
"pre") sanity ;;
esac

62
mysql-oracle/files/evolinux-defaults.cnf
Unescape View File

@ -0,0 +1,62 @@
[mysqld]
###### Connexions
# Maximum de connexions concurrentes (defaut = 100)... provoque un "Too many connections"
max_connections = 250
# Maximum de connexions en attente en cas de max_connections atteint (defaut = 50)
back_log = 100
# Maximum d'erreurs avant de blacklister un hote
max_connect_errors = 10
# Loguer les requetes trop longues
slow_query_log = 1
slow_query_log_file = /var/log/mysql/mysql-slow.log
long_query_time = 10
###### Tailles
# Taille reservee au buffer des index MyIsam
# A ajuster selon les resultats
key_buffer_size = 512M
# Taille max des paquets envoyés/reçus … provoque un "Packet too large"
max_allowed_packet = 64M
# Taille de la memoire reserve pour un thread
thread_stack = 192K
# A mettre le nombre de threads CPU alloues pour MySQL
thread_cache_size = 1
# Taille maximum des tables de type MEMORY
max_heap_table_size = 64M
###### Cache
# max_connections x nbre max de tables dans une jointure (defaut = 64)
table_open_cache = 4096
table_definition_cache = 4096
# Taille max des requetes cachees (defaut = 1M)
query_cache_limit = 8M
# Taille reservee pour le cache (defaut = 0)
query_cache_size = 256M
# Type de requetes a cacher (defaut = 1 : tout peut etre cache)
query_cache_type = 1
# Cache tables
max_heap_table_size = 128M
tmp_table_size = 128M
###### InnoDB
# Si InnoDB n'est pas utilise... le desactiver
#skip-innodb
# En general, il est plus optimum d'avoir un fichier par table
innodb_file_per_table
# Taille memoire allouee pour le cache des datas et index
# A ajuster en fonction de sa RAM (si serveur dedie a MySQL, on peut aller jusqu'a 80%)
innodb_buffer_pool_size = 512M
# Nombre maximum de threads systeme concurents
innodb_thread_concurrency = 16
# Ajuste la valeur des logs InnoDB
# (attention, il faut ensuite stopper MySQL et effacer les fichiers ib_logfile*)
#innodb_log_file_size = 128M
#innodb_log_files_in_group = 2
###### Misc
# charset utf8 par defaut
character-set-server=utf8
collation-server=utf8_general_ci
# Patch MySQL 5.5.53
secure-file-priv = ""

160
mysql-oracle/files/my-add.sh
Unescape View File

@ -0,0 +1,160 @@
#!/bin/sh
usage() {
echo "Usage: $0 [ -d <database> -u <user> [-p <password>] [-f] ]"
}
interactive() {
echo "Add an account / database in MySQL"
echo "Enter the name of the new database"
read db
if ( is_db $db ); then
echo "Database $db already exist !" >&2
exit 1
fi
echo "Enter account with all right on this new database"
echo "(you can use existant account)"
read user
if ( is_user $user ); then
echo "Warning, account already exists, update password ? [N/y]"
read confirm
if [ "${confirm}" = "y" ] || [ "${confirm}" = "Y" ]; then
echo -n "Enter new password for existant MySQL account (empty for random): "
read password
if [ -z "${password}" ]; then
password=$(apg -n1)
fi
fi
else
echo -n "Enter new password for new MySQL account (empty for random): "
read password
if [ -z "${password}" ]; then
password=$(apg -n1)
fi
fi
mysql_add $db $user $password
}
cli() {
while getopts ":d:u:p:f" opt; do
case "$opt" in
d)
db=$OPTARG
;;
u)
user=$OPTARG
;;
p)
password=$OPTARG
;;
f)
force="true"
;;
*)
usage
exit 1
;;
esac
done
shift $((OPTIND-1))
if [ -z "${db}" ]; then
usage
exit 1
fi
if [ -z "${user}" ]; then
usage
exit 1
fi
if ( is_db $db ); then
echo "Database $db already exist !" >&2
exit 1
fi
if ( is_user $user ); then
if [ -z "${force}" ]; then
if [ -n "${password}" ]; then
echo "User $user already exist, update password with -f !" >&2
exit 1
fi
else
if [ -z "${password}" ]; then
password=$(apg -n1)
fi
fi
else
if [ -z "${password}" ]; then
password=$(apg -n1)
fi
fi
mysql_add $db $user $password >/dev/null
}
is_db() {
db=$1
mysql mysql -Ne "SHOW DATABASES;"|grep -q "^${db}$"
exit $?
}
is_user() {
user=$1
nb_user=$(mysql mysql -Ne "SELECT COUNT(User) from user WHERE User='${user}';")
if [ $nb_user -gt 0 ]; then
exit 0
else
exit 1
fi
}
mysql_add() {
db=$1
user=$2
password=$3
echo -n "Create '${db}' database ..."
mysql -e "CREATE DATABASE ${db};"
if [ $? -eq 0 ]; then
echo "OK"
else
echo "KO"
exit 1
fi
if [ -z "${password}" ]; then
password=$(mysql mysql -Ne "SELECT Password FROM user WHERE User='${user}' AND Host='localhost';")
echo -n "Grant '${user}' to '${db}' database ..."
mysql -e "GRANT ALL PRIVILEGES ON ${db}.* TO ${user}@localhost IDENTIFIED BY PASSWORD '${password}';"
grant=$?
else
echo -n "Grant '${user}' to '${db}' database with password '${password}' ..."
mysql -e "GRANT ALL PRIVILEGES ON ${db}.* TO ${user}@localhost IDENTIFIED BY '${password}';"
grant=$?
fi
if [ $grant -eq 0 ]; then
echo "OK"
else
echo "KO"
exit 1
fi
echo -n "Flush Mysql privileges ..."
mysql -e "FLUSH PRIVILEGES;"
if [ $? -eq 0 ]; then
echo "OK"
else
echo "KO"
exit 1
fi
}
main() {
if [ $# = 0 ]; then
interactive
else
cli $@
fi
}
main $@

BIN
mysql-oracle/files/mysql-apt-config_0.8.9-1_all.deb
View File

Binary file not shown.

5
mysql-oracle/files/mysql-optimize.sh
Unescape View File

@ -0,0 +1,5 @@
#!/bin/sh
# Do some weekly optimizations.
# Reset the cache to avoid fragmentation.
mysql --defaults-extra-file=/etc/mysql/debian.cnf -e "RESET QUERY CACHE;"

50
mysql-oracle/files/mysqltuner.cron.sh
Unescape View File

@ -0,0 +1,50 @@
#!/bin/bash
set -e
export TERM=screen
mem=$(free -m | grep Mem: | tr -s ' ' | cut -d ' ' -f2)
swap=$(free -m | grep Swap: | tr -s ' ' | cut -d ' ' -f2)
template=$(mktemp --tmpdir=/tmp evomysqltuner.XXX)
body=$(mktemp --tmpdir=/tmp evomysqltuner.XXX)
clientmail=$(grep EVOMAINTMAIL /etc/evomaintenance.cf | cut -d'=' -f2)
hostname=$(grep HOSTNAME /etc/evomaintenance.cf | cut -d'=' -f2)
hostname=${hostname%%.evolix.net}
# If hostname is composed with -, remove the first part.
if [[ $hostname =~ "-" ]]; then
hostname=$(echo $hostname | cut -d'-' -f2-)
fi
# Remove temporary files on exit.
trap "rm $template $body" EXIT
# Add port here if you have more than one instance!
instances="3306"
for instance in $instances; do
mysqltuner --port $instance --host 127.0.0.1 --forcemem $mem --forceswap $swap \
| aha > /var/www/mysqlreport_${instance}.html
cat << EOT > $template
Content-Type: text/plain; charset="utf-8"
Reply-To: Équipe Evolix <equipe@evolix.fr>
From: Équipe Evolix <equipe@evolix.net>
To: $clientmail
Subject: Rapport MySQL instance $instance pour votre serveur $hostname
EOT
cat << EOT > $body
Bonjour,
Veuillez trouver ci-joint un rapport MySQL.
Celui-ci permet d'identifier aisément si des optimisations MySQL sont possibles.
N'hésitez pas à nous indiquer par mail ou ticket quelles variables vous souhaiter
optimiser.
Veuillez noter qu'il faudra redémarrer MySQL pour appliquer de nouveaux paramètres.
Bien à vous,
--
Rapport automatique Evolix
EOT
mutt -x -e 'set send_charset="utf-8"' -H $template \
-a /var/www/mysqlreport_${instance}.html < $body
done
chmod 644 /var/www/mysqlreport*html

2328
mysql-oracle/files/mytop
Unescape View File

@ -0,0 +1,2328 @@
#!/usr/bin/perl -w
#
# $Id: mytop,v 1.91 2012/01/18 16:49:12 mgrennan Exp $
=pod
=head1 NAME
mytop - display MySQL server performance info like `top'
=cut
## most of the POD is at the bottom of the file
use 5.005;
use strict;
use DBI;
use Getopt::Long;
use DBD::mysql;
use Socket;
$main::VERSION = "1.91";
$|=1;
$0 = 'mytop';
my $WIN = ($^O eq 'MSWin32') ? 1 : 0;
## Test for color support.
eval { require Term::ANSIColor; };
my $HAS_COLOR = $@ ? 0 : 1;
$HAS_COLOR = 0 if $WIN;
## Test of Time::HiRes support
eval { require Time::HiRes };
my $HAS_TIME = $@ ? 0 : 1;
my $debug = 0;
## Try to lower our priority (which, who, pri)
setpriority(0,0,10) unless $WIN;
## Prototypes
sub Clear();
sub GetData();
sub GetQPS();
sub FullQueryInfo($);
sub Explain($);
sub PrintTable(@);
sub PrintHelp();
sub Sum(@);
sub commify($);
sub make_short($);
sub Hashes($);
sub Execute($);
sub StringOrRegex($);
sub GetInnoDBStatus();
sub GetCmdSummary();
sub GetShowVariables();
sub GetShowStatus();
sub cmd_s;
sub cmd_S;
sub cmd_q;
sub FindProg($);
## Default Config Values
my %config = (
batchmode => 0,
color => 1,
db => 'test',
delay => 5,
filter_user => qr/.?/,
filter_db => qr/.?/,
filter_host => qr/.?/,
filter_state => qr/.?/,
header => 1,
help => 0,
host => 'localhost',
idle => 1,
long => 120,
long_nums => 0,
mode => 'top',
prompt => 0,
pass => '',
port => 3306,
resolve => 0,
slow => 10, # slow query time
socket => '',
sort => 1, # default or reverse sort ("s")
user => 'root'
);
my %qcache = (); ## The query cache--used for full query info support.
my %ucache = (); ## The user cache--used for full killing by user
my %dbcache = (); ## The db cache. This should be merged at some point.
my %statcache = (); ## The show status cache for GetShowStatus()
my (%STATUS, %OLD_STATUS); # header stuff.
my $CLEAR = $WIN ? '': `clear`;
## Term::ReadKey values
my $RM_RESET = 0;
my $RM_NOBLKRD = 3; ## using 4 traps Ctrl-C :-(
## Read the user's config file, if it exists.
my $config = "$ENV{HOME}/.mytop";
if (-e $config)
{
if (open CFG, "<$config")
{
while (<CFG>)
{
next if /^\s*$/; ## skip blanks
next if /^\s*#/; ## skip comments
chomp;
if (/(\S+)\s*=\s*(.*\S)/)
{
$config{lc $1} = $2 if exists $config{lc $1};
}
}
close CFG;
}
}
## Command-line args.
use vars qw($opt_foo);
Getopt::Long::Configure('no_ignore_case', 'bundling');
GetOptions(
"color!" => \$config{color},
"user|u=s" => \$config{user},
"pass|password|p=s" => \$config{pass},
"database|db|d=s" => \$config{db},
"host|h=s" => \$config{host},
"port|P=i" => \$config{port},
"socket|S=s" => \$config{socket},
"delay|s=i" => \$config{delay},
"batch|batchmode|b" => \$config{batchmode},
"header!" => \$config{header},
"idle|i!" => \$config{idle},
"resolve|r!" => \$config{resolve},
"prompt!" => \$config{prompt},
"long=i" => \$config{long},
"long_nums!" => \$config{long_nums},
"mode|m=s" => \$config{mode},
"slow=i" => \$config{slow},
"sort=s" => \$config{sort}
);
## User may have put the port with the host.
if ($config{host} =~ s/:(\d+)$//)
{
$config{port} = $1;
}
## Don't use Term::ReadKey unless running interactively.
if (not $config{batchmode})
{
require Term::ReadKey;
Term::ReadKey->import();
}
## User may want to disable color.
if ($HAS_COLOR and not $config{color})
{
$HAS_COLOR = 0;
}
if ($HAS_COLOR)
{
import Term::ANSIColor ':constants';
}
else
{
*RESET = sub { };
*YELLOW = sub { };
*RED = sub { };
*MAGENTA = sub { };
*GREEN = sub { };
*BLUE = sub { };
*WHITE = sub { };
*BOLD = sub { };
}
my $RESET = RESET() || '';
my $YELLOW = YELLOW() || '';
my $RED = RED() || '';
my $MAGENTA = MAGENTA() || '';
my $GREEN = GREEN() || '';
my $BLUE = BLUE() || '';
my $WHITE = WHITE() || '';
my $BOLD = BOLD() || '';
## Connect
my $dsn;
## Socket takes precedence.
$dsn ="DBI:mysql:database=$config{db};mysql_read_default_group=mytop;";
if ($config{socket} and -S $config{socket})
{
$dsn .= "mysql_socket=$config{socket}";
}
else
{
$dsn .= "host=$config{host};port=$config{port}";
}
if ($config{prompt})
{
print "Password: ";
ReadMode(2);
chomp($config{pass} = <STDIN>);
ReadMode(0);
print "\n";
}
my $dbh = DBI->connect($dsn, $config{user}, $config{pass},
{ PrintError => 0 });
if (not ref $dbh)
{
my $Error = <<EODIE
Cannot connect to MySQL server. Please check the:
* database you specified "$config{db}" (default is "test")
* username you specified "$config{user}" (default is "root")
* password you specified "$config{pass}" (default is "")
* hostname you specified "$config{host}" (default is "localhost")
* port you specified "$config{port}" (default is 3306)
* socket you specified "$config{socket}" (default is "")
The options my be specified on the command-line or in a ~/.mytop
config file. See the manual (perldoc mytop) for details.
Here's the exact error from DBI. It might help you debug:
$DBI::errstr
EODIE
;
die $Error;
}
ReadMode($RM_RESET) unless $config{batchmode};
## Get static data
my $db_version;
my $db_release;
my $have_query_cache;
my @variables = Hashes("show variables");
foreach (@variables)
{
if ($_->{Variable_name} eq "version")
{
$db_version = $_->{Value};
$db_release = int(sprintf("%d",$db_version));
next;
}
if ($_->{Variable_name} eq "have_query_cache")
{
# if ($_->{Value} eq 'YES')
if ($_->{Value} eq 'YES' or $_->{Value} eq 'DEMAND') # http://freshmeat.net/users/jerjones
{
$have_query_cache = 1;
}
else
{
$have_query_cache = 0;
}
next;
}
}
#########################################################################
##
## The main loop
##
#########################################################################
ReadMode($RM_NOBLKRD) unless $config{batchmode};
while (1)
{
my $key;
if ($config{mode} eq 'qps')
{
GetQPS();
$key = ReadKey(1);
next unless $key;
if ($key =~ /t/i)
{
$config{mode} = 'top';
}
if ($key =~ /q/)
{
cmd_q();
}
next;
}
if ($config{mode} eq 'top')
{
GetData();
last if $config{batchmode};
$key = ReadKey($config{delay});
next unless $key;
}
elsif ($config{mode} eq 'cmd')
{
GetCmdSummary();
last if $config{batchmode};
$key = ReadKey($config{delay});
next unless $key;
}
elsif ($config{mode} eq 'innodb')
{
GetInnoDBStatus();
last if $config{batchmode};
$key = ReadKey($config{delay});
next unless $key;
}
elsif ($config{mode} eq 'status')
{
GetShowStatus();
last if $config{batchmode};
$key = ReadKey($config{delay});
next unless $key;
}
##
## keystroke command processing (if we get this far)
##
if ($key eq '!')
{
Execute("stop slave");
Execute("set global sql_slave_skip_counter=1");
Execute("start slave");
}
# t - top
if ($key =~ /t/i)
{
$config{mode} = 'top';
}
## q - quit
if ($key eq 'q')
{
cmd_q();
}
if ($key eq 'D')
{
require Data::Dumper;
print Data::Dumper::Dumper([\%config]);
ReadKey(0);
}
## l - change long running hightling
if ($key eq 'l')
{
cmd_l();
next;
}
## m - mode swtich to qps
if ($key eq 'm')
{
$config{mode} = 'qps';
Clear() unless $config{batchmode};
print "Queries Per Second [hit q to exit this mode]\n";
next;
}
## c - mode swtich to command summary
if ($key eq 'c')
{
$config{mode} = 'cmd';
Clear() unless $config{batchmode};
print "Command Summary [hit q to exit this mode]\n";
next;
}
## C - change Color on and off
if ($key eq 'C')
{
if ( $HAS_COLOR )
{
$HAS_COLOR = 0;
}
else
{
$HAS_COLOR = 1;
}
}
## s - seconds of delay
if ($key eq 's')
{
cmd_s();
next;
}
if ($key eq 'S')
{
cmd_S();
next;
}
## R - resolve hostnames
if ($key eq 'R')
{
if ($config{resolve})
{
$config{resolve} = 0;
}
else
{
$config{resolve} = 1;
}
}
## t - username based filter
if ($key eq 't')
{
ReadMode($RM_RESET);
print RED(), "Which state (blank for all, /.../ for regex): ", RESET();
$config{filter_state} = StringOrRegex(ReadLine(0));
ReadMode($RM_NOBLKRD);
next;
}
## u - username based filter
if ($key eq 'u')
{
ReadMode($RM_RESET);
print RED(), "Which user (blank for all, /.../ for regex): ", RESET();
$config{filter_user} = StringOrRegex(ReadLine(0));
ReadMode($RM_NOBLKRD);
next;
}
## d - database name based filter
if ($key eq 'd')
{
ReadMode($RM_RESET);
print RED(), "Which database (blank for all, /.../ for regex): ",
RESET();
$config{filter_db} = StringOrRegex(ReadLine(0));
ReadMode($RM_NOBLKRD);
next;
}
## h - hostname based filter
if ($key eq 'h')
{
ReadMode($RM_RESET);
print RED(), "Which hostname (blank for all, /.../ for regex): ",
RESET();
$config{filter_host} = StringOrRegex(ReadLine(0));
ReadMode($RM_NOBLKRD);
next;
}
## E - Show full Replication Error
if ($key eq 'E')
{
my($data) = Hashes('SHOW SLAVE STATUS');
Clear();
print "Error is: $data->{Last_Error}\n";
print RED(), "-- paused. press any key to resume --", RESET();
ReadKey(0);
next;
}
## F - remove all filters
if ($key eq 'F')
{
$config{filter_host} = qr/.?/;
$config{filter_db} = qr/.?/;
$config{filter_user} = qr/.?/;
$config{filter_state} = qr/.?/;
print RED(), "-- display unfiltered --", RESET();
sleep 1;
next;
}
## p - pause
if ($key eq 'p')
{
print RED(), "-- paused. press any key to resume --", RESET();
ReadKey(0);
next;
}
## i - idle toggle
if ($key =~ /i/)
{
if ($config{idle})
{
$config{idle} = 0;
$config{sort} = 1;
print RED(), "-- idle (sleeping) processed filtered --", RESET();
sleep 1;
}
else
{
$config{idle} = 1;
$config{sort} = 0;
print RED(), "-- idle (sleeping) processed unfiltered --", RESET();
sleep 1;
}
}
## I - InnoDB status
if ($key =~ 'I')
{
$config{mode} = 'innodb';
Clear() unless $config{batchmode};
print "InnoDB Status [hit q to exit this mode]\n";
next;
}
## o - sort order
if ($key =~ /o/)
{
if ($config{sort})
{
$config{sort} = 0;
print RED(), "-- sort order reversed --", RESET();
sleep 1;
}
else
{
$config{sort} = 1;
print RED(), "-- sort order reversed --", RESET();
sleep 1;
}
}
## ? - help
if ($key eq '?')
{
Clear();
PrintHelp();
ReadKey(0);
next;
}
## k - kill
if ($key eq 'k')
{
ReadMode($RM_RESET);
print RED(), "Thread id to kill: ", RESET();
my $id = ReadLine(0);
$id =~ s/\s//g;
if ($id =~ /^\d+$/)
{
Execute("KILL $id");
}
else
{
print RED(), "-- invalid thread id --", RESET();
sleep 1;
}
ReadMode($RM_NOBLKRD);
next;
}
## K - kill based on a username
if ($key =~ /K/)
{
ReadMode($RM_RESET);
print RED(), "User to kill: ", RESET();
my $user = ReadLine(0);
$user =~ s/\s//g;
if ($user =~ /^\S+$/)
{
for my $pid (keys %ucache)
{
next unless $ucache{$pid} eq $user;
Execute("KILL $pid");
select(undef, undef, undef, 0.2);
}
}
else
{
print RED(), "-- invalid thread id --", RESET();
sleep 1;
}
ReadMode($RM_NOBLKRD);
}
## f - full info
if ($key =~ /f/)
{
ReadMode($RM_RESET);
print RED(), "Full query for which thread id: ", RESET();
my $id = ReadLine(0);
chomp $id;
FullQueryInfo($id);
ReadMode($RM_NOBLKRD);
print RED(), "-- paused. press any key to resume or (e) to explain --",
RESET();
my $key = ReadKey(0);
if ($key eq 'e')
{
Explain($id);
print RED(), "-- paused. press any key to resume --", RESET();
ReadKey(0);
}
next;
}
## e - explain
if ($key =~ /e/)
{
ReadMode($RM_RESET);
print RED(), "Explain which query (id): ", RESET();
my $id = ReadLine(0);
chomp $id;
Explain($id);
ReadMode($RM_NOBLKRD);
print RED(), "-- paused. press any key to resume --", RESET();
ReadKey(0);
next;
}
## r - reset status counters
if ($key =~ /r/)
{
Execute("FLUSH STATUS");
print RED(), "-- counters reset --", RESET();
sleep 1;
next;
}
## H - header toggle
if ($key eq 'H')
{
if ($config{header})
{
$config{header} = 0;
}
else
{
$config{header}++;
}
}
## # - magic debug key
if ($key eq '#')
{
$debug = 1;
}
if ($key eq 'V')
{
GetShowVariables();
print RED(), "-- paused. press any key to resume --", RESET();
ReadKey(0);
}
if ($key eq 'S')
{
$config{mode} = 'status';
}
}
ReadMode($RM_RESET) unless $config{batchmode};
exit;
#######################################################################
sub Clear()
{
if (not $WIN)
{
print "$CLEAR"
}
else
{
print "\n" x 90; ## dumb hack for now. Anyone know how to
## clear the screen in dos window on a Win32
## system??
}
}
my $last_time;
sub GetData()
{
## Get terminal info
my $now_time;
%qcache = (); ## recycle memory
%dbcache = ();
my ($width, $height, $wpx, $hpx, $lines_left);
if (not $config{batchmode})
{
($width, $height, $wpx, $hpx) = GetTerminalSize();
$lines_left = $height - 2;
}
else
{
$height = 999_999; ## I hope you don't have more than that!
$lines_left = 999_999;
$width = 80;
}
##
## Header stuff.
##
if ($config{header})
{
my @recs = "";
if ( $db_release > 4 )
{
@recs = Hashes("show global status");
}
else
{
@recs = Hashes("show status");
}
## if the server died or we lost connectivity
if (not @recs)
{
ReadMode($RM_RESET);
exit 1;
}
## get high-res or low-res time
my ($t_delta);
if ($HAS_TIME)
{
$now_time = Time::HiRes::gettimeofday();
}
else
{
$now_time = time;
}
if ($last_time and $last_time != $now_time)
{
$t_delta = $now_time - $last_time;
}
%OLD_STATUS = %STATUS;
foreach my $ref (@recs)
{
my $key = $ref->{Variable_name};
my $val = $ref->{Value};
$STATUS{$key} = $val;
}
## Compute Key Cache Hit Stats
$STATUS{Key_read_requests} ||= 1; ## can't divide by zero next
my $cache_hits_percent = (100-($STATUS{Key_reads}/$STATUS{Key_read_requests}) * 100);
$cache_hits_percent = sprintf("%2.2f",$cache_hits_percent);
## Query Cache info for <= Ver. 4.1
##
## mysql> show status like 'qcache%';
## +-------------------------+----------+
## | Variable_name | Value |
## +-------------------------+----------+
## | Qcache_queries_in_cache | 81 |
## | Qcache_inserts | 4961668 |
## | Qcache_hits | 1374170 |
## | Qcache_not_cached | 5656249 |
## | Qcache_free_memory | 33164800 |
## | Qcache_free_blocks | 2 |
## | Qcache_total_blocks | 168 |
## +-------------------------+----------+
##
## Query Cache info for => Ver. 5.0
##
## mysql> show status like 'qcache%';
## +-------------------------+------------+
## | Variable_name | Value |
## +-------------------------+------------+
## | Qcache_free_blocks | 37652 |
## | Qcache_free_memory | 110289712 |
## | Qcache_hits | 1460617356 |
## | Qcache_inserts | 390563495 |
## | Qcache_lowmem_prunes | 6414172 |
## | Qcache_not_cached | 93002420 |
## | Qcache_queries_in_cache | 66558 |
## | Qcache_total_blocks | 192031 |
## +-------------------------+------------+
my $query_cache_hits = 0;
my $query_cache_hits_per_sec = 0;
my $now_query_cache_hits_per_sec = 0;
if ($have_query_cache)
{
$query_cache_hits = $STATUS{Qcache_hits};
$query_cache_hits_per_sec = $STATUS{Qcache_hits} / $STATUS{Uptime};
if (defined $last_time and $last_time != $now_time)
{
my $q_delta = $STATUS{Qcache_hits} - $OLD_STATUS{Qcache_hits};
$now_query_cache_hits_per_sec = sprintf "%.2f", $q_delta / $t_delta;
}
}
open L, "</proc/loadavg";
my $l = <L>;
close L;
chomp $l;
$last_time = $now_time;
## Server Uptime in meaningful terms...
my $time = $STATUS{Uptime};
my ($d,$h,$m,$s) = (0, 0, 0, 0);
$d += int($time / (60*60*24)); $time -= $d * (60*60*24);
$h += int($time / (60*60)); $time -= $h * (60*60);
$m += int($time / (60)); $time -= $m * (60);
$s += int($time);
my $uptime = sprintf("%d+%02d:%02d:%02d", $d, $h, $m, $s);
## Queries per second...
my $avg_queries_per_sec = sprintf("%.2f", $STATUS{Questions} / $STATUS{Uptime});
my $num_queries = $STATUS{Questions};
my @t = localtime(time);
my $current_time = sprintf "[%02d:%02d:%02d]", $t[2], $t[1], $t[0];
my $host_width = 75;
my $up_width = $width - $host_width;
Clear() unless $config{batchmode};
print RESET();
printf "%-${host_width}s%${up_width}s\n",
"MySQL on $config{host} ($db_version)",
"load $l up $uptime $current_time";
$lines_left--;
printf " Queries: %-6s qps: %4.0f Slow: %7s Se/In/Up/De(%%): %02.0f/%02.0f/%02.0f/%02.0f \n",
make_short( $STATUS{Questions} ), # q total
$STATUS{Questions} / $STATUS{Uptime}, # qps, average
make_short( $STATUS{Slow_queries} ), # slow
# hmm. a Qcache hit is really a select and should be counted.
100 * ($STATUS{Com_select} + ($STATUS{Qcache_hits}||0) ) / $STATUS{Questions},
100 * ($STATUS{Com_insert} + $STATUS{Com_replace} ) / $STATUS{Questions},
100 * ($STATUS{Com_update} ) / $STATUS{Questions},
100 * $STATUS{Com_delete} / $STATUS{Questions};
$lines_left--;
if ($t_delta)
{
my $q_diff = ( $STATUS{Questions} - $OLD_STATUS{Questions} );
# print("q_diff: $STATUS{Questions} - $OLD_STATUS{Questions} / $t_delta = $q_diff\n");
printf(" Sorts: %5.0f qps now: %4.0f Slow qps: %3.1f Threads: %4.0f (%4.0f/%4.0f) %02.0f/%02.0f/%02.0f/%02.0f \n",
( $STATUS{Sort_rows} - $OLD_STATUS{Sort_rows} ) / $t_delta,
( $STATUS{Questions} - $OLD_STATUS{Questions} ) / $t_delta,
( # slow now (qps)
($STATUS{Slow_queries} ) ?
( $STATUS{Slow_queries} - $OLD_STATUS{Slow_queries} ) / $t_delta :
0
),
$STATUS{Threads_connected},
$STATUS{Threads_running},
$STATUS{Threads_cached},
(100 * ($STATUS{Com_select} - $OLD_STATUS{Com_select} +
($STATUS{Qcache_hits}||0) - ($OLD_STATUS{Qcache_hits}||0)
) ) / ($q_diff ),
(100 * ($STATUS{Com_insert} - $OLD_STATUS{Com_insert} +
$STATUS{Com_replace} - $OLD_STATUS{Com_replace}
) ) / ($q_diff ),
(100 * ($STATUS{Com_update} - $OLD_STATUS{Com_update}) ) / ($q_diff ),
(100 * ($STATUS{Com_delete} - $OLD_STATUS{Com_delete}) ) / ($q_diff ),
);
}
else
{
print "\n";
}
$lines_left--;
if ($have_query_cache and $STATUS{Com_select} and $query_cache_hits)
{
printf(" Cache Hits: %-5s Hits/s: %4.1f Hits now: %5.1f Ratio: ",
make_short($STATUS{Qcache_hits}), # cache hits
$STATUS{Qcache_hits} / $STATUS{Uptime}, # hits / sec
($t_delta) ? ($STATUS{Qcache_hits} - $OLD_STATUS{Qcache_hits}) / $t_delta : 0, # Hits Now
);
my($Ratio) = 100 * ($STATUS{Qcache_hits}) / ($STATUS{Qcache_hits} + $STATUS{Com_select} );
if ($HAS_COLOR)
{
print YELLOW() if ($Ratio < 80.0);
print RED() if ($Ratio < 50.0);
print MAGENTA() if ($Ratio < 20.0);
}
printf("%4.1f%% ",$Ratio);
if ($HAS_COLOR)
{
print RESET();
}
print " Ratio now: ";
my($Ratio_now) = ($t_delta) ? # ratio now
100 * ($STATUS{Qcache_hits} - $OLD_STATUS{Qcache_hits} ) /
( ($STATUS{Com_select} + $STATUS{Qcache_hits} -
($OLD_STATUS{Qcache_hits} + $OLD_STATUS{Com_select})
) || 1) : 0;
if ($HAS_COLOR)
{
print GREEN() if ($Ratio_now => 80.0);
print YELLOW() if ($Ratio_now < 80.0);
print RED() if ($Ratio_now < 50.0);
print MAGENTA() if ($Ratio_now < 20.0);
}
printf("%4.1f%% \n",$Ratio_now);
if ($HAS_COLOR)
{
print RESET();
}
}
$lines_left--;
printf(" Key Efficiency: %2.1f%% Bps in/out: %5s/%5s ",
$cache_hits_percent,
make_short($STATUS{Bytes_received} / $STATUS{Uptime} ),
make_short($STATUS{Bytes_sent} / $STATUS{Uptime}));
printf("Now in/out: %5s/%5s",
make_short(($STATUS{Bytes_received} - $OLD_STATUS{Bytes_received}) / $t_delta ),
make_short(($STATUS{Bytes_sent} - $OLD_STATUS{Bytes_sent}) / $t_delta ))
if ($t_delta);
print "\n";
$lines_left--;
my($data) = Hashes('show global variables like "read_only"');
if ($data->{Value} ne "OFF")
{
print RED() if ($HAS_COLOR) ;
print " ReadOnly";
RESET() if ($HAS_COLOR);
}
my($data) = Hashes('SHOW SLAVE STATUS');
if (defined($data->{Master_Host}))
{
if (defined($data->{Seconds_Behind_Master}))
{
if ($HAS_COLOR) {
print GREEN();
print YELLOW() if ($data->{Seconds_Behind_Master} > 60);
print MAGENTA() if ($data->{Seconds_Behind_Master} > 360);
}
}
print " Replication ";
print "IO:$data->{Slave_IO_Running} ";
print "SQL:$data->{Slave_SQL_Running} ";
print RESET() if ($HAS_COLOR);
if (defined($data->{Seconds_Behind_Master}))
{
if ($HAS_COLOR) {
print GREEN();
print YELLOW() if ($data->{Seconds_Behind_Master} > 60);
print MAGENTA() if ($data->{Seconds_Behind_Master} > 360);
}
print "Delay: $data->{Seconds_Behind_Master} sec.";
} else {
my $free = $width - 45;
my $Err = substr $data->{Last_Error},0 ,$free;
printf(" ERR: %-${free}s", $Err) if ( $Err ne "" );
}
print WHITE() if ($HAS_COLOR);
print "\n";
$lines_left--;
}
print "\n";
}
if (not $config{batchmode} and not $config{header})
{
Clear();
print RESET();
}
##
## Threads
##
#my $sz = $width - 52;
my @sz = (9, 9, 15, 10, 10, 6, 8);
my $used = scalar(@sz) + Sum(@sz);
my $free = $width - $used;
print BOLD() if ($HAS_COLOR);
printf "%9s %9s %15s %10s %10s %6s %8s %-${free}s\n",
'Id','User','Host/IP','DB','Time', 'Cmd', 'State', 'Query';
print RESET() if ($HAS_COLOR);
## Id User Host DB
printf "%9s %9s %15s %10s %10s %6s %8s %-${free}s\n",
'--','----','-------','--','----', '---', '-----', '----------';
$lines_left -= 2;
my $proc_cmd = "show full processlist";
my @data = Hashes($proc_cmd);
foreach my $thread (@data)
{
last if not $lines_left;
## Drop Domain Name, unless it looks like an IP address. If
## it's an IP, we'll strip the port number because it's rarely
## interesting.
my $is_ip = 0;
if ($thread->{Host} =~ /^(\d{1,3}\.){3}(\d{1,3})(:\d+)?$/)
{
$thread->{Host} =~ s/:.*$//;
$is_ip = 1;
}
else
{
$thread->{Host} =~ s/^([^.]+).*/$1/;
}
## Otherwise, look up the IP (if resolve is set) and strip the
## name
if ($is_ip and $config{resolve})
{
$thread->{Host} =~ s/:\d+$//;
# my $host = $thread->{Host};
my $host = gethostbyaddr(inet_aton($thread->{Host}), AF_INET);
# $host =~ s/^([^.]+).*/$1/;
$thread->{Host} = $host;
}
## Fix possible undefs
$thread->{db} ||= '';
$thread->{Info} ||= '';
$thread->{Time} ||= 0 ;
$thread->{Id} ||= 0 ;
$thread->{User} ||= '';
$thread->{Command} ||= '';
$thread->{Host} ||= '';
$thread->{State} ||= "";
## alter double hyphen comments so they don't break
## the query when newlines are removed - http://freshmeat.net/users/jerjones
$thread->{Info} =~ s~\s--(.*)$~ /* $1 */ ~mg;
## Normalize spaces -- mostly disabled for now. This can
## break EXPLAIN if you try to explain a mangled query. It
## may be re-enabled later as an option.
## leading space removal
$thread->{Info} =~ s/^\s*//;
if (1)
{
## remove newlines and carriage returns
$thread->{Info} =~ s/[\n\r]//g;
## collpase whitespace
$thread->{Info} =~ s/\s+/ /g;
}
## stow it in the cache
$qcache{$thread->{Id}} = $thread->{Info};
$dbcache{$thread->{Id}} = $thread->{db};
$ucache{$thread->{Id}} = $thread->{User};
}
## Sort by idle time (closest thing to CPU usage I can think of).
my @sorted;
if (not $config{sort})
{
@sorted = sort { $a->{Time} <=> $b->{Time} } @data
}
else
{
@sorted = sort { $b->{Time} <=> $a->{Time} } @data
}
foreach my $thread (@sorted)
{
# Check to see if we can skip out. We skip out if we know the
# given line doesn't match.
next if (($thread->{Command} eq "Sleep")
and
(not $config{idle}));
next if (($thread->{Command} eq "Binlog Dump")
and
(not $config{idle}));
next if (($thread->{Command} eq "Daemon")
and
(not $config{idle}));
next if ($thread->{User} !~ $config{filter_user});
next if ($thread->{db} !~ $config{filter_db});
next if ($thread->{Host} !~ $config{filter_host});
next if ($thread->{State} !~ $config{filter_state});
$thread->{State} = trim(sprintf("%8.8s",$thread->{State}));
# Otherwise, print.
my $smInfo;
if ($thread->{Info})
{
$smInfo = substr $thread->{Info}, 0, $free;
}
# if ($thread->{State})
# {
# $smInfo = substr $thread->{State}, 0, $free;
# }
else
{
$smInfo = "";
}
if ($HAS_COLOR)
{
print YELLOW() if $thread->{Command} eq 'Query';
print WHITE() if $thread->{Command} eq 'Sleep';
print GREEN() if $thread->{Command} eq 'Connect';
print BOLD() if $thread->{Time} > $config{slow};
print MAGENTA() if $thread->{Time} > $config{long};
}
printf "%9d %9.9s %15.15s %10.10s %10d %6.6s %8.8s %-${free}.${free}s\n",
$thread->{Id}, $thread->{User}, $thread->{Host}, $thread->{db},
$thread->{Time}, $thread->{Command}, $thread->{State}, $smInfo;
print RESET() if $HAS_COLOR;
$lines_left--;
last if $lines_left == 0;
}
}
###########################################################################
my $questions;
sub GetQPS()
{
my($data) = Hashes('SHOW STATUS LIKE "Questions"');
my $num = $data->{Value};
if (not defined $questions) ## first time?
{
$questions = $num;
return;
}
my $qps = $num - $questions;
$questions = $num;
print "$qps\n";
}
###########################################################################
sub GetQcacheSummary()
{
}
###########################################################################
sub GetInnoDBStatus()
{
if (not $config{pager})
{
if (not $config{pager} = FindProg('less'))
{
$config{pager} = FindProg('more');
}
}
my @data = Hashes("SHOW INNODB STATUS");
open P, "|$config{pager}" or die "$!";
print keys %{$data[0]};
print $data[0]->{Status},"\n";
close P;
}
###########################################################################
my %prev_data;
sub GetCmdSummary()
{
my ($width, $height, $wpx, $hpx, $lines_left);
if (not $config{batchmode})
{
($width, $height, $wpx, $hpx) = GetTerminalSize();
$lines_left = $height - 2;
}
else
{
$height = 999_999; ## I hope you don't have more than that!
$lines_left = 999_999;
$width = 80;
}
# Variable_name and Value pairs come back...
my @data = Hashes("SHOW STATUS LIKE 'Com_%'");
my %cmd_data;
my %cmd_delta;
my %cmd_pct;
my %cmd_delta_pct;
my $total;
my $delta_total;