运维首页 > 数据运维 > MySQL 运维管理工具–pt(Percona-toolkit)

MySQL 运维管理工具–pt(Percona-toolkit)

2024年 7月 25日 74.7k 0

Table of Contents

  • 一.检查和安装与Perl相关的模块
  • 二.下载和安装percona toolkit的包
    • 2.1 下载和安装percona toolkit的包
    • 2.2 安装percona-toolkit工具包
    • 2.3 percona-toolkit工具介绍
      • 2.3.1 pt-archiver(归档表)
      • 2.3.2 pt-online-schema-change(在线修改表结构)
      • 2.3.3 pt-table-checksum
      • 2.3.4 pt-table-sync
      • 2.3.5 pt-deadlock-logger(死锁检测)
      • 2.3.6 pt-duplicate-key-checker（主键冲突检测）
      • 2.3.7 pt-kill（杀进程）
        • 2.3.7.1 测试kill空闲链接
        • 2.3.7.2 kill查询时间超过20秒的进程
        • 2.3.7.3 Kill掉 select IFNULl.*语句开头的SQL
  • 参考:

一.检查和安装与Perl相关的模块

PT工具是使用Perl语言编写和执行的，所以需要系统中有Perl环境。
依赖包检查命令为：

yum -y install perl-DBI perl-DBD-MySQL perl-Time-HiRes perl-IO-Socket-SSL

二.下载和安装percona toolkit的包

2.1 下载和安装percona toolkit的包

参照：https://www.percona.com/doc/percona-toolkit/3.0/installation.html#installing-percona-toolkit-on-red-hat-or-centos

yum -y install http://www.percona.com/downloads/percona-release/redhat/0.1-4/percona-release-0.1-4.noarch.rpm

查看可以安装的包

[root@10-31-1-119 src]# yum list | grep percona-toolkit
percona-toolkit.noarch 2.2.20-1 percona-release-noarch
percona-toolkit.x86_64 3.2.1-1.el7 percona-release-x86_64
percona-toolkit-debuginfo.x86_64 3.0.13-1.el7 percona-release-x86_64
[root@10-31-1-119 src]#

2.2 安装percona-toolkit工具包

yum install percona-toolkit

遇到报错

[root@10-31-1-119 src]# yum install percona-toolkit -y
已加载插件：fastestmirror, langpacks
Loading mirror speeds from cached hostfile
正在解决依赖关系
--> 正在检查事务
---> 软件包 percona-toolkit.x86_64.0.3.2.1-1.el7 将被 安装
--> 正在处理依赖关系 perl(Digest::MD5)，它被软件包 percona-toolkit-3.2.1-1.el7.x86_64 需要
--> 正在检查事务
---> 软件包 perl-Digest-MD5.x86_64.0.2.52-3.el7 将被 安装
--> 正在处理依赖关系 perl(Digest::base) >= 1.00，它被软件包 perl-Digest-MD5-2.52-3.el7.x86_64 需要
--> 正在检查事务
---> 软件包 perl-Digest.noarch.0.1.17-245.el7 将被 安装
--> 解决依赖关系完成

依赖关系解决

============================================================================================================================================================================================================
Package 架构 版本 源 大小
============================================================================================================================================================================================================
正在安装:
percona-toolkit x86_64 3.2.1-1.el7 percona-release-x86_64 17 M
为依赖而安装:
perl-Digest noarch 1.17-245.el7 base 23 k
perl-Digest-MD5 x86_64 2.52-3.el7 base 30 k

事务概要
============================================================================================================================================================================================================
安装 1 软件包 (+2 依赖软件包)

总下载量：17 M
安装大小：17 M
Downloading packages:
(1/3): perl-Digest-MD5-2.52-3.el7.x86_64.rpm | 30 kB 00:00:00
(2/3): perl-Digest-1.17-245.el7.noarch.rpm | 23 kB 00:00:00
percona-toolkit-3.2.1-1.el7.x8 FAILED 51% [=======================================- ] 4.8 B/s | 8.6 MB 487:30:28 ETA
http://repo.percona.com/release/7/RPMS/x86_64/percona-toolkit-3.2.1-1.el7.x86_64.rpm: [Errno 12] Timeout on http://repo.percona.com/release/7/RPMS/x86_64/percona-toolkit-3.2.1-1.el7.x86_64.rpm: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds')
正在尝试其它镜像。
percona-toolkit-3.2.1-1.el7.x8 FAILED 96% [========================================================================== ] 5.7 B/s | 16 MB 31:35:32 ETA
http://repo.percona.com/release/7/RPMS/x86_64/percona-toolkit-3.2.1-1.el7.x86_64.rpm: [Errno 12] Timeout on http://repo.percona.com/release/7/RPMS/x86_64/percona-toolkit-3.2.1-1.el7.x86_64.rpm: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds')
正在尝试其它镜像。
warning: /var/cache/yum/x86_64/7/percona-release-x86_64/packages/percona-toolkit-3.2.1-1.el7.x86_64.rpm: Header V4 RSA/SHA256 Signature, key ID 8507efa5: NOKEY==========-] 3.0 kB/s | 17 MB 00:00:01 ETA
percona-toolkit-3.2.1-1.el7.x86_64.rpm 的公钥尚未安装
(3/3): percona-toolkit-3.2.1-1.el7.x86_64.rpm | 17 MB 00:01:59
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
总计 4.8 kB/s | 17 MB 00:59:47
从 file:///etc/pki/rpm-gpg/RPM-GPG-KEY-Percona 检索密钥
导入 GPG key 0xCD2EFD2A:
用户ID : "Percona MySQL Development Team "
指纹 : 430b df5c 56e7 c94e 848e e60c 1c4c bdcd cd2e fd2a
软件包 : percona-release-0.1-4.noarch (@/percona-release-0.1-4.noarch)
来自 : /etc/pki/rpm-gpg/RPM-GPG-KEY-Percona

percona-toolkit-3.2.1-1.el7.x86_64.rpm 的公钥尚未安装

失败的软件包是：percona-toolkit-3.2.1-1.el7.x86_64
GPG 密钥配置为：file:///etc/pki/rpm-gpg/RPM-GPG-KEY-Percona

解决方案

[root@10-31-1-119 src]# yum update percona-release
已加载插件：fastestmirror, langpacks
Loading mirror speeds from cached hostfile
正在解决依赖关系
--> 正在检查事务
---> 软件包 percona-release.noarch.0.0.1-4 将被 升级
---> 软件包 percona-release.noarch.0.1.0-24 将被 更新
--> 解决依赖关系完成

依赖关系解决

============================================================================================================================================================================================================
Package 架构 版本 源 大小
============================================================================================================================================================================================================
正在更新:
percona-release noarch 1.0-24 percona-release-noarch 19 k

事务概要
============================================================================================================================================================================================================
升级 1 软件包

总下载量：19 k
Is this ok [y/d/N]: y
Downloading packages:
No Presto metadata available for percona-release-noarch
percona-release-1.0-24.noarch.rpm | 19 kB 00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
正在更新 : percona-release-1.0-24.noarch 1/2
* Enabling the Percona Original repository
All done!
* Enabling the Percona Release repository
All done!
The percona-release package now contains a percona-release script that can enable additional repositories for our newer products.

For example, to enable the Percona Server 8.0 repository use:

percona-release setup ps80

Note: To avoid conflicts with older product versions, the percona-release setup command may disable our original repository for some products.

For more information, please visit:
https://www.percona.com/doc/percona-repo-config/percona-release.html

清理 : percona-release-0.1-4.noarch 2/2
验证中 : percona-release-1.0-24.noarch 1/2
验证中 : percona-release-0.1-4.noarch 2/2

更新完毕:
percona-release.noarch 0:1.0-24

完毕！
[root@10-31-1-119 src]#
[root@10-31-1-119 src]# yum install percona-toolkit -y
已加载插件：fastestmirror, langpacks
Loading mirror speeds from cached hostfile
percona-release-noarch | 2.9 kB 00:00:00
percona-release-x86_64 | 2.9 kB 00:00:00
prel-release-x86_64 | 2.9 kB 00:00:00
prel-release-x86_64/7/primary_db | 1.1 kB 00:00:00
正在解决依赖关系
--> 正在检查事务
---> 软件包 percona-toolkit.x86_64.0.3.2.1-1.el7 将被 安装
--> 正在处理依赖关系 perl(Digest::MD5)，它被软件包 percona-toolkit-3.2.1-1.el7.x86_64 需要
--> 正在检查事务
---> 软件包 perl-Digest-MD5.x86_64.0.2.52-3.el7 将被 安装
--> 正在处理依赖关系 perl(Digest::base) >= 1.00，它被软件包 perl-Digest-MD5-2.52-3.el7.x86_64 需要
--> 正在检查事务
---> 软件包 perl-Digest.noarch.0.1.17-245.el7 将被 安装
--> 解决依赖关系完成

依赖关系解决

============================================================================================================================================================================================================
Package 架构 版本 源 大小
============================================================================================================================================================================================================
正在安装:
percona-toolkit x86_64 3.2.1-1.el7 percona-release-x86_64 17 M
为依赖而安装:
perl-Digest noarch 1.17-245.el7 base 23 k
perl-Digest-MD5 x86_64 2.52-3.el7 base 30 k

事务概要
============================================================================================================================================================================================================
安装 1 软件包 (+2 依赖软件包)

总计：17 M
安装大小：17 M
Downloading packages:
warning: /var/cache/yum/x86_64/7/percona-release-x86_64/packages/percona-toolkit-3.2.1-1.el7.x86_64.rpm: Header V4 RSA/SHA256 Signature, key ID 8507efa5: NOKEY
从 file:///etc/pki/rpm-gpg/PERCONA-PACKAGING-KEY 检索密钥
导入 GPG key 0x8507EFA5:
用户ID : "Percona MySQL Development Team (Packaging key) "
指纹 : 4d1b b29d 63d9 8e42 2b21 13b1 9334 a25f 8507 efa5
软件包 : percona-release-1.0-24.noarch (@percona-release-noarch)
来自 : /etc/pki/rpm-gpg/PERCONA-PACKAGING-KEY
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
正在安装 : perl-Digest-1.17-245.el7.noarch 1/3
正在安装 : perl-Digest-MD5-2.52-3.el7.x86_64 2/3
正在安装 : percona-toolkit-3.2.1-1.el7.x86_64 3/3
验证中 : perl-Digest-1.17-245.el7.noarch 1/3
验证中 : percona-toolkit-3.2.1-1.el7.x86_64 2/3
验证中 : perl-Digest-MD5-2.52-3.el7.x86_64 3/3

已安装:
percona-toolkit.x86_64 0:3.2.1-1.el7

作为依赖被安装:
perl-Digest.noarch 0:1.17-245.el7 perl-Digest-MD5.x86_64 0:2.52-3.el7

完毕！
[root@10-31-1-119 src]#

2.3 percona-toolkit工具介绍

有的32个命令，可以分为7大类

工具类别 工具命令 工具作用 备注
开发类 pt-duplicate-key-checker 列出并删除重复的索引和外键
开发类 pt-online-schema-change 在线修改表结构
开发类 pt-query-advisor 分析查询语句，并给出建议，有bug 已废弃
开发类 pt-show-grants 规范化和打印权限
开发类 pt-upgrade 在多个服务器上执行查询，并比较不同
性能类 pt-index-usage 分析日志中索引使用情况，并出报告
性能类 pt-pmp 为查询结果跟踪，并汇总跟踪结果
性能类 pt-visual-explain 格式化执行计划
性能类 pt-table-usage 分析日志中查询并分析表使用情况
配置类 pt-config-diff 比较配置文件和参数
配置类 pt-mysql-summary 对mysql配置和status进行汇总
配置类 pt-variable-advisor 分析参数，并提出建议
监控类 pt-deadlock-logger 提取和记录mysql死锁信息
监控类 pt-fk-error-logger 提取和记录外键信息
监控类 pt-mext 并行查看status样本信息
监控类 pt-query-digest 分析查询日志，并产生报告 常用命令
监控类 pt-trend 按照时间段读取slow日志信息 已废弃
复制类 pt-heartbeat 监控mysql复制延迟
复制类 pt-slave-delay 设定从落后主的时间
复制类 pt-slave-find 查找和打印所有mysql复制层级关系
复制类 pt-slave-restart 监控salve错误，并尝试重启salve
复制类 pt-table-checksum 校验主从复制一致性
复制类 pt-table-sync 高效同步表数据
系统类 pt-diskstats 查看系统磁盘状态
系统类 pt-fifo-split 模拟切割文件并输出
系统类 pt-summary 收集和显示系统概况
系统类 pt-stalk 出现问题时，收集诊断数据
系统类 pt-sift 浏览由pt-stalk创建的文件
系统类 pt-ioprofile 查询进程IO并打印一个IO活动表
实用类 pt-archiver 将表数据归档到另一个表或文件中
实用类 pt-find 查找表并执行命令
实用类 pt-kill Kill掉符合条件的sql 常用命令
实用类 pt-align 对齐其他工具的输出
实用类 pt-fingerprint 将查询转成密文

2.3.1 pt-archiver(归档表)

# 重要参数
--limit 100 每次取100行数据用pt-archive处理
--txn-size 100 设置100行为一个事务提交一次，
--where 'id
mysql> select sysdate;
ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id: 14188
Current database: *** NONE ***

ERROR 1054 (42S22): Unknown column 'sysdate' in 'field list'
mysql>
mysql>
-- session 2
[root@10-31-1-120 ~]# mysql -uroot -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 14180
Server version: 5.7.31-log MySQL Community Server (GPL)

Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>
mysql>
mysql>
mysql> use test;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> show tables;
+----------------+
| Tables_in_test |
+----------------+
| deadlocks |
| t1 |
+----------------+
2 rows in set (0.00 sec)

mysql> begin;
Query OK, 0 rows affected (0.00 sec)

mysql> select * from t1;
+------+------+
| id | name |
+------+------+
| 1 | a |
| 2 | b |
| 3 | c |
+------+------+
3 rows in set (0.00 sec)

mysql> select current_timestamp();
ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id: 14189
Current database: test

+---------------------+
| current_timestamp() |
+---------------------+
| 2020-08-31 10:35:54 |
+---------------------+
1 row in set (0.00 sec)

可以看到，未完成的事务因为空闲5秒，也被kill了，生产环境需谨慎

2.3.7.2 kill查询时间超过20秒的进程

数据准备

mysql> select count(*) from t1;
+----------+
| count(*) |
+----------+
| 5889738 |
+----------+
1 row in set (30.54 sec)

kill 语句准备

/usr/bin/pt-kill --user=lepus --password=lepus --host=10.31.1.112 --busy-time 20 --match-info "SELECT|select" --victim all --interval 5 --kill --daemonize -S --pid=/tmp/ptkill.pid --print --log=/tmp/pt-kill.log &

测试记录

[root@10-31-1-119 ~]# /usr/bin/pt-kill --user=lepus --password=lepus --host=10.31.1.112 --busy-time 20 --match-info "SELECT|select" --victim all --interval 5 --kill --daemonize -S --pid=/tmp/ptkill.pid --print --log=/tmp/pt-kill.log &
[2] 15172
[root@10-31-1-119 ~]#
[2]+ 完成 /usr/bin/pt-kill --user=lepus --password=lepus --host=10.31.1.112 --busy-time 20 --match-info "SELECT|select" --victim all --interval 5 --kill --daemonize -S --pid=/tmp/ptkill.pid --print --log=/tmp/pt-kill.log
[root@10-31-1-119 ~]#
[root@10-31-1-119 ~]# ps -ef | grep pt-kill
root 15173 1 0 10:51 ? 00:00:00 perl /usr/bin/pt-kill --user=lepus --password=lepus --host=10.31.1.112 --busy-time 20 --match-info SELECT|select --victim all --interval 5 --kill --daemonize -S --pid=/tmp/ptkill.pid --print --log=/tmp/pt-kill.log
root 15208 31644 0 10:52 pts/1 00:00:00 grep --color=auto pt-kill
[root@10-31-1-119 ~]#

mysql> select count(*) from t1 order by name;
ERROR 2013 (HY000): Lost connection to MySQL server during query
mysql>

2.3.7.3 Kill掉 select IFNULl.*语句开头的SQL

kill语句

pt-kill --user=lepus --password=lepus --host=10.31.1.112 --victims all --busy-time=0 --match-info="select IFNULl.*" --interval 1 --kill --daemonize --pid=/tmp/ptkill.pid --print --log=/tmp/pt-kill.log &

测试记录

[root@10-31-1-119 ~]# pt-kill --user=lepus --password=lepus --host=10.31.1.112 --victims all --busy-time=0 --match-info="select IFNULl.*" --interval 1 --kill --daemonize --pid=/tmp/ptkill.pid --print --log=/tmp/pt-kill.log &
[2] 15609
[root@10-31-1-119 ~]# Overwriting PID file /tmp/ptkill.pid because the PID that it contains, 13171, is not running at /usr/bin/pt-kill line 2420.
Overwriting PID file /tmp/ptkill.pid because the PID that it contains, 13171, is not running at /usr/bin/pt-kill line 2420.

[2]+ 完成 pt-kill --user=lepus --password=lepus --host=10.31.1.112 --victims all --busy-time=0 --match-info="select IFNULl.*" --interval 1 --kill --daemonize --pid=/tmp/ptkill.pid --print --log=/tmp/pt-kill.log

mysql> select IFNULl(id,'NULL'),name from t1;
ERROR 2013 (HY000): Lost connection to MySQL server during query
mysql>

查看日志

[root@10-31-1-119 ~]# more /tmp/pt-kill.log
*******************************************************************
Using the default of SSL_verify_mode of SSL_VERIFY_NONE for client
is deprecated! Please set SSL_verify_mode to SSL_VERIFY_PEER
possibly with SSL_ca_file|SSL_ca_path for verification.
If you really don't want to verify the certificate and keep the
connection open to Man-In-The-Middle attacks please set
SSL_verify_mode explicitly to SSL_VERIFY_NONE in your application.
*******************************************************************
at /usr/bin/pt-kill line 5761.
# 2020-08-31T10:35:03 KILL 14177 (Sleep 233 sec) NULL
# 2020-08-31T10:35:03 KILL 14180 (Sleep 167 sec) NULL
# 2020-08-31T10:35:48 KILL 14188 (Sleep 9 sec) NULL
# 2020-08-31T10:36:03 KILL 14189 (Sleep 9 sec) NULL
*******************************************************************
Using the default of SSL_verify_mode of SSL_VERIFY_NONE for client
is deprecated! Please set SSL_verify_mode to SSL_VERIFY_PEER
possibly with SSL_ca_file|SSL_ca_path for verification.
If you really don't want to verify the certificate and keep the
connection open to Man-In-The-Middle attacks please set
SSL_verify_mode explicitly to SSL_VERIFY_NONE in your application.
*******************************************************************
at /usr/bin/pt-kill line 5761.
*******************************************************************
Using the default of SSL_verify_mode of SSL_VERIFY_NONE for client
is deprecated! Please set SSL_verify_mode to SSL_VERIFY_PEER
possibly with SSL_ca_file|SSL_ca_path for verification.
If you really don't want to verify the certificate and keep the
connection open to Man-In-The-Middle attacks please set
SSL_verify_mode explicitly to SSL_VERIFY_NONE in your application.
*******************************************************************
at /usr/bin/pt-kill line 5761.

# A software update is available:
# 2020-08-31T10:53:06 KILL 10663 (Query 22 sec) select count(*) from t1 order by name
# 2020-08-31T10:56:29 KILL 10680 (Query 0 sec) select IFNULl(id,'NULL'),name from t1
[root@10-31-1-119 ~]#

参考:

1.https://www.cnblogs.com/zishengY/p/6852280.html
2.https://www.jianshu.com/p/36ace5c2bc8b
3.https://www.cnblogs.com/zhoujinyi/p/3392800.html

转载自：https://www.modb.pro/db/31907

大树
数据运维
0 0

相关文章

Oracle如何使用授予和撤销权限的语法和示例
Awesome Project: 探索 MatrixOrigin 云原生分布式数据库
下载丨66页PDF，云和恩墨技术通讯（2024年7月刊）
社区版oceanbase安装
Oracle 导出CSV工具-sqluldr2
ETL数据集成丨快速将MySQL数据迁移至Doris数据库

发布评论

大树

这个人很懒，什么都没有留下～

文章8568 评论0