sysbench1.0.14压力测试对比

2023年 7月 15日 69.3k 0

sysbench是基于LuaJIT的可脚本化多线程基准测试工具。它最常用于数据库基准测试,但也可用于创建不涉及数据库服务器的任意复杂工作负载。

  • sysbench附带了以下功能:oltp_*.lua:类似OLTP的数据库基准测试的集合fileio:文件系统级基准测试cpu: 简单的CPU基准测试memory:内存访问基准测试threads:基于线程的调度程序基准测试mutex:一个POSIX互斥体基准测试
  • 特点:可以统计速率和延迟,包括延迟百分点数形成简单的排序即使有数千个并发,开销也很低。sysbench能够每秒生成,跟踪数以万计的事件通过lua脚本可以实现预定义的测试条件,能够轻松完成新的基础测试

更多请访问:https://github.com/akopytov/sysbench

1,安装

curl -s https://packagecloud.io/install/repositories/akopytov/sysbench/script.rpm.sh | sudo bash
sudo yum -y install sysbench

2,zabbix-agent安装

curl -Lk https://raw.githubusercontent.com/LinuxEA-Mark/zabbix3.0.2-complete-works/master/zabbix-install/zabbix-install.sh|bash -s load 10.10.240.114

授权用户监控

GRANT SELECT ON *.* TO 'zabbix'@'127.0.0.1' IDENTIFIED BY 'password';

下载监控脚本(zabbix模板在github上)

curl -Lk https://raw.githubusercontent.com/LinuxEA-Mark/zabbix3.0.2-complete-works/master/mysql_QPTS/mariadb/install_db_status.sh|bash
  • 提供的lua脚本

    [root@DS-VM-Node49 ~]# ll /usr/local/sysbench/share/sysbench/
    总用量 60
    -rwxr-xr-x 1 root root  1446 5月  21 14:45 bulk_insert.lua
    -rw-r--r-- 1 root root 14468 5月  21 14:45 oltp_common.lua
    -rwxr-xr-x 1 root root  1290 5月  21 14:45 oltp_delete.lua
    -rwxr-xr-x 1 root root  2415 5月  21 14:45 oltp_insert.lua
    -rwxr-xr-x 1 root root  1265 5月  21 14:45 oltp_point_select.lua
    -rwxr-xr-x 1 root root  1649 5月  21 14:45 oltp_read_only.lua
    -rwxr-xr-x 1 root root  1824 5月  21 14:45 oltp_read_write.lua
    -rwxr-xr-x 1 root root  1118 5月  21 14:45 oltp_update_index.lua
    -rwxr-xr-x 1 root root  1127 5月  21 14:45 oltp_update_non_index.lua
    -rwxr-xr-x 1 root root  1440 5月  21 14:45 oltp_write_only.lua
    -rwxr-xr-x 1 root root  1933 5月  21 14:45 select_random_points.lua
    -rwxr-xr-x 1 root root  2132 5月  21 14:45 select_random_ranges.lua

    3,创建一个表

    mysql> create database db_20180528;
    Query OK, 1 row affected (0.00 sec)

    4,授权压测用户权限

    GRANT all  ON db_20180528.* TO 'mark'@'%' identified by 'password';  

    5,压测的参数

    sysbench /usr/share/sysbench/oltp_read_write.lua 
    --mysql-host=10.10.240.145   # IP 
    --mysql-port=3336            # 端口
    --mysql-user=mark            # 用户名
    --mysql-password=password    # 密码
    --mysql-db=db_20180528       # 库名
    --db-driver=mysql            # 数据库类型
    --tables=10                     # 生成 10 个测试表
    --table-size=3000000            # 每个测试表填充数据量为 100000 
    --report-interval=10         # 定期以秒为单位报告具有指定间隔的中间统计信息 0禁用中间报告[0]
    --threads=250                # 线程数
    --time=60                       # 执行时长为 120秒
    prepare

    其中run执行真正的压测,cleanup用来清除数据和表,一般先用prepare构建表结构,如下:

    sysbench /usr/share/sysbench/oltp_read_write.lua 
    --mysql-host=10.10.240.145 
    --mysql-port=3336 
    --mysql-user=mark 
    --mysql-password=password 
    --mysql-db=db20180528_3336 
    --db-driver=mysql 
    --tables=10 
    --table-size=1000000 
    --report-interval=10 
    --threads=80 
    --time=60 
    run

    实际prepare的表结构可能是这样的:

    MariaDB [sbtest]>  desc sbtest.sbtest1;
    +-------+------------------+------+-----+---------+----------------+
    | Field | Type             | Null | Key | Default | Extra          |
    +-------+------------------+------+-----+---------+----------------+
    | id    | int(10) unsigned | NO   | PRI | NULL    | auto_increment |
    | k     | int(10) unsigned | NO   | MUL | 0       |                |
    | c     | char(120)        | NO   |     |         |                |
    | pad   | char(60)         | NO   |     |         |                |
  • rows in set (0.00 sec)

    上面的测试命令代表的是:对mysql进行oltp基准测试,表数量10,每表行数约10w(几乎delete多少就会insert的多少),并且是非事务的只读测试,持续60s,并发线程数80个。

6,负载均衡只读压测

需要说明的选项:我们可以把threads依次递增(成倍数的递加),或者调整my.cnf参数,比较效果。另外需要注意的是,大部分mysql中间件对事务的处理,默认都是把sql发到主库执行,所以只读测试需要加上skip-trx=on来跳过测试中的显式事务。

请注意:我们使用maxscale的Readconnroute路由负载进行测试,集群MariaDB Galera Cluster

sysbench /usr/share/sysbench/oltp_read_only.lua 
--mysql-host=10.10.240.145 
--mysql-port=3336 
--mysql-user=mark 
--mysql-password=password 
--mysql-db=db20180528_3336 
--db-driver=mysql 
--tables=10 
--table-size=1000000 
--report-interval=10 
--threads=80 
--time=60 
--skip-trx=on 
run

注意:先使用prepare,完成后使用run,清楚数据使用cleanup,prepare一次可以多次重复使用结果如下:

[ 10s ] thds: 80 tps: 2050.04 qps: 28756.83 (r/w/o: 28756.83/0.00/0.00) lat (ms,95%): 112.67 err/s: 0.00 reconn/s: 0.00
[ 20s ] thds: 80 tps: 2322.46 qps: 32507.76 (r/w/o: 32507.76/0.00/0.00) lat (ms,95%): 121.08 err/s: 0.00 reconn/s: 0.00
[ 30s ] thds: 80 tps: 2841.30 qps: 39772.25 (r/w/o: 39772.25/0.00/0.00) lat (ms,95%): 74.46 err/s: 0.00 reconn/s: 0.00
[ 40s ] thds: 80 tps: 3247.04 qps: 45465.87 (r/w/o: 45465.87/0.00/0.00) lat (ms,95%): 66.84 err/s: 0.00 reconn/s: 0.00
[ 50s ] thds: 80 tps: 3294.92 qps: 46130.73 (r/w/o: 46130.73/0.00/0.00) lat (ms,95%): 69.29 err/s: 0.00 reconn/s: 0.00
[ 60s ] thds: 80 tps: 3324.43 qps: 46544.00 (r/w/o: 46544.00/0.00/0.00) lat (ms,95%): 71.83 err/s: 0.00 reconn/s: 0.00
SQL statistics:
    queries performed:
        read:                            2392432
        write:                           0
        other:                           0
        total:                           2392432
    transactions:                        170888 (2845.82 per sec.)
    queries:                             2392432 (39841.44 per sec.)
    ignored errors:                      0      (0.00 per sec.)
    reconnects:                          0      (0.00 per sec.)

General statistics:
    total time:                          60.0471s
    total number of events:              170888

Latency (ms):
         min:                                    6.66
         avg:                                   28.09
         max:                                 2301.15
         95th percentile:                       71.83
         sum:                              4801074.58

Threads fairness:
    events (avg/stddev):           2136.1000/730.42
    execution time (avg/stddev):   60.0134/0.01

在用sysbench压的时候,在mysql后端会话里有时看到大量的query cache lock,如果使用的是uniform取样,最好把查询缓存关掉。当然如果是做两组性能对比压测,因为都受这个因素影响,关心也不大。

负载均衡的监控图指标如下rorr.png

6,混合读写

混合读写我们使用maxscale的Readwritesplit读写分离进行测试,集群MariaDB Galera Cluster,逻辑主从如下:

Server changed state: server1[10.10.240.202:3306]: new_master. [Running] -> [Master, Synced, Running]
Server changed state: server2[10.10.240.203:3306]: new_slave. [Running] -> [Slave, Synced, Running]
Server changed state: server3[10.10.240.146:3306]: new_slave. [Running] -> [Slave, Synced, Running]

在1.0版本中混合读写oltp_read_write.lua这个脚本即可我们创建db20180528_3337库进行混合读写的测试

sysbench /usr/share/sysbench/oltp_read_write.lua 
--mysql-host=10.10.240.145 
--mysql-port=3337 
--mysql-user=mark 
--mysql-password=password 
--mysql-db=db20180528_3337 
--db-driver=mysql 
--tables=10 
--table-size=1000000 
--report-interval=10 
--threads=80 
--time=60 
--skip-trx=on 
run

注意:先使用prepare,完成后使用run,清楚数据使用cleanup,prepare一次可以多次重复使用注意:如果不加oltp-skip-trx=on,那么所有查询都会发往主库,但如果在有写入的情况下使用--oltp-skip-trx=on跳过BEGIN和COMMIT

结果如下:

[ 10s ] thds: 80 tps: 132.06 qps: 2495.40 (r/w/o: 1957.85/267.63/269.92) lat (ms,95%): 1506.29 err/s: 0.00 reconn/s: 0.00
[ 20s ] thds: 80 tps: 161.21 qps: 2903.16 (r/w/o: 2258.92/325.62/318.62) lat (ms,95%): 1561.52 err/s: 0.00 reconn/s: 0.00
[ 30s ] thds: 80 tps: 163.90 qps: 2948.16 (r/w/o: 2293.07/339.90/315.20) lat (ms,95%): 1836.24 err/s: 0.00 reconn/s: 0.00
[ 40s ] thds: 80 tps: 191.70 qps: 3450.23 (r/w/o: 2685.83/397.60/366.80) lat (ms,95%): 1401.61 err/s: 0.00 reconn/s: 0.00
[ 50s ] thds: 80 tps: 202.30 qps: 3640.71 (r/w/o: 2832.33/431.59/376.79) lat (ms,95%): 1479.41 err/s: 0.00 reconn/s: 0.00
[ 60s ] thds: 80 tps: 242.71 qps: 4367.04 (r/w/o: 3394.31/513.12/459.61) lat (ms,95%): 1304.21 err/s: 0.00 reconn/s: 0.00
SQL statistics:
    queries performed:
        read:                            154266
        write:                           22895
        other:                           21181
        total:                           198342
    transactions:                        11019  (182.90 per sec.)
    queries:                             198342 (3292.19 per sec.)
    ignored errors:                      0      (0.00 per sec.)
    reconnects:                          0      (0.00 per sec.)

General statistics:
    total time:                          60.2444s
    total number of events:              11019

Latency (ms):
         min:                                    9.71
         avg:                                  436.60
         max:                                 4126.11
         95th percentile:                     1506.29
         sum:                              4810859.89

Threads fairness:
    events (avg/stddev):           137.7375/15.14
    execution time (avg/stddev):   60.1357/0.06

监控如下:混合读写.png试试update

sysbench /usr/share/sysbench/oltp_update_index.lua 
--mysql-host=10.10.240.145 
--mysql-port=3337 
--mysql-user=mark 
--mysql-password=password 
--mysql-db=db20180528_3337 
--db-driver=mysql 
--tables=10 
--table-size=1000000 
--report-interval=10 
--threads=80 
--time=60 
--skip-trx=on 

注意:先使用prepare,完成后使用run,清楚数据使用cleanup,prepare一次可以多次重复使用

生成报告执行结果

sysbench 1.0.14 (using bundled LuaJIT 2.1.0-beta2)

Running the test with following options:
Number of threads: 80
Report intermediate results every 10 second(s)
Initializing random number generator from current time


Initializing worker threads...

Threads started!
[ 10s ] thds: 80 tps: 1249.32 qps: 1249.32 (r/w/o: 0.00/482.19/767.13) lat (ms,95%): 320.17 err/s: 0.00 reconn/s: 0.00
[ 20s ] thds: 80 tps: 1672.80 qps: 1672.80 (r/w/o: 0.00/646.40/1026.40) lat (ms,95%): 253.35 err/s: 0.00 reconn/s: 0.00
[ 30s ] thds: 80 tps: 1909.94 qps: 1909.94 (r/w/o: 0.00/740.21/1169.72) lat (ms,95%): 227.40 err/s: 0.00 reconn/s: 0.00
[ 40s ] thds: 80 tps: 2265.18 qps: 2265.18 (r/w/o: 0.00/868.45/1396.72) lat (ms,95%): 186.54 err/s: 0.00 reconn/s: 0.00
[ 50s ] thds: 80 tps: 2587.02 qps: 2587.02 (r/w/o: 0.00/986.64/1600.37) lat (ms,95%): 186.54 err/s: 0.00 reconn/s: 0.00
[ 60s ] thds: 80 tps: 2562.74 qps: 2562.74 (r/w/o: 0.00/990.11/1572.62) lat (ms,95%): 153.02 err/s: 0.00 reconn/s: 0.00
SQL statistics:
    queries performed:
        read:                            0      //总select数量
        write:                           47181  //总update、insert、delete语句数量
        other:                           75371  //commit、unlock tables以及其他mutex的数量
        total:                           122552
    transactions:                        122552 (2040.15 per sec.) //通常需要关注的数字(TPS)
    queries:                             122552 (2040.15 per sec.)
    ignored errors:                      0      (0.00 per sec.) //忽略的错误数
    reconnects:                          0      (0.00 per sec.)

General statistics:
    total time:                          60.0684s  //即max-time指定的压测实际
    total number of events:              122552    //总的事件数,一般与transactions相同

Latency (ms):
         min:                                    0.34
         avg:                                   39.19  //95%的语句的平均响应时间
         max:                                 2665.16
         95th percentile:                      215.44
         sum:                              4802756.15

Threads fairness:
    events (avg/stddev):           1531.9000/148.18
    execution time (avg/stddev):   60.0345/0.02

updater监控如下:混合写.png

sysbench还可以对文件系统IO测试,CPU性能测试,以及内存分配与传输速度测试这里并不做介绍,sysbench可以作为硬件参数的一个测试对比。至于吞吐连和读写tps需要自己计算

相关文章

Oracle如何使用授予和撤销权限的语法和示例
Awesome Project: 探索 MatrixOrigin 云原生分布式数据库
下载丨66页PDF,云和恩墨技术通讯(2024年7月刊)
社区版oceanbase安装
Oracle 导出CSV工具-sqluldr2
ETL数据集成丨快速将MySQL数据迁移至Doris数据库

发布评论