mysqlutilities对比两个库数据一致性

2024年 5月 28日 50.6k 0

1.安装mysql-utilities

首先yum源安装python, 之后根据python版本下载安装mysql-connector-python
yum install python
python --version
python 2.6.6

下载地址:https://downloads.mysql.com/archives/c-python/
rpm -ivh mysql-connector-python-2.1.6-1.el6.x86_64.rpm
which python

之后安装 mysql-utilities
下载地址:https://downloads.mysql.com/archives/utilities/
[root@redhat1 ~]# rpm -ivh mysql-utilities-1.6.5-1.el6.noarch.rpm
warning: mysql-utilities-1.6.5-1.el6.noarch.rpm: Header V3 DSA/SHA1 Signature, key ID 5072e1f5: NOKEY
Preparing… ########################################### [100%]
1:mysql-utilities ########################################### [100%]
mysql-utilities安装后会有2个命令工具:mysqldbcompare、mysqldiff

[root@redhat1 bin]# which mysqldbcompare
/usr/bin/mysqldbcompare

2.Mysqldbcompare工具

2.1mysqldbcompare的语法如下:

$ mysqldbcompare --server1=user:pass@host:port:socket --server2=user:pass@host:port:socket db1:db2
以上参数中:

--server1:MySQL服务器1配置。
--server2:MySQL服务器2配置。如果是同一服务器,--server2可以省略。
db1:db2:要比较的两个数据库。如果比较不同服务器上的同名数据库,可以省略:db2。
--all:比较所有两服务器上所有的同名数据库。--exclude排除无需比较的数据库。
--run-all-tests:运行完整比较,遇到第一次差异时不停止。
--changes-for=:修改对象。例如--changes-for=server2,那么对比以sever1为主,生成的差异的修改也是针对server2的对象的修改。
-d DIFFTYPE,--difftype=DIFFTYPE:差异的信息显示的方式,有[unified|context|differ|sql],默认是unified。如果使用sql,那么就直接生成差异的SQL,这样非常方便。
--show-reverse:在生成的差异修改里面,同时会包含server2和server1的修改。
--skip-table-options:保持表的选项不变,即对比的差异里面不包括表名、AUTO_INCREMENT、ENGINE、CHARSET等差异。
--skip-diff:跳过对象定义比较检查。所谓对象定义,就是CREATE语句()里面的部分,--skip-table-options是()外面的部分。
--skip-object-compare:默认情况下,先检查两个数据库中相互缺失的对象,再对都存在对象间的差异。这个参数的作用就是,跳过第一步,不检查相互缺失的对象。
--skip-checksum-table:数据一致性验证时跳过CHECKSUM TABLE。
--skip-data-check:跳过数据一致性验证。
--skip-row-count:跳过字段数量检查。
使用mysqldbcompare找出差异化数据

2.2可对比的内容:

可对比出2个库中的对象个数是否一致;
可对比出表数据量区别,有主键的表可列出差异的数据,无主键表只提示数据量不一致。
可对比出有主键的表的实际数据的区别,有主键的表一行记录中某非主键字段值不一致会列出。
无法对比出非主键表数据量一致,某行记录数据值不一致 。

2.3个人测试:

以server2为准,看server1的区别在哪:
/usr/bin/mysqldbcompare --server1=root:123456@192.168.56.101:3306 --server2=root:123456@192.168.56.101:3306 --run-all-test --changes-for=server1 --difftype=sql test:test1

以server1 为准,看server2的区别在哪,列出server2的修改sql:
/usr/bin/mysqldbcompare --server1=root:123456@192.168.56.101:3306 --server2=root:123456@192.168.56.101:3306 --run-all-test --changes-for=server2 --difftype=sql test:test1

2.3.1一开始 test2,test3,test4表不存在于test1库。 mn2,mn1 没有主键和自增。

[root@redhat1 bin]# /usr/bin/mysqldbcompare --server1=root:123456@192.168.56.101:3306 --server2=root:123456@192.168.56.101:3306 --run-all-test --changes-for=server2 --difftype=sql test:test1
# WARNING: Using a password on the command line interface can be insecure.
# server1 on 192.168.56.101: ... connected.
# server2 on 192.168.56.101: ... connected.
# Checking databases test on server1 and test1 on server2
#
# WARNING: Objects in server1.test but not in server1.test1:
# TABLE: test4
# TABLE: test3
# TABLE: test2
#
# Defn Row Data
# Type Object Name Diff Count Check
# -------------------------------------------------------------------------
# TABLE mn1 FAIL pass -
# - Compare table checksum FAIL
# - Find row differences SKIP
#
# Transformation for --changes-for=server2:
#

ALTER TABLE `test1`.`mn1`
ADD PRIMARY KEY(`id`),
CHANGE COLUMN id id int(11) NOT NULL,
AUTO_INCREMENT=13;

# The table mn1 does not have an usable Index or primary key.

# TABLE mn2 FAIL pass -
# - Compare table checksum FAIL
# - Find row differences SKIP
#
# Transformation for --changes-for=server2:
#

ALTER TABLE `test1`.`mn2`
ADD PRIMARY KEY(`id`),
CHANGE COLUMN id id int(11) NOT NULL,
AUTO_INCREMENT=34;

# The table mn2 does not have an usable Index or primary key.

# Database consistency check failed.
#
# ...done

2.3.2所有表结构都一致,test2表在test1库中无数据。

[root@redhat1 ~]# /usr/bin/mysqldbcompare --server1=root:123456@192.168.56.101:3306 --server2=root:123456@192.168.56.101:3306 --run-all-test --changes-for=server2 --difftype=sql test:test1
# WARNING: Using a password on the command line interface can be insecure.
# server1 on 192.168.56.101: ... connected.
# server2 on 192.168.56.101: ... connected.
# Checking databases test on server1 and test1 on server2
#
# Defn Row Data
# Type Object Name Diff Count Check
# -------------------------------------------------------------------------
# TABLE mn1 pass pass -
# - Compare table checksum FAIL
# - Find row differences pass
# TABLE mn2 pass pass -
# - Compare table checksum FAIL
# - Find row differences pass
# TABLE test2 pass FAIL -
# - Compare table checksum FAIL
# - Find row differences SKIP
#
# Row counts are not the same among `test`.`test2` and `test1`.`test2`.
#
# The table test2 does not have an usable Index or primary key.

# TABLE test3 pass pass -
# - Compare table checksum FAIL
# - Find row differences SKIP
#
# The table test3 does not have an usable Index or primary key.

# TABLE test4 pass pass -
# - Compare table checksum FAIL
# - Find row differences SKIP
#
# The table test4 does not have an usable Index or primary key.

# Database consistency check failed.
#
# ...done
[root@redhat1 ~]#

2.3.3test,test1 库完全一致: 表无主键 数据量对比会显示skip 。

[root@redhat1 ~]# /usr/bin/mysqldbcompare --server1=root:123456@192.168.56.101:3306 --server2=root:123456@192.168.56.101:3306 --run-all-test --changes-for=server2 --difftype=sql test:test1
# WARNING: Using a password on the command line interface can be insecure.
# server1 on 192.168.56.101: ... connected.
# server2 on 192.168.56.101: ... connected.
# Checking databases test on server1 and test1 on server2
#
# Defn Row Data
# Type Object Name Diff Count Check
# -------------------------------------------------------------------------
# TABLE mn1 pass pass -
# - Compare table checksum FAIL
# - Find row differences pass
# TABLE mn2 pass pass -
# - Compare table checksum FAIL
# - Find row differences pass
# TABLE test2 pass pass -
# - Compare table checksum FAIL
# - Find row differences SKIP
#
# The table test2 does not have an usable Index or primary key.

# TABLE test3 pass pass -
# - Compare table checksum FAIL
# - Find row differences SKIP
#
# The table test3 does not have an usable Index or primary key.

# TABLE test4 pass pass -
# - Compare table checksum FAIL
# - Find row differences SKIP
#
# The table test4 does not have an usable Index or primary key.

# Database consistency check failed.
#
# ...done

2.3.4 test2,test4, mn1 表的数据量在test库都比test1库中多:

有主键的会详细列出具体的数据差异,没有主键的只提示数据量不一致

[root@redhat1 ~]# /usr/bin/mysqldbcompare --server1=root:123456@192.168.56.101:3306 --server2=root:123456@192.168.56.101:3306 --run-all-test --changes-for=server2 --difftype=sql test:test1
# WARNING: Using a password on the command line interface can be insecure.
# server1 on 192.168.56.101: ... connected.
# server2 on 192.168.56.101: ... connected.
# Checking databases test on server1 and test1 on server2
#
# Defn Row Data
# Type Object Name Diff Count Check
# -------------------------------------------------------------------------
# TABLE mn1 FAIL FAIL -
# - Compare table checksum FAIL
# - Find row differences FAIL
#
# Transformation for --changes-for=server2:
#

ALTER TABLE `test1`.`mn1`
AUTO_INCREMENT=18;

# Row counts are not the same among `test`.`mn1` and `test1`.`mn1`.
#
# Transformation for --changes-for=server2:
#

INSERT INTO `test1`.`mn1` (`id`, `name`, `address`) VALUES('16', 'dds', 'country4');
INSERT INTO `test1`.`mn1` (`id`, `name`, `address`) VALUES('17', 'dfdds', 'country4');
INSERT INTO `test1`.`mn1` (`id`, `name`, `address`) VALUES('15', 'fdds', 'country4');
INSERT INTO `test1`.`mn1` (`id`, `name`, `address`) VALUES('13', 'fd', 'country4');
INSERT INTO `test1`.`mn1` (`id`, `name`, `address`) VALUES('14', 'fds', 'country4');

# TABLE mn2 pass pass -
# - Compare table checksum FAIL
# - Find row differences pass
# TABLE test2 pass FAIL -
# - Compare table checksum FAIL
# - Find row differences SKIP
#
# Row counts are not the same among `test`.`test2` and `test1`.`test2`.
#
# The table test2 does not have an usable Index or primary key.

# TABLE test3 pass pass -
# - Compare table checksum FAIL
# - Find row differences SKIP
#
# The table test3 does not have an usable Index or primary key.

# TABLE test4 pass FAIL -
# - Compare table checksum FAIL
# - Find row differences SKIP
#
# Row counts are not the same among `test`.`test4` and `test1`.`test4`.
#
# The table test4 does not have an usable Index or primary key.

# Database consistency check failed.
#
# ...done

2.3.5所有表 数据量完全一致, mn1表一行记录 非主键字段值不一样:

[root@redhat1 ~]# /usr/bin/mysqldbcompare --server1=root:123456@192.168.56.101:3306 --server2=root:123456@192.168.56.101:3306 --run-all-test --changes-for=server2 --difftype=sql test:test1
# WARNING: Using a password on the command line interface can be insecure.
# server1 on 192.168.56.101: ... connected.
# server2 on 192.168.56.101: ... connected.
# Checking databases test on server1 and test1 on server2
#
# Defn Row Data
# Type Object Name Diff Count Check
# -------------------------------------------------------------------------
# TABLE mn1 pass pass -
# - Compare table checksum FAIL
# - Find row differences FAIL
#
# Transformation for --changes-for=server2:
#

# Data differences found among rows:
UPDATE `test1`.`mn1` SET `name` = 'dfdds' WHERE `id` = '18';

# TABLE mn2 pass pass -
# - Compare table checksum FAIL
# - Find row differences pass
# TABLE test2 pass pass -
# - Compare table checksum FAIL
# - Find row differences SKIP
#
# The table test2 does not have an usable Index or primary key.

# TABLE test3 pass pass -
# - Compare table checksum FAIL
# - Find row differences SKIP
#
# The table test3 does not have an usable Index or primary key.

# TABLE test4 pass pass -
# - Compare table checksum FAIL
# - Find row differences SKIP
#
# The table test4 does not have an usable Index or primary key.

# Database consistency check failed.
#
# ...done

2.3.6无主键表test3 有一行记录值不一致: 非主键表数据量一致,某行记录数据值不一致 无法对比出。

[root@redhat1 ~]# /usr/bin/mysqldbcompare --server1=root:123456@192.168.56.101:3306 --server2=root:123456@192.168.56.101:3306 --run-all-test --changes-for=server2 --difftype=sql test:test1
# WARNING: Using a password on the command line interface can be insecure.
# server1 on 192.168.56.101: ... connected.
# server2 on 192.168.56.101: ... connected.
# Checking databases test on server1 and test1 on server2
#
# Defn Row Data
# Type Object Name Diff Count Check
# -------------------------------------------------------------------------
# TABLE mn1 pass pass -
# - Compare table checksum FAIL
# - Find row differences FAIL
#
# Transformation for --changes-for=server2:
#

# Data differences found among rows:
UPDATE `test1`.`mn1` SET `name` = 'dfdds' WHERE `id` = '18';

# TABLE mn2 pass pass -
# - Compare table checksum FAIL
# - Find row differences pass
# TABLE test2 pass pass -
# - Compare table checksum FAIL
# - Find row differences SKIP
#
# The table test2 does not have an usable Index or primary key.

# TABLE test3 pass pass -
# - Compare table checksum FAIL
# - Find row differences SKIP
#
# The table test3 does not have an usable Index or primary key.

# Database consistency check failed.
#
# ...done

3.mysqldiff工具:

对比表对象不一致工具

mysqldiff --server1=user:pass@host:port:socket --server2=user:pass@host:port:socket db1.object1:db2.object1 db3:db4
$ mysqldbcompare --server1=user:pass@host:port:socket --server2=user:pass@host:port:socket db1:db2

3.1mysqldiff语法有两个用法:

db1:db2:如果只指定数据库,那么就将两个数据库中互相缺少的对象显示出来,不比较对象里面的差异。这里的对象包括表、存储过程、函数、触发器等。
db1.object1:db2.object1:如果指定了具体表对象,那么就会详细对比两个表的差异,包括表名、字段名、备注、索引、大小写等所有的表相关的对象。
接下来看一些主要的参数:

--server1:配置server1的连接。
--server2:配置server2的连接。
--character-set:配置连接时用的字符集,如果不显示配置默认使用character_set_client。
--width:配置显示的宽度。
--skip-table-options:保持表的选项不变,即对比的差异里面不包括表名、AUTO_INCREMENT、ENGINE、CHARSET等差异。
-d DIFFTYPE,--difftype=DIFFTYPE:差异的信息显示的方式,有[unified|context|differ|sql],默认是unified。如果使用sql,那么就直接生成差异的SQL,这样非常方便。
--changes-for=:修改对象。例如--changes-for=server2,那么对比以sever1为主,生成的差异的修改也是针对server2的对象的修改。
--show-reverse:在生成的差异修改里面,同时会包含server2和server1的修改。

/usr/bin/mysqldiff --server1=root:123456@192.168.56.101:3306 --server2=root:123456@192.168.56.101:3306 --changes-for=server2 --difftype=sql test.test4:test1.test4

3.2可对比的内容:

可对比出2个库中的对象个数是否一致;
可对比出2个库中表对象的结构是否一致(多列少列,字段长度,表字符集);
无法对比2个库中的表的数据量;

3.3个人测试

3.3.1一开始2个库数据量完全一致,mn1表一行记录 非主键字段值不一样:

无法对比出 非主键值不一样的数据

[root@redhat1 ~]# /usr/bin/mysqldiff --server1=root:123456@192.168.56.101:3306 --server2=root:123456@192.168.56.101:3306 --changes-for=server2 --difftype=sql test.test4:test1.test4
# WARNING: Using a password on the command line interface can be insecure.
# server1 on 192.168.56.101: ... connected.
# server2 on 192.168.56.101: ... connected.
# Comparing test.test4 to test1.test4 [PASS]
# Success. All objects are the same.
[root@redhat1 ~]#
[root@redhat1 ~]# /usr/bin/mysqldiff --server1=root:123456@192.168.56.101:3306 --server2=root:123456@192.168.56.101:3306 --changes-for=server2 --difftype=sql test:test1
# WARNING: Using a password on the command line interface can be insecure.
# server1 on 192.168.56.101: ... connected.
# server2 on 192.168.56.101: ... connected.
# Comparing `test` to `test1` [PASS]
# Comparing `test`.`mn1` to `test1`.`mn1` [PASS]
# Comparing `test`.`mn2` to `test1`.`mn2` [PASS]
# Comparing `test`.`test2` to `test1`.`test2` [PASS]
# Comparing `test`.`test3` to `test1`.`test3` [PASS]
# Comparing `test`.`test4` to `test1`.`test4` [PASS]
# Success. All objects are the same.

3.3.2mn1表一行记录 非主键字段值不一样,test.test4表数据量比test1库多:

无法对比数据量差异

[root@redhat1 ~]# /usr/bin/mysqldiff --server1=root:123456@192.168.56.101:3306 --server2=root:123456@192.168.56.101:3306 --changes-for=server2 --difftype=sql test:test1
# WARNING: Using a password on the command line interface can be insecure.
# server1 on 192.168.56.101: ... connected.
# server2 on 192.168.56.101: ... connected.
# Comparing `test` to `test1` [PASS]
# Comparing `test`.`mn1` to `test1`.`mn1` [PASS]
# Comparing `test`.`mn2` to `test1`.`mn2` [PASS]
# Comparing `test`.`test2` to `test1`.`test2` [PASS]
# Comparing `test`.`test3` to `test1`.`test3` [PASS]
# Comparing `test`.`test4` to `test1`.`test4` [PASS]
# Success. All objects are the same.

3.3.3mn1表一行记录 非主键字段值不一样,test.test4 id int(15),test1.test4 id int(11):

能对比出字段长度不一致

[root@redhat1 ~]# /usr/bin/mysqldiff --server1=root:123456@192.168.56.101:3306 --server2=root:123456@192.168.56.101:3306 --changes-for=server2 --difftype=sql test:test1
# WARNING: Using a password on the command line interface can be insecure.
# server1 on 192.168.56.101: ... connected.
# server2 on 192.168.56.101: ... connected.
# Comparing `test` to `test1` [PASS]
# Comparing `test`.`mn1` to `test1`.`mn1` [PASS]
# Comparing `test`.`mn2` to `test1`.`mn2` [PASS]
# Comparing `test`.`test2` to `test1`.`test2` [PASS]
# Comparing `test`.`test3` to `test1`.`test3` [PASS]
# Comparing `test`.`test4` to `test1`.`test4` [FAIL]
# Transformation for --changes-for=server2:
#

ALTER TABLE `test1`.`test4`
CHANGE COLUMN id id int(15) NULL;

# Compare failed. One or more differences found.
[root@redhat1 ~]#

3.3.4test4 字符集不一致, test.test4=utf8 , test1.test4=gbk :

可以对比出字符集区别

[root@redhat1 ~]# /usr/bin/mysqldiff --server1=root:123456@192.168.56.101:3306 --server2=root:123456@192.168.56.101:3306 --changes-for=server2 --difftype=sql test:test1
# WARNING: Using a password on the command line interface can be insecure.
# server1 on 192.168.56.101: ... connected.
# server2 on 192.168.56.101: ... connected.
# Comparing `test` to `test1` [PASS]
# Comparing `test`.`mn1` to `test1`.`mn1` [PASS]
# Comparing `test`.`mn2` to `test1`.`mn2` [PASS]
# Comparing `test`.`test2` to `test1`.`test2` [PASS]
# Comparing `test`.`test3` to `test1`.`test3` [PASS]
# Comparing `test`.`test4` to `test1`.`test4` [FAIL]
# Transformation for --changes-for=server2:
#

ALTER TABLE `test1`.`test4`
COLLATE=utf8mb4_unicode_ci;
# Compare failed. One or more differences found.

3.3.5Test1库中缺少test4表:

[root@redhat1 ~]# /usr/bin/mysqldiff --server1=root:123456@192.168.56.101:3306 --server2=root:123456@192.168.56.101:3306 --changes-for=server2 --difftype=sql test:test1
# WARNING: Using a password on the command line interface can be insecure.
# server1 on 192.168.56.101: ... connected.
# server2 on 192.168.56.101: ... connected.
# WARNING: Objects in server1.test but not in server1.test1:
# TABLE: test4
# Compare failed. One or more differences found.

3.3.6test1.test4表多一列address列:

[root@redhat1 ~]# /usr/bin/mysqldiff --server1=root:123456@192.168.56.101:3306 --server2=root:123456@192.168.56.101:3306 --changes-for=server2 --difftype=sql test:test1
# WARNING: Using a password on the command line interface can be insecure.
# server1 on 192.168.56.101: ... connected.
# server2 on 192.168.56.101: ... connected.
# Comparing `test` to `test1` [PASS]
# Comparing `test`.`mn1` to `test1`.`mn1` [PASS]
# Comparing `test`.`mn2` to `test1`.`mn2` [PASS]
# Comparing `test`.`test2` to `test1`.`test2` [PASS]
# Comparing `test`.`test3` to `test1`.`test3` [PASS]
# Comparing `test`.`test4` to `test1`.`test4` [FAIL]
# Transformation for --changes-for=server2:
#

ALTER TABLE `test1`.`test4`
DROP COLUMN address;

# Compare failed. One or more differences found.

4.0 总结

mysql-utilities应对mysql5.6 5.7版本使用。但还是用的很少。最后更新时间是2018年6月份。到8.0版本基本被抛弃。有一些情况也可以很好的使用起来。

相关文章

Oracle如何使用授予和撤销权限的语法和示例
Awesome Project: 探索 MatrixOrigin 云原生分布式数据库
下载丨66页PDF,云和恩墨技术通讯(2024年7月刊)
社区版oceanbase安装
Oracle 导出CSV工具-sqluldr2
ETL数据集成丨快速将MySQL数据迁移至Doris数据库

发布评论