手把手教你安装 MySQL InnoDB Cluster 8.0.34

MySQL InnoDB Cluster 提供完整的高可用性 MySQL 解决方案。通过使用 MySQL Shell 附带的 AdminAPI，您可以轻松配置和管理至少三个 MySQL 服务器的组实例以充当 InnoDB 集群。

InnoDB Cluster 主要使用如下三大主件：
● MySQL Shell
● MySQL服务器和组复制
● MySQL Router

InnoDB 集群中的每个 MySQL 服务器实例都运行 MySQL Group Replication，它提供了在 InnoDB 集群内复制数据的机制，并具有内置的故障切换。Admin API 消除了在 InnoDB 集群中直接使用 Group Replication 的需要，但有关更多信息，请参阅解释详细信息的 Group Replication。从 MySQL 8.0.27 开始，您还可以设置 InnoDB ClusterSet，通过将主 InnoDB 集群与其他位置（如不同的数据中心）的一个或多个副本链接，为 InnoDB 集群部署提供容灾能力。

MySQL Router 可以根据您部署的集群自动配置，将客户端应用程序透明地连接到服务器实例。在服务器实例发生意外故障的情况下，群集会自动重新配置。在默认的单主模式中，InnoDB 集群有一个读写服务器实例-主实例。多个辅助服务器实例是主服务器的副本。如果主服务器出现故障，辅助服务器将自动升级为主服务器。MySQL Router 检测到这一点，并将客户端应用程序转发到新的主应用程序。

如下是官方 MySQL InnoDB Cluster 架构简图：

下图是我三台 Linux7 部署架构图，IP 分别为 56、57、58，56 Node1 为主节点，并安装 MySQL 和 router，57 Node2、58 Node3 为从节点,也安装 MySQL 和 router，并在 Node3 安装了 shell。Node3 的 shell 用来安装 InnoDB Cluster，其他每个节点的 router 地址则作为高可用地址分配给 K8S 应用容器。当然官方建议将 router 和应用容器一起部署，我这么部署和使用也没有太大的问题。

下面开始使用最传统的方式从零开始安装 MySQL InnoDB Cluster 8.0.34 单主模式，Linux X86 操作系统已安装好，并且 yum 源配置完成，1T /data 文件系统已挂载，所需软件包已下载并上传到服务器。也可参考去年的主从搭建文档----Linux 环境搭建 MySQL8.0.28 主从同步环境。

--所需的软件包
https://dev.mysql.com/downloads/mysql/8.0.html

mysql-8.0.34-linux-glibc2.12-x86_64.tar.xz
mysql-shell-8.0.34-1.el7.x86_64.rpm
mysql-router-8.0.34-linux-glibc2.12-x86_64.tar.xz


一、系统准备

以下 6 小节操作需要在三台主机执行。

1、centos7 关闭防火墙

systemctl stop firewalld.service 或者 systemctl stop firewalld
systemctl disable firewalld.service 或者 systemctl disable firewalld
systemctl status firewalld


2、关闭 selinux

getenforce
setenforce 0

SELINUX=disabled


3、绑定 /etc/hosts 解析(示例如下)

vim /etc/hosts

192.168.27.56 jiekexu-my-56
192.168.27.57 jiekexu-my-57
192.168.27.58 jiekexu-my-58


4、安装 yum 包

yum -y groupinstall "DeveLopment tools"
yum -y install ncurses ncurses-devel openssl-devel bison gcc gcc-c++ make


5、清理系统自带 MariaDB 环境

CentOS 7 版本的系统默认自带安装了 MariaDB，需要先清理。

## 查询已安装的mariadb
rpm -qa |grep mariadb
或
yum list installed | grep mariadb
## 卸载 mariadb 包，文件名为上述命令查询出来的文件
rpm -e --nodeps mariadb-libs-5.5.68-1.el7.x86_64
yum -y remove mariadb-libs.x86_64


6、内核参数修改

[root@jiekexu-my-56 ~]# cat>>/etc/sysctl.conf /etc/security/limits.conf /etc/pam.d/login /etc/profile select * from performance_schema.replication_group_members;
+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
| CHANNEL_NAME | MEMBER_ID | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION | MEMBER_COMMUNICATION_STACK |
+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
| group_replication_applier | b9f52d68-52e2-11ee-953c-fa163e0e1ce8 |192.168.27.56 | 33066 | ONLINE | PRIMARY | 8.0.34 | XCom |
+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
1 rows in set (0.00 sec)


4、其他节点加入组复制（其他节点执行）

Node2 和 Node3 执行

START GROUP_REPLICATION;


5、启动完成之后在看 Group 信息

SELECT * FROM performance_schema.replication_group_members;

mysql> select * from performance_schema.replication_group_members;
+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
| CHANNEL_NAME | MEMBER_ID | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION | MEMBER_COMMUNICATION_STACK |
+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
| group_replication_applier | b9f52d68-52e2-11ee-953c-fa163e0e1ce8 |192.168.27.56 | 33066 | ONLINE | PRIMARY | 8.0.34 | XCom |
| group_replication_applier | eaae5cec-52e2-11ee-a03c-fa163e0e1c84 |192.168.27.57 | 33066 | ONLINE | SECONDARY | 8.0.34 | XCom |
| group_replication_applier | eea965a9-52e2-11ee-8027-fa163e0e1c05 |192.168.27.58 | 33066 | ONLINE | SECONDARY | 8.0.34 | XCom |
+---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
3 rows in set (0.00 sec)


6、连接 Node1 主库创建 root@’%’ 用户

节点1

mysql -uroot -p -P 33066 -S /data/mysqldb/socket/mysql.sock

create user root@'%' identified by 'root@123';
grant all privileges on *.* to root@'%' with grant option;
flush privileges;


============================================
部署多实例需要添加如下两个参数(否则端口冲突)
mysqlx_port
mysqlx_socket

节点加入集群报错时，节点需要设置如下参数(使用默认密码插件时需要执行)

set global group_replication_recovery_get_public_key=on;

============================================

四、安装 MySQL Shell

初始化实例及创建纳管库，库名 mysql_innodb_cluster_metadata，只需要在一台中间件服务器安装即可，这里选择 Node3。

1、root 用户安装 MySQL Shell

# rpm -ivh mysql-shell-8.0.34-1.el7.x86_64.rpm

--这里下载的是 rpm 包，如果你下载的是二进制版本，可直接解压配置环境变量即可。
# tar -zxvf mysql-shell-8.0.34-linux-glibc2.12-x86-64bit.tar.gz
# ln -s mysql-shell-8.0.34-linux-glibc2.12-x86-64bit mysql-shell
# export PATH=$PATH:/app/mysql-shell/bin


2、连接数据库，添加纳管

在第一次连接，创建纳管时，需要从 primary 连接进去。否则会破坏 read_only（root 用户会破坏 super_read_only)。

mysqlsh root@192.168.27.56:33066 --py


添加纳管
dba.create_cluster(‘mysqlrouter’)

MySQL 192.168.27.56:33066 ssl Py > dba.create_cluster('mysqlrouter')
A new InnoDB Cluster will be created on instance '192.168.27.56:33066'.

3、查看集群状态

c=dba.get_cluster()
c.status()

MySQL 192.168.27.56:33066 ssl Py > c.status()
{
"clusterName": "mysqlrouter",
"defaultReplicaSet": {
"name": "default",
"primary": "192.168.27.56:33066",
"ssl": "DISABLED",
"status": "OK",
"statusText": "Cluster is ONLINE and can tolerate up to ONE failure.",
"topology": {
"192.168.27.56:33066": {
"address": "192.168.27.56:33066",
"memberRole": "PRIMARY",
"mode": "R/W",
"readReplicas": {},
"replicationLag": "applier_queue_applied",
"role": "HA",
"status": "ONLINE",
"version": "8.0.34"
},
"192.168.27.57:33066": {
"address": "192.168.27.57:33066",
"memberRole": "SECONDARY",
"mode": "R/O",
"readReplicas": {},
"replicationLag": "applier_queue_applied",
"role": "HA",
"status": "ONLINE",
"version": "8.0.34"
},
"192.168.27.58:33066": {
"address": "192.168.27.58:33066",
"memberRole": "SECONDARY",
"mode": "R/O",
"readReplicas": {},
"replicationLag": "applier_queue_applied",
"role": "HA",
"status": "ONLINE",
"version": "8.0.34"
}
},
"topologyMode": "Single-Primary"
},
"groupInformationSourceMember": "192.168.27.56:33066"
}
MySQL 192.168.27.56:33066 ssl Py >


4、MySQL Shell 的其他方法

dba.help()
help dba

MySQL Shell 工具集
help util


5、删除纳管

本次不需要执行，只要删除元数据数据库即可，从 primary 连接。

dba.drop_metadata_schema()


6、查看纳管数据库

mysql_innodb_cluster_metadata

mysql> show databases;
+-------------------------------+
| Database |
+-------------------------------+
| information_schema |
| mysql |
| mysql_innodb_cluster_metadata |
| performance_schema |
| sys |
+-------------------------------+
5 rows in set (0.00 sec)

Database changed
mysql> show tables;
+-----------------------------------------+
| Tables_in_mysql_innodb_cluster_metadata |
+-----------------------------------------+
| async_cluster_members |
| async_cluster_views |
| clusters |
| clusterset_members |
| clusterset_views |
| clustersets |
| instances |
| router_rest_accounts |
| routers |
| schema_version |
| v2_ar_clusters |
| v2_ar_members |
| v2_clusters |
| v2_cs_clustersets |
| v2_cs_members |
| v2_cs_router_options |
| v2_gr_clusters |
| v2_instances |
| v2_router_rest_accounts |
| v2_routers |
| v2_this_instance |
+-----------------------------------------+
21 rows in set (0.00 sec)


五、MySQL Router 安装配置

1、单独三节点安装 MySQL Router

在主节点服务器配置 mysql 用户执行

$ tar -xvf mysql-router-8.0.34-linux-glibc2.12-x86_64.tar.xz
$ mv mysql-router-8.0.34-linux-glibc2.12-x86_64 mysqlrouter
$ mkdir -p /data/mysqlrouter/mysqlrouter6446


/app/mysqlrouter/bin/mysqlrouter

2、初始化 MySQL Router

/app/mysqlrouter/bin/mysqlrouter --bootstrap root@192.168.27.56:33066 --directory /data/mysqlrouter/mysqlrouter6446 --conf-base-port 6446 --conf-use-sockets --force

(注意：--conf-base-port 6446 这个参数是 mysqlrouter 参数 默认是 6446)


[mysql@jiekexu-my-56 data]$ /app/mysqlrouter/bin/mysqlrouter --bootstrap root@192.168.27.56:33066 --directory /data/mysqlrouter/mysqlrouter6446 --conf-base-port 6446 --conf-use-sockets --force
Please enter MySQL password for root:
# Bootstrapping MySQL Router instance at '/data/mysqlrouter/mysqlrouter6446'...

- Read/Write Connections: jiekexu-my-56:6448, /data/mysqlrouter/mysqlrouter6446/mysqlx.sock
- Read/Only Connections: jiekexu-my-56:6449, /data/mysqlrouter/mysqlrouter6446/mysqlxro.sock
=====================================================================
--./mysqlrouter_passwd set /app/percona/mysqlrouter3306/mysqlrouter.pwd rdsroot 输入 rest 认证密码(curl 命令查询 MGR 集群状态 非必须)


3、启动 MySQL Router

[mysql@jiekexu-my-57 mysqlrouter6446]$ cd /data/mysqlrouter/mysqlrouter6446
[mysql@jiekexu-my-57 mysqlrouter6446]$ ./start.sh
[mysql@jiekexu-my-57 mysqlrouter6446]$ PID 26919 written to '/data/mysqlrouter/mysqlrouter6446/mysqlrouter.pid'
stopping to log to the console. Continuing to log to filelog

--除了上面脚本启动外，按提示中如下命令也可以启动：
/app/mysqlrouter/bin/mysqlrouter -c /data/mysqlrouter/mysqlrouter6446/mysqlrouter.conf


查看日志
cat mysqlrouter.log

[mysql@jiekexu-my-56 log]$ cat mysqlrouter.log
2023-09-14 18:28:12 io INFO [7ff61678e780] starting 16 io-threads, using backend 'linux_epoll'
2023-09-14 18:28:12 http_server INFO [7ff61678e780] listening on 0.0.0.0:8443
2023-09-14 18:28:12 metadata_cache_plugin INFO [7ff5d8ff9700] Starting Metadata Cache
2023-09-14 18:28:12 metadata_cache INFO [7ff5d8ff9700] Connections using ssl_mode 'PREFERRED'
2023-09-14 18:28:12 metadata_cache INFO [7ff616702700] Starting metadata cache refresh thread
2023-09-14 18:28:12 routing INFO [7ff5937fe700] [routing:bootstrap_x_rw] started: routing strategy = first-available
2023-09-14 18:28:12 routing INFO [7ff5937fe700] [routing:bootstrap_x_rw] started: listening using /data/mysqlrouter/mysqlrouter6446/mysqlx.sock
2023-09-14 18:28:12 routing INFO [7ff5937fe700] Start accepting connections for routing routing:bootstrap_x_rw listening on 6448 and named socket /data/mysqlrouter/mysqlrouter6446/mysqlx.sock
2023-09-14 18:28:12 routing INFO [7ff598ff9700] [routing:bootstrap_rw] started: routing strategy = first-available
2023-09-14 18:28:12 routing INFO [7ff5997fa700] [routing:bootstrap_ro] started: routing strategy = round-robin-with-fallback
2023-09-14 18:28:12 routing INFO [7ff593fff700] [routing:bootstrap_x_ro] started: routing strategy = round-robin-with-fallback
2023-09-14 18:28:12 routing INFO [7ff598ff9700] [routing:bootstrap_rw] started: listening using /data/mysqlrouter/mysqlrouter6446/mysql.sock
2023-09-14 18:28:12 routing INFO [7ff5997fa700] [routing:bootstrap_ro] started: listening using /data/mysqlrouter/mysqlrouter6446/mysqlro.sock
2023-09-14 18:28:12 routing INFO [7ff598ff9700] Start accepting connections for routing routing:bootstrap_rw listening on 6446 and named socket /data/mysqlrouter/mysqlrouter6446/mysql.sock
2023-09-14 18:28:12 routing INFO [7ff5997fa700] Start accepting connections for routing routing:bootstrap_ro listening on 6447 and named socket /data/mysqlrouter/mysqlrouter6446/mysqlro.sock
2023-09-14 18:28:12 routing INFO [7ff593fff700] [routing:bootstrap_x_ro] started: listening using /data/mysqlrouter/mysqlrouter6446/mysqlxro.sock
2023-09-14 18:28:12 routing INFO [7ff593fff700] Start accepting connections for routing routing:bootstrap_x_ro listening on 6449 and named socket /data/mysqlrouter/mysqlrouter6446/mysqlxro.sock
2023-09-14 18:28:12 metadata_cache INFO [7ff616702700] Connected with metadata server running on 192.168.27.56:33066
2023-09-14 18:28:12 metadata_cache INFO [7ff616702700] Potential changes detected in cluster after metadata refresh (view_id=0)
2023-09-14 18:28:12 metadata_cache INFO [7ff616702700] Metadata for cluster 'mysqlrouter' has 3 member(s), single-primary:
2023-09-14 18:28:12 metadata_cache INFO [7ff616702700] 192.168.27.56:33066 / 33060 - mode=RW
2023-09-14 18:28:12 metadata_cache INFO [7ff616702700] 192.168.27.57:33066 / 33060 - mode=RO
2023-09-14 18:28:12 metadata_cache INFO [7ff616702700] 192.168.27.58:33066 / 33060 - mode=RO


4、使用 Router 登录 mysql 测试

mysql -uroot -p -h192.168.27.56 -P 6446（6446是 mysqlrouer 读写端口）
mysql -uroot -p -h192.168.27.56 -P 6447（6447是 mysqlrouer 只读端口）


======================================

5、MySQL Router systemctl 启动配置

vim /etc/systemd/system/mysqlrouter@3306.service

Restart=no


6、只读参数设置

修改参数文件，将三个节点中的参数文件中关于只读的最后两行参数注释去掉。
注意：在《MySQL 实战》一书中说明单主模式下，组复制会自动将 Secondary 节点的 read_only 和 super_read_only 设置为 ON，感兴趣的小伙伴可以测试一下，我这里已经搭建完成了才看到没法测试了。

[mysql@jiekexu-my-56 log]$ ps -ef | grep mysql
root 11252 10549 0 17:37 pts/0 00:00:00 su - mysql
mysql 11253 11252 0 17:37 pts/0 00:00:00 -bash
mysql 11540 11253 0 17:43 pts/0 00:00:00 /bin/sh /app/mysql8.0.34/bin/mysqld_safe --defaults-file=/data/mysqldb/conf/mysql.conf --user=mysql
mysql 12786 11540 0 17:43 pts/0 00:00:26 /app/mysql8.0.34/bin/mysqld --defaults-file=/data/mysqldb/conf/mysql.conf --basedir=/app/mysql8.0.34 --datadir=/data/mysqldb/data --plugin-dir=/app/mysql8.0.34/lib/plugin --log-error=/data/mysqldb/log/mysqld.log --pid-file=/data/mysqldb/pid/mysqld.pid --socket=/data/mysqldb/socket/mysql.sock --port=33066
mysql 14279 1 0 18:28 pts/0 00:00:03 /app/mysqlrouter/bin/mysqlrouter -c /data/mysqlrouter/mysqlrouter6446/mysqlrouter.conf
mysql 14532 11253 0 18:35 pts/0 00:00:00 ps -ef
mysql 14533 11253 0 18:35 pts/0 00:00:00 grep --color=auto mysql
[mysql@jiekexu-my-56 log]$ vim /data/mysqldb/conf/mysql.conf
read_only=1
super_read_only=1


然后登录两个从库，执行从库只读设置（注意只在从库执行，主库不用执行）。

mysql> set global read_only=on;
Query OK, 0 rows affected (0.00 sec)
mysql> set global super_read_only=on;
Query OK, 0 rows affected (0.00 sec)
mysql> show master status;
+------------------+----------+--------------+------------------+--------------------------------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+--------------------------------------------+
| mysql_bin.000001 | 397716 | | | 957e8af0-bc63-11ea-bb19-005056a52572:1-438 |
+------------------+----------+--------------+------------------+--------------------------------------------+
1 row in set (0.00 sec)


7、可能遇到的问题

mysql 高版本可能存在 note 提示
可以看到，集群的状态都有一个问题：

"instanceErrors": [

"NOTE: The required parallel-appliers settings are not enabled on the instance. Use dba.configureInstance() to fix it."


这个错误也提示了修复的方式，只要通过 mysqlshell 单独连上 MGR 中的每个实例，运行下即可修复： dba.configureInstance()

再根据提示，将 mysql 的参数修正下，或者将这个参数加到参数文件中，本文则是直接加到参数文件中了。

set persist binlog_transaction_dependency_tracking='WRITESET';


下面是以前搭建的 MySQL 8.0.30 也是第一次遇到这个问题，特此记录一下。

[mysql@ts-mysql-101 ~]$ mysqlsh
MySQL Shell 8.0.30

Do you want to perform the required configuration changes? [y/n]: y
Configuring instance...
The instance '192.168.221.101:33066' was configured to be used in an InnoDB cluster.
MySQL 192.168.221.101:33066 ssl JS > cluster.status()
{
"clusterName": "mysqlrouter",
"defaultReplicaSet": {
"name": "default",
"primary": "192.168.221.101:33066",
"ssl": "DISABLED",
"status": "OK",
"statusText": "Cluster is ONLINE and can tolerate up to ONE failure.",
"topology": {
"192.168.221.101:33066": {
"address": "192.168.221.101:33066",
"memberRole": "PRIMARY",
"mode": "R/W",
"readReplicas": {},
"replicationLag": "applier_queue_applied",
"role": "HA",
"status": "ONLINE",
"version": "8.0.30"
},
"192.168.221.102:33066": {
"address": "192.168.221.102:33066",
"instanceErrors": [
"NOTE: The required parallel-appliers settings are not enabled on the instance. Use dba.configureInstance() to fix it."
],
"memberRole": "SECONDARY",
"mode": "R/O",
"readReplicas": {},
"replicationLag": "applier_queue_applied",
"role": "HA",
"status": "ONLINE",
"version": "8.0.30"
},
"192.168.221.103:33066": {
"address": "192.168.221.103:33066",
"instanceErrors": [
"NOTE: The required parallel-appliers settings are not enabled on the instance. Use dba.configureInstance() to fix it."
],
"memberRole": "SECONDARY",
"mode": "R/O",
"readReplicas": {},
"replicationLag": "applier_queue_applied",
"role": "HA",
"status": "ONLINE",
"version": "8.0.30"
}
},
"topologyMode": "Single-Primary"
},
"groupInformationSourceMember": "192.168.221.101:33066"
}


参考链接

https://dev.mysql.com/doc/mysql-shell/8.0/en/mysql-innodb-cluster.html


全文完，希望可以帮到正在阅读的你，如果觉得此文对你有帮助，可以分享给你身边的朋友，同事，你关心谁就分享给谁，一起学习共同进步~~~

欢迎关注我的公众号【JiekeXu DBA之路】，第一时间一起学习新知识！
————————————————————————————
公众号：JiekeXu DBA之路
CSDN ：https://blog.csdn.net/JiekeXu
墨天轮：https://www.modb.pro/u/4347
腾讯云：https://cloud.tencent.com/developer/user/5645107
————————————————————————————