资源池化支持同城 dorado 双集群部署方式：dd 模拟(手动部署+无 cm)、cm 模拟(手动部署 dd 模拟+有 cm)、磁阵(手动部署)、集群管理工具部署
1.集群间切换
基于《资源池化+同城dorado双集群》部署方式，集群间切换设计如下：
1.1.主备集群状态
前提条件：已经部署资源池化同城双集群环境
| 集群中心 | 端 | 节点类型 | local role | run mode |
| 生产中心 | 主端 | 主节点0 | primary | primary (资源池化+传统主) |
| 备节点1 | standby | normal (资源池化+传统单机) | ||
| 容灾中心 | 备端 | 首备节点0 | standby | standby(资源池化+传统备) |
| 从备节点1 | standby | normal (资源池化+传统单机) |
local role 从系统函数 pg_stat_get_stream_replications 中获取的 local_role 参数：
openGauss=# select * from pg_stat_get_stream_replications();
local_role | static_connections | db_state | detail_information
------------+--------------------+----------+--------------------
Primary | 1 | Normal | Normal
(1 row)
Tips:run mode 指数据库内核运行模式是primary还是standby还是normal，是t_thrd.postmaster_cxt.HaShmData->current_mode或t_thrd.xlog_cxt.server_mode参数指代的主备运行模式类型
1.2.failover
以下提到的/home/omm/ss_hatest/dn0为数据库dn目录，解释如下：
| 集群中心 | 端 | 节点类型 | local role | dn目录 |
| 生产中心 | 主端 | 主节点0 | primary | /home/omm/ss_hatest/dn0 |
| 备节点1 | standby | /home/omm/ss_hatest/dn1 | ||
| 容灾中心 | 备端 | 首备节点0 | Main Standby | /home/omm/ss_hatest1/dn0 |
| 从备节点1 | standby | /home/omm/ss_hatest1/dn1 |
双集群间failover即主集群故障，备集群升为主集群的过程，操作过程如下：
(1) kill 主集群 将主集群节点全部 kill 掉 (2) stop 备集群
gs_ctl stop -D /home/omm/ss_hatest1/dn0
gs_ctl stop -D /home/omm/ss_hatest1/dn1
(3) 备集群设置 cluster_run_mode
gs_guc set -Z datanode -D /home/omm/ss_hatest1/dn0 -c "cluster_run_mode=cluster_primary"
(4) 切换远程同步复制主从端 如果是cm模拟部署方式(博客：博客资源池化同城dorado双集群部署二之cm模拟部署)，不需要在管控平台切换同步复制对方向的操作。
如果是om部署方式(博客：资源池化同城dorado双集群部署四之om部署)，则在拉起集群之前，需要在管控平台切换同步复制对方向的操作，操作如下： 登录到备存储管控平台，操作data protection -> luns -> remote replication pairs(远程复制对) -> 找到远程同步复制xlog对应的lun -> More -> Primary/Standby Switchover，操作完后，即可看到Local Resource从Secondary变成Primary。
(5) 以主集群模式重启备集群的节点
gs_ctl start -D /home/omm/ss_hatest1/dn0 -M primary
gs_ctl start -D /home/omm/ss_hatest1/dn1
(5) 查询新主集群
gs_ctl query -D /home/omm/ss_hatest1/dn0
1.2.switchover
双集群间switchover即主集群降为备集群，备集群升为主集群的过程，操作过程如下：
(1) stop 主集群
gs_ctl stop -D /home/omm/ss_hatest/dn0
gs_ctl stop -D /home/omm/ss_hatest/dn1
(2) stop 备集群
gs_ctl stop -D /home/omm/ss_hatest1/dn0
gs_ctl stop -D /home/omm/ss_hatest1/dn1
(3) 备集群设置 cluster_run_mode
gs_guc set -Z datanode -D /home/omm/ss_hatest1/dn0 -c "cluster_run_mode=cluster_primary"
(4) 切换远程同步复制主从端 如果是cm模拟部署方式(博客：博客资源池化同城dorado双集群部署二之cm模拟部署)，不需要在管控平台切换同步复制对方向的操作。
如果是om部署方式(博客：资源池化同城dorado双集群部署四之om部署)，则在拉起集群之前，需要在管控平台切换同步复制对方向的操作，操作如下： 登录到备存储管控平台，操作data protection -> luns -> remote replication pairs(远程复制对) -> 找到远程同步复制xlog对应的lun -> More -> Primary/Standby Switchover，操作完后，即可看到Local Resource从Secondary变成Primary。
(5) 以主集群模式重启备集群的节点
gs_ctl start -D /home/omm/ss_hatest1/dn0 -M primary
gs_ctl start -D /home/omm/ss_hatest1/dn1
(5) 查询新主集群
gs_ctl query -D /home/omm/ss_hatest1/dn0
(6) 主集群设置 cluster_run_mode=cluster_standby
gs_guc set -Z datanode -D /home/zx/ss_hatest/dn0 -c "cluster_run_mode=cluster_standby"
(7) 以备集群模式重启备集群的节点
gs_ctl start -D /home/omm/ss_hatest/dn0 -M standby
gs_ctl start -D /home/omm/ss_hatest/dn1
(8) 查询新备集群
gs_ctl query -D /home/omm/ss_hatest/dn0
2. 主集群内切换
2.1.failover
该章节介绍基于cm模拟部署方式的集群内切换，om部署方式的双集群和资源池化原有集群内切换方法一样。 主集群内failover即主集群主节点降为备节点，备节点升为主节点的过程，操作过程如下：
(1) 检查节点状态 查询状态
主集群主节点0
gs_ctl query -D /home/omm/ss_hatest/dn0
HA state:
local_role : Primary
static_connections : 1
db_state : Normal
detail_information : Normal
Senders info:
sender_pid : 1456376
local_role : Primary
peer_role : StandbyCluster_Standby
peer_state : Normal
state : Streaming
sender_sent_location : 2/5C8
sender_write_location : 2/5C8
sender_flush_location : 2/5C8
sender_replay_location : 2/5C8
receiver_received_location : 2/5C8
receiver_write_location : 2/5C8
receiver_flush_location : 2/5C8
receiver_replay_location : 2/5C8
sync_percent : 100%
sync_state : Async
sync_priority : 0
sync_most_available : Off
channel : 127.0.0.1:6600-->127.0.0.1:43350
Receiver info:
No information
主集群备节点1
gs_ctl query -D /home/omm/ss_hatest/dn1
HA state:
local_role : Standby
static_connections : 0
db_state : Normal
detail_information : Normal
Senders info:
No information
Receiver info:
No information
备集群首备节点0
gs_ctl query -D /home/omm/ss_hatest1/dn0
HA state:
local_role : Main Standby
static_connections : 1
db_state : Normal
detail_information : Normal
Senders info:
No information
Receiver info:
receiver_pid : 1901181
local_role : Standby
peer_role : Primary
peer_state : Normal
state : Normal
sender_sent_location : 2/A458
sender_write_location : 2/A458
sender_flush_location : 2/A458
sender_replay_location : 2/A458
receiver_received_location : 2/A458
receiver_write_location : 2/A458
receiver_flush_location : 2/A458
receiver_replay_location : 2/A458
sync_percent : 100%
channel : 127.0.0.1:41952127.0.0.1:37904
Receiver info:
No information
备集群首备节点0
[zx@node1host54 pg_log]$ gs_ctl query -D /home/zx/ss_hatest1/dn0
[2023-04-24 15:53:44.305][3878378][][gs_ctl]: gs_ctl query ,datadir is /home/zx/ss_hatest1/dn0
HA state:
local_role : Main Standby
static_connections : 2
db_state : Normal
detail_information : Normal
Senders info:
No information
Receiver info:
receiver_pid : 3816277
local_role : Standby
peer_role : Primary
peer_state : Normal
state : Normal
sender_sent_location : 2/43EA798
sender_write_location : 2/43EA798
sender_flush_location : 2/43EA798
sender_replay_location : 2/43EA798
receiver_received_location : 2/43EA798
receiver_write_location : 2/43EA798
receiver_flush_location : 2/43EA798
receiver_replay_location : 2/43EA798
sync_percent : 100%
channel : 127.0.0.1:37904
