Kubernetes Helm快速部署Redis Cluster & Redisinsight
软件 | 版本 | 注解 | |
---|---|---|---|
helm | v3.4.0 | ||
Kubernetes | v1.24.0 | ||
Redis Cluster | 7.0.11 | ||
Chart | 8.6.4 |
添加Helm chart
<span>[</span> <span>root@k8s</span> <span>-</span> <span>01</span> <span> </span> <span>~]]#</span> <span> helm repo add bitnami https</span> <span>:</span> <span>//charts.bitnami.com/bitnami</span>
查询 Redis 资源
<span>[</span> <span>root@k8s</span> <span>-</span> <span>01</span> <span> </span> <span>~]#</span> <span> helm search repo redis</span>
<span>NAME CHART VERSION APP VERSION DESCRIPTION</span>
<span>bitnami</span> <span>/</span> <span>redis </span> <span>17.11</span> <span>.</span> <span>6</span> <span> </span> <span>7.0</span> <span>.</span> <span>11</span> <span> </span> <span>Redis</span> <span>(</span> <span>R</span> <span>)</span> <span> </span> <span>is</span> <span> an open source</span> <span>,</span> <span> advanced key</span> <span>-</span> <span>value </span> <span>...</span>
<span>bitnami</span> <span>/</span> <span>redis</span> <span>-</span> <span>cluster </span> <span>8.6</span> <span>.</span> <span>4</span> <span> </span> <span>7.0</span> <span>.</span> <span>11</span> <span> </span> <span>Redis</span> <span>(</span> <span>R</span> <span>)</span> <span> </span> <span>is</span> <span> an open source</span> <span>,</span> <span> scalable</span> <span>,</span> <span> distribut</span> <span>...</span>
拉取 chart 到本地
<span>#拉取最新</span>
<span>helm pull bitnami</span> <span>/</span> <span>redis</span> <span>-</span> <span>cluster</span>
<span>#选择版本拉取</span>
<span>helm pull bitnami</span> <span>/</span> <span>redis</span> <span>-</span> <span>cluster </span> <span>--</span> <span>version </span> <span>8.6</span> <span>.</span> <span>4</span>
- 默认拉取完毕就是
redis-cluster-8.6.4.tgz
解压helm包
<span> tar xf redis</span> <span>-</span> <span>cluster</span> <span>-</span> <span>8.6</span> <span>.</span> <span>4.tgz</span>
数据存储需要创建sc,我这里有默认的sc就不创建,例子如下
<span>[</span> <span>root@k8s</span> <span>-</span> <span>01</span> <span> nfs</span> <span>]#</span> <span> cat sc</span> <span>.</span> <span>yaml</span>
<span>kind</span> <span>:</span> <span> </span> <span>StorageClass</span>
<span>apiVersion</span> <span>:</span> <span> storage</span> <span>.</span> <span>k8s</span> <span>.</span> <span>io</span> <span>/</span> <span>v1</span>
<span>metadata</span> <span>:</span>
<span> annotations</span> <span>:</span>
<span> storageclass</span> <span>.</span> <span>kubernetes</span> <span>.</span> <span>io</span> <span>/</span> <span>is</span> <span>-</span> <span>default</span> <span>-</span> <span>class</span> <span>:</span> <span> </span> <span>"true"</span>
<span> name</span> <span>:</span> <span> nfs</span> <span>-</span> <span>storage</span>
<span>provisioner</span> <span>:</span> <span> nfs</span> <span>-</span> <span>provisioner</span>
<span>volumeBindingMode</span> <span>:</span> <span> </span> <span>Immediate</span>
<span>#我这里是nfs的sc,我们将nfs sc设置为默认存储后,后面就不需要在修改value.yaml中的sc</span>
如果不想使用nfs sc,就自行创建其它存储类型的sc
<span>[</span> <span>root@k8s</span> <span>-</span> <span>01</span> <span> </span> <span>~]#</span> <span> kubectl </span> <span>get</span> <span> sc</span>
<span>NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE</span>
<span>nfs</span> <span>-</span> <span>storage </span> <span>(</span> <span>default</span> <span>)</span> <span> nfs</span> <span>-</span> <span>provisioner </span> <span>Delete</span> <span> </span> <span>Immediate</span> <span> </span> <span>false</span> <span> </span> <span>48d</span>
修改Values.yaml
<span>cd redis</span> <span>-</span> <span>cluster</span> <span>/</span>
<span>vim values</span> <span>.</span> <span>yaml</span>
<span> storageClass</span> <span>:</span> <span> </span> <span>""</span> <span> </span> <span>#前面设置了默认的sc,如果不想使用默认sc,这里可以额外填写</span>
<span> redis</span> <span>:</span>
<span> password</span> <span>:</span> <span> </span> <span>"redis123"</span> <span> </span> <span>#Redis密码</span>
<span>service</span> <span>:</span>
<span> </span> <span>## @param service.ports.redis Kubernetes Redis service port</span>
<span> </span> <span>##</span>
<span> ports</span> <span>:</span>
<span> redis</span> <span>:</span> <span> </span> <span>6379</span> <span> </span> <span># 定义Cluster redis 服务端口</span>
<span>#下面的探测时间不修改,第一次启动的时候会让Redis Cluster自动重启</span>
<span>redis</span> <span>:</span>
<span> livenessProbe</span> <span>:</span> <span> </span> <span># 修改了 livenessProbe 的探测时间</span>
<span> enabled</span> <span>:</span> <span> </span> <span>true</span>
<span> initialDelaySeconds</span> <span>:</span> <span> </span> <span>60</span>
<span> periodSeconds</span> <span>:</span> <span> </span> <span>30</span>
<span> timeoutSeconds</span> <span>:</span> <span> </span> <span>10</span>
<span> successThreshold</span> <span>:</span> <span> </span> <span>1</span>
<span> failureThreshold</span> <span>:</span> <span> </span> <span>5</span>
<span> readinessProbe</span> <span>:</span> <span> </span> <span># 修改了 readinessProbe 的探测时间</span>
<span> enabled</span> <span>:</span> <span> </span> <span>true</span>
<span> initialDelaySeconds</span> <span>:</span> <span> </span> <span>60</span>
<span> periodSeconds</span> <span>:</span> <span> </span> <span>30</span>
<span> timeoutSeconds</span> <span>:</span> <span> </span> <span>10</span>
<span> successThreshold</span> <span>:</span> <span> </span> <span>1</span>
<span> failureThreshold</span> <span>:</span> <span> </span> <span>5</span>
<span> startupProbe</span> <span>:</span> <span> </span> <span># 修改了 startupProbe 的探测时间</span>
<span> enabled</span> <span>:</span> <span> </span> <span>false</span>
<span> path</span> <span>:</span> <span> </span> <span>/</span>
<span> initialDelaySeconds</span> <span>:</span> <span> </span> <span>300</span>
<span> periodSeconds</span> <span>:</span> <span> </span> <span>30</span>
<span> timeoutSeconds</span> <span>:</span> <span> </span> <span>10</span>
<span> failureThreshold</span> <span>:</span> <span> </span> <span>6</span>
<span> successThreshold</span> <span>:</span> <span> </span> <span>1</span>
<span>metrics</span> <span>:</span> <span> </span> <span>#redis metrics,可以选择开启,会自动为我们创建一个Export</span>
<span> </span> <span>## @param metrics.enabled Start a side-car prometheus exporter</span>
<span> </span> <span>##</span>
<span> enabled</span> <span>:</span> <span> </span> <span>false</span>
<span>cluster</span> <span>:</span>
<span> </span> <span>## nodes:是包括副本在内的节点总数。</span>
<span> </span> <span>## 这意味着将有 3 个主节点和 3 个副本节点</span>
<span> </span> <span>##(由于 replicas 默认设置为 1,每个主节点将有 1 个副本)。</span>
<span> </span> <span>## 因此,nodes = numberOfMasterNodes + numberOfMasterNodes * replicas</span>
<span> nodes</span> <span>:</span> <span> </span> <span>6</span>
<span> replicas</span> <span>:</span> <span> </span> <span>1</span>
<span>...</span>
<span>#其它参数可以自行修改</span>
创建Redis Cluster集群
<span>[</span> <span>root@k8s</span> <span>-</span> <span>01</span> <span> redis</span> <span>-</span> <span>cluster</span> <span>]#</span> <span> cd redis</span> <span>-</span> <span>cluster</span>
<span>[</span> <span>root@k8s</span> <span>-</span> <span>01</span> <span> redis</span> <span>-</span> <span>cluster</span> <span>]#</span> <span> kubectl create ns redis</span>
<span>[</span> <span>root@k8s</span> <span>-</span> <span>01</span> <span> redis</span> <span>-</span> <span>cluster</span> <span>]#</span> <span> helm install redis</span> <span>-</span> <span>cluster </span> <span>-</span> <span>n redis </span> <span>.</span>
<span>NAME</span> <span>:</span> <span> redis</span> <span>-</span> <span>cluster</span>
<span>LAST DEPLOYED</span> <span>:</span> <span> </span> <span>Tue</span> <span> </span> <span>Jun</span> <span> </span> <span>27</span> <span> </span> <span>14</span> <span>:</span> <span>31</span> <span>:</span> <span>38</span> <span> </span> <span>2023</span>
<span>NAMESPACE</span> <span>:</span> <span> redis</span>
<span>STATUS</span> <span>:</span> <span> deployed</span>
<span>REVISION</span> <span>:</span> <span> </span> <span>1</span>
<span>TEST SUITE</span> <span>:</span> <span> </span> <span>None</span>
<span>NOTES</span> <span>:</span>
<span>CHART NAME</span> <span>:</span> <span> redis</span> <span>-</span> <span>cluster</span>
<span>CHART VERSION</span> <span>:</span> <span> </span> <span>8.6</span> <span>.</span> <span>4</span>
<span>APP VERSION</span> <span>:</span> <span> </span> <span>7.0</span> <span>.</span> <span>11</span> <span>**</span> <span> </span> <span>Please</span> <span> be patient </span> <span>while</span> <span> the chart </span> <span>is</span> <span> being deployed </span> <span>**</span>
<span>To</span> <span> </span> <span>get</span> <span> your password run</span> <span>:</span>
<span> </span> <span>export</span> <span> REDIS_PASSWORD</span> <span>=</span> <span>$</span> <span>(</span> <span>kubectl </span> <span>get</span> <span> secret </span> <span>--</span> <span>namespace</span> <span> </span> <span>"redis"</span> <span> redis</span> <span>-</span> <span>cluster </span> <span>-</span> <span>o jsonpath</span> <span>=</span> <span>"{.data.redis-password}"</span> <span> </span> <span>|</span> <span> base64 </span> <span>-</span> <span>d</span> <span>)</span>
<span>You</span> <span> have deployed a </span> <span>Redis</span> <span>®</span> <span> </span> <span>Cluster</span> <span> accessible only </span> <span>from</span> <span> within you </span> <span>Kubernetes</span> <span> </span> <span>Cluster</span> <span>.</span> <span>INFO</span> <span>:</span> <span> </span> <span>The</span> <span> </span> <span>Job</span> <span> to create the cluster will be created</span> <span>.</span> <span>To</span> <span> connect to your </span> <span>Redis</span> <span>®</span> <span> cluster</span> <span>:</span>
<span>1.</span> <span> </span> <span>Run</span> <span> a </span> <span>Redis</span> <span>®</span> <span> pod that you can </span> <span>use</span> <span> </span> <span>as</span> <span> a client</span> <span>:</span>
<span>kubectl run </span> <span>--</span> <span>namespace</span> <span> redis redis</span> <span>-</span> <span>cluster</span> <span>-</span> <span>client </span> <span>--</span> <span>rm </span> <span>--</span> <span>tty </span> <span>-</span> <span>i </span> <span>--</span> <span>restart</span> <span>=</span> <span>'Never'</span> <span> </span>
<span> </span> <span>--</span> <span>env REDIS_PASSWORD</span> <span>=</span> <span>$REDIS_PASSWORD </span>
<span>--</span> <span>image docker</span> <span>.</span> <span>io</span> <span>/</span> <span>bitnami</span> <span>/</span> <span>redis</span> <span>-</span> <span>cluster</span> <span>:</span> <span>7.0</span> <span>.</span> <span>11</span> <span>-</span> <span>debian</span> <span>-</span> <span>11</span> <span>-</span> <span>r21 </span> <span>--</span> <span> bash</span>
<span>2.</span> <span> </span> <span>Connect</span> <span> </span> <span>using</span> <span> the </span> <span>Redis</span> <span>®</span> <span> CLI</span> <span>:</span>
<span>redis</span> <span>-</span> <span>cli </span> <span>-</span> <span>c </span> <span>-</span> <span>h redis</span> <span>-</span> <span>cluster </span> <span>-</span> <span>a $REDIS_PASSWORD</span>
创建完成后可以使用下面的命令查看helm集群
<span>[</span> <span>root@k8s</span> <span>-</span> <span>01</span> <span> </span> <span>~]#</span> <span> helm </span> <span>-</span> <span>n redis list</span>
<span>NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION</span>
<span>redis</span> <span>-</span> <span>cluster redis </span> <span>1</span> <span> </span> <span>2023</span> <span>-</span> <span>06</span> <span>-</span> <span>27</span> <span> </span> <span>14</span> <span>:</span> <span>31</span> <span>:</span> <span>38.202235845</span> <span> </span> <span>+</span> <span>0800</span> <span> CST deployed redis</span> <span>-</span> <span>cluster</span> <span>-</span> <span>8.6</span> <span>.</span> <span>4</span> <span> </span> <span>7.0</span> <span>.</span> <span>11</span>
检查Pod状态
<span>[</span> <span>root@k8s</span> <span>-</span> <span>01</span> <span> </span> <span>~]#</span> <span> kubectl </span> <span>get</span> <span> pod </span> <span>-</span> <span>n redis</span>
<span>NAME READY STATUS RESTARTS AGE</span>
<span>redis</span> <span>-</span> <span>cluster</span> <span>-</span> <span>0</span> <span> </span> <span>1</span> <span>/</span> <span>1</span> <span> </span> <span>Running</span> <span> </span> <span>1</span> <span> </span> <span>(</span> <span>102m</span> <span> ago</span> <span>)</span> <span> </span> <span>103m</span>
<span>redis</span> <span>-</span> <span>cluster</span> <span>-</span> <span>1</span> <span> </span> <span>1</span> <span>/</span> <span>1</span> <span> </span> <span>Running</span> <span> </span> <span>1</span> <span> </span> <span>(</span> <span>102m</span> <span> ago</span> <span>)</span> <span> </span> <span>103m</span>
<span>redis</span> <span>-</span> <span>cluster</span> <span>-</span> <span>2</span> <span> </span> <span>1</span> <span>/</span> <span>1</span> <span> </span> <span>Running</span> <span> </span> <span>1</span> <span> </span> <span>(</span> <span>102m</span> <span> ago</span> <span>)</span> <span> </span> <span>103m</span>
<span>redis</span> <span>-</span> <span>cluster</span> <span>-</span> <span>3</span> <span> </span> <span>1</span> <span>/</span> <span>1</span> <span> </span> <span>Running</span> <span> </span> <span>1</span> <span> </span> <span>(</span> <span>102m</span> <span> ago</span> <span>)</span> <span> </span> <span>103m</span>
<span>redis</span> <span>-</span> <span>cluster</span> <span>-</span> <span>4</span> <span> </span> <span>1</span> <span>/</span> <span>1</span> <span> </span> <span>Running</span> <span> </span> <span>1</span> <span> </span> <span>(</span> <span>102m</span> <span> ago</span> <span>)</span> <span> </span> <span>103m</span>
<span>redis</span> <span>-</span> <span>cluster</span> <span>-</span> <span>5</span> <span> </span> <span>1</span> <span>/</span> <span>1</span> <span> </span> <span>Running</span> <span> </span> <span>1</span> <span> </span> <span>(</span> <span>102m</span> <span> ago</span> <span>)</span> <span> </span> <span>103m</span>
检查svc
<span>[</span> <span>root@k8s</span> <span>-</span> <span>01</span> <span> </span> <span>~]#</span> <span> kubectl </span> <span>get</span> <span> svc </span> <span>-</span> <span>n redis</span>
<span>NAME TYPE CLUSTER</span> <span>-</span> <span>IP EXTERNAL</span> <span>-</span> <span>IP PORT</span> <span>(</span> <span>S</span> <span>)</span> <span> AGE</span>
<span>redis</span> <span>-</span> <span>cluster </span> <span>ClusterIP</span> <span> </span> <span>10.96</span> <span>.</span> <span>64.112</span> <span> </span> <span><none></span> <span> </span> <span>6379</span> <span>/</span> <span>TCP </span> <span>103m</span>
<span>redis</span> <span>-</span> <span>cluster</span> <span>-</span> <span>headless </span> <span>ClusterIP</span> <span> </span> <span>None</span> <span> </span> <span><none></span> <span> </span> <span>6379</span> <span>/</span> <span>TCP</span> <span>,</span> <span>16379</span> <span>/</span> <span>TCP </span> <span>103m</span>
<span>[</span> <span>root@k8s</span> <span>-</span> <span>01</span> <span> </span> <span>~]#</span>
集群内部访问
<span>默认</span> <span> redis</span> <span>-</span> <span>cluster</span> <span>-</span> <span>0</span> <span> </span> <span>和</span> <span> redis</span> <span>-</span> <span>cluster</span> <span>-</span> <span>1</span> <span> </span> <span>为主从,</span> <span>0</span> <span>主</span> <span>1</span> <span>从</span>
<span>默认</span> <span> redis</span> <span>-</span> <span>cluster</span> <span>-</span> <span>2</span> <span> </span> <span>和</span> <span> redis</span> <span>-</span> <span>cluster</span> <span>-</span> <span>3</span> <span> </span> <span>为主从,</span> <span>0</span> <span>主</span> <span>1</span> <span>从</span>
<span>默认</span> <span> redis</span> <span>-</span> <span>cluster</span> <span>-</span> <span>4</span> <span> </span> <span>和</span> <span> redis</span> <span>-</span> <span>cluster</span> <span>-</span> <span>5</span> <span> </span> <span>为主从,</span> <span>0</span> <span>主</span> <span>1</span> <span>从</span>
<span>所以集群主节点为:</span> <span>redis</span> <span>-</span> <span>cluster</span> <span>-</span> <span>0</span> <span> redis</span> <span>-</span> <span>cluster</span> <span>-</span> <span>2</span> <span> redis</span> <span>-</span> <span>cluster</span> <span>-</span> <span>4</span>
<span>所以集群从节点为:</span> <span>redis</span> <span>-</span> <span>cluster</span> <span>-</span> <span>1</span> <span> redis</span> <span>-</span> <span>cluster</span> <span>-</span> <span>3</span> <span> redis</span> <span>-</span> <span>cluster</span> <span>-</span> <span>5</span>
访问测试
我这里直接在node节点上使用redis-cli测试,当然也可以像上面那样启动一个容器来测试
<span>[</span> <span>root@k8s</span> <span>-</span> <span>node2 </span> <span>~]#</span> <span> redis</span> <span>-</span> <span>cli </span> <span>-</span> <span>c </span> <span>-</span> <span>h </span> <span>10.96</span> <span>.</span> <span>64.112</span> <span> </span> <span>-</span> <span>p </span> <span>6379</span> <span> </span> <span>-</span> <span>a redis123</span>
<span>#这里的IP我采用的是Cluster IP</span>
<span>10.96</span> <span>.</span> <span>64.112</span> <span> </span> <span>:</span> <span>6379</span> <span>></span> <span> KEYS </span> <span>*</span>
<span>1</span> <span>)</span> <span> </span> <span>"ab"</span>
<span>2</span> <span>)</span> <span> </span> <span>"jjj"</span>
查看集群信息
<span>cluster info</span>
<span>cluster nodes</span>
创建Redisinsight
RedisInsight 是一个直观且高效的 Redis GUI,允许您与数据库交互并管理数据 - 内置对 Redis 模块的支持。
RedisInsight 亮点:
- 浏览、过滤、可视化您的键值 Redis 数据结构并查看不同格式的键值(包括 JSON、Hex、ASCII 等)
- 对列表、哈希、字符串、集合、排序集和流的 CRUD 支持
- RedisJSON的 CRUD 支持
- Profiler - 实时分析发送到 Redis 的每条命令
- SlowLog - 基于Slowlog命令分析Redis实例中的慢操作
- Pub/Sub - 支持Redis pub/sub,支持订阅频道并向频道发布消息
- 批量操作 - 根据浏览器或树视图中设置的过滤器批量删除键
- Workbench 简介 - 先进的命令行界面,具有智能命令自动完成、复杂数据可视化和对原始模式的支持
- 命令自动完成支持RediSearch、RedisJSON、RedisGraph、RedisTimeSeries、RedisAI
- RediSearch索引、查询和聚合的可视化
- 能够构建自己的数据可视化插件
- Redis 功能的内置点击指南
- 正式支持Redis OSS、Redis Cloud。可与 Microsoft Azure Redis 缓存配合使用(即将推出官方支持)。
- 适用于 macOS(包括 M1)、Windows 和 Linux
使用Redisinsight来快速管理Redis集群
<span>cat </span> <span>>></span> <span>RedisInsight</span> <span>.</span> <span>yaml </span> <span><<</span> <span>EOF</span>
<span>apiVersion</span> <span>:</span> <span> apps</span> <span>/</span> <span>v1</span>
<span>kind</span> <span>:</span> <span> </span> <span>Deployment</span>
<span>metadata</span> <span>:</span> <span> </span>
<span> name</span> <span>:</span> <span> redisinsight</span>
<span> </span> <span>namespace</span> <span>:</span> <span> redis</span>
<span>spec</span> <span>:</span>
<span> replicas</span> <span>:</span> <span> </span> <span>1</span>
<span> selector</span> <span>:</span> <span> </span>
<span> matchLabels</span> <span>:</span>
<span> app</span> <span>:</span> <span> redisinsight</span>
<span> </span> <span>template</span> <span>:</span> <span> </span>
<span> metadata</span> <span>:</span> <span> </span>
<span> labels</span> <span>:</span> <span> </span>
<span> app</span> <span>:</span> <span> redisinsight</span>
<span> spec</span> <span>:</span>
<span> containers</span> <span>:</span>
<span> </span> <span>-</span> <span> name</span> <span>:</span> <span> redisinsight</span>
<span> image</span> <span>:</span> <span> redislabs</span> <span>/</span> <span>redisinsight</span> <span>:</span> <span>1.12</span> <span>.</span> <span>1</span>
<span> imagePullPolicy</span> <span>:</span> <span> </span> <span>IfNotPresent</span>
<span> ports</span> <span>:</span> <span> </span>
<span> </span> <span>-</span> <span> containerPort</span> <span>:</span> <span> </span> <span>8001</span>
<span> volumeMounts</span> <span>:</span> <span> </span>
<span> </span> <span>-</span> <span> name</span> <span>:</span> <span> db</span>
<span> mountPath</span> <span>:</span> <span> </span> <span>/</span> <span>db</span>
<span> volumes</span> <span>:</span>
<span> </span> <span>-</span> <span> name</span> <span>:</span> <span> db</span>
<span> emptyDir</span> <span>:</span> <span> </span> <span>{}</span>
<span>---</span>
<span>apiVersion</span> <span>:</span> <span> v1</span>
<span>kind</span> <span>:</span> <span> </span> <span>Service</span>
<span>metadata</span> <span>:</span>
<span> name</span> <span>:</span> <span> redisinsight</span> <span>-</span> <span>service</span>
<span> </span> <span>namespace</span> <span>:</span> <span> redis</span>
<span>spec</span> <span>:</span>
<span> type</span> <span>:</span> <span> </span> <span>NodePort</span>
<span> ports</span> <span>:</span>
<span> </span> <span>-</span> <span> port</span> <span>:</span> <span> </span> <span>80</span>
<span> targetPort</span> <span>:</span> <span> </span> <span>8001</span>
<span> nodePort</span> <span>:</span> <span> </span> <span>31888</span>
<span> selector</span> <span>:</span>
<span> app</span> <span>:</span> <span> redisinsight</span>
<span>EOF</span>
创建redisinsight
<span>kubectl apply </span> <span>-</span> <span>f </span> <span>RedisInsight</span> <span>.</span> <span>yaml</span>
如果需要持久化自行添加即可
默认 http://nodeport IP:31888 访问
<span>redis</span> <span>-</span> <span>cluster</span> <span>-</span> <span>headless</span> <span>.</span> <span>redis</span> <span>.</span> <span>svc</span> <span>.</span> <span>cluster</span> <span>.</span> <span>local</span>
<span>#redis-cluster-headless svc名称</span>
<span>#redis namespace名称</span>