基于gitea+K3s实现DevOps/CI/CD

对于需要自建Git服务实现CI/CD及DevOps的，gitea无疑是个非常好的选择。而对于小型运维团队来说，使用K3s能减少大量的维护成本。

gitea和K3s都是非常轻量和易于安装的。

gitea文档：docs.gitea.com/zh-cn/

K3s文档：docs.k3s.io/zh/

安装gitea

参考文档：docs.gitea.com/zh-cn/insta…

cd /usr/local
mkdir gitea
vim docker-compose.yml

version: "3"

networks:
  gitea:
    external: false

services:
  server:
    image: gitea/gitea:1.20.4
    container_name: gitea
    environment:
      - USER_UID=1000
      - USER_GID=1000
    restart: always
    networks:
      - gitea
    volumes:
      - ./gitea:/data
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
    ports:
      - "3000:3000"
      - "222:22"

# 启动gitea
docker compose up -d

启用action

vim gitea/gitea/conf/app.ini
------
#末尾添加
[actions]
ENABLED=true
------
## 重启gitea
docker restart gitea

访问http://your-ip:3000 ,配置管理员账号密码和其他相应内容，点击安装即可。安装完成后如下：

建议将服务暴漏为https域名，更能节省时间。以 gitea.mydomain.com 为例。

安装act runner

参考文档：docs.gitea.com/zh-cn/usage…

获取token

到管理后台-actions-Runner中，点击创建runner，复制token，如下

# 例如如下token
AaJkPUFeQs37hpGm6SHUIqHx3Tw8kmm0xX5gJv6f

注册runner

mkdir runner
cd runner
vim docker-compose.yml

version: "3.8"
services:
  runner:
    image: gitea/act_runner:nightly
    container_name: gitea_runner
    restart: always
    environment:
      CONFIG_FILE: /config.yaml
      GITEA_INSTANCE_URL: "https://your-ip:3000"
      GITEA_RUNNER_REGISTRATION_TOKEN: "AaJkPUFeQs37hpGm6SHUIqHx3Tw8kmm0xX5gJv6f"
      GITEA_RUNNER_NAME: "my-runner"
      GITEA_RUNNER_LABELS: "${RUNNER_LABELS}"#需要特殊环境部署的，可以修改该标签，或在web控制台修改
    volumes:
      - ./config.yaml:/config.yaml
      - ./data:/data
      - /var/run/docker.sock:/var/run/docker.sock
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
    ports:
      - "8088:8088"

为加快构建速度，一般来说我们需要允许缓存：创建config.yaml，允许缓存，请注意修改your-ip为runner所在服务器的内网ip。

vim config.yaml

# Example configuration file, it's safe to copy this as the default config file without any modification.

# You don't have to copy this file to your instance,
# just run `./act_runner generate-config > config.yaml` to generate a config file.

log:
  # The level of logging, can be trace, debug, info, warn, error, fatal
  level: info

runner:
  # Where to store the registration result.
  file: .runner
  # Execute how many tasks concurrently at the same time.
  capacity: 1
  # Extra environment variables to run jobs.
  envs:
    A_TEST_ENV_NAME_1: a_test_env_value_1
    A_TEST_ENV_NAME_2: a_test_env_value_2
    RUNNER_TOOL_CACHE: /toolcache
  # Extra environment variables to run jobs from a file.
  # It will be ignored if it's empty or the file doesn't exist.
  env_file: .env
  # The timeout for a job to be finished.
  # Please note that the Gitea instance also has a timeout (3h by default) for the job.
  # So the job could be stopped by the Gitea instance if it's timeout is shorter than this.
  timeout: 3h
  # Whether skip verifying the TLS certificate of the Gitea instance.
  insecure: false
  # The timeout for fetching the job from the Gitea instance.
  fetch_timeout: 5s
  # The interval for fetching the job from the Gitea instance.
  fetch_interval: 2s
  # The labels of a runner are used to determine which jobs the runner can run, and how to run them.
  # Like: ["macos-arm64:host", "ubuntu-latest:docker://node:16-bullseye", "ubuntu-22.04:docker://node:16-bullseye"]
  # If it's empty when registering, it will ask for inputting labels.
  # If it's empty when execute `deamon`, will use labels in `.runner` file.
  labels: []

cache:
  # Enable cache server to use actions/cache.
  enabled: true
  # The directory to store the cache data.
  # If it's empty, the cache data will be stored in $HOME/.cache/actcache.
  dir: ""
  # The host of the cache server.
  # It's not for the address to listen, but the address to connect from job containers.
  # So 0.0.0.0 is a bad choice, leave it empty to detect automatically.
  host: "your-ip"
  # The port of the cache server.
  # 0 means to use a random available port.
  port: 8088
  # The external cache server URL. Valid only when enable is true.
  # If it's specified, act_runner will use this URL as the ACTIONS_CACHE_URL rather than start a server by itself.
  # The URL should generally end with "/".
  external_server: ""

container:
  # Specifies the network to which the container will connect.
  # Could be host, bridge or the name of a custom network.
  # If it's empty, act_runner will create a network automatically.
  network: ""
  # Whether to use privileged mode or not when launching task containers (privileged mode is required for Docker-in-Docker).
  privileged: false
  # And other options to be used when the container is started (eg, --add-host=my.gitea.url:host-gateway).
  options:
  # The parent directory of a job's working directory.
  # If it's empty, /workspace will be used.
  workdir_parent:
  # Volumes (including bind mounts) can be mounted to containers. Glob syntax is supported, see https://github.com/gobwas/glob
  # You can specify multiple volumes. If the sequence is empty, no volumes can be mounted.
  # For example, if you only allow containers to mount the `data` volume and all the json files in `/src`, you should change the config to:
    # valid_volumes:
  #   - data
  #   - /src/*.json
  # If you want to allow any volume, please use the following configuration:
  # valid_volumes:
  #   - '**'
  valid_volumes: []
  # overrides the docker client host with the specified one.
  # If it's empty, act_runner will find an available docker host automatically.
  # If it's "-", act_runner will find an available docker host automatically, but the docker host won't be mounted to the job containers and service containers.
  # If it's not empty or "-", the specified docker host will be used. An error will be returned if it doesn't work.
  docker_host: ""
  # Pull docker image(s) even if already present
  force_pull: false

host:
  # The parent directory of a job's working directory.
  # If it's empty, $HOME/.cache/act/ will be used.
  workdir_parent:

docker compose up -d

重新刷新页面，可看到新的runner

安装k3s

参考文档docs.k3s.io/zh/quick-st…

在需要部署应用的服务器(例如192.168.0.100)上安装K3s,安装命令

curl -sfL https://rancher-mirror.rancher.cn/k3s/k3s-install.sh | INSTALL_K3S_MIRROR=cn sh -

如果暴漏为https域名，可以省略以下步骤

私有仓库镜像配置

如果是配置了https域名的可以忽略本步

docker允许访问gitea仓库

vim /etc/docker/daemon.json 

{
  "log-driver": "json-file",
  "log-opts": {"max-size": "100m", "max-file": "100"},
  "insecure-registries": [
   "your-ip:3000"
  ]
}

k3s允许访问gitea

vim /etc/rancher/k3s/registries.yaml

mirrors:
  gitea.mydomain.com:
    endpoint:
      - "http://your-ip:3000"
configs:
  "your-ip:3000":
    auth:
      username: your-gitea-name # this is the registry username
      password: your-gitea-password # this is the registry password

systemctl restart docker
systemctl restart k3s

持续集成

以下步骤将会以ruoyi-vue-plus为例。

启用action

点击右上角+-迁移外部仓库将ruoyi-vue-plus仓库迁移到本地gitea中。待迁移完成后，点击项目设置-仓库-启用actions,点击更新仓库设置按钮，开启项目action.

获取gitea部署token

在个人信息及配置-应用中生成一个token，注意权限选择repository和user都是read，其他不选。

请注意令牌只会显示一次，复制保管好。

添加CI文件

以最新分支5.X为例，切换到5.X分支，在项目中添加.gitea/workflows/main.yml文件

name: Ruoyi Vue Plus
run-name: CI/CD runner
on:
  push:
    # Sequence of patterns matched against refs/heads
    branches:
      - 5.X

jobs:
  ci-and-cd:
    runs-on: ubuntu-latest
    container:
      image: catthehacker/ubuntu:act-latest
    env:
      # cache bug https://gitea.com/gitea/act_runner/issues/70
      RUNNER_TOOL_CACHE: /toolcache
      REGISTRY: gitea.mydomain.com
      APP_NAME: ruoyi-admin
    steps:
      - name: checkout from git
        uses: actions/checkout@v3
      - name: Set up Java
        uses: graalvm/setup-graalvm@v1
        with:
          java-version: '17.0.7'
          distribution: 'graalvm'
          cache: 'maven'
      - name: Build with Maven
        run: mvn package -DskipTests -Pprod
      - name: Extract metadata (tags, labels) for Docker
        id: meta
        uses: docker/metadata-action@v4
        with:
          images: ${{ env.REGISTRY }}/devops/${{ env.APP_NAME }}
      - name: Login to Docker Hub
        uses: docker/login-action@v2
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ secrets.DOCKERHUB_USERNAME }}
          password: ${{ secrets.DOCKERHUB_PASSWORD }}
      - name: Build and push
        uses: docker/build-push-action@master
        with:
          context: ./ruoyi-admin
          push: ${{ github.event_name != 'pull_request' }}
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
      - name: Deploy to k8s
        uses: D3rHase/ssh-command-action@v0.2.2
        with:
          host: ${{secrets.REMOTE_HOST}}
          user: ${{secrets.REMOTE_USER}}
          private_key: ${{secrets.REMOTE_PRIVATE_KEY}}
          command: kubectl apply -f http://${{ secrets.DEVOPS_GITEA_TOKEN }}@${{ env.REGISTRY }}/devops/ruoyi-vue-plus/raw/branch/master/.deploy/deployment.yml && kubectl rollout restart deployment ${{ env.APP_NAME }}

设置action密钥

可以在个人中心，action中设置全局的，也可以在仓库中设置对应仓库的。

在actions中，添加密钥，名称为DOCKERHUB_USERNAME和DOCKERHUB_PASSWORD，分别对应你的gitea账号和密码。添加DEVOPS_GITEA_TOKEN，即上一步得到的token。

添加密钥REMOTE_HOST为需要部署应用的服务器ip（192.168.0.100）,REMOTE_USER为root，REMOTE_PRIVATE_KEY为允许登录的ssh key

创建k8s secret

kubectl create secret docker-registry dockerhub-id --docker-server= --docker-username= --docker-password= --docker-email=

创建deployment

在.deploy目录下创建文件deployment.yml文件。

---
## k8s默认是访问不了外部服务的，需要代理,以mysql和redis为例，代理myservice名称到192.168.0.100，配置文件中对应的ip修改为myservice
apiVersion: v1
kind: Endpoints
metadata:
  name: myservice
subsets:
  - addresses:
      - ip: 192.168.0.100
    ports:
      - port: 6379
        protocol: TCP
        name: redis
      - port: 3306
        protocol: TCP
        name: mysql
---
apiVersion: v1
kind: Service
metadata:
  name: myservice
spec:
  ports:
    - port: 6379
      targetPort: 6379
      protocol: TCP
      name: redis
    - port: 3306
      targetPort: 3306
      protocol: TCP
      name: mysql

---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ruoyi-admin
spec:
  replicas: 2
  revisionHistoryLimit: 3
  selector:
    matchLabels:
      app: ruoyi-admin
  template:
    metadata:
      labels:
        app: ruoyi-admin
    spec:
      containers:
        - image: gitea.mydomain.com/devops/ruoyi-admin:5.X
          name: ruoyi-admin
          imagePullPolicy: Always
          readinessProbe:
            httpGet:
              path: /actuator/health
              port: 8080
            initialDelaySeconds: 1
            successThreshold: 5
          ports:
            - containerPort: 8080
      imagePullSecrets:
        - name: dockerhub-id

---

apiVersion: v1
kind: Service
metadata:
  name: ruoyi-admin
spec:
  type: NodePort
  externalTrafficPolicy: Local
  ports:
    - name: http-8080
      protocol: TCP
      port: 8080
      nodePort: 30088
  selector:
    app: ruoyi-admin

修改application-prod.yml

由于k8s默认是访问不了外部服务的，需要代理,以mysql和redis为例，代理myservice名称到192.168.0.100，配置文件中对应的ip修改为myservice。

修改mysql的ip为myservice,如下：

          url: jdbc:mysql://myservice:3306/ruoyi-vue?useUnicode=true&characterEncoding=utf8&zeroDateTimeBehavior=convertToNull&useSSL=true&serverTimezone=GMT%2B8&autoReconnect=true&rewriteBatchedStatements=true

redis同上

推送并部署

提交代码，查看actions是否成功。运行成功后，访问http://192.168.0.100:30088 即可访问成功。

Q&A

Q:actions中的Set up job慢，github访问超时

A:将github action仓库导入到本地gitea，修改ci文件为本地url，例如checkout，将 github.com/actions/che… 同步到本地，修改ci为

    steps:
      - name: checkout from git
        uses: https://gitea.mydomain.com/devops/checkout@v3

其他同理。

Q：我需要部署的服务器是私有的，需要VPN才能访问，怎么办？

A：如果该私有服务器能访问到gitea，只需要在该私有服务器上部署一个runner，修改该runner的GITEA_RUNNER_LABELS标签，例如GITEA_RUNNER_LABELS: test-runner然后修改ci文件，指定cd步骤为test-runner即可。

ci:

name: Ruoyi Vue Plus
run-name: CI/CD runner
on:
  push:
    # Sequence of patterns matched against refs/heads
    branches:
      - 5.X

jobs:
  ci:
    runs-on: ubuntu-latest
    container:
      image: catthehacker/ubuntu:act-latest
    env:
      # cache bug https://gitea.com/gitea/act_runner/issues/70
      RUNNER_TOOL_CACHE: /toolcache
      REGISTRY: gitea.mydomain.com
      APP_NAME: ruoyi-admin
    steps:
      - name: checkout from git
        uses: actions/checkout@v3
      - 同上，将Deploy to k8s步骤移到后面
  cd:
    needs: ci
    runs-on: test-runner # 指定runner
    container:
      image: catthehacker/ubuntu:act-latest
    env:
      # cache bug https://gitea.com/gitea/act_runner/issues/70
      RUNNER_TOOL_CACHE: /toolcache
      REGISTRY: gitea.mydomain.com
      APP_NAME: ruoyi-admin
    steps:
    steps:
      - name: Deploy to k8s
        uses: D3rHase/ssh-command-action@v0.2.2
        with:
          host: ${{secrets.REMOTE_HOST}} 
          user: ${{secrets.REMOTE_USER}} 
          private_key: ${{secrets.REMOTE_PRIVATE_KEY}} 
          command: kubectl apply -f http://${{ secrets.DEVOPS_GITEA_TOKEN }}@${{ env.REGISTRY }}/devops/ruoyi-vue-plus/raw/branch/master/.deploy/deployment.yml && kubectl rollout restart deployment ${{ env.APP_NAME }}