025从零搭建微服务文件服务（一）

写在最前

如果这个项目让你有所收获，记得 Star 关注哦，这对我是非常不错的鼓励与支持。

源码地址（后端）：gitee.com/csps/mingyu…

源码地址（前端）：gitee.com/csps/mingyu…

文档地址：gitee.com/csps/mingyu…

对象存储服务

对象存储服务（Object Storage Service，简称 OSS）是一种云计算服务，用于存储和管理大规模数据、多媒体文件、备份和归档数据等。它采用了对象存储的方式，将数据以对象的形式存储在云端，并为用户提供了可靠、高可用、高扩展性、低成本的存储解决方案。

以下是对象存储服务的一些关键特点和功能：

对象存储服务通常用于各种云计算场景，包括网站托管、数据备份和恢复、多媒体存储和分发、大数据分析等。它提供了可靠的数据存储和管理解决方案，帮助用户降低存储成本、提高数据可用性，并支持灵活的数据访问和操作。

国内常用的 OSS 服务

开源的 OSS 服务

愿景

本文件服务将兼容所有S3协议的云厂商均支持，测试集成厂商如下：Minio、阿里云 OSS（aliyun）、七牛Kodo（qiniu）、腾讯 COS（qcloud）

接下来先从 Minio 开始～

Minio

Minio 是一个开源的对象存储服务器，专注于提供 S3 兼容的存储服务。Minio 的主要用途包括构建私有云对象存储解决方案、存储和管理大规模数据、备份和归档数据、构建容器化应用程序的持久性存储等。它是一个强大而灵活的对象存储服务器，适用于各种不同的应用和场景。

它具有以下主要特点和功能：

Docker 部署 Minio

docker-compose 部署 minio，版本 RELEASE.2023-09-04T19-57-37Z

version: '3.8'
services:  
  mingyue-minio:
    image: minio/minio:RELEASE.2023-09-04T19-57-37Z
    container_name: mingyue-minio
    ports:
      - 5000:9000 # api 端口
      - 5001:9001 # 控制台端口
    environment:
      MINIO_ACCESS_KEY: minioadmin  #管理后台用户名
      MINIO_SECRET_KEY: minioadmin  #管理后台密码，最小8个字符
    volumes:
      - ./minio/data:/data              #映射当前目录下的data目录至容器内/data目录
      - ./minio/config:/root/.minio/     #映射配置目录
    command: server --console-address ':9001' /data  #指定容器中的目录 /data
    privileged: true
    restart: always

登录访问

创建桶

上传测试

上传完成

访问测试

http://minio服务器ip:5000/存储目录/文件名

http://mingyue-minio:5000/mingyue/logo_sm.jpg，页面报错，因为匿名(游客没登录)用户没有访问权限

This XML file does not appear to have any style information associated with it. The document tree is shown below.

  AccessDenied
  Access Denied.
  logo_sm.jpg
  mingyue
  /mingyue/logo_sm.jpg
  1782D77A870F45C7
  dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8

如果我们需要我们上传的文件可以被匿名用户访问，那么需要添加访问权限 - Anonymous Access

再次访问测试，就可以看到上传的图片了

创建 Access Key

Access Key 用于后续接口调用的身份认证使用。例：

Access Key：d6zVm5AP07uGCqSmsTxe

Secret Key：Vsm6qQDHgGchukEpyEoeX3dTe7fic60nTi8D9a0I

新建 mingyue-common-oss 模块

添加依赖

    
        com.csp.mingyue
        mingyue-common-core
    
​
    
        com.amazonaws
        aws-java-sdk-s3
    

创建配置类

@Data
@ConfigurationProperties(prefix = "oss")
public class OssProperties {
​
  /**
   * 配置key
   */
  private String configKey;
​
  /**
   * 域名
   */
  private String endpoint;
​
  /**
   * 自定义域名
   */
  private String domain;
​
  /**
   * 前缀
   */
  private String prefix;
​
  /**
   * ACCESS_KEY
   */
  private String accessKey;
​
  /**
   * SECRET_KEY
   */
  private String secretKey;
​
  /**
   * 存储空间名
   */
  private String bucketName;
​
  /**
   * 存储区域
   */
  private String region;
​
  /**
   * 是否https（Y:是;N:否）
   */
  private String isHttps;
​
  /**
   * 桶权限类型(0:private;1:public;2:custom)
   */
  private String accessPolicy;
​
}

新建 mingyue-oss-biz.yml Nacos 配置

# 文件服务配置
oss:
  configKey: minio
  endpoint: mingyue-minio:5000
  domain:
  prefix:
  accessKey: d6zVm5AP07uGCqSmsTxe
  secretKey: Vsm6qQDHgGchukEpyEoeX3dTe7fic60nTi8D9a0I
  bucketName: mingyue
  region: 
  isHttps: N
  accessPolicy: 1

编写 Oss Client

所有兼容S3协议的云厂商均支持(阿里云 腾讯云 七牛云 minio)

public class OssClient {
​
  private final String configKey;
​
  private final OssProperties properties;
​
  private final AmazonS3 client;
​
  public OssClient(String configKey, OssProperties ossProperties) {
    this.configKey = configKey;
    this.properties = ossProperties;
    try {
      AwsClientBuilder.EndpointConfiguration endpointConfig = new AwsClientBuilder.EndpointConfiguration(
          properties.getEndpoint(), properties.getRegion());
​
      AWSCredentials credentials = new BasicAWSCredentials(properties.getAccessKey(), properties.getSecretKey());
      AWSCredentialsProvider credentialsProvider = new AWSStaticCredentialsProvider(credentials);
      ClientConfiguration clientConfig = new ClientConfiguration();
      if (OssConstant.IS_HTTPS.equals(properties.getIsHttps())) {
        clientConfig.setProtocol(Protocol.HTTPS);
      }
      else {
        clientConfig.setProtocol(Protocol.HTTP);
      }
      AmazonS3ClientBuilder build = AmazonS3Client.builder().withEndpointConfiguration(endpointConfig)
          .withClientConfiguration(clientConfig).withCredentials(credentialsProvider)
          .disableChunkedEncoding();
      if (!StrUtil.containsAny(properties.getEndpoint(), OssConstant.CLOUD_SERVICE)) {
        // minio 使用https限制使用域名访问 需要此配置 站点填域名
        build.enablePathStyleAccess();
      }
      this.client = build.build();
​
      createBucket();
    }
    catch (Exception e) {
      if (e instanceof OssException) {
        throw e;
      }
      throw new OssException("配置错误! 请检查系统配置:[" + e.getMessage() + "]");
    }
  }
​
  public void createBucket() {
    try {
      String bucketName = properties.getBucketName();
      if (client.doesBucketExistV2(bucketName)) {
        return;
      }
      CreateBucketRequest createBucketRequest = new CreateBucketRequest(bucketName);
      AccessPolicyType accessPolicy = getAccessPolicy();
      createBucketRequest.setCannedAcl(accessPolicy.getAcl());
      client.createBucket(createBucketRequest);
      client.setBucketPolicy(bucketName, getPolicy(bucketName, accessPolicy.getPolicyType()));
    }
    catch (Exception e) {
      throw new OssException("创建Bucket失败, 请核对配置信息:[" + e.getMessage() + "]");
    }
  }
​
  public UploadResult upload(byte[] data, String path, String contentType) {
    return upload(new ByteArrayInputStream(data), path, contentType);
  }
​
  public UploadResult upload(InputStream inputStream, String path, String contentType) {
    if (!(inputStream instanceof ByteArrayInputStream)) {
      inputStream = new ByteArrayInputStream(IoUtil.readBytes(inputStream));
    }
    try {
      ObjectMetadata metadata = new ObjectMetadata();
      metadata.setContentType(contentType);
      metadata.setContentLength(inputStream.available());
      PutObjectRequest putObjectRequest = new PutObjectRequest(properties.getBucketName(), path, inputStream,
          metadata);
      // 设置上传对象的 Acl 为公共读
      putObjectRequest.setCannedAcl(getAccessPolicy().getAcl());
      client.putObject(putObjectRequest);
    }
    catch (Exception e) {
      throw new OssException("上传文件失败，请检查配置信息:[" + e.getMessage() + "]");
    }
    return UploadResult.builder().fileUrl(getUrl() + "/" + path).fileName(path).build();
  }
​
  public UploadResult upload(File file, String path) {
    try {
      PutObjectRequest putObjectRequest = new PutObjectRequest(properties.getBucketName(), path, file);
      // 设置上传对象的 Acl 为公共读
      putObjectRequest.setCannedAcl(getAccessPolicy().getAcl());
      client.putObject(putObjectRequest);
    }
    catch (Exception e) {
      throw new OssException("上传文件失败，请检查配置信息:[" + e.getMessage() + "]");
    }
    return UploadResult.builder().fileUrl(getUrl() + "/" + path).fileName(path).build();
  }
​
  public void delete(String path) {
    path = path.replace(getUrl() + "/", "");
    try {
      client.deleteObject(properties.getBucketName(), path);
    }
    catch (Exception e) {
      throw new OssException("删除文件失败，请检查配置信息:[" + e.getMessage() + "]");
    }
  }
​
  public UploadResult uploadSuffix(byte[] data, String suffix, String contentType) {
    return upload(data, getPath(properties.getPrefix(), suffix), contentType);
  }
​
  public UploadResult uploadSuffix(InputStream inputStream, String suffix, String contentType) {
    return upload(inputStream, getPath(properties.getPrefix(), suffix), contentType);
  }
​
  public UploadResult uploadSuffix(File file, String suffix) {
    return upload(file, getPath(properties.getPrefix(), suffix));
  }
​
  /**
   * 获取文件元数据
   * @param path 完整文件路径
   */
  public ObjectMetadata getObjectMetadata(String path) {
    path = path.replace(getUrl() + "/", "");
    S3Object object = client.getObject(properties.getBucketName(), path);
    return object.getObjectMetadata();
  }
​
  public InputStream getObjectContent(String path) {
    path = path.replace(getUrl() + "/", "");
    S3Object object = client.getObject(properties.getBucketName(), path);
    return object.getObjectContent();
  }
​
  public String getUrl() {
    String domain = properties.getDomain();
    String endpoint = properties.getEndpoint();
    String header = OssConstant.IS_HTTPS.equals(properties.getIsHttps()) ? "https://" : "http://";
    // 云服务商直接返回
    if (StrUtil.containsAny(endpoint, OssConstant.CLOUD_SERVICE)) {
      if (StrUtil.isNotBlank(domain)) {
        return header + domain;
      }
      return header + properties.getBucketName() + "." + endpoint;
    }
    // minio 单独处理
    if (StrUtil.isNotBlank(domain)) {
      return header + domain + "/" + properties.getBucketName();
    }
    return header + endpoint + "/" + properties.getBucketName();
  }
​
  public String getPath(String prefix, String suffix) {
    // 生成uuid
    String uuid = IdUtil.fastSimpleUUID();
    // 文件路径
    String path = DateUtil.today() + "/" + uuid;
    if (StrUtil.isNotBlank(prefix)) {
      path = prefix + "/" + path;
    }
    return path + suffix;
  }
​
  public String getConfigKey() {
    return configKey;
  }
​
  /**
   * 获取私有 URL 链接
   * @param objectKey 对象KEY
   * @param second 授权时间
   */
  public String getPrivateUrl(String objectKey, Integer second) {
    GeneratePresignedUrlRequest generatePresignedUrlRequest = new GeneratePresignedUrlRequest(
        properties.getBucketName(), objectKey).withMethod(HttpMethod.GET)
            .withExpiration(new Date(System.currentTimeMillis() + 1000L * second));
    URL url = client.generatePresignedUrl(generatePresignedUrlRequest);
    return url.toString();
  }
​
  /**
   * 检查配置是否相同
   */
  public boolean checkPropertiesSame(OssProperties properties) {
    return this.properties.equals(properties);
  }
​
  /**
   * 获取当前桶权限类型
   * @return 当前桶权限类型 code
   */
  public AccessPolicyType getAccessPolicy() {
    return AccessPolicyType.getByType(properties.getAccessPolicy());
  }
​
  private static String getPolicy(String bucketName, PolicyType policyType) {
    StringBuilder builder = new StringBuilder();
    builder.append("{n"Statement": [n{n"Action": [n");
    if (policyType == PolicyType.WRITE) {
      builder.append(""s3:GetBucketLocation",n"s3:ListBucketMultipartUploads"n");
    }
    else if (policyType == PolicyType.READ_WRITE) {
      builder.append(""s3:GetBucketLocation",n"s3:ListBucket",n"s3:ListBucketMultipartUploads"n");
    }
    else {
      builder.append(""s3:GetBucketLocation"n");
    }
    builder.append("],n"Effect": "Allow",n"Principal": "*",n"Resource": "arn:aws:s3:::");
    builder.append(bucketName);
    builder.append(""n},n");
    if (policyType == PolicyType.READ) {
      builder.append(
          "{n"Action": [n"s3:ListBucket"n],n"Effect": "Deny",n"Principal": "*",n"Resource": "arn:aws:s3:::");
      builder.append(bucketName);
      builder.append(""n},n");
    }
    builder.append("{n"Action": ");
    switch (policyType) {
      case WRITE:
        builder.append(
            "[n"s3:AbortMultipartUpload",n"s3:DeleteObject",n"s3:ListMultipartUploadParts",n"s3:PutObject"n],n");
        break;
      case READ_WRITE:
        builder.append(
            "[n"s3:AbortMultipartUpload",n"s3:DeleteObject",n"s3:GetObject",n"s3:ListMultipartUploadParts",n"s3:PutObject"n],n");
        break;
      default:
        builder.append(""s3:GetObject",n");
        break;
    }
    builder.append(""Effect": "Allow",n"Principal": "*",n"Resource": "arn:aws:s3:::");
    builder.append(bucketName);
    builder.append("/*"n}n],n"Version": "2012-10-17"n}n");
    return builder.toString();
  }
​
}

编写 Oss Factory

采用工厂设计，为后续引入阿里云、腾讯云、七牛云等不同OSS服务做准备

@Slf4j
@Component
@RequiredArgsConstructor
public class OssFactory {
​
  private static final Map CLIENT_CACHE = new ConcurrentHashMap();
​
  private final OssProperties ossProperties;
​
  /**
   * 获取实例
   */
  public OssClient instance() {
    String configKey = ossProperties.getConfigKey();
​
    // 配置不相同则重新构建
    CLIENT_CACHE.put(configKey, new OssClient(configKey, ossProperties));
    log.info("创建 OSS 实例 key => {}", configKey);
​
    return CLIENT_CACHE.get(configKey);
  }
​
}

OSS 文件服务

OSS 对象存储服务逻辑接口实现

@Override
public SysOssVo upload(MultipartFile file) {
  String originalFilename = file.getOriginalFilename();
  String suffix = StrUtil.sub(originalFilename, originalFilename.lastIndexOf("."), originalFilename.length());
  OssClient storage = ossFactory.instance();
  UploadResult uploadResult;
  try {
    uploadResult = storage.uploadSuffix(file.getBytes(), suffix, file.getContentType());
  }
  catch (IOException e) {
    throw new ServiceException(e.getMessage());
  }
​
  // 保存文件信息
  SysOssVo vo = new SysOssVo();
  BeanUtil.copyProperties(uploadResult, vo);
  vo.setOssId(System.currentTimeMillis());
​
  return vo;
}

对象存储服务

@Slf4j
@Tag(name = "对象存储服务")
@RestController
@RequestMapping("oss")
@RequiredArgsConstructor
public class OssController {
​
  private final OssService ossService;
​
  @PostMapping(value = "/upload", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
  @Operation(summary = "上传文件")
  public R upload(@RequestPart("file") MultipartFile file) {
    if (ObjectUtil.isNull(file)) {
      return R.fail("上传文件不能为空");
    }
​
    SysOssVo oss = ossService.upload(file);
    UploadVo vo = new UploadVo();
    vo.setFileUrl(oss.getUrl());
    vo.setFileName(oss.getFileName());
    vo.setOssId(oss.getOssId());
​
    return R.ok(vo);
  }
​
}

启动服务

启动服务，打开接口文档，上传文件测试

http://mingyue-gateway:9100/webjars/swagger-ui/index.html?urls.primaryName=oss#/%E5%AF%B9%E8%B1%A1%E5%AD%98%E5%82%A8%E6%9C%8D%E5%8A%A1/upload

成功后接口返回如下：

{
  "code": 200,
  "msg": "操作成功",
  "data": {
    "fileUrl": "http://mingyue-minio:5000/mingyue/2023-09-11/a6987303827148b983c84ac415240480.png",
    "fileName": "2023-09-11/a6987303827148b983c84ac415240480.png",
    "ossId": 1694414731976
  }
}

改进空间

1. 上传文件后未保存文件信息，无法溯源

解决方案：添加 OSS 文件存储表

2. 配置通过 Nacos 单一 OSS 服务支持

解决方案：添加 OSS 配置表

小结

本节完成了 Minio 的部署，并且通过编码的方式将文件上传至 Minio。接下来先围绕【改进空间】优化上传服务，然后再接入其他 OSS 存储服务，逐步完善文件服务。