Limiting bucket upload types using Cloud Functions

Preface

I believe that many masters have discovered the vulnerability of arbitrary file upload in storage buckets. However, due to the characteristics of storage buckets, the harm of such arbitrary file upload is lower than that of traditional sites, but it still has certain risks. For example, it can be used for phishing or hanging black pages, etc.

The conventional way to fix this type of risk is to limit the file upload type in the backend code. There are already corresponding articles on the Internet. This article will explore another method, which is to use cloud functions to limit the file upload type of the storage bucket.

This article will take the scenario of limiting the storage bucket to upload only pictures as an example. As for how to limit other types, you only need to slightly modify the function code to achieve it. In order to better understand the content of this article, a simple flowchart is drawn here as follows.

Steps

Here, we take Alibaba Cloud as an example. First, we create an event function in the Function Compute FC service. The region of the function must be consistent with the region of the target storage bucket.

Select Python as the running environment, save the following code as index.py file, compress it into a ZIP package, and then upload the ZIP package to the cloud function.

import os
import json
import oss2
import imghdr


def handler(event, context):
    # 获取临时访问凭证并进行认证
    creds = context.credentials
    auth = oss2.StsAuth(creds.access_key_id, creds.access_key_secret, creds.security_token)

    # 获取上传文件的相关信息
    evt_lst = json.loads(event)
    evt = evt_lst('events')(0)
    bucket_name = evt('oss')('bucket')('name')
    object_key = evt('oss')('object')('key')
    endpoint = 'oss-' + evt('region') + '-internal.aliyuncs.com'
    bucket = oss2.Bucket(auth, endpoint, bucket_name)

    # 获取上传文件的后缀类型
    upload_file_suffix_type = os.path.splitext(object_key)(1).replace('.', '')

    # 获取上传文件的内容类型
    head_result = bucket.head_object(object_key)
    upload_file_content_type = head_result.headers.get('Content-Type')

    # 获取上传文件的文件头类型
    upload_file_content = bucket.get_object(object_key).read()
    upload_file_header_type = imghdr.what(None, h=upload_file_content(:32))

    print(f'(+) 文件上传路径：oss://{bucket_name}/{object_key}')
    print(f'(+) 文件头类型：{upload_file_header_type}')
    print(f'(+) 文件后缀类型：{upload_file_suffix_type}')
    print(f'(+) 文件 Content-Type：{upload_file_content_type}')

    # 允许上传的文件类型列表
    allowed_types = ('jpg', 'jpeg', 'png', 'gif')
    allowed_content_types = ('image/jpeg', 'image/png', 'image/gif')

    # 检查文件类型是否合规
    Compliant = 0
    print('(+) 正在检查上传的文件是否合规 ……')
    if upload_file_header_type not in allowed_types:
        print('(-) 文件头类型检查不通过。')
    else:
        print('(+) 文件头类型检查通过。')
        if upload_file_suffix_type not in allowed_types:
            print('(-) 文件后缀类型检查不通过。')
        else:
            print('(+) 文件后缀类型检查通过。')
            if upload_file_content_type not in allowed_content_types:
                print('(-) 文件 Content Type 检查不通过。')
            else:
                print('(+) 文件 Content Type 检查通过。')
                Compliant = 1

    # 删除不允许上传的文件
    if Compliant:
        print(f'(+) 文件 oss://{bucket_name}/{object_key} 检查通过。')
    else:
        print(f'(-) 文件 oss://{bucket_name}/{object_key} 检查不通过，正在删除该文件。')
        bucket.delete_object(object_key)
        print(f'(!) 文件 oss://{bucket_name}/{object_key} 已被删除。')

Since we need to call the GetObject and DeleteObject APIs in the Python code here, we also need to configure a role with at least these two operation permissions for this function.

Here we create a role in IAM. The minimum permissions required for this role are as follows:

{
  "Version": "1",
  "Statement": (
    {
      "Effect": "Allow",
      "Action": (
        "oss:GetObject",
        "oss:DeleteObject"
      ),
      "Resource": "acs:oss:oss-<your_bucket_region>:<your_account_id>:<your_bucket_name>/*"
    }
  )
}

The trust policy is as follows:

{
  "Statement": (
    {
      "Action": "sts:AssumeRole",
      "Effect": "Allow",
      "Principal": {
        "Service": (
          "fc.aliyuncs.com"
        )
      }
    }
  ),
  "Version": "1"
}

After the role is created, select the role name we want to use in the function in “Advanced Configuration”. You can keep the default or modify other places according to your needs. Finally, click the “Create” button of the function.

After creating the cloud function, we also need to create a trigger for this function so that the function can be executed when a file is uploaded to the bucket.

On the function configuration page, select Trigger, click Create Trigger, select OSS for the trigger type, select the bucket for which you want to restrict file uploads as the bucket name, select PutObject for the trigger event, use the default role as the role, and click OK. All configurations are now complete.

At this point, we upload a normal file to the bucket and we can see that all three checks are passed, so the file can be uploaded normally.

If you upload a file with the suffix html, the check will fail and the cloud function will delete the file from the storage bucket.

If the uploaded file name has a png suffix, the Content-Type is image/jpeg, but the Body part is not a picture type, it will also fail to upload.

So far, we have basically been able to use cloud functions to limit the file upload types. However, there are still some areas that can be optimized and some limitations that are left for readers to explore.

Areas that can be optimized:

1. If you want to identify other types of files, such as zip, txt and other formats, the imghdr library used here is not enough. Imghdr can only identify the type of image files. In this case, you can use a third-party library, such as the python-magic library.
2. The strategy adopted in the code here is to directly delete non-compliant files when they are detected. In actual scenarios, direct deletion may have certain risks, especially when there is a problem of overlapping buckets with the same name. Therefore, appropriate adjustments need to be made here based on your own business situation.

Where there are limitations:

Since the OSS trigger in the cloud function can only be executed asynchronously, the file has already been uploaded to the bucket when the cloud function is triggered. Therefore, there is no way to solve the problem of overwriting when uploading files with the same name. This problem currently seems to be able to be solved only through backend code.

Summarize

This article provides a new solution for limiting the types of files uploaded to a bucket, but it also has certain limitations.

In addition, I noticed that in the current bucket policy, *.png can be used to limit the file upload suffix under the resource path. However, there is currently no way to limit the Content Type. I also hope that cloud vendors can add a policy to limit the Content Type in the bucket policy, so that it is more convenient to limit the file upload type of the bucket, and it will become easier to improve the security of the bucket.

However, if you want a stricter restriction policy, cloud functions or backend code still have greater freedom.

This article has been updated to the T Wiki Cloud Security Library. The article address is:wiki.teamssix.com/cloudservice/s3/limiting-bucket-upload-types-using-cloud-functions.html

For more information, please follow my personal WeChat public account: TeamsSix