How to generate AWS S3 Presigned URLs

Learn how to generate Fuga Object Store, AWS S3 compatible Presigned URLs using Python.

Presigned URLs according to boto3.amazonaws.com:

A user who does not have AWS credentials or permission to access an S3 object can be granted temporary access by using a presigned URL.

Prerequisites:

• Fuga Cloud EC2 credentials
• Python 3.6+
• boto3 python package (tested with boto3 1.9.244 and botocore 1.12.244)

Create a new Python file named generate-presigned-url.py with the following content:

#!/usr/bin/env python3

import os
import sys
import json
from argparse import ArgumentParser
from urllib.parse import urlencode, quote_plus

import boto3

with open('credentials.json', 'r') as fd:
    credentials = json.loads(fd.read())

def main():

    parser = ArgumentParser(description='Creates a Presigned URL')
    parser.add_argument('--bucket-name',
                        dest='bucket_name',
                        action='store',
                        required=True,
                        help='the name of the bucket to upload to')
    parser.add_argument('--object-name',
                        dest='object_name',
                        action='store',
                        required=True,
                        help='the name of the object to upload')
    args = parser.parse_args()

    s3 = boto3.client('s3',
        endpoint_url=credentials.get('endpoint_url'),
        aws_access_key_id=credentials.get('access_key'),
        aws_secret_access_key=credentials.get('secret_key'),
    )

    response = s3.generate_presigned_url(
        ClientMethod='put_object',
        Params={'Bucket': args.bucket_name, 'Key': args.object_name},
        ExpiresIn=3600,
    )

    print(f"curl -i --request PUT --upload-file {args.object_name} '{response}'")

if __name__ == '__main__':
    main()

Create a new JSON file named credentials.json with the following content, update the contents to match your setup you can find the endpoints in my.fuga dashboard.

{
  "access_key": "<your ec2 credential access>",
  "secret_key": "<your ec2 credential secret>",
  "endpoint_url": "https://<api_endpoint>"
}

Install the required Python packages and run the script to generate a Presigned-URL:

% pip install --user boto3
% python3.7 ./generate-presigned-url.py --bucket-name my-first-bucket --object-name plain.txt
curl -i --request PUT --upload-file plain.txt 'https://<api_endpoint>/my-first-bucket/plain.txt?....'

The generated curl command with Presigned-URL can be shared with others in order to upload a file named plain.txt without sharing your EC2 credentials. The URL will expire within 60 minutes.

Create a new TXT file named plain.txt and upload it using the previously generated curl command with Presigned-URL:

% date > plain.txt
% curl -i --request PUT --upload-file plain.txt 'https://<api_endpoints>/my-first-bucket/plain.txt?....'
HTTP/2 200
...

A response code of "200" indicates the file was successfully uploaded.

For information and examples see boto3.amazonaws.com/../s3-presigned-urls.html.