How to generate AWS S3 Presigned URLs

Learn how to generate Fuga Object Store, AWS S3 compatible Presigned URLs using Python.

Presigned URLs according to boto3.amazonaws.com:

A user who does not have AWS credentials or permission to access an S3 object can be granted temporary access by using a presigned URL.

Prerequisites:

  • Fuga Cloud EC2 credentials
  • Python 3.6+
  • boto3 python package (tested with boto3 1.9.244 and botocore 1.12.244)

Create a new Python file named generate-presigned-url.py with the following content:

#!/usr/bin/env python3

import os
import sys
import json
from argparse import ArgumentParser
from urllib.parse import urlencode, quote_plus

import boto3

with open('credentials.json', 'r') as fd:
credentials = json.loads(fd.read())

def main():

parser = ArgumentParser(description='Creates a Presigned URL')
parser.add_argument('--bucket-name',
dest='bucket_name',
action='store',
required=True,
help='the name of the bucket to upload to')
parser.add_argument('--object-name',
dest='object_name',
action='store',
required=True,
help='the name of the object to upload')
args = parser.parse_args()

s3 = boto3.client('s3',
endpoint_url=credentials.get('endpoint_url'),
aws_access_key_id=credentials.get('access_key'),
aws_secret_access_key=credentials.get('secret_key'),
)

response = s3.generate_presigned_url(
ClientMethod='put_object',
Params={'Bucket': args.bucket_name, 'Key': args.object_name},
ExpiresIn=3600,
)

print(f"curl -i --request PUT --upload-file {args.object_name} '{response}'")

if __name__ == '__main__':
main()

Create a new JSON file named credentials.json with the following content, update the contents to match your setup:

{
"access_key": "<your ec2 credential access>",
"secret_key": "<your ec2 credential secret>",
"endpoint_url": "https://object.api.ams.fuga.cloud:443"
}

Install the required Python packages and run the script to generate a Presigned-URL:

% pip install --user boto3
% python3.7 ./generate-presigned-url.py --bucket-name my-first-bucket --object-name plain.txt
curl -i --request PUT --upload-file plain.txt 'https://object.api.ams.fuga.cloud:443/my-first-bucket/plain.txt?....'

The generated curl command with Presigned-URL can be shared with others in order to upload a file named plain.txt without sharing your EC2 credentials. The URL will expire within 60 minutes.

Create a new TXT file named plain.txt and upload it using the previously generated curl command with Presigned-URL:

% date > plain.txt
% curl -i --request PUT --upload-file plain.txt 'https://object.api.ams.fuga.cloud:443/my-first-bucket/plain.txt?....'
HTTP/2 200
...

A response code of "200" indicates the file was successfully uploaded.

For information and examples see boto3.amazonaws.com/../s3-presigned-urls.html.