Skip to content

How to connect to the Fuga Cloud Object Store

Estimated time to read: 8 minutes

Using the Fuga object storage is a good way to store lots of data in use cases where speed isn’t the most important factor, but scalability is. First, we need some setup.

API keys

  1. Click on Access tab
  2. Click on Credentials, and then select the EC2/S3 credentials at the top. Overview of S3 credentials

Now you can find the S3 credentials of your current selected environment. At each credential you can find in which region the credentials are deployed.

To create a new S3 credential, click on EC2 / S3 credential.

  1. To list and create API keys for the Object Store we need the Fuga OpenStack CLI. If you haven’t installed it yet check out our tutorial.
  2. After following the tutorial for your operating system it’s time to list the credentials. We need the credentials to access our Object Store container in later steps.
  3. To list all the credentials, enter the command: 
    openstack ec2 credentials list
+--------+--------+------------+---------+
| Access | Secret | Project ID | User ID |
+--------+--------+------------+---------+
| 9999.. | 9999.. | 99999999.. | 99999.. |
+--------+--------+------------+---------+
    • If the list of credentials is empty, a new pair of credentials can be generated with: 
          openstack ec2 credentials create
  4. Either copy the Access and Secret key to the clipboard or save them temporarily

Creating an Object Store container

  1. Click on Storage tab
  2. Click on Create a container and give your new container a fitting name.

Now we can use the Object Store inside the browser, but if we want to use it outside the browser, we need some more setup.

To create a new bucket: 

$ openstack container create my-bucket2

To list your buckets: 

$ openstack container list
+------------------+
| Name             |
+------------------+
| my-bucket        |
+------------------+

Now you have created a new bucket with the CLI tools, this one is usable with S3 and Swift.

To install AWS CLI we can use: 

$ pip3 install awscli

Then the next step is to configure the AWS CLI: 

$ aws configure --profile=fuga
AWS Access Key ID     [None]: <access id>
AWS Secret Access     [None]: <secret>
Default region name   [None]: <press enter>
Default output format [None]: <press enter>

To create a new bucket: 

$ aws --profile=fuga --endpoint=<fuga_s3_endpoint> s3api create-bucket --bucket test

To list your current buckets: 

$ aws --profile=fuga --endpoint=<fuga_s3_endpoint> s3api list-buckets
{
    "Buckets": [
        {
            "Name": "test",
            "CreationDate": "2021-10-20T12:34:37.650Z"
        },
    ],
    "Owner": {
        "DisplayName": "owner name",
        "ID": "owner ID."
    }
}

Now you have created a new bucket with the AWS CLI tools, this one is usable with S3 and Swift.

Using the object store

Using your own application

To use your own application, it needs to be compatible with S3 or Swift (Then you have to use the application or team credentials).

To configure the object store it requires mostly 3 things:

  • The endpoint:

    • This can be found in Access, then go to API Endpoints and search for S3.

  • The access key

  • The access secret

Using the EC2 key pair and S3 URL the S3 object can now be accessed using any S3 client or by using one of the various libraries available. Below is a code example using the boto library in Python.

from boto.s3.key import Key
import boto
import boto.s3.connection

access_key = '99999999999999999999999999999999'
secret_key = '99999999999999999999999999999999'
filename = 'test.txt'

conn = boto.connect_s3(
aws_access_key_id = access_key,
aws_secret_access_key = secret_key,
host = '<fuga_s3_endpoint',
port = <fuga_s3_port>,
calling_format = boto.s3.connection.OrdinaryCallingFormat(),
)

bucket = conn.get_bucket('mybucket')

k = Key(bucket)
k.key = filename
k.set_contents_from_filename(filename)
k.make_public()

We will be creating a .NET core project using a template with MVC.

  1. Create a new folder for the .NET Core project: 
    mkdir fugadotnet
  2. Open the folder in your terminal with the following command: 
    cd fugadotnet

  3. Create a new empty .NET Core ASP project by executing the following command: 

    dotnet new web
    An empty web project will be in your fugadotnet directory.

  4. Create a new folder inside your fugadotnet folder called Controllers: 

    mkdir Controllers

  5. Create a new folder inside your fugadotnet folder called Views: 
    mkdir Views
  6. Go inside the Views folder with the following command: 
    cd Views

  7. Inside the Views folder, create a new folder called ObjectStore.

    Enter the following command: 

    mkdir ObjectStore
    This is the name of the controller that we will create in the next chapter, feel free to change this.

  8. Open this project in your favorite IDE or text editor, for example, Visual Code or Atom.

Adding the AWS .NET S3 SDK to your project

The Fuga Object Store is Amazon S3 compatible, that’s why we can use the official Amazon SDK. We will be installing the AWS S3 SDK package to our .NET Core project.

  1. In your terminal execute the following command in your project root: 
    dotnet add package AWSSDK.S3 --version 3.3.18.3

Creating a new Controller

We will be creating a Controller, this will contain most of the logic for the Object Store.

  1. Inside your Controllers folder, create a new file called ObjectStoreController.cs within your text editor or with the following command: 
    touch ObjectStoreController.cs
  2. Open the ObjectStoreController.cs with your text editor and paste in the following code: 
    using System;
using Microsoft.AspNetCore.Mvc;
using System.IO;
using Amazon.S3;
using Amazon.S3.Model;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Http;

namespace fugadotnet.Controllers
{
    public class ObjectStoreController : Controller
    {
        private string container = "YOUR_CONTAINER_NAME_HERE";

        private AmazonS3Client fuga = Startup.GetS3Client();
        public IActionResult Index()
        {
            return View();
        }

        [HttpPost]
        public IActionResult Index(string name)
        {
            if (HttpContext.Request.Form.Files != null)
            {
                var files = HttpContext.Request.Form.Files;
                foreach (var file in files)
                {
                    Console.WriteLine(file.FileName);
                    if (file.Length > 0)
                    {
                        UploadObjectToObjectStore(file);
                    }
                }
            }
            return View();
        }

        public IActionResult Get([FromQuery(Name = "name")] string name)
        {
            ViewData["content"] = GetObject(name).Result;
            return View();
        }

        async Task<string> GetObject(string name)
        {
            GetObjectRequest request = new GetObjectRequest
            {
                BucketName = container,
                Key = name
            };
            using (GetObjectResponse response = await fuga.GetObjectAsync(request))
            using (Stream responseStream = response.ResponseStream)
            using (StreamReader reader = new StreamReader(responseStream))
            {
                return reader.ReadToEnd();
            }
        }
        async Task UploadObjectToObjectStore(IFormFile file)
        {
            var result = string.Empty;
            using (var reader = new StreamReader(file.OpenReadStream()))
            {
                result = reader.ReadToEnd();
            }
            var request = new PutObjectRequest
            {
                BucketName = container,
                Key = file.FileName,
                ContentBody = result
            };

            await fuga.PutObjectAsync(request);
        }
    }
}
  3. Inside your ObjectStoreController.cs, edit the container variable to your container name.
  4. Save the file.

Creating the views

  1. Inside your fugadotnet/Views/ObjectStore folder, create two views with the following command: 
    touch Index.cshtml Get.cshtml
  2. Open Index.cshtml with your text editor and paste in the following: 
    <h1>Fuga Object Store file upload</h1>
<form method="post" enctype="multipart/form-data" asp-controller="ObjectStore" asp-action="Index">
<div class="form-group">
    <div class="col-md-10">
        <p>Upload one or more files using this form:</p>
        <input type="file" name="files" multiple />
    </div>
</div>
<div class="form-group">
    <div class="col-md-10">
        <input type="submit" value="Upload" />
    </div>
</div>
</form>
  3. Save the file.
  4. Open Get.cshtml and paste in the following: 
        <b>@ViewData["content"]</b>
  5. Save the file.

Attaching our controller to the project and initializing the connection

Before

  1. Open the Startup.cs file located in the project root and paste in the following: 
    using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Amazon.Runtime;
using Amazon.S3;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.HttpsPolicy;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;

namespace fugadotnet
{
    public class Startup
    {
        private static AmazonS3Client client;
        public Startup(IConfiguration configuration)
        {
            Configuration = configuration;
            AmazonS3Config config = new AmazonS3Config();
            config.ServiceURL = "<fuga_s3_endpoint>";
            config.ForcePathStyle = true;
            config.SignatureVersion = "2";
            BasicAWSCredentials cred = new BasicAWSCredentials($"{Configuration["AccessKey"]}",
                                                                $"{Configuration["SecretKey"]}");
            client = new AmazonS3Client(cred, config);
        }

        public static AmazonS3Client GetS3Client()
        {
            return client;
        }

        public IConfiguration Configuration { get; }

        public void ConfigureServices(IServiceCollection services)
        {
            services.AddMvc().SetCompatibilityVersion(CompatibilityVersion.Version_2_1);
        }

        public void Configure(IApplicationBuilder app, IHostingEnvironment env)
        {
            if (env.IsDevelopment())
            {
                app.UseDeveloperExceptionPage();
            }

            app.UseMvc(routes =>
            {
                routes.MapRoute(
                    name: "default",
                    template: "{controller=ObjectStore}/{action=Index}/{id?}");
            });
        }
    }
}
  2. Save the file.

Configuring access to your Fuga Object Store container

  1. In your project root, create a new file called appsettings.json: 
    touch appsettings.json
  2. Copy paste in the following settings: 
    {
"AccessKey": "YOUR_ACCESS_KEY",
"SecretKey": "YOUR_SECRET_KEY"
}
  3. Replace the YOUR_ACCESS_KEY and YOUR_SECRET_KEY with the correct data.
  4. Save the file.

Testing the application

  1. Run the application by executing the following command inside your project root folder: 
    dotnet run
  2. Your web browser will show you a file upload page.
  3. Upload a text file, for example, test.txt with Hello, World as content.
  4. Browse to https://localhost:5001/ObjectStore/Get?name=test.txt

Hello, World will be displayed.

Mount the object store as a file system

If you are using Linux we can mount the Object Store using s3fs. With MacOS and Windows, we can use Cyberduck to connect to the Object Store.

This section shows you how to mount your Fuga object storage to your instance, this allows you to use your object storage like a volume.

  1. SSH into your instance or open a local terminal so that we can save the credentials.
  2. To create a new empty file for our credentials, enter the following command: 
    sudo touch /etc/passwd-s3fs
  3. To open the new file in a text editor like Nano, enter the following command: 
    sudo nano /etc/passwd-s3fs

  4. Paste (Ctrl + Shift + V in Nano or Vim) in your access key and secret key into the new file in the following format:

    • your_accesskey:your_secret key

      Example: 9f2XXXXX36314b1eaf64XXXXX0e74e3:0541e02XXXXXc79b25b6e0XXXXX32e0

  5. To save the file, first press Ctrl + X and if asked to save changes press ‘Y’. When asked for a filename to write, simply press enter.

  6. After saving our keys we should change the file permissions of this file to 600. Execute the following command: 
    sudo chmod 600 /etc/passwd-s3fs

Installing s3fs

  1. SSH into your instance or open a local terminal
  2. To install the required packages run the following command: 
    sudo apt-get install s3fs
  3. Before we can actually mount our Fuga Object Store container we first need to create a mounting point. For this tutorial, we will just mount the container to the standard /mnt. Start by creating an empty folder: 
    sudo mkdir /mnt

  4. To make our container mount automatically we can edit the fstab file. Open the fstab file with our favorite editor: 

    sudo nano /etc/fstab

    • Add the following line in your fstab file, make sure to replace YOUR_CONTAINER with the name of your container that you created on the Fuga dashboard: 
      s3fs#YOUR_CONTAINER /mnt fuse _netdev,allow_other,use_path_request_style,url=https://core.fuga.cloud:8080 0 0
    • To save the file, first press Ctrl + X and if asked to save changes press ‘Y’. When asked for a filename to write, simply press enter.

  5. To instantly mount your object storage enter the following command: 

    sudo mount -a

  6. You can find your Object Store at the mount point /mnt.

Now you’re done configuring your Fuga Object Store in Linux!

Fuga object storage is compatible with Amazon S3. Chances are, your favorite S3 client supports the Object Store. In this tutorial, we’re using the open source Cyberduck client. To connect to the Object Store we need some data:

The Fuga S3 URL: https://core.fuga.cloud:8080
The Access Key and the Secret Key we created or listed in the first paragraph

Next, create a new connection in Cyberduck. Select Amazon S3 and fill out the Server (S3 URL, without https:// and without 8080), poort: 8080, Username (Access Key) and Password (Secret Key) and click connect.

That’s it. You should be able to see your containers. You can now drag and drop items to and from containers in your object store.

Enjoy working with the object store!