During the agile design and architecture process of your new (web) service, two key questions will often be:
- How are we going to deal with object storage?
- Is there a better way than getting locked into Amazon S3 etc.?
Yes, and yes! Hello cloud-native world, meet Minio, your private, on-your-own-terms object storage solution, on any public cloud or server.
This blog post will guide you on how to run your cloud object storage on any server with Minio and Cloud 66 backed by Kubernetes.
What is Object storage?
Object storage, also known as object-based storage, is a computer data storage architecture that manages data as objects, as opposed to other storage architectures like file systems, which manage data as a file hierarchy, and block storage, which manages data as blocks within sectors and tracks.
Each object typically includes the data itself, a variable amount of metadata, and a globally unique identifier.
What is Minio?
Minio is an object storage server, compatible with the Amazon S3 cloud storage service. It is best suited for storing unstructured data such as photos, videos, log files, backups, and container/VM images. Object size can range from a few KBs to a maximum of 5TB.
Minio is like owning your own S3 Cloud on your premises, under your data privacy laws, and on your own terms—and it's very easy to setup with Cloud 66 on any public cloud or your own servers!
Run Minio locally for testing
Before running Minio in production, let's give it a test spin on your local box.
Just install docker and docker-compose (instructions here) and use the following docker-compose.yml.
version: '3.1'
services:
minio:
image: minio/minio
command: minio server /data
volumes:
# mount to local disk
- ./data:/data
ports:
# expose the ports
- "9000:9000"
environment:
# setup the credentials
MINIO_ACCESS_KEY: some_random_access_key
MINIO_SECRET_KEY: very_secret_key
Run docker-compose up and hit the end-point http://localhost:9000, then use the MINIO_ACCESS_KEY and MINIO_SECRET_KEY credentials to login.
BAM! You have your own S3-like object storage!
How to use it
Minio is compatible with all S3 commands—if your service can talk to S3 you can talk to Minio! No code changes.
Let's use an S3 command line tool called s3cmd as an example. You can find all the details here.
To play around with your local instalment, edit the following fields in your s3cmd configuration file ~/.s3cfg
# Setup endpoint
host_base = localhost:9000
host_bucket = localhost:9000
bucket_location = us-east-1
use_https = false
# Setup access keys
access_key = some_random_access_key
secret_key = very_secret_key
# Enable S3 v4 signature APIs
signature_v2 = False
Create a bucket in your brand new object storage:
s3cmd mb s3://hello-world-bucket
Add some files to your new bucket:
s3cmd put FILE [FILE...] s3://hello-world-bucket
List the contents of your bucket:
s3cmd ls s3://hello-world-bucket
Or use the UI to manage your buckets and files.
Run in production
It's time to run production on any cloud provider or on your own server.
We are going to deploy Minio on our Cloud 66 for Containers offering—a complete solution for building, running, and maintaining containerized apps in production. This will allow you to run a multi-tenant setup on your own server(s).
- Sign-up to Cloud 66, of course ;-)
- Create a new Project and use the following
service.yml
version: 2
services:
minio:
image: minio/minio:RELEASE.2017-08-05T00-00-53Z
command: minio server /data
volumes:
- /mnt/minio/data:/data
- /mnt/minio/config:/root/.minio
env_vars:
MINIO_ACCESS_KEY: change_to_random_string
MINIO_SECRET_KEY: change_to_random_string
ports:
- 9000:80:443
A service.yml is a Cloud 66 service definition file that tells you which services we want to run on your infrastructure. (For more about service.yml, check our documentation.)
Given the ephemeral nature of containers, it’s important to consider storage solutions to avoid data loss. That's the reason you find the volume statement in service.yml
volumes:
- /mnt/minio/data:/data
- /mnt/minio/config:/root/.minio
Choose to deploy in a production environment
Choose which provider you want to deploy (make sure you entered your provider credentials) or bring your own server using our registered server feature.
Hit deploy! and wait until we're done with all the heavy lifting. After a couple of minutes, you will have deployed your own object storage, backed by Kubernetes!
Yes! Kubernetes in Space.
Securing Minio with SSL
Of course, this instalment is not secure because we are using HTTP and not HTTPS. Cloud 66 support Let's Encrypt (FREE!) SSL, or you can bring your own SSL.
Again, we take care of all the heavy lifting and when the SSL certificate is created and installed, you have your Minio running with SSL!
Look! Green lock! ;-)
Test drive production
To start using your production Minio service. Edit the following fields in your s3cmd configuration file ~/.s3cfg
# Setup endpoint
host_base = [your_domain_name]
host_bucket = [your_domain_name]
bucket_location = us-east-1
# use HTTPS !!!
use_https = true
# Setup access keys
access_key = some_random_access_key
secret_key = very_secret_key
# Enable S3 v4 signature APIs
signature_v2 = False
Check if you can create a bucket in your brand new object storage:
s3cmd mb s3://hello-world-bucket
BAM! Object storage at your service.
Summary
Deploying your own S3-compatible object storage is as simple as buying a pair of shoes. It's fun running Minio on any server! Give us a spin a let us know what you think of our services.
Start using the space!