How do I use Terraform with Wasabi?

Terraform is an open-source tool that allows you to manage your infrastructure as code. Below are the instructions to use Terraform with Wasabi. Note that these examples discuss the use of Wasabi's us-east-1 storage regions. To use other Wasabi storage regions, please use the appropriate Wasabi service URL as described in this article.

Note that Terraform has a built-in mechanism to check few flags (via AWS S3 plugin model) wherein some features are not currently supported on Wasabi which is supported on AWS S3 and hence the deployment of the IAAS model would fail in those cases. This KB doc demonstrates the IAAS including automation deployment using both AWS & MinIO plugin to overcome that limitation

Install Terraform

1. Download Terraform

2. Install by unzipping the downloaded file and run the executable file

3. Verify by typing the below command in a new terminal

terraform --version

4. Terraform describes the infrastructure in a .tf configuration file. Below is a sample config file that connects to Wasabi and uploads the state file to the bucket:

Terraform must store state about your managed infrastructure and configuration. This state is used by Terraform to map real-world resources to your configuration, keep track of metadata, and improve performance for large infrastructures.

This state is stored by default in a local file named "terraform.tfstate",

terraform {
    required_version = ">= 0.11, < 0.12"
    backend "s3" {
    skip_requesting_account_id = true
    skip_credentials_validation = true
    skip_get_ec2_platforms = true
    skip_metadata_api_check = true
    access_key = "<WASABI-ACCESS-KEY>"
    secret_key = "<WASABI-SECRECT-KEY>"
    endpoint = "https://s3.wasabisys.com"
    region = "us-east-1"
    bucket = "<BUCKET>"
    key = "<PATH-IN-BUCKET>/terraform.tfstate"
  }
}

The below example shows how to create an S3 Bucket on Wasabi using Terraform

Note that this example discusses the use of Wasabi's us-east-1 storage region. To use other Wasabi storage regions, please use the appropriate Wasabi service URL as described in this article.

provider aws {
  region = "us-east-1"
  access_key = "XXXXXXXX"
  secret_key = "XXXXXXXXXXXXXXXX"

  endpoints {
    sts = "https://sts.wasabisys.com"
    iam = "https://iam.wasabisys.com"
    s3 = "https://s3.wasabisys.com"
  }

  s3_force_path_style = true
  skip_get_ec2_platforms = true
}
resource aws_s3_bucket username-terraform-us-east-1-testing-1 {
  bucket = "username-terraform-us-east-1-testing-1"
  acl = "private"
}

Note: using the method above with the AWS service provider to create buckets currently returns an error like this:

This error occurs when using a different S3 implementation (in this case, Wasabi) some features have been disabled due to not being completely implemented on the server side.

The Terraform team is currently working on a fix where they support applying a skip to non-supported features on the server-side for example 'replication_config'. Terraform has an open issue being worked on here which is a similar case with AWS acceleration causing issues: https://github.com/hashicorp/terraform-provider-aws/issues/14775

What we recommend as a workaround is using MinIO and this MinIO plugin to connect to Wasabi as an AWS gateway. MinIO is an application that can directly connect to Wasabi and overcome some of the issues/limitations that we are currently facing with Terraform's AWS provider.

------------------------------------------------------------------------------------------------------

Install MinIO

1. Download MinIO Server (We will be using Unbuntu 20.04 in our example below)

2. Install by following the instructions for your specific OS.

3. Configure the access key & secret key by exporting them:

export MINIO_ROOT_USER=<access-key>

export MINIO_ROOT_PASSWORD=<secret-key>

Steps to execute:

Once you have Installed Terraform and MinIO

Run the following shell script to start the Minio server and connect to your bucket on Wasabi as an s3 gateway using the service URLs provided here: What are the service URLs for Wasabi's different storage regions?
```
./minio gateway s3 https://<service-url>
```
In this example, we are using the us-east-1 endpoint. Please be sure to use the correct region endpoint URL based on the location of your bucket
```
./minio gateway s3 https://s3.wasabisys.com
```
NOTE: If you see this error:

make sure that you have RUN the following before starting the gateway

export MINIO_ROOT_USER=<access-key>

export MINIO_ROOT_PASSWORD=<secret-key>

This will run a MinIO Server at https://localhost:9000 Enter your access-key and secret-key here if you wish to navigate your buckets on the Wasabi account.

Running the server should look something like this:

NOTE: Please do not close this terminal as we will need to keep it open for the remainder of the operation.

Create a main.tf file with the following data in your preferred location.
This file contains the code to set up the plugins and resolve dependencies required for the execution.

# Plugin required to create buckets through minio.
terraform {
  required_providers {
    minio = {
      source = "aminueza/minio"
      version = ">= 1.0.0"
    }
  }
  required_version = ">= 0.13"
}

# minio provider for creating buckets.
provider "minio" {
  minio_server = var.minio_server
  minio_region = var.region
  minio_access_key = var.access_key
  minio_secret_key = var.secret_key

Create variables.tf file with the following data.
This file contains variables such as an access key, secret key, and region which are used by the main.tf file.

Replace the access key and secret key with your own credentials.

The region here should be the same as the region that you started the MinIO server, so in our case it is us-east-1.

# region for aws and minio
variable "region" {
  description = "Default region"
  default = "us-east-1"
}

# server location for local minio server
variable "minio_server" {
  description = "Default MINIO host and port"
  default = "localhost:9000"
}

# access key
variable "access_key" {
  description = "User"
  default = "<insert access key here>"
}

# secret key
variable "secret_key" {
  description = "Secret user"
  default = "<insert secret key here>"

Create a wasabi.tf file with the following data:
This file contains the code to create a simple bucket on Wasabi. Replace the bucket field with your test bucket name. In our case, we are naming it "rv-test-terraform-bucket-minio".
```
resource "minio_s3_bucket" "bucket" {
bucket = "<insert bucket name here>"
acl = "private"
}
```
Now we need to run this script. Open another terminal and navigate to your preferred location where you created the 'main.tf', 'variables.tf' and 'wasabi.tf' scripts and run the following commands.
Go to the location where the .tf scripts are present and run the following commands:
terraform init
This will initialize the provider plugins.
terraform plan
This will see if any changes are required for your infrastructure.
terrafrom apply
This will apply to the planned action.
NOTE: It will prompt you to type yes. Type yes and enter.
If you navigate to your account using the Wasabi Management Console you will see the bucket has been created.

---------------------------------------------------------------------------------------------------

AUTOMATION using Terraform

NOTE: This infrastructure automation example follows the exact model detailed in this KB document

How to automate infrastructure design on Wasabi

1. For automation, follow the steps above to install Terraform and MinIO and run the MinIO server.

2. Create the following ".tf" files:

main.tf

We will be using this to create all configurations for plugins. Two important plugins are:

'minio' for creating buckets
'aws' provider with the custom endpoint to create IAM groups, users, and policies

# Plugin required to create buckets through minio.
terraform {
  required_providers {
    minio = {
      source = "aminueza/minio"
      version = ">= 1.0.0"
    }
  }
  required_version = ">= 0.13"
}

# AWS provider to connect sts, iam and s3
provider aws {
  alias = "wasabi"
  region = var.region

  #coded credentials from variables file
  access_key = var.access_key
  secret_key = var.secret_key

  endpoints {
    sts = "https://sts.wasabisys.com"
    iam = "https://iam.wasabisys.com"
    s3 = "https://s3.wasabisys.com"
  }

  # skip checks
  s3_force_path_style = true
  skip_get_ec2_platforms = true
  skip_credentials_validation = true
  skip_metadata_api_check = true
  skip_requesting_account_id = true
}

# minio provider for creating buckets.
provider "minio" {
  minio_server = var.minio_server
  minio_region = var.region
  minio_access_key = var.access_key
  minio_secret_key = var.secret_key
}

variables.tf

Replace the access key and secret key with your credentials.
The region here should be the same as the region that you started the MinIO server with so in our case it is us-east-1.
Replace the usernames with a list of usernames required.

NOTE: The name-addon appends "wasabi-technologies-" to the user as dictated by the policy in the wasabi.tf file.

# region for aws and minio
variable "region" {
  description = "Default region"
  default = "us-east-1"
}

# server location for local minio server
variable "minio_server" {
  description = "Default MINIO host and port"
  default = "localhost:9000"
}

# access key
variable "access_key" {
  description = "User"
  default = "<insert access key here>"
}

# secret key
variable "secret_key" {
  description = "Secret user"
  default = "<insert secret key here>"
}

# prefix appended to bucket name and policy
variable "name-addon" {
  type = string
  description = "a prefix added to the usernames"
  default = "wasabi-technologies-" # see the policy in wasabi.tf file and chage that as well
}

# list of usernames to be added to the group
variable "usernames" {
  description = "usernames to be created on Wasabi"
  default = ["<insert usernames here>"] # e.g. ["user1","user2"]
}

# group name.
variable "group-name" {
  description = "name of the group users will be added to."
  default = "restricted-access-group"
}

wasabi.tf

This file contains the code for the automation.

# 1. create IAM users based on the variables list
resource "aws_iam_user" "iam_users" {
  for_each = toset(var.usernames)
  name = each.key
  provider = aws.wasabi
  force_destroy = true
}

# 2. create access key for created user
resource "aws_iam_access_key" "iam_user_keys" {
  for_each = aws_iam_user.iam_users
  user = each.key
  provider = aws.wasabi
  depends_on = [
    aws_iam_user.iam_users]
}

# 3. Create buckets
resource "minio_s3_bucket" "bucket" {
  for_each = toset(var.usernames)
  bucket = format("%s%s", var.name-addon, each.key)
  acl = "private"
}

# 4. Create Group
resource "aws_iam_group" "restricted_access_group" {
  name = var.group-name
  provider = aws.wasabi
}

# 5. Create Policy
resource "aws_iam_policy" "restricted_access_policy" {
  name = "restricted-access-policy"

  policy = jsonencode(
  {
    "Version": "2012-10-17",
    "Statement": [
      {
        Effect: "Allow",
        Action: "s3:ListAllMyBuckets",
        Resource: "arn:aws:s3:::*"
      },
      {
        Effect: "Allow",
        Action: "s3:*",
        Resource: [
          "arn:aws:s3:::wasabi-technologies-$${aws:username}",
          "arn:aws:s3:::wasabi-technologies-$${aws:username}/*"
        ]
      }
    ]
  }
  )
  provider = aws.wasabi
}

# 6. Attach policy to group
resource "aws_iam_policy_attachment" "test-attach" {
  name = "test-attachment"
  groups = [
    aws_iam_group.restricted_access_group.name]
  policy_arn = aws_iam_policy.restricted_access_policy.arn
  provider = aws.wasabi
  depends_on = [
    aws_iam_policy.restricted_access_policy,
    aws_iam_group.restricted_access_group]
}

# 7. Attach user to group
resource "aws_iam_group_membership" "add_users_to_group" {
  name = "tf-testing-group-membership"
  for_each = aws_iam_user.iam_users
  users = var.usernames
  group = aws_iam_group.restricted_access_group.name
  provider = aws.wasabi
  depends_on = [
    aws_iam_group.restricted_access_group,
    aws_iam_user.iam_users]
}

output.tf

This file is used to output a secret key created for the users.

output "secret" {
  value = tomap({
  for k, s in aws_iam_access_key.iam_user_keys : k => [
    s.id,
    s.secret]
  })
}

3. terraform init
This will initialize the provider plugins

4. terraform plan
This will see if any changes are required for your infrastructure.

5. terrafrom apply
This will apply to the planned action.

Have more questions? Submit a request