Moved to OCI

I have completely migrated my blog from iPage.com to Oracle Cloud Infrastructure. There were some struggles over the last few weeks but I have learned how to solve the problems.

I will start to blog about the struggles I had to migrate my blog. One thing that I have accomplished is Dockerize WordPress + NGINX. This is something I have always wanted to do. My struggles paid off and my blog runs so much faster. It runs on a VM with 1 CPU with 2 GB memory on AMD. I will see how much it costs and will talk about it here.

My site runs on Ubuntu. I was able to clean up some mess that accumulated over the years in the database too.

Will blog again.

How to List available Shapes on OCI

Here is an easy way to list available shapes on Oracle Cloud Infrastrucuture.

oci compute shape list -c $tenancy_id --query "data[*]".{'shape:shape'} --output table

As of 6/5/2001, here is the list of available shapes on OCI.

+------------------------+
| shape                  |
+------------------------+
| BM.Standard.A1.160     |
| BM.Standard2.52        |
| BM.Optimized3.36       |
| BM.Standard.E4.128     |
| BM.Standard.E3.128     |
| BM.Standard.E2.64      |
| BM.Standard1.36        |
| VM.Optimized3.Flex     |
| VM.Standard.E4.Flex    |
| VM.Standard.E3.Flex    |
| VM.Standard.A1.Flex    |
| VM.Standard2.1         |
| VM.Standard2.2         |
| VM.Standard2.4         |
| VM.Standard2.8         |
| VM.Standard2.16        |
| VM.Standard2.24        |
| VM.Standard.E2.1       |
| VM.Standard.E2.2       |
| VM.Standard.E2.4       |
| VM.Standard.E2.8       |
| VM.Standard1.1         |
| VM.Standard1.2         |
| VM.Standard1.4         |
| VM.Standard1.8         |
| VM.Standard1.16        |
| BM.Standard.A1.160     |
| BM.Standard2.52        |
| BM.Optimized3.36       |
| BM.Standard.E4.128     |
| BM.Standard.E3.128     |
| BM.Standard.E2.64      |
| BM.Standard1.36        |
| VM.Optimized3.Flex     |
| VM.Standard.E4.Flex    |
| VM.Standard.E3.Flex    |
| VM.Standard.A1.Flex    |
| VM.Standard2.1         |
| VM.Standard2.2         |
| VM.Standard2.4         |
| VM.Standard2.8         |
| VM.Standard2.16        |
| VM.Standard2.24        |
| VM.Standard.E2.1       |
| VM.Standard.E2.2       |
| VM.Standard.E2.4       |
| VM.Standard.E2.8       |
| VM.Standard1.1         |
| VM.Standard1.2         |
| VM.Standard1.4         |
| VM.Standard1.8         |
| VM.Standard1.16        |
| BM.Standard.A1.160     |
| BM.Standard2.52        |
| BM.Optimized3.36       |
| BM.Standard.E4.128     |
| BM.Standard.E3.128     |
| BM.Standard.E2.64      |
| BM.Standard1.36        |
| VM.Optimized3.Flex     |
| VM.Standard.E4.Flex    |
| VM.Standard.E3.Flex    |
| VM.Standard.A1.Flex    |
| VM.Standard2.1         |
| VM.Standard2.2         |
| VM.Standard2.4         |
| VM.Standard2.8         |
| VM.Standard2.16        |
| VM.Standard2.24        |
| VM.Standard.E2.1       |
| VM.Standard.E2.2       |
| VM.Standard.E2.4       |
| VM.Standard.E2.8       |
| VM.Standard.E2.1.Micro |
| VM.Standard1.1         |
| VM.Standard1.2         |
| VM.Standard1.4         |
| VM.Standard1.8         |
| VM.Standard1.16        |
+------------------------+

Checking a Port is Open

I have a VM in Japan on Oracle Cloud Infrastructure but I cannot seem to reach the target port on a resource.

I tried the following command and the port 22 seems to be open.

nc -zv [my public IP address] 22

The command give me…

Connection to [my public IP address] port 22 [tcp/ssh] succeeded!

But when I try 3128 which is the target port I want to reach, it times out. Hmm the routing may not be going well. I checked the port on the local machine and a process is listening and it is open at the OS level. nc -zv is a useful tool to check if ports are open.

How to Flatten OCI CLI Data

OCI CLI is a very convenient way to get your cloud resource data. Quite often, the data that gets returned from the API is nested. For example, when you get the data on your VM, the command and the data that gets returned looks like this.

oci compute instance list -c $tenancy_id --region ap-tokyo-1

* I have my instance right under the root compartment (tenancy).

Here is the data that gets returned by executing the command.

{
  "data": [
    {
      <SNIP>
      "display-name": "tokyo-proxy",
      "extended-metadata": {},
      "fault-domain": "FAULT-DOMAIN-1",
      "freeform-tags": {},    
      <SNIP>
      "lifecycle-state": "STOPPED",
      "metadata": {
        "ssh_authorized_keys": "ssh-rsa fakepublickeyhere  hiriumi@hayato-mac-mini.local\n"
      },
      "region": "ap-tokyo-1",
      "shape": "VM.Standard.E3.Flex",
      "shape-config": {
        "gpu-description": null,
        "gpus": 0,
        "local-disk-description": null,
        "local-disks": 0,
        "local-disks-total-size-in-gbs": null,
        "max-vnic-attachments": 2,
        "memory-in-gbs": 2.0,
        "networking-bandwidth-in-gbps": 1.0,
        "ocpus": 1.0,
        "processor-description": "2.25 GHz AMD EPYC\u2122 7742 (Rome)"
      },
      "source-details": {
        "boot-volume-size-in-gbs": null,
        "image-id": "ocid1.image.oc1.ap-tokyo-1.aaaaaaaavnulbem7sdol6pzlqadzwrcej2yyg3negvbz7ullnsoaspluzydq",
        "kms-key-id": null,
        "source-type": "image"
      },
      "system-tags": {},
      "time-created": "2020-11-14T07:29:26.033000+00:00",
      "time-maintenance-reboot-due": null
    }
  ]
}

Notice that the highlighted lines are the nested data. To limit the data to a few data points, you could do something like this.

oci compute instance list --region ap-tokyo-1 -c $tenancy_id \
--query 'data[]'.{'name:"display-name",shape_config:"shape-config"'} --output table

By adding –query option, you can limit the data and by adding –output table, you can format the data in an easy-to-see format but the output does not look good because the shape-config node has child nodes.

------------------------------------------------------------------------------------------+
| name        | shape_config                                                                                                                                                                                                                                                                                 |
+-------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| tokyo-proxy | {'ocpus': 1.0, 'memory-in-gbs': 2.0, 'processor-description': '2.25 GHz AMD EPYC™ 7742 (Rome)', 'networking-bandwidth-in-gbps': 1.0, 'max-vnic-attachments': 2, 'gpus': 0, 'gpu-description': None, 'local-disks': 0, 'local-disks-total-size-in-gbs': None, 'local-disk-description': None} |
+-------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

By executing the following command, we can flatten the child nodes and limit the data to display even further.

oci compute instance list -c $tenancy_id --region ap-tokyo-1 \
--query 'data[]'.{'name:"display-name",cores:"shape-config".ocpus,memory:"shape-config"."memory-in-gbs"'} --output table

Here is the sample output.

+-------+--------+-------------+
| cores | memory | name        |
+-------+--------+-------------+
| 1.0   | 2.0    | tokyo-proxy |
+-------+--------+-------------+

It took me a while to figure this out so I thought I’d share it here in case someone is looking for a way to flatten the nested data using OCI CLI.

How to Provision a VM on OCI with Flexible Shape

OCI has a lot of shapes to choose from. What is a Shape? Here is a quote from the official site.

A shape is a template that determines the number of CPUs, amount of memory, and other resources that are allocated to an instance.

So what if you cannot find the shape you would like to use? It is possible to use this VM.Standard.E3.Flex shape and specify the size of the memory and CPUs. Here is an example I am using to spin up a proxy server in Tokyo.

resource oci_core_instance tokyo-proxy {
    compartment_id      = var.tenancy_ocid
    display_name        = "tokyo-proxy"
    availability_domain = "GUMM:AP-TOKYO-1-AD-1"
    shape               = "VM.Standard.E3.Flex"

    shape_config {
        memory_in_gbs = 2
        ocpus = 1
    }
    
    source_details {
        source_type = "image"
        source_id   = var.config["image_ocid"]
        boot_volume_size_in_gbs = 50
    }
    create_vnic_details {
        assign_public_ip = "true"
        hostname_label   = "tokyo-proxy"
        subnet_id        = var.config["public_subnet_ocid"]
    }
    metadata = {
        ssh_authorized_keys = file(var.public_ssh_key)
    }
}

This provisions quite a small instance in ap-tokyo-1 region, which I am planning to use as a proxy server to watch movies from Japan. Even a smaller spec would do but I will go with it for the time being.

I believe the boot volume size cannot be lower than 50 GB. You can read the official documentation regarding the minimum boot volume size.

How to List Images for a Region in OCI CLI

I am in the process of creating my own infrastructure in Oracle Cloud Infrastructure (OCI). I’ve set up a VCN (virtual cloud network) in Tokyo region and now I want to spin up a VM. I need to know which image to use and Terraform OCI provider requires image ID. Here is a way to list available images with their IDs (OCID) in OCI CLI.

oci compute image list -c $TENANCY_ID --region ap-tokyo-1 \
--query "data[*]".{'name:"display-name",id:id'} --output table

Here is the kind of data you’d get.

| ocid1.image.oc1.ap-tokyo-1.aaaaaaaaec34d6ybpedtu5sxhtwvurvdy4kp7bqr4ijnqiwvh75xxmkqnufq | Windows-Server-2019-Standard-Edition-VM-Gen2-2020.10.22-0              |
| ocid1.image.oc1.ap-tokyo-1.aaaaaaaatdmex2izbi7d7zdznvkyxdeygamojfnd3kroplg4d3p4i5rdnnra | Windows-Server-2019-Standard-Edition-VM-Gen2-2020.09.21-0              |
<SNIP>
| ocid1.image.oc1.ap-tokyo-1.aaaaaaaayqu2pdreelb4dtqyrroiyi5vzc4fkpgh24dnrhfdhbk2irkhetyq | Canonical-Ubuntu-20.04-Minimal-2020.08.24-0                            |
| ocid1.image.oc1.ap-tokyo-1.aaaaaaaahk3krxqgimom7cy2z4b5lsoakm6bhmnbaaaincvgtuu4wivntxjq | Canonical-Ubuntu-20.04-2020.10.14-0                                    |
| ocid1.image.oc1.ap-tokyo-1.aaaaaaaa3ioe7z5wmi7lfk4xyoret7tmlyr2g2jowsqzh4hz6qcd5pakdtda | Canonical-Ubuntu-20.04-2020.09.07-0                                    |
| ocid1.image.oc1.ap-tokyo-1.aaaaaaaax6wjwsktl7kpl3zw3gfn34cqp76b36usraojs4jors3lmfmg52ba | Canonical-Ubuntu-20.04-2020.08.21-0                                    |

Take a note of the ID for the image you want to use and save it in one of the Terraform files to use it.

For a convenience, you may want to know how to list Shapes as well.

oci compute shape list -c $TENANCY_ID \
--query "data[*]".{'shape:shape,memory:"memory-in-gbs",ocpus:ocpus'} \
--output table
| 768.0  | 52.0  | BM.Standard2.52        |
| 2048.0 | 128.0 | BM.Standard.E3.128     |
<SNIP>
| 64.0   | 8.0   | VM.Standard.E2.8       |
| 1.0    | 1.0   | VM.Standard.E2.1.Micro |
| 56.0   | 8.0   | VM.Standard1.8         |

How to List All Available Regions in OCI CLI

When you use Terraform, you are going to want to know the name of the region to use. I’ll show you in a easy-to-see way.

oci iam region list \
--query "data[*]".{'key:key,name:name'} --output table

The command above shows the following output.

+-----+----------------+
| key | name           |
+-----+----------------+
| AMS | eu-amsterdam-1 |
| BOM | ap-mumbai-1    |
| DXB | me-dubai-1     |
| FRA | eu-frankfurt-1 |
| GRU | sa-saopaulo-1  |
| HYD | ap-hyderabad-1 |
| IAD | us-ashburn-1   |
| ICN | ap-seoul-1     |
| JED | me-jeddah-1    |
| KIX | ap-osaka-1     |
| LHR | uk-london-1    |
| MEL | ap-melbourne-1 |
| NRT | ap-tokyo-1     |
| PHX | us-phoenix-1   |
| SJC | us-sanjose-1   |
| SYD | ap-sydney-1    |
| YNY | ap-chuncheon-1 |
| YUL | ca-montreal-1  |
| YYZ | ca-toronto-1   |
| ZRH | eu-zurich-1    |
+-----+----------------+

It is possible to view available regions from the web console though it’s paged. Click the region -> Manage Regions to view.

It is wise to subscribe to only the regions you are going to use. Subscribing to regions do not cost at all, however.

What I Wish I Had Known about Terraform When I Started

I feel extremely lucky that I am working for the company that I am working for. It’s a job and there are things I don’t like but I am doing what I have wanted to do. I think Terraform is an important technology to manage cloud resources, so I am listing the things I wish I had known

  1. Terraform is a compiled go-lang executable.
  2. Terraform communicates with Terraform state file. It reads from and writes to the state file.
  3. Terraform communicates with cloud through provider.
  4. Terraform state file can be stored locally or on Object Storage.
  5. Terraform can import existing cloud resources to the state file.

The list can go on. I will add more when they pop in my mind.

How to Start Terraform with Oracle Cloud Infrastructure (OCI)

Terraform, as you may know, is a very convenient way to manage resources in cloud services and Oracle Cloud Infrastructure (OCI) is one of them. Infrastructure as code is the way to go for sure. I’m not going to discuss how great it is here but I will introduce how to get started with Terraform with OCI.

Prerequisites

  1. OCI Account. Free Oracle Cloud Promotion is available from here if you don’t have one.
  2. You already have OCI CLI installed and it is in a working state. Please go through this blog article if you have not.

Download and Install Terraform

You can download Terraform from here. Terraform is a single executable so installation is simply place it somewhere and give the location to $PATH.

terraform --version

I am using Terraform v0.13.4 as of October 2020.

Steps

Terraform can use the credential that is already set up for OCI CLI. I will show how to do it below.

Create a directory where you store all the Terraform files. e.g. ~/dev/terraform

Now Create terraform.tfvars with the following contents.

user_profile = "DEFAULT"

The “DEFAULT” indicates the profile to use for Terraform to communicate with OCI in ~/.oci/config file created when you setup for OCI CLI.

Create provider.tf file with the following contents.

provider "oci" {
    alias = "us-ashburn-1"
    region = "us-ashburn-1"
    tenancy_ocid = var.tenancy_id
    config_file_profile = var.user_profile
}

Create variables.tf with the following contents. Replace the tenancy_id with your own.

variable "tenancy_id" {
    default = "ocid1.tenancy.oc1..aaaaaaaaca7lntmtszny3mgfhmzb5jb5oi6xucnrb7z6emorxdzheos4m4pa"
}

variable "user_profile" {
    default = ""
}

variable "fingerprint" {
    default =  ""
}

variable "private_key_path" {
    default = ""
}

variable "user_ocid" {
    default = ""
}

At this point, you should have the following 3 files in the directory.

  • provider.tf
  • terraform.tfvars
  • variables.tf

Now execute the following command to initialize terraform.

terraform init

The command creates .terraform directory and it has some stuff you really shouldn’t modify.

Then execute this.

terraform plan

You should see…

No changes. Infrastructure is up-to-date.

Let’s see if we can create a compartment. In case you don’t know, a compartment is a logical group of resources on OCI.

Create a file compartments.tf with the following contents. Replace the value of compartment_id to your tenancy ID. It indicates the parent compartment of the compartment you are planning to create. In this case, the compartment is going to be created at the root level of the tenancy, so I am specifying the tenancy ID.

resource "oci_identity_compartment" "blog" {
    compartment_id = "ocid1.tenancy.oc1..aaaaaaaaca7lntmtszny3mgfhmzb5jb5oi6xucnrb7z6emorxdzheos4m4pa"
    description = "blog compartment"
    name = "blog"
    enable_delete = "true"
}

Once it’s done, execute terraform plan to show what Terraform is going to do.

terraform plan

Here is the plan output.

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # oci_identity_compartment.blog will be created
  + resource "oci_identity_compartment" "blog" {
      + compartment_id = "ocid1.tenancy.oc1..aaaaaaaaca7lntmtszny3mgfhmzb5jb5oi6xucnrb7z6emorxdzheos4m4pa"
      + defined_tags   = (known after apply)
      + description    = "blog compartment"
      + enable_delete  = true
      + freeform_tags  = (known after apply)
      + id             = (known after apply)
      + inactive_state = (known after apply)
      + is_accessible  = (known after apply)
      + name           = "blog"
      + state          = (known after apply)
      + time_created   = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Once it looks good, apply the change by executing terraform apply.

terraform apply

Enter yes when prompted and hit enter. If you see it on OCI web console, you should actually see the new compartment under the root.

One thing to note is that terraform apply generates terraform.tfstate file. Do not delete or manually modify it. It has the information of your OCI resources that you are managing with Terraform.

Terraform can pretty much manage all resources in OCI. The OCI provider reference of Terraform can be found here. Now that my Terraform can talk to my OCI tenancy, I plan to manage resources with it.