How to Flatten OCI CLI Data

OCI CLI is a very convenient way to get your cloud resource data. Quite often, the data that gets returned from the API is nested. For example, when you get the data on your VM, the command and the data that gets returned looks like this.

oci compute instance list -c $tenancy_id --region ap-tokyo-1

* I have my instance right under the root compartment (tenancy).

Here is the data that gets returned by executing the command.

{
  "data": [
    {
      <SNIP>
      "display-name": "tokyo-proxy",
      "extended-metadata": {},
      "fault-domain": "FAULT-DOMAIN-1",
      "freeform-tags": {},    
      <SNIP>
      "lifecycle-state": "STOPPED",
      "metadata": {
        "ssh_authorized_keys": "ssh-rsa fakepublickeyhere  hiriumi@hayato-mac-mini.local\n"
      },
      "region": "ap-tokyo-1",
      "shape": "VM.Standard.E3.Flex",
      "shape-config": {
        "gpu-description": null,
        "gpus": 0,
        "local-disk-description": null,
        "local-disks": 0,
        "local-disks-total-size-in-gbs": null,
        "max-vnic-attachments": 2,
        "memory-in-gbs": 2.0,
        "networking-bandwidth-in-gbps": 1.0,
        "ocpus": 1.0,
        "processor-description": "2.25 GHz AMD EPYC\u2122 7742 (Rome)"
      },
      "source-details": {
        "boot-volume-size-in-gbs": null,
        "image-id": "ocid1.image.oc1.ap-tokyo-1.aaaaaaaavnulbem7sdol6pzlqadzwrcej2yyg3negvbz7ullnsoaspluzydq",
        "kms-key-id": null,
        "source-type": "image"
      },
      "system-tags": {},
      "time-created": "2020-11-14T07:29:26.033000+00:00",
      "time-maintenance-reboot-due": null
    }
  ]
}

Notice that the highlighted lines are the nested data. To limit the data to a few data points, you could do something like this.

oci compute instance list --region ap-tokyo-1 -c $tenancy_id \
--query 'data[]'.{'name:"display-name",shape_config:"shape-config"'} --output table

By adding –query option, you can limit the data and by adding –output table, you can format the data in an easy-to-see format but the output does not look good because the shape-config node has child nodes.

------------------------------------------------------------------------------------------+
| name        | shape_config                                                                                                                                                                                                                                                                                 |
+-------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| tokyo-proxy | {'ocpus': 1.0, 'memory-in-gbs': 2.0, 'processor-description': '2.25 GHz AMD EPYC™ 7742 (Rome)', 'networking-bandwidth-in-gbps': 1.0, 'max-vnic-attachments': 2, 'gpus': 0, 'gpu-description': None, 'local-disks': 0, 'local-disks-total-size-in-gbs': None, 'local-disk-description': None} |
+-------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

By executing the following command, we can flatten the child nodes and limit the data to display even further.

oci compute instance list -c $tenancy_id --region ap-tokyo-1 \
--query 'data[]'.{'name:"display-name",cores:"shape-config".ocpus,memory:"shape-config"."memory-in-gbs"'} --output table

Here is the sample output.

+-------+--------+-------------+
| cores | memory | name        |
+-------+--------+-------------+
| 1.0   | 2.0    | tokyo-proxy |
+-------+--------+-------------+

It took me a while to figure this out so I thought I’d share it here in case someone is looking for a way to flatten the nested data using OCI CLI.

How to Provision a VM on OCI with Flexible Shape

OCI has a lot of shapes to choose from. What is a Shape? Here is a quote from the official site.

A shape is a template that determines the number of CPUs, amount of memory, and other resources that are allocated to an instance.

So what if you cannot find the shape you would like to use? It is possible to use this VM.Standard.E3.Flex shape and specify the size of the memory and CPUs. Here is an example I am using to spin up a proxy server in Tokyo.

resource oci_core_instance tokyo-proxy {
    compartment_id      = var.tenancy_ocid
    display_name        = "tokyo-proxy"
    availability_domain = "GUMM:AP-TOKYO-1-AD-1"
    shape               = "VM.Standard.E3.Flex"

    shape_config {
        memory_in_gbs = 2
        ocpus = 1
    }
    
    source_details {
        source_type = "image"
        source_id   = var.config["image_ocid"]
        boot_volume_size_in_gbs = 50
    }
    create_vnic_details {
        assign_public_ip = "true"
        hostname_label   = "tokyo-proxy"
        subnet_id        = var.config["public_subnet_ocid"]
    }
    metadata = {
        ssh_authorized_keys = file(var.public_ssh_key)
    }
}

This provisions quite a small instance in ap-tokyo-1 region, which I am planning to use as a proxy server to watch movies from Japan. Even a smaller spec would do but I will go with it for the time being.

I believe the boot volume size cannot be lower than 50 GB. You can read the official documentation regarding the minimum boot volume size.

How to List Images for a Region in OCI CLI

I am in the process of creating my own infrastructure in Oracle Cloud Infrastructure (OCI). I’ve set up a VCN (virtual cloud network) in Tokyo region and now I want to spin up a VM. I need to know which image to use and Terraform OCI provider requires image ID. Here is a way to list available images with their IDs (OCID) in OCI CLI.

oci compute image list -c $TENANCY_ID --region ap-tokyo-1 \
--query "data[*]".{'name:"display-name",id:id'} --output table

Here is the kind of data you’d get.

| ocid1.image.oc1.ap-tokyo-1.aaaaaaaaec34d6ybpedtu5sxhtwvurvdy4kp7bqr4ijnqiwvh75xxmkqnufq | Windows-Server-2019-Standard-Edition-VM-Gen2-2020.10.22-0              |
| ocid1.image.oc1.ap-tokyo-1.aaaaaaaatdmex2izbi7d7zdznvkyxdeygamojfnd3kroplg4d3p4i5rdnnra | Windows-Server-2019-Standard-Edition-VM-Gen2-2020.09.21-0              |
<SNIP>
| ocid1.image.oc1.ap-tokyo-1.aaaaaaaayqu2pdreelb4dtqyrroiyi5vzc4fkpgh24dnrhfdhbk2irkhetyq | Canonical-Ubuntu-20.04-Minimal-2020.08.24-0                            |
| ocid1.image.oc1.ap-tokyo-1.aaaaaaaahk3krxqgimom7cy2z4b5lsoakm6bhmnbaaaincvgtuu4wivntxjq | Canonical-Ubuntu-20.04-2020.10.14-0                                    |
| ocid1.image.oc1.ap-tokyo-1.aaaaaaaa3ioe7z5wmi7lfk4xyoret7tmlyr2g2jowsqzh4hz6qcd5pakdtda | Canonical-Ubuntu-20.04-2020.09.07-0                                    |
| ocid1.image.oc1.ap-tokyo-1.aaaaaaaax6wjwsktl7kpl3zw3gfn34cqp76b36usraojs4jors3lmfmg52ba | Canonical-Ubuntu-20.04-2020.08.21-0                                    |

Take a note of the ID for the image you want to use and save it in one of the Terraform files to use it.

For a convenience, you may want to know how to list Shapes as well.

oci compute shape list -c $TENANCY_ID \
--query "data[*]".{'shape:shape,memory:"memory-in-gbs",ocpus:ocpus'} \
--output table
| 768.0  | 52.0  | BM.Standard2.52        |
| 2048.0 | 128.0 | BM.Standard.E3.128     |
<SNIP>
| 64.0   | 8.0   | VM.Standard.E2.8       |
| 1.0    | 1.0   | VM.Standard.E2.1.Micro |
| 56.0   | 8.0   | VM.Standard1.8         |