How to Install Docker Compose 2 on Intel x86_64 Processor

I’m guessing most of the people out there still use Intel x86_64 processor. I am writing a blog article on how to install Docker Compose 2 on it.

  1. Create a directory.
    mkdir -p ~/.docker/cli-plugins/
  2. Download Docker Compose 2 binary for x86_64.
    curl -SL https://github.com/docker/compose/releases/download/v2.2.3/docker-compose-linux-x86_64 -o ~/.docker/cli-plugins/docker-compose
  3. Give the binary file execute permission.
    chmod +x ~/.docker/cli-plugins/docker-compose
  4. Check if the binary has been installed successfully.
    docker compose version
  5. You should an output like the image below.

Back to Linux Mint

I had my old PC laying around doing nothing for about a year. I am reviving it by putting it back at my desk and installing Linux Mint.

I love the fact that once you install the OS, pretty much everything is good to go. There is no spending so much time to get certain device to work because of driver issues. I can just start to use everything I want.

I cannot emphasize hard enough how great Linux Mint is. I would think it is up for prime time to take over the desktop world. I understand non-tech people may say they cannot play games with Linux. I guess they can just stick with Windows but Linux is very very capable of running computations. This blog runs on Linux, and even really big enterprise sized applications run on Linux. My daily work revolves around it and I am having so much fun.

I am thinking… If you are a tech and don’t even bother to learn Linux, you would have a serious disadvantage as an engineer.

Just recently, my own blog really helped me perform a great work and all of the articles that helped it were related to Linux. Obviously, my company uses Linux for all of our production purposes and I am so glad I spent a lot of time digging into it and blogged about it.

With the latest toy of vagrant, I am having so much fun!

How to Convert Bytes to Appropriate Unit in Python

You sometimes get raw bytes for size of files. It’s hard to read and need a way to convert it. Here is the way you could do it.

def convert_size(size_bytes):
    if size_bytes == 0:
        return "0B"
    size_name = ("B", "KB", "MB", "GB", "TB", "PB", "EB", "ZB", "YB")
    i = int(math.floor(math.log(size_bytes, 1024)))
    p = math.pow(1024, i)
    s = round(size_bytes / p, 2)
    return "%s %s" % (s, size_name[i])

I don’t claim that I wrote it but I found it somewhere on the Internet but I can’t remember the source…

How to Calculate File Checksum in Python

Checksum is finger print of a file. If two files have the same checksum, they are identical files.

There are several types of checksum and the ones we use the most are MD5 and SHA256 based on my experience. Here is the sample code in Python.

import hashlib

class playground(unittest.TestCase):
    def test_sha256(self):
        file_name = 'test.jpg'
        with open(file_name, 'rb') as file:
            bytes = file.read()
            hash = hashlib.sha256(bytes).hexdigest()
            print(hash)

    def test_md5(self):
        file_name = 'test.jpg'
        with open(file_name, 'rb') as file:
            print(hashlib.md5(file.read()).hexdigest())

As a result, the SHA256 of test.jpg is c91834ce2d9e57edb6ccd118c10e5fb3b0eacfb8a8ecda73ae6680ced50009de and MD5 is 61a6ef0dd8c606fc0eb4c676f0e4296a

You don’t need Python to calculate these checksums. Here is the example you can run on your Mac or Linux. I’m sure you can run it on your Linux Subsystem on Windows as well.

❯ md5 test.jpg
MD5 (test.jpg) = 61a6ef0dd8c606fc0eb4c676f0e4296a

❯ openssl dgst -sha256 test.jpg
SHA256(test.jpg)= c91834ce2d9e57edb6ccd118c10e5fb3b0eacfb8a8ecda73ae6680ced50009de

Vagrant

I had a chance to learn vagrant at work. It is created by Hashicorp which is the creator of Terraform. I love what they have done. You can easily provision a VM locally and manage them. Looks like it requires VirtualBox as its hypervisor. You may want to install it beforehand. I learned that VirtualBox and Hyper-V can co-exist if you are on Windows.

To install Oracle Linux 7 VM, you can…

vagrant init oraclelinux/7 https://oracle.github.io/vagrant-projects/boxes/oraclelinux/7.json

You may want to decide which directory you init it from because it places Vagrantfile which has configurations of the VM. You can make changes to it as you see fit.

To start the vagrant VM, execute…

vagrant up

Just like that you can spin up a VM. To get into the VM via ssh, just enter…

vagrant ssh

You can do port forwarding to the VM so you can spin up test environment on the VM.

This is the best toy I have had in the last year or so. 🙂

Edit: There are other types of providers that Vagrant supports. Hyper-V is one of them, so it should work with Hyper-V as well. https://www.vagrantup.com/docs/providers

How to Install Docker Engine with Ansible on Oracle Linux 7

I am wanting to make sure Docker engine is present on my Jenkins host. To prove that Docker is not installed yet, here is the result of sudo systemctl status docker on the Jenkins host.

Unit docker.service could not be found.

Here is the Ansible playbook I came up with. The target host is an Oracle Linux 7 on ARM64 processor.

---
- name: Install docker
  gather_facts: No
  hosts: jenkins

  tasks:
    - name: Install Docker
      yum:
        name: docker-engine
        state: installed
      become: yes
    
    - name: Enable/Start Docker
      systemd:
        name: docker
        enabled: yes
        state: started
      become: yes

    - name: Add user vagrant to docker group
      user:
        name: vagrant
        groups: docker
        append: yes
      become: yes

Here is the result when I execute it.

ansible-playbook playbooks/docker-ce.yaml
PLAY [Install docker] ****************************************************************************************************************************

TASK [Install Docker] ****************************************************************************************************************************
changed: [jenkins.hayato-iriumi.net]

TASK [Enable/Start Docker] ***********************************************************************************************************************
changed: [jenkins.hayato-iriumi.net]

TASK [Add user vagrant to docker group] **********************************************************************************************************
ok: [jenkins.hayato-iriumi.net]

PLAY RECAP ***************************************************************************************************************************************
jenkins.hayato-iriumi.net  : ok=3    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

When I ssh into the Jenkins host and check if Docker daemon is running here is the output.

[opc@jenkins ~]$ sudo systemctl status docker
● docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
   Active: active (running) since Mon 2022-01-03 04:55:11 GMT; 1min 37s ago
     Docs: https://docs.docker.com
 Main PID: 18476 (dockerd)
    Tasks: 8
   Memory: 37.5M
   CGroup: /system.slice/docker.service
           └─18476 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

Jan 03 04:55:10 jenkins dockerd[18476]: time="2022-01-03T04:55:10.694835968Z" level=warning msg="Your kernel does not support cgroup blkio weight"
Jan 03 04:55:10 jenkins dockerd[18476]: time="2022-01-03T04:55:10.695237969Z" level=warning msg="Your kernel does not support cgroup bl..._device"
Jan 03 04:55:10 jenkins dockerd[18476]: time="2022-01-03T04:55:10.695923611Z" level=info msg="Loading containers: start."
Jan 03 04:55:11 jenkins dockerd[18476]: time="2022-01-03T04:55:11.014165992Z" level=info msg="Default bridge (docker0) is assigned with...address"
Jan 03 04:55:11 jenkins dockerd[18476]: time="2022-01-03T04:55:11.130298822Z" level=info msg="Loading containers: done."
Jan 03 04:55:11 jenkins dockerd[18476]: time="2022-01-03T04:55:11.147750103Z" level=warning msg="Not using native diff for overlay2, th...overlay2
Jan 03 04:55:11 jenkins dockerd[18476]: time="2022-01-03T04:55:11.148281344Z" level=info msg="Docker daemon" commit=9bb540d graphdriver...03.11-ol
Jan 03 04:55:11 jenkins dockerd[18476]: time="2022-01-03T04:55:11.148597025Z" level=info msg="Daemon has completed initialization"
Jan 03 04:55:11 jenkins systemd[1]: Started Docker Application Container Engine.
Jan 03 04:55:11 jenkins dockerd[18476]: time="2022-01-03T04:55:11.208274084Z" level=info msg="API listen on /var/run/docker.sock"
Hint: Some lines were ellipsized, use -l to show in full.

Contained Ansible Environment

When installing Ansible on Linux, ansible.cfg and hosts files are at /etc/ansible. Instead of changing the files there, I am going to create a more contained Ansible environment.

I am doing this because of the following 3 reasons.

  • The configuration and host list does not get affected by external changes.
  • The changes can be traced with source control like Git.
  • The changes can be reviewed by your peers for any error.

Let’s create the ansible.cfg file.

ansible-config init --disabled -t all > ansible.cfg

Now I am going to create hosts file in the same directory with the following content.

[jenkins]
jenkins.hayato-iriumi.net ansible_user=opc
 
[test]
ansibletest.westcentralus.cloudapp.azure.com ansible_user=azureuser

Now add or uncomment the following line in ansible.cfg file.

inventory=./hosts

Test run.

ansible all -m ping

As long as you can access those hosts from your ssh, the command should be successful.

ansibletest.westcentralus.cloudapp.azure.com | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/libexec/platform-python"
    },
    "changed": false,
    "ping": "pong"
}
jenkins.hayato-iriumi.net | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    },
    "changed": false,
    "ping": "pong"
}

How to do sudo with Ansible

Now I have the following content in my test.yaml playbook. I want Ansible to write a file under /etc directory.

- name: My playbook
  hosts: test
  tasks:
     - name: Leaving a mark
       command: "touch /etc/foo"

When I execute it like the following, I get an error.

ansible-playbook-3 test.yaml
fatal: [ansibletest.westcentralus.cloudapp.azure.com]: FAILED! => {"changed": true, "cmd": ["touch", "/etc/foo"], "delta": "0:00:00.003802", "end": "2022-01-01 02:53:01.519156", "msg": "non-zero return code", "rc": 1, "start": "2022-01-01 02:53:01.515354", "stderr": "touch: cannot touch '/etc/foo': Permission denied", "stderr_lines": ["touch: cannot touch '/etc/foo': Permission denied"], "stdout": "", "stdout_lines": []}

Obviously, azureuser does not have permission to write out the file, so how do we do sudo in Ansible?

You just have to pass --become parameter to do sudo in Ansible.

ansible-playbook-3 test.yaml --become
PLAY [My playbook] *******************************************************************************************************************************

TASK [Gathering Facts] ***************************************************************************************************************************
ok: [ansibletest.westcentralus.cloudapp.azure.com]

TASK [Leaving a mark] ****************************************************************************************************************************
[WARNING]: Consider using the file module with state=touch rather than running 'touch'.  If you need to use command because file is insufficient
you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message.
changed: [ansibletest.westcentralus.cloudapp.azure.com]

PLAY RECAP ***************************************************************************************************************************************
ansibletest.westcentralus.cloudapp.azure.com : ok=2    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

After executing it, I checked the file and it was created.

How to Dry Run Ansible Playbook

I have test.yaml playbook with the following content.

- name: My playbook
  hosts: all
  tasks:
     - name: Leaving a mark
       command: "touch /tmp/ansible_was_here"

I would like to do dry run using this playbook. Here is what I can do.

ansible-playbook-3 test.yaml --check

Here is the output.

[opc@ansible-primary ansible]$ ansible-playbook-3 test.yaml --check

PLAY [My playbook] *******************************************************************************************************************************

TASK [Gathering Facts] ***************************************************************************************************************************
[WARNING]: Platform linux on host jenkins.pub.ashburn.oraclevcn.com is using the discovered Python interpreter at /usr/bin/python, but future
installation of another Python interpreter could change this. See
https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html for more information.
ok: [jenkins.pub.ashburn.oraclevcn.com]
ok: [ansibletest.westcentralus.cloudapp.azure.com]

TASK [Leaving a mark] ****************************************************************************************************************************
skipping: [jenkins.pub.ashburn.oraclevcn.com]
skipping: [ansibletest.westcentralus.cloudapp.azure.com]

PLAY RECAP ***************************************************************************************************************************************
ansibletest.westcentralus.cloudapp.azure.com : ok=1    changed=0    unreachable=0    failed=0    skipped=1    rescued=0    ignored=0
jenkins.pub.ashburn.oraclevcn.com : ok=1    changed=0    unreachable=0    failed=0    skipped=1    rescued=0    ignored=0

It skipped the actual execution on the hosts specified in /etc/ansbile/hosts and returned ok for each host.

Managed Node in Azure

I provisioned a Red Hat Linux in Azure. When you provision a Linux host in Azure, the default user is azureuser. I would like to manage the host from the Ansible node on OCI. Here is the /etc/ansible/hosts file that I modified.

[jenkins]
jenkins.pub.ashburn.oraclevcn.com

[test]
ansibletest.westcentralus.cloudapp.azure.com

When I test the managed nodes, I get the following error.

ansibletest.westcentralus.cloudapp.azure.com | UNREACHABLE! => {
    "changed": false,
    "msg": "Failed to connect to the host via ssh: Permission denied (publickey,gssapi-keyex,gssapi-with-mic).",
    "unreachable": true
}

This is because Ansible tries to access the host with the default user instead of azureuser. To specify a certain user to be used on the managed node, here is what you can do.

[jenkins]
jenkins.pub.ashburn.oraclevcn.com

[test]
ansibletest.westcentralus.cloudapp.azure.com ansible_user=azureuser

If you run the following commnad…

ansible-3 all -m ping

Output:

jenkins.pub.ashburn.oraclevcn.com | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    },
    "changed": false,
    "ping": "pong"
}
ansibletest.westcentralus.cloudapp.azure.com | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/libexec/platform-python"
    },
    "changed": false,
    "ping": "pong"
}

This way, it uses azureuser to access the host in Azure.