Reverse String

Changing from install and configuring server products to some coding.

Let’s reverse a char array.

Input: [“a”, “b”, “c”, “d”, “e”, “f”]
Output: [“f”, “e”, “d”, “c”, “b”, “a”]

I’m writing the code in Java.

    public void reverseString(char[] s) {
        int i = 0;
        int j = s.length - 1;
        
        while (i < j)
        {
            char tmp = s[i];
            s[i] = s[j];
            s[j] = tmp;
            
            i++;
            j--;
        }
    }

This algorithm is probably O(n) in time complexity and O(1) in space complexity as it does not create another set of array to store the result.

Just a little bit of fun of coding. 🙂

How to Install Docker CE on CentOS 7

I’ve worked on Docker for a while but I have never written a blog about Docker as much as I’ve wanted. I will blog about how to install Docker CE on CentOS 7. I’m writing this blog based on the official documentation here.

Prerequisite

There are a few prerequisite packages to install.

$ sudo yum install -y yum-utils device-mapper-persistent-data lvm2

Install Repository

We have to register Docker’s yum repository now.

$ sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo

Install and Start Docker CE

$ sudo yum install docker-ce docker-ce-cli containerd.io
$ sudo systemctl enable docker
$ sudo systemctl start docker

Test Docker

Run the following command to test if Docker was installed correctly.

$ sudo docker run hello-world

You will see an output like the image below.

It will download the hello-world image and run it.

If you execute the command below, you will see the container that ran and exited.

$ sudo docker ps -a

The container is no longer useful, so let’s remove it. Replace the [CONTAINER ID] with what you see in the result of the previous command.

$ sudo docker rm [CONTAINER ID]

Run Docker Commands without sudo

It’s quite a bit more typing if you have to “sudo” for Docker commands on top of entering your password often by using it. There is a way to omit sudo when running Docker commands.

$ sudo usermod -aG docker $USER
$ exit

You have to exit the SSH session and log back in to be able to execute docker commands without sudo.

Removing Downloaded Image

Let’s see the list of images on the Docker host by executing the command below. I will omit sudo from here on.

$ docker images

Let’s remove it because we really don’t need it anymore.

$ docker rmi [IMAGE ID]

Recap

We went over the installation and the very basic test of Docker. What I’ve written here should get you going with Docker on CentOS 7.

How to Install Jenkins on AWS

I previously wrote an article on how to install Jenkins on Azure. I’ve also blogged about following related topics.

Now I’m going to try to write about Jenkins installation and configuration on AWS. I obviously don’t want to just reinvent the wheel or completely do things wrong, so I did my search and I found a documentation here. I’m going to refer to it and work on an instance actually on my personal AWS account.

Create EC2 (VM) Instance

Let’s create a VM instance to host the Jenkins master. I’m going to assume that you’ve already set up an AWS account.

With your browser, navigate to aws.com and click on EC2 under All services –> Compute.

Make sure to select Oregon as the region at the upper right area.

Click Launch Instance button.

I’m going to select “Amazon Linux 2 AMI (HVM), SSD Volume Type”.

In the next step, I’m going to select the VM template of t2 micro of vCPUs 1 and 1 GB memory. I think it’s enough to run an instance of Jenkins master but I would not select this configuration in actual enterprise environments for its sheer size to scale. Click Review and Launch button.

Click Launch button in the next step.

You are now prompted to select an existing key pair or create a new one. I am creating a new one for this one and naming it jenkins_master. Make sure to download it to your desktop for later use for your SSH session. Finally click Launch instances button.

SSH into the Created Instance

Let’s SSH into the created instance. First, navigate to EC2 -> Instances and select the instance you just created. And click Connect button.

By clicking Connect button, you will see the instruction on how to SSH into the instance like the following.

Open terminal and navigate to the directory where you store the .pem file and copy and paste the command in the Example: section.

ssh -i "jenkins_master.pem" ec2-user@yourhost.compute.amazonaws.com

Once the command is successful, you will see a screen like the following.

As you can see, Amazon Linux is an offspring of CentOS so run sudo yum -y update to update the packages.

Installing Jenkins Master

Now we can install Jenkins master. The latest version of Java supported by Jenkins is Java 11. I’m going to install OpenJDK 11 first.

$ sudo amazon-linux-extras install java-openjdk11

To check if Open JDK has been installed, execute the following command.

$ java --version

Now we are going to import Jenkins repo so that we can use yum to install Jenkins.

$ curl --silent --location http://pkg.jenkins-ci.org/redhat-stable/jenkins.repo | sudo tee /etc/yum.repos.d/jenkins.repo

$ sudo rpm --import https://jenkins-ci.org/redhat/jenkins-ci.org.key

Now execute the command to install Jenkins.

sudo yum install jenkins

Start Jenkins.

$ sudo systemctl start jenkins

Make sure that the Jenkins daemon starts by itself after reboot.

$ sudo systemctl enable jenkins

See if the Jenkins process is up and running by executing the following command.

You should see something like the following image.

$ ps aux | grep jenkins

–httpPort=8080 indicates that Jenkins master is listening to port 8080 with plain HTTP. I will review the configurations

Configuring Jenkins

We know that Jenkins exposes its UI on port 8080/tcp. I would like to use the default port of 80 for HTTP traffic for this instance of Jenkins. So I’m going to change the listening port by modifying the configuration file at /etc/sysconfig/jenkins

$ sudo vi /etc/sysconfig/jenkins

Now let’s make sure that JENKINS_PORT=”8080″ is present in the file. This means that you will be able to access the Jenkins UI via HTTP at the port 8080. Restart Jenkins master just in case.

$ sudo systemctl restart jenkins

Opening a Port

To access the running instance of Jenkins master on your browser, the port 80/tcp has to be open on AWS side. Here is how you can do it.

Navigate to EC2 dashboard and click Security Groups under NETWORK & SECURITY on the left menu.

Click Create Security Group button and you will see a popup like the image below. Click Add Rule button and select Custom TCP on Type column. Enter 8080 in the Port Range column. Click Create button.

Now we are going to assign this security group to the instance we created previously. Navigate to Instances and make sure that the target instance is checked. Click Actions button and select Networking –> Change Security Groups.

Select the security group that was just created and click Assign Security Groups button.

This opens the port 8080 to the target instance. Let’s try to access it!

There are two ways to access it. If you navigate to the EC2 Instances page and select the target instance, you will see the detailed information about the instance. You can see the public DNS and its IP address so enter an URL like below.

http://[your public DNS]:8080

When it’s successful, you should see a UI asking you to unlock Jenkins.

I have already documented the way to configure Jenkins here in my previous post, so please refer to it from here on.

Recap

We have learned how to spin up a VM and actually install Jenkins master on it on AWS. There are a lot more to get to the point where Jenkins master may be secured and usable on AWS. The principle is pretty much the same as the one on Azure, such as enabling SSL and getting slaves communicate with it via the secured communication. I will talk more about it later in my blog.

Getting Jenkins Jobs Recursively in Python via API

I’ve wanted to come up with a code that can get the whole list of Jenkins jobs recursively. I was able to write it but I’m not really liking it… View it in GitHub.

    def get_jobs(self, recursive=False):
        r = None
        if self.base_url.endswith("/"):
            api_url = self.base_url + "api/json"
        else:
            api_url = self.base_url + "/api/json"

        try:
            urllib3.disable_warnings()
            headers = {'Content-Type': 'application/json'}
            r = requests.get(url=api_url, headers=headers, auth=(self.username, self.api_token), verify=ssl.CERT_NONE)
            jenkins = json.loads(r.text)

            result = {}

            if recursive:
                folders = []
                while True:
                    self.process_result(jenkins['jobs'], folders, result)
                    if len(folders) == 0:
                        break

                    for index, folder in enumerate(folders):
                        r = requests.get(url=f"{folder}api/json", headers=headers,
                                         auth=(self.username, self.api_token), verify=ssl.CERT_NONE)
                        jenkins = json.loads(r.text)
                        self.process_result(jenkins['jobs'], folders, result)
                        folders.pop(index)
                    if len(folders) == 0:
                        break
            else:
                for job in jenkins['jobs']:
                    if job['_class'] != 'com.cloudbees.hudson.plugins.folder.Folder':
                        result[job['url']] = job['name']

            return result
        finally:
            if r is not None:
                r.close()

    def process_result(self, jobs, folders, result):
        for job in jobs:
            if job['_class'] == 'com.cloudbees.hudson.plugins.folder.Folder':
                folders.append(job['url'])
            else:
                result[job['url']] = job['name']

The reason I’m not really liking it because I’m repeating process_result twice in the code. At least I was successful in what I wanted to do and I didn’t use recursive function for recursive result but there has to be a better way to accomplish it. I’m going to dig more into this.

Edit: I realized that the reason why I have repeat the process_result function twice in the code is because Python doesn’t have support do…while. I could be wrong but I’m thinking that’s the core of the issue but if anyone out there can suggest a better way, please let me know. 🙂

How to Get config.xml of a Jenkins Job with Python

Each Jenkins job is represented by XML. You can get the XML file using an URL like this.

https://your-server/job/your_job/config.xml

Actually by saving it, you can back up your Jenkins job as XML file. Here is a function that can return you the XML content.

import requests
import urllib3
import ssl

class jenkins_server:
    def __init__(self, base_url, username, api_token):
        self.base_url = base_url
        self.username = username
        self.api_token = api_token
        self.crumb = None

    def get_job_xml(self, job_path):
        r = None

        if self.base_url.endswith("/"):
            api_url = f"{self.base_url}{job_path}"
        else:
            api_url = f"{self.base_url}/{job_path}"

        try:
            urllib3.disable_warnings()
            headers = {'Content-Type': 'application/xml'}
            r = requests.get(url=api_url, headers=headers, auth=(self.username, self.api_token), verify=ssl.CERT_NONE)
            return r.text

        except Exception as e:
            print("Error occurred: ", str(e))
        finally:
            if r is not None:
                r.close()

Here is the test code.

    def test_get_job_xml(self):
        jenkins = jenkins_server(self.config['jenkins_server']['base_url'],
                                 self.config['jenkins_server']['username'],
                                 self.config['jenkins_server']['api_token'])

        xml = jenkins.get_job_xml("job/win-test/config.xml")

        print(xml)

Notice that I implemented a configuration file with the test so that I can reuse the data in different test cases.

The XML content looks like this.

<?xml version='1.1' encoding='UTF-8'?>
<project>
  <description></description>
  <keepDependencies>false</keepDependencies>
  <properties/>
  <scm class="hudson.scm.NullSCM"/>
  <assignedNode>win</assignedNode>
  <canRoam>false</canRoam>
  <disabled>false</disabled>
  <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding>
  <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding>
  <triggers/>
  <concurrentBuild>false</concurrentBuild>
  <builders>
    <hudson.tasks.BatchFile>
      <command>dir</command>
    </hudson.tasks.BatchFile>
  </builders>
  <publishers/>
  <buildWrappers/>
</project>

Racap

This technique looks kind of useless but it has a huge potential if you have 500 Jenkins legacy jobs that needs to be managed without DSL. Jenkins API allows us to update jobs by us uploading the XML files. It means you could make mass changes to the XML files and upload them.

The ideal way to manage many Jenkins jobs is to use Job DSL and there is no doubt about it, but for legacy Jenkins jobs, this technique will lead us to something really advantageous.

How to Get CentOS 7 to Join a Windows Domain

I previously wrote an article on setting up a DNS ANAME for prepare for CentOS 7 to join a Windows domain. It’s a prerequisite for the steps in this article.

Steps

Let’s ssh into the CentOS 7 as root. I’m planning to use the VM as a Docker host, so I named it as dockerhost01.

$ ssh root@dockerhost01.homenet.iriumi.ad

Now I’m going to install the prerequisite packages using yum.

# yum install sssd realmd oddjob oddjob-mkhomedir adcli samba-common samba-common-tools krb5-workstation openldap-clients policycoreutils-python -y

Now using realm, we will join the domain. Make sure the credential you are using in the following command should be from the Windows domain. Make sure to replace [domain admin user] to an actual user.

# realm join --user=[domain admin user] homenet.iriumi.ad

You can alternatively add -v option to show verbose information.

Execute the following command to confirm that the OS is not a part of the Windows domain.

# realm list

Let’s see if I can check a user.

# id hiriumi@homenet.iriumi.ad

It shows the ID of the user and also which AD domain groups the user belongs to.

I would have to use [myusername]@homenet.iriumi.ad as my login name, which I would like to do. I’m going to make some changes in the configuration. Open up the configuration file executing the following command.

# vi /etc/sssd/sssd.conf

Change use_fully_qualified_names to False and I’m going to change fallback_homedir to /home/%u instead of /home/%u@%d

use_fully_qualified_names indicates whether you want to use [your username]@domain.foo.com as your username or not. fallback_homedir indicates how and where you want SSSD to create your home directory. My sssd.conf looks like the following.

[sssd]                                         
domains = homenet.iriumi.ad                    
config_file_version = 2                        
services = nss, pam                            
                                               
[domain/homenet.iriumi.ad]                     
ad_domain = homenet.iriumi.ad                  
krb5_realm = HOMENET.IRIUMI.AD                 
realmd_tags = manages-system joined-with-samba 
cache_credentials = True                       
id_provider = ad                               
krb5_store_password_if_offline = True          
default_shell = /bin/bash                      
ldap_id_mapping = True                         
use_fully_qualified_names = False              
fallback_homedir = /home/%u                    
access_provider = ad                           

Save the change and get out (:wq).

Next restart SSSD.

# systemctl restart sssd

At this point, you can logout and ssh back into CentOS 7 as a domain user. Let’s try it.

$ ssh [your username]@dockerhost01.homenet.iriumi.ad

If you enter pwd, it created /home/[your username] directory as your home directory.

Now when you execute a command that requires sudoer permission, you will get message like the following.

[your username] is not in the sudoers file.  This incident will be reported.

We are going to logout and ssh back into it as root to fix this issue.

Add a new file at /etc/sudoers.d/sudoers

# vi /etc/sudoers.d/sudoers

Add a line like the following in the file. I’m going add a single user as a sudoer for now.

[your username] ALL=(ALL) ALL

You can also add AD groups as sudoers by adding a line like below.

%domain\ admins@homenet.iriumi.ad ALL=(ALL) ALL

Now that you added yourself as a sudoer, logout as root and login as the domain user. You should be able to execute sudo commands.

Recap

Having a centralized credential manager like Windows domain controller is quite essential to efficiently manage many servers. I have introduced a way to get CentOS 7 to join a Windows domain but this technique can be used for RedHat line of Linux distro. I’m not sure how it can be done for Debian/Ubuntu line of Linux distro and it might be a good topic for another blog article.

Adding a DNS ANAME on Windows Server Core

I’m getting ready to have an integrated environment on my server. I have a Windows domain controller up and running and I’m about to get my CentOS 7 join the domain.

Before I can go on, CentOS 7 needs to be able to communicate with the DNS server that I created on the Windows Server Core. The IP address of the DNS server in my network is 192.168.1.26. And the domain name is homenet.iriumi.ad.

A DNS server can have multiple zones. Let’s see what kind of zones I have by executing the following command.

$ Get-DnsServerZone

I have the following zones on my DNS server.

Now let’s see what DNS ANAMEs we have.

$ Get-DnsServerResourceRecord -ZoneName "homenet.iriumi.ad"

It gives you the list of DNS entries in the zone. I have a CentOS 7 host that I have assigned a static IP address to and I’m going to make sure I can resolve it.

$ Add-DnsServerResourceRecordA -ZoneName "homenet.iriumi.ad" -AllowUpdateAny -Name "dockerhost01" -IPv4Address "192.168.1.27"

By executing the command above, as long as a machine can talk to the DNS server, it can resolve dockerhost01.homenet.iriumi.ad to 192.168.1.27. In other words, dockerhost01.homenet.iriumi.ad is mapped to 192.168.1.27.

If necessary, you can remove the DNS entry by executing the following command.

$ Remove-DnsServerResourceRecord -ZoneName "homenet.iriumi.ad" -RRType "A" -Name "dockerhost01"

Next, I’m going to ssh into my CentOS 7 VM and then configure it so that it asks the DNS server on Windows Server Core to resolve names.

# vi /etc/sysconfig/network-script/ifcfg-[your network interface]

In the text file, add the following entry. Change the IP address and the domain name to fit your environment, obviously.

DNS1=192.168.1.26
DNS2=208.67.222.222
DNS3=208.67.220.220
DOMAIN=homenet.iriumi.ad

I have DNS1 point to my Windows Server Core with DNS server. And DNS2 and DNS3 are pointing to OpenDNS. Save and get out by :wq in vi.

Restart the network by executing the following command.

# systemctl restart network

Once that’s done, the system writes these data in /etc/resolv.conf. Check it by executing the following command.

# cat /etc/resolv.conf

Now try pinging dockerhost01.homenet.iriumi.ad and the IP address is resolved and get a response.

Now we are ready to get this host to join the Windows domain!

Jenkins and Python

There is a nice Python library that allows you to manipulate Jenkins master but having more dependencies can make it cumbersome to manage systems. For some simple things, I’m for writing my own script from scratch.

Jenkins has nice set of REST-like API interfaces. I’ve actually created an open source tool in WPF and C# around the mechanism before. I know there will be time when I will want some simple Python scripts to manipulate Jenkins jobs/builds, so I’m going to write some of it just for my reference. Hope this will help someone out there.

I have these 5 Jenkins jobs on my Jenkins instance on Azure. Let’s see if I can get the list using Python.

It may not be necessary to mention this but I’m writing the code in Python 3. It seems that they are really doing away with version 2, so… The following class lets you get the list of jobs. View code in GitHub.

import requests
import base64
import urllib3
import sys
import ssl

class jenkins_server:
    def __init__(self, base_url, username, api_token):
        self.base_url = base_url
        self.username = username
        self.api_token = api_token
        self.crumb = None

    def get_jobs(self):
        r = None
        api_url = None
        if self.base_url.endswith("/"):
            api_url = self.base_url + "api/json"
        else:
            api_url = self.base_url + "/api/json"

        try:
            urllib3.disable_warnings()
            headers = {}
            headers['Content-Type'] = 'application/json'
            r = requests.get(url=api_url, headers=headers, auth=(self.username, self.api_token), verify=ssl.CERT_NONE)

            return r.json()
        except Exception as e:
            print("Error occurred: ", str(e))
        finally:
            if r is not None:
                r.close()

Please remember that requests is not in the standard library, so you may need to install it with pip. Another thing to note is that I could not figure out a proper way to verify the SSL cert. I tried many things but I was spending too much time so I’m setting verify=ssl.CERT_NONE. I’m hoping to figure it out later when I have more time.

Here is the sample test code.

    def test_get_jobs(self):
        jenkins = jenkins_server("https://jks.westus.cloudapp.azure.com", "yourusername", "api token here")
        jobs = jenkins.get_jobs()
        for job in jobs['jobs']:
            print(f"name: {job['name']}")

Here is the output from the sample code.

name: build-jenkins-toolset-dev
name: linux-prod
name: linux-test
name: pipeline-test
name: win-test

I will dig deeper in Jenkins API with Python later in my blog.