bash muscle (1)

I should build my bash muscle. I’m going to do just that gradually here.

What if I wanted to find files that contain certain text in them? Let’s try the command below.

$ grep -rl 'test' | grep groovy

-r (–recursive) means “Read all files under each directory, recursively. Follow all symbolic links, unlike -r” is what’s in man page for grep.

-l (–files-with-matches) “Suppress normal output; instead print the name of each input file from which output would normally have been printed. The scanning will stop on the first match.

The | grep groovy part filters the files that contains groovy in the file name.

By running the command, you would get a result like the following.

docker/master/initial-setup.groovy

And the file certainly contains the text “test” in it.

This command is kind of like Find in Files in Windows but it’s better with grep capability. Definitely a command to remember.

What’s taking up the space?

There was a case where one of my CentOS servers running out of space at the root which is mapped to /dev/sda5. I didn’t think I was using it much but the df -h was telling me otherwise. So here is what I did.

$ cd /
$ sudo du -h --max-depth=1

The command shows which directory is taking up how much space in human readable format. Thanks to that, I figured out what the problem was. 🙂

How to Install Jenkins on AWS

I previously wrote an article on how to install Jenkins on Azure. I’ve also blogged about following related topics.

Now I’m going to try to write about Jenkins installation and configuration on AWS. I obviously don’t want to just reinvent the wheel or completely do things wrong, so I did my search and I found a documentation here. I’m going to refer to it and work on an instance actually on my personal AWS account.

Create EC2 (VM) Instance

Let’s create a VM instance to host the Jenkins master. I’m going to assume that you’ve already set up an AWS account.

With your browser, navigate to aws.com and click on EC2 under All services –> Compute.

Make sure to select Oregon as the region at the upper right area.

Click Launch Instance button.

I’m going to select “Amazon Linux 2 AMI (HVM), SSD Volume Type”.

In the next step, I’m going to select the VM template of t2 micro of vCPUs 1 and 1 GB memory. I think it’s enough to run an instance of Jenkins master but I would not select this configuration in actual enterprise environments for its sheer size to scale. Click Review and Launch button.

Click Launch button in the next step.

You are now prompted to select an existing key pair or create a new one. I am creating a new one for this one and naming it jenkins_master. Make sure to download it to your desktop for later use for your SSH session. Finally click Launch instances button.

SSH into the Created Instance

Let’s SSH into the created instance. First, navigate to EC2 -> Instances and select the instance you just created. And click Connect button.

By clicking Connect button, you will see the instruction on how to SSH into the instance like the following.

Open terminal and navigate to the directory where you store the .pem file and copy and paste the command in the Example: section.

ssh -i "jenkins_master.pem" ec2-user@yourhost.compute.amazonaws.com

Once the command is successful, you will see a screen like the following.

As you can see, Amazon Linux is an offspring of CentOS so run sudo yum -y update to update the packages.

Installing Jenkins Master

Now we can install Jenkins master. The latest version of Java supported by Jenkins is Java 11. I’m going to install OpenJDK 11 first.

$ sudo amazon-linux-extras install java-openjdk11

To check if Open JDK has been installed, execute the following command.

$ java --version

Now we are going to import Jenkins repo so that we can use yum to install Jenkins.

$ curl --silent --location http://pkg.jenkins-ci.org/redhat-stable/jenkins.repo | sudo tee /etc/yum.repos.d/jenkins.repo

$ sudo rpm --import https://jenkins-ci.org/redhat/jenkins-ci.org.key

Now execute the command to install Jenkins.

sudo yum install jenkins

Start Jenkins.

$ sudo systemctl start jenkins

Make sure that the Jenkins daemon starts by itself after reboot.

$ sudo systemctl enable jenkins

See if the Jenkins process is up and running by executing the following command.

You should see something like the following image.

$ ps aux | grep jenkins

–httpPort=8080 indicates that Jenkins master is listening to port 8080 with plain HTTP. I will review the configurations

Configuring Jenkins

We know that Jenkins exposes its UI on port 8080/tcp. I would like to use the default port of 80 for HTTP traffic for this instance of Jenkins. So I’m going to change the listening port by modifying the configuration file at /etc/sysconfig/jenkins

$ sudo vi /etc/sysconfig/jenkins

Now let’s make sure that JENKINS_PORT=”8080″ is present in the file. This means that you will be able to access the Jenkins UI via HTTP at the port 8080. Restart Jenkins master just in case.

$ sudo systemctl restart jenkins

Opening a Port

To access the running instance of Jenkins master on your browser, the port 80/tcp has to be open on AWS side. Here is how you can do it.

Navigate to EC2 dashboard and click Security Groups under NETWORK & SECURITY on the left menu.

Click Create Security Group button and you will see a popup like the image below. Click Add Rule button and select Custom TCP on Type column. Enter 8080 in the Port Range column. Click Create button.

Now we are going to assign this security group to the instance we created previously. Navigate to Instances and make sure that the target instance is checked. Click Actions button and select Networking –> Change Security Groups.

Select the security group that was just created and click Assign Security Groups button.

This opens the port 8080 to the target instance. Let’s try to access it!

There are two ways to access it. If you navigate to the EC2 Instances page and select the target instance, you will see the detailed information about the instance. You can see the public DNS and its IP address so enter an URL like below.

http://[your public DNS]:8080

When it’s successful, you should see a UI asking you to unlock Jenkins.

I have already documented the way to configure Jenkins here in my previous post, so please refer to it from here on.

Recap

We have learned how to spin up a VM and actually install Jenkins master on it on AWS. There are a lot more to get to the point where Jenkins master may be secured and usable on AWS. The principle is pretty much the same as the one on Azure, such as enabling SSL and getting slaves communicate with it via the secured communication. I will talk more about it later in my blog.

How to Get CentOS 7 to Join a Windows Domain

I previously wrote an article on setting up a DNS ANAME for prepare for CentOS 7 to join a Windows domain. It’s a prerequisite for the steps in this article.

Steps

Let’s ssh into the CentOS 7 as root. I’m planning to use the VM as a Docker host, so I named it as dockerhost01.

$ ssh root@dockerhost01.homenet.iriumi.ad

Now I’m going to install the prerequisite packages using yum.

# yum install sssd realmd oddjob oddjob-mkhomedir adcli samba-common samba-common-tools krb5-workstation openldap-clients policycoreutils-python -y

Now using realm, we will join the domain. Make sure the credential you are using in the following command should be from the Windows domain. Make sure to replace [domain admin user] to an actual user.

# realm join --user=[domain admin user] homenet.iriumi.ad

You can alternatively add -v option to show verbose information.

Execute the following command to confirm that the OS is not a part of the Windows domain.

# realm list

Let’s see if I can check a user.

# id hiriumi@homenet.iriumi.ad

It shows the ID of the user and also which AD domain groups the user belongs to.

I would have to use [myusername]@homenet.iriumi.ad as my login name, which I would like to do. I’m going to make some changes in the configuration. Open up the configuration file executing the following command.

# vi /etc/sssd/sssd.conf

Change use_fully_qualified_names to False and I’m going to change fallback_homedir to /home/%u instead of /home/%u@%d

use_fully_qualified_names indicates whether you want to use [your username]@domain.foo.com as your username or not. fallback_homedir indicates how and where you want SSSD to create your home directory. My sssd.conf looks like the following.

[sssd]                                         
domains = homenet.iriumi.ad                    
config_file_version = 2                        
services = nss, pam                            
                                               
[domain/homenet.iriumi.ad]                     
ad_domain = homenet.iriumi.ad                  
krb5_realm = HOMENET.IRIUMI.AD                 
realmd_tags = manages-system joined-with-samba 
cache_credentials = True                       
id_provider = ad                               
krb5_store_password_if_offline = True          
default_shell = /bin/bash                      
ldap_id_mapping = True                         
use_fully_qualified_names = False              
fallback_homedir = /home/%u                    
access_provider = ad                           

Save the change and get out (:wq).

Next restart SSSD.

# systemctl restart sssd

At this point, you can logout and ssh back into CentOS 7 as a domain user. Let’s try it.

$ ssh [your username]@dockerhost01.homenet.iriumi.ad

If you enter pwd, it created /home/[your username] directory as your home directory.

Now when you execute a command that requires sudoer permission, you will get message like the following.

[your username] is not in the sudoers file.  This incident will be reported.

We are going to logout and ssh back into it as root to fix this issue.

Add a new file at /etc/sudoers.d/sudoers

# vi /etc/sudoers.d/sudoers

Add a line like the following in the file. I’m going add a single user as a sudoer for now.

[your username] ALL=(ALL) ALL

You can also add AD groups as sudoers by adding a line like below.

%domain\ admins@homenet.iriumi.ad ALL=(ALL) ALL

Now that you added yourself as a sudoer, logout as root and login as the domain user. You should be able to execute sudo commands.

Recap

Having a centralized credential manager like Windows domain controller is quite essential to efficiently manage many servers. I have introduced a way to get CentOS 7 to join a Windows domain but this technique can be used for RedHat line of Linux distro. I’m not sure how it can be done for Debian/Ubuntu line of Linux distro and it might be a good topic for another blog article.

Adding a DNS ANAME on Windows Server Core

I’m getting ready to have an integrated environment on my server. I have a Windows domain controller up and running and I’m about to get my CentOS 7 join the domain.

Before I can go on, CentOS 7 needs to be able to communicate with the DNS server that I created on the Windows Server Core. The IP address of the DNS server in my network is 192.168.1.26. And the domain name is homenet.iriumi.ad.

A DNS server can have multiple zones. Let’s see what kind of zones I have by executing the following command.

$ Get-DnsServerZone

I have the following zones on my DNS server.

Now let’s see what DNS ANAMEs we have.

$ Get-DnsServerResourceRecord -ZoneName "homenet.iriumi.ad"

It gives you the list of DNS entries in the zone. I have a CentOS 7 host that I have assigned a static IP address to and I’m going to make sure I can resolve it.

$ Add-DnsServerResourceRecordA -ZoneName "homenet.iriumi.ad" -AllowUpdateAny -Name "dockerhost01" -IPv4Address "192.168.1.27"

By executing the command above, as long as a machine can talk to the DNS server, it can resolve dockerhost01.homenet.iriumi.ad to 192.168.1.27. In other words, dockerhost01.homenet.iriumi.ad is mapped to 192.168.1.27.

If necessary, you can remove the DNS entry by executing the following command.

$ Remove-DnsServerResourceRecord -ZoneName "homenet.iriumi.ad" -RRType "A" -Name "dockerhost01"

Next, I’m going to ssh into my CentOS 7 VM and then configure it so that it asks the DNS server on Windows Server Core to resolve names.

# vi /etc/sysconfig/network-script/ifcfg-[your network interface]

In the text file, add the following entry. Change the IP address and the domain name to fit your environment, obviously.

DNS1=192.168.1.26
DNS2=208.67.222.222
DNS3=208.67.220.220
DOMAIN=homenet.iriumi.ad

I have DNS1 point to my Windows Server Core with DNS server. And DNS2 and DNS3 are pointing to OpenDNS. Save and get out by :wq in vi.

Restart the network by executing the following command.

# systemctl restart network

Once that’s done, the system writes these data in /etc/resolv.conf. Check it by executing the following command.

# cat /etc/resolv.conf

Now try pinging dockerhost01.homenet.iriumi.ad and the IP address is resolved and get a response.

Now we are ready to get this host to join the Windows domain!

How to Assign Static IP Address on CentOS 7

When provisioning a server, it’s usually a good practice to assign static IP addresses. I have provisioned a CentOS 7 and ran yum update -y to update the default packages.

Checking Your Current IP Address

I have logged in to the console as root so that I can check the current IP address. Enter the following command.

# ip addr

You can see that this machine has 192.168.1.107. This IP address was obtained from the DHCP server. I’m going to give it 192.168.1.27. Fortunately, CentOS 7 comes with an utility to make it easier for us to assign a static IP address.

Assign a Static IP Address

Enter nmtui in your terminal.

# nmtui

You will see a UI like the image below.

Hit Enter key and you will see one or more network interface for you to configure. Select the network interface and hit Enter key.

Navigate to <Show> for IPv4 and hit Enter key. And then change Automatic to Manual.

Now navigate your cursor to Addresses <Add…> and hit enter.

Add 192.168.1.27 for the static IP address (it depends on your environment), 192.168.1.1 for its gateway and we will use OpenDNS (208.67.222.222 208.67.220.220) for DNS servers. Navigate your cursor to OK at the lower right corner of the screen and hit Enter key.

Restart network daemon by entering the following command.

# systemctl restart network

At this point, the terminal might look frozen because the OS’s IP address has changed to a different one if you SSH’ed into it. Check the IP address again by entering ip addr on it. You now should see the new static IP address you just configured.