How to Make Linux Jenkins Slave a Daemon

I wrote an article on creating a Jenkins slave on Linux. The method was to just create a bash script file that requires to be executed by hand. And it wouldn’t survive restarting the host, so what I need to do is to make the script a daemon (service).

Here is what I did before configuring the daemon.

  1. Provision a Ubuntu host on Azure (it doesn’t matter where you provision the host as long as your Jenkins master on the public Internet and secured).
  2. Update the system. (sudo apt update && sudo apt upgrade)
  3. Install OpenJDK. (sudo apt install openjdk-11-jre-headless)
  4. Open port 50000 (Inboud and Outbound) to the host. I am opening all protocols.

Creating a Daemon

We will create a script at home directory first. To contain everything for Jenkins slave, I am creating /home/azureuser/jenkins-slave directory. You can create jenkins-slave or whatever the directory name you would like anywhere.

Then create slave.sh in /home/azureuser/jenkins-slave directory with the following content. Change the URL and the secret acccording to the Jenkins node you have created on Jenkins master. Make the script executable by executing chmod +x slave.sh.

java -jar agent.jar -jnlpUrl https://jenkins.hayato-iriumi.net/computer/linux%2Dnode/jenkins-agent.jnlp -secret 136fa14dcc4013727e24c9f1a9b84127d7c7ca0cfa15e22c1e1d4e0140122529 -workDir "./slave"
exit 0

Also make sure you download agent.jar from Jenkins master to /home/azureuser/jenkins-slave directory. Also user opessl and keytool to trust the SSL cert. You can refer to the previous blog article on how to use keytool.

Now create /etc/systemd/system/jenkins-slave.service file with the following content. sudo vim /etc/systemd/system/jenkins-slave.service

[Unit]
Description=JenkinsSlave

[Service]
User=azureuser
WorkingDirectory=/home/azureuser/jenkins-slave
ExecStart=/bin/bash /home/azureuser/jenkins-slave/slave.sh
Restart=always

[Install]
WantedBy=multi-user.target

Enable the daemon with the following command.

sudo systemctl enable jenkins-slave.service

Now start the daemon with the following command.

sudo systemctl start jenkins-slave.service

Check the status of it.

sudo systemctl status jenkins-slave.service

Result:

● jenkins-slave.service - JenkinsSlave
     Loaded: loaded (/etc/systemd/system/jenkins-slave.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2021-10-22 22:10:03 UTC; 8min ago
   Main PID: 14698 (bash)
      Tasks: 19 (limit: 486)
     Memory: 103.3M
     CGroup: /system.slice/jenkins-slave.service
             ├─14698 /bin/bash /home/azureuser/jenkins-slave/slave.sh
             └─14699 java -jar agent.jar -jnlpUrl https://jenkins.hayato-iriumi.net/computer/linux%2Dnode/jenkins-agent.jnlp -secret 136fa14dcc40>

Oct 22 22:10:05 jenkins-slave bash[14699]: Oct 22, 2021 10:10:05 PM hudson.remoting.jnlp.Main$CuiListener status
Oct 22 22:10:05 jenkins-slave bash[14699]: INFO: Handshaking
Oct 22 22:10:05 jenkins-slave bash[14699]: Oct 22, 2021 10:10:05 PM hudson.remoting.jnlp.Main$CuiListener status
Oct 22 22:10:05 jenkins-slave bash[14699]: INFO: Connecting to jenkins.hayato-iriumi.net:50000
Oct 22 22:10:05 jenkins-slave bash[14699]: Oct 22, 2021 10:10:05 PM hudson.remoting.jnlp.Main$CuiListener status
Oct 22 22:10:05 jenkins-slave bash[14699]: INFO: Trying protocol: JNLP4-connect
Oct 22 22:10:06 jenkins-slave bash[14699]: Oct 22, 2021 10:10:06 PM hudson.remoting.jnlp.Main$CuiListener status
Oct 22 22:10:06 jenkins-slave bash[14699]: INFO: Remote identity confirmed: da:ca:7a:5a:1e:ac:df:56:81:96:8a:d7:71:d9:5e:4c
Oct 22 22:10:06 jenkins-slave bash[14699]: Oct 22, 2021 10:10:06 PM hudson.remoting.jnlp.Main$CuiListener status
Oct 22 22:10:06 jenkins-slave bash[14699]: INFO: Connected

When the connection is established to the Jenkins master, here is what it looks like.

With this method, it survives restarting the host.

If you try to list the services on the host with the following command, you will see the item in the list.

sudo systemctl list-units --type=service

How to Create Jenkins Slave on Linux

Most of the articles I find on creating a permanent Jenkins slave on Linux requires the slave node to be exposed to public Internet. I want the Linux slave to be pinging Jenkins master just like Windows service. Here is the way I came up with.

Install Prerequisites

  • Java (sudo dnf install java-11-openjdk.x86_64)
    • Check if the Java has been installed. (java -version)
      Result:
openjdk version "11.0.12" 2021-07-20 LTS
OpenJDK Runtime Environment 18.9 (build 11.0.12+7-LTS)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.12+7-LTS, mixed mode, sharing)

Add a Permanent Node

Login to Jenkins master and click Manage Jenkins -> Manage Nodes and Clouds. Click New Node. And then give the node a name (like linux-node), select Permanent Agent and click OK.

And then, click Save button. If you navigate to the node that you just created, you should see something like…

We will take copy this line.

java -jar agent.jar -jnlpUrl https://jenkins.hayato-iriumi.net/computer/linux%2Dnode/jenkins-agent.jnlp -secret 136fa14dcc4013727e24c9f1a9b84127d7c7ca0cfa15e22c1e1d4e0140122529 -workDir ""

Now, we’ll have to download agent.jar from the Jenkins master and upload the file to the slave machine. Just click on the agent.jar link to download it.

I have the agent.jar file in Downloads directory, so here is the command to upload the agent.jar file to the slave machine.

scp ./Downloads/agent.jar hiriumi@192.168.1.29:~

Now, ssh into the slave machine.

ssh hiriumi@192.169.1.29

Trust SSL Certificate

If your Jenkins master has SSL implemented, it’s a good practice to trust the SSL certificate. Here is how you can download the certificate on your slave machine.

openssl s_client -showcerts -connect jenkins.hayato-iriumi.net:443 < /dev/null | openssl x509 -outform DER > jenkins.hayato-iriumi.net.cer

Now use keytool to trust it.

sudo keytool -trustcacerts \
-keystore "/etc/java/java-11-openjdk/java-11-openjdk-11.0.12.0.7-0.el8_4.x86_64/lib/security/cacerts" \
-storepass changeit -alias jenkins -import -file \
"/home/hiriumi/jenkins.hayato-iriumi.net.cer"

Create a Script File and Execute

Paste the Java command you get from the node page in to slave.sh

java -jar agent.jar \
-jnlpUrl https://jenkins.hayato-iriumi.net/computer/linux%2Dnode/jenkins-agent.jnlp \
-secret 136fa14dcc4013727e24c9f1a9b84127d7c7ca0cfa15e22c1e1d4e0140122529 -workDir "./slave"

Make sure slave.sh is executable by adding execute flag on the file.

chmod +x slave.sh

If you execute the slave.sh file, it starts to communicate with the Jenkins master and starts to serve as one of the Jenkins slave.

./slave.sh

Once the connection is successful, you will see something like the following.

Now, this method does not survive restarting the slave machine. Now that the communication is successful, I will look into making this script a daemon.

Where Cloud Makes Sense

There are some Cloud providers out there. AWS by Amazon being one of the most famous one. There is Azure by Microsoft and there is OCI (Oracle Cloud Infrastructure) by Oracle. Cloud is nothing but computers on the other side of the Internet is what I used to think, but it’s a lot more.

I mean imagine a situation where you want to run a high demanded service? You would have to hire bunch of experienced engineers, invest in infrastructure, compute resources and then you have to design networking, how applications can be deployed, and how they can be monitored… It is not hard to imagine the amount of money that you would have to spend up front is big.

And the service your company have may experience big spike by holiday season demand but other times, not so much. If you host your application on-premise, the money you need to invest is big throughout the year.

However, if you do it on Cloud, you spend pretty much nothing up front. You can scale up your infrastructure depending on the load but you can terminate the resources that you don’t use other times. I think Cloud can save you so much.

Plus, Cloud providers does hardware maintenance for you. If an SSD fails, they replace it for you at no cost. They are specialized in it. And they keep adding services so that you can use them.

When you think about the infrastructure that they have, it could cost you hundreds of millions of dollars if not a billion. I heard that the initial investment that Larry Ellison did for Oracle Cloud Infrastructure was around 2 billion dollars.

Imagine your company has to pay that much money up front to create your own Cloud? I mean, paying 2 billion dollars doesn’t mean you are gonna get good scalable Cloud. You have to have the right engineers at the right place.

The more I learn about Cloud, the more I believe in it and it amazes me how it can change the IT world.

How to Install VLC on Oracle Linux 8

VLC is an essential software to play any video files. I use it on all the computers that I use. Oracle Linux 8 does not have VLC by default, so here is the way to install VLC.

sudo dnf install https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm
sudo dnf install https://download1.rpmfusion.org/free/el/rpmfusion-free-release-8.noarch.rpm
sudo dnf install vlc

Enjoy!

Installing JetBrains Products on Oracle Linux 8

I am trying to configure Oracle Linux 8 as my spare laptop. I need to install JetBrains products on it. I tried to install JetBrains Toolbox but it wouldn’t work. It’s packaged as AppImage file, so it should be pretty easy but when I execute it, a blank white window shows up and disappears.

So I looked for an alternative way to install JetBrains products. I installed snapd on it with the following command.

sudo dnf install snapd

Then, I searched for the JetBrains products like the following.

snap search jetbrains

Result:

Name                       Version   Publisher   Notes    Summary
pycharm-community          2021.2.2  jetbrains✓  classic  PyCharm Community Edition
phpstorm                   2021.2.3  jetbrains✓  classic  PhpStorm
pycharm-professional       2021.2.2  jetbrains✓  classic  PyCharm Professional Edition
intellij-idea-community    2021.2.3  jetbrains✓  classic  Capable & Ergonomic Java IDE
intellij-idea-ultimate     2021.2.3  jetbrains✓  classic  Capable & Ergonomic Java IDE for Enterprise, Web & Mobile Development
webstorm                   2021.2.2  jetbrains✓  classic  WebStorm
datagrip                   2021.2.4  jetbrains✓  classic  DataGrip
clion                      2021.2.3  jetbrains✓  classic  A cross-platform IDE for C and C++
pycharm-educational        2021.2.2  jetbrains✓  classic  Easy and Professional Tool to Learn & Teach Programming with Python
rubymine                   2021.2.3  jetbrains✓  classic  The Most Intelligent Ruby and Rails IDE
space                      2021.2.0  jetbrains✓  -        Desktop Application for JetBrains Space
rider                      2021.2.2  jetbrains✓  classic  A fast & powerful cross-platform .NET IDE
goland                     2021.2.3  jetbrains✓  classic  GoLand
intellij-idea-educational  2021.2.2  jetbrains✓  classic  IntelliJ IDEA Educational Edition
kotlin                     1.5.31    jetbrains✓  classic  Command line Kotlin compiler

The first application I want to install is PyCharm, so I ran the following command to install it.

snap install pycharm-professional

If you search pycharm in your GNOME UI, you will be able to start to use it.

I still would like to use JetBrains’ Toolbox so I posted my question in their support forum to resolve

Before I posted it, I did a fair bit of research. Toolbox is packaged as AppImage, so you can check the command options like the following.

./jetbrains-toolbox --appimage-help

I learned that you can even extract files from the image like the following.

./jetbrains-toolbox --appimage-extract

I did digging into the extracted files but I could not find a solution for it. Oh well, I can use JetBrains’ products anyway, so I’m happy for now.

How to Change Java Version on Oracle Linux 8

I had Java 8 on my Oracle Linux 8 machine but I wanted to change it to Open JDK 11. Here is what I did. The following instruction will probably work for RedHat descendants.

First, I had to find a way to install Open JDK. I ran the following command to get the right package information.

dnf search java-11

Here is the result.

java-11-openjdk.x86_64 : OpenJDK 11 Runtime Environment
java-11-openjdk.src : OpenJDK 11 Runtime Environment
java-11-openjdk-demo.x86_64 : OpenJDK 11 Demos
java-11-openjdk-devel.x86_64 : OpenJDK 11 Development Environment
java-11-openjdk-headless.x86_64 : OpenJDK 11 Headless Runtime
                                : Environment
java-11-openjdk-javadoc.x86_64 : OpenJDK 11 API documentation
java-11-openjdk-javadoc-zip.x86_64 : OpenJDK 11 API documentation
                                   : compressed in a single archive
java-11-openjdk-jmods.x86_64 : JMods for OpenJDK 11
java-11-openjdk-src.x86_64 : OpenJDK 11 Source Bundle
java-11-openjdk-static-libs.x86_64 : OpenJDK 11 libraries for static
                                   : linking

To install Open JDK 11, run the following command.

sudo dnf install java-11-openjdk.x86_64

Now I want to switch my default Java version to the one I just installed.

sudo alternatives --config java

alternatives lists the available Java installations and it gives you options to select from.

Once you make the selection, check the default Java version like the following.

java -version

Result:

openjdk version "11.0.12" 2021-07-20 LTS
OpenJDK Runtime Environment 18.9 (build 11.0.12+7-LTS)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.12+7-LTS, mixed mode, sharing)

How to Get a Row in SQLite with Python

I want to get the value of id where col1 = buhi and col2 = buhi.

Here is the Python code I came up with.

conn = sqlite3.connect('test.sqlite')
cur = conn.cursor()
params = ('buhi', 'buhi')
cur.execute('''SELECT id, col1, col2 FROM HOGE 
    WHERE col1 = ? AND col2 = ?''', params)
print(cur.fetchone())

cur.fetchone() returns the matching data in tuple.

(5, 'buhi', 'buhi')

So if you want to use any specific value, you can access it like…

cur.fetchone()[0]

Edit: If no row matches the SQL statement, fetchone() returns None, so it may be necessary to check if the data exists like the following.

row = cur.fetchone('''SELECT id, col1, col2 FROM HOGE 
    WHERE col1 = ? AND col2 = ?''', params)
if row is not None:
    print(row)

Surface Pro 7 Replacement

My 1 and a half year old Surface Pro 7 had gotten broken. It would turn on but I wouldn’t see anything on the screen. So I looked for a way to get it repaired and Microsoft wanted me to send it to a repair center and get it replaced with a refurbished one for like $500. If I had gone for a new laptop with the spec I wanted, it would have cost more than $1000, so I went for the option.

I sent it back to Microsoft’s repair center and I received the replacement yesterday. I was happy to receive it and I immediately turned it on and went through the initial setup process.

After hearing the annoying Cortana’s narration, I got it connected to my WiFi and then tried to move on to associate my Surface Pro 7 with my Microsoft account but it continuously failed on “Checking the connection to Microsoft. This might take a while.” and it would take me back to connecting to WiFi. It turned out to be endless loop of connecting it to my WiFi and the message. Reinstalling Windows 10 did not make any difference.

I even tried to install Linux Mint to see if it boots OK but it did not. I was able to boot Surface Pro 7 with the bootable Linux Mint installation USB but after I installed it, it did not. I am guessing there is something wrong with a part of hardware. When I tried to look for a solution on the network device issue, Microsoft wanted me to run a diagnostic tool on it. It was impossible

From what I read, Microsoft said that I would receive a refurbished replacement. I don’t mind refurbished product but I would recommend that they do more extensive testing before they ship the replacement.

This resulted in me having to create another support case and send it back to Microsoft. But this time, they are not charging me at all because it’s still under warranty.

My old Macbook Pro lasted for about 10 years. During that time, I had to pay some to get it repaired. As for Surface Pro 7, it broke in one and a half year. I might have gotten a lemon but as much as I like Surface Pro 7, I am not a happy customer.

But with the most economical option, I will try to use Surface Pro 7 as long as the next replacement works for me.

New Dell Laptop

I have received a new Dell laptop from the company I work for. This is for a project that has to do with Oracle Linux. I feel like Christmas already. I really appreciate the opportunity to work on it.

I will write about some of the things I can expose here in my blog. My goal is to create an Oracle Linux environment where I can do my dev work with Cinnamon DE on it!