How to Update SQLite Database with Python

I was trying to update a table in SQLite database and this worked for me.

    def test_update(self):
        conn = sqlite3.connect('test.sqlite')
        cur = conn.cursor()

        params = ('buhi', 'buhi', 1)

        cur.execute('''UPDATE HOGE 
        SET col1 = ?,
        col2 = ?
        WHERE id = ?
        ''', params)
        conn.commit()

As you can see, all the parameters can be passed as a tuple and the ?’s represent the values that will be replaced by the actual values.

The last line of the code will actually updates the data in the database.

How to Add a Control to System.Windows.Controls.MenuItem Programmatically in WPF

I’m working on a new feature for Jenkins Toolset. It’s the feature that allows you to invoke a job with parameters. I need to add parameters to MenuItem according to parameter types. I have not completed the feature yet but I figured out a way to add controls programmatically, so here is the sample code.

var g = new System.Windows.Controls.Grid();
g.ColumnDefinitions.Add(new System.Windows.Controls.ColumnDefinition());
g.ColumnDefinitions.Add(new System.Windows.Controls.ColumnDefinition());

var lblText = new System.Windows.Controls.TextBlock();
g.Children.Add(lblText);
lblText.Text = $"{p.Name} ";
System.Windows.Controls.Grid.SetColumn(lblText, 0);

var txtBox = new System.Windows.Controls.TextBox();
txtBox.Width = 100;
g.Children.Add(txtBox);
System.Windows.Controls.Grid.SetColumn(txtBox, 1);

parItem.Header = g;

Here is the sample UI.

It took me a while to figure this out, so I thought I’d share it here in my blog. 🙂

The Right Ports

nmap is a very useful tool to check the open ports. Yeah, bad guys could use it too but you want to make sure the host you have exposed to the Internet has the minimal number of ports open. When I scan my own host that hosts this blog site like nmap hayato-iriumi.net, I get the following output.

Starting Nmap 7.80 ( https://nmap.org ) at 2021-09-23 18:27 PDT
Nmap scan report for hayato-iriumi.net (150.136.86.255)
Host is up (0.097s latency).
Not shown: 996 filtered ports
PORT     STATE SERVICE
22/tcp   open  ssh
80/tcp   open  http
443/tcp  open  https
8080/tcp open  http-proxy

I have the 4 ports open intentionally for my own management of my site. This makes me think what I should actually do down the road. I should close 22 and use a bastion to SSH into the host for management.

8080 is open for another management reason. Obviously, 80 is open for HTTP connection which redirects traffic to 443 (SSL, HTTPS). If you do nmap google.com, you can see port 80 and 443 are open to public as well.

Starting Nmap 7.80 ( https://nmap.org ) at 2021-09-23 18:28 PDT
Nmap scan report for google.com (142.250.69.206)
Host is up (0.026s latency).
Other addresses for google.com (not scanned): 2607:f8b0:400a:805::200e
rDNS record for 142.250.69.206: sea30s08-in-f14.1e100.net
Not shown: 998 filtered ports
PORT    STATE SERVICE
80/tcp  open  http
443/tcp open  https

Here is the first paragraph of nmap man page. It tells you what it’s supposed to do.

Nmap (“Network Mapper”) is an open source tool for network exploration and security auditing. It was designed to rapidly scan large networks, although it works fine against single hosts. Nmap uses raw IP packets in novel ways to determine what hosts are available on the network, what services (application name and version) those hosts are offering, what operating systems (and OS versions) they are running, what type of packet filters/firewalls are in use, and dozens of other characteristics. While Nmap is commonly used for security audits, many systems and network administrators find it useful for routine tasks such as network inventory, managing service upgrade schedules, and monitoring host or service uptime.

man page for nmap is pretty big so there must be a lot we can do with this tool.

Tech Support Scam

I’m guessing whoever come to read my blog is quite technical. I’m unpleasantly surprised that tech support scam has been going on on a large scale against innocent old people. I can’t stand such injustice. I have been so into this YouTube channel by Jim Browning. He has amazing skills to track them down.

I never thought such scams could make millions but they do. When you see “Virus Detected!” kind of message with a phone number by merely accessing a website, please do not call that number. They are there to fraud and rip off money from you. Please do not get fooled by the scary sounds or the alert messages. They are all fake written in JavaScript and HTML.

I hope this spreads at least some awareness of the scam industry mainly going on in India.

Jenkins Toolset Explained 2

One of the most useful feature of Jenkins Toolset is Compare. You can compare 2 jobs with your favorite diff tool. I use Beyond Compare for it.

You can see the differences in these 2 jobs in text. And you can sync all or parts of the job and push the change to the Jenkins master side right there on the tool.

This could be (or was) useful when I had to make similar change to an existing job from a new one.

The example is only showing the comparing 2 jobs on the same Jenkins master but it is possible to compare 2 different jobs on 2 different Jenkins master too.

To be continued…

Jenkins Toolset Explained 1

Like I wrote earlier, I created a WPF software to manage Jenkins jobs relatively easier if not a lot easier. In this article, I will explain the details of the features.

Architecture

Jenkins Toolset utilizes Jenkins’ REST APIs. The communication is through only HTTP or HTTPS. It supports both.

For the authentication, it uses token. We’ll go over how to do it below.

And you can make changes to the downloaded XML config files directly and the application keeps track on the changes.

Starting

When you start Jenkins Toolset, you have to enter the root URL for the target Jenkins instance.

When you hit enter there, you get a following error message at the bottom of the screen.

Jenkins Toolset doesn’t know who you are, so enter your username to the Jenkins instance.

Once you enter your username, you can click on Get Token button to get to the screen where you can generate your token to access the instance of the Jenkins via Jenkins Toolset.

Copy the token and paste it to the textbox on Jenkins Toolset. Once you have your username and token, you can now start to get the data from the Jenkins instance.

Edit Jobs

You can directly edit jobs in the XML form using Notepad++ or any text editor you would like. If you right click on a job or multiple jobs, you can open them in XML format.

Once, you make changes to the XML file, the changed jobs are marked in red meaning the change is still local and it’s ready to be uploaded.

You can right click the job in red and select Jenkins Server –> Push Job Changes to apply the change to the server side.

When you click Push Job Changes menu item, you get prompted if you are sure to push the change to the server side.

When the job push is successful, you will see a message and the red job turns back to the normal color.

And then, you can open the job with the browser and you can see the changes applied. This feature is very effective if you have hundreds of jobs and you want to mass update something and you wanna do it by doing find and replace operation. Notepad++ can do regex find and replace for each opened file.

To be continued…

Multi Threading with map in Python

I wrote a blog article regarding advanced map with Python earlier and I found multi threading process using map with ThreadPoolExecutor. If you want to run multiple long running processes at the same time in multiple threads, this example may be for you. Here is the sample code.

import unittest
from concurrent import futures
from concurrent.futures import ThreadPoolExecutor
from itertools import repeat
import time

class playground(unittest.TestCase):

    def long_running_function(self, a, n):
        print(f"Executing {n}...")
        time.sleep(3)
        return a ** n

    def test_advanced_map(self):
        start_time = time.time()
        n = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
        with ThreadPoolExecutor(max_workers=3) as executor:
            fs = map(executor.submit, repeat(self.long_running_function), repeat(2), n)
            results = futures.wait(list(fs))
            for result in results.done:
                if result.done():
                    print(f"{result.result()}")
                if result.exception() is not None:
                    print(f"{result.exception()}")

        elapsed_time = time.time() - start_time
        print(f"took {elapsed_time}")

Output:

Executing 1...
Executing 2...
Executing 3...
Executing 4...
Executing 5...
Executing 6...
Executing 7...
Executing 8...
Executing 9...
Executing 10...

took 12.019917964935303

So that long_running_function takes 2 parameters. a is a fixed parameter and n is for each element of the n list.

So if you executed the long_running_function sequentially for each element, it would take 30 seconds because the function sleeps for 3 seconds but using 3 threads at a time, it took only 12 seconds. 10 elements should equal to 4 batches, hence 12 seconds. If there are hundreds or thousands of items, you can imagine this method can be a huge gain.

In test_advanced_map function, it instantiates ThreadPoolExecutor with max_workers value 3. It means 3 threads run at the same time. Depending on your program, you can increase the number but be careful not to increase the number too much because it can hog your memory.

And using map function, you execute the executor.submit function. You pass long_running_function as a fixed parameter as a function to be executed. And then, you pass repeat(2) and n for the parameter a for the long_running_function. You can execute a function for each element of list using map function in Python. I wrote about it before here.

Lastly, this method can handle returned results and unhandled exceptions. I believe this is the best way to do multi threading in Python. I could be wrong, so please comment if you have better ideas! 🙂

New Stuff

I am using Visual Studio 2012 Preview 3.1 on Windows 11. I didn’t plan on it but I just installed them when I saw them available on my personal laptop. I wouldn’t do it to my work laptop but I don’t mind being bleeding edge on my personal machine.

I like new stuff. It’s been my nature for decades in my software engineer career. I believe it has been doing good to me. Being a software engineer, you always need to keep up with new technology. The technology I used to work with a decade or 2 ago is completely different from what I deal with today.

If you find it hard to do, software engineer might not be the career for you. If you find it fun to do, then you have a good chance to be a good software engineer.

I’m not saying new stuff is always better. I think balance is very important. Using everything new is not good and sticking with old way only is not good either. It’s a matter of balance.

This blog is a place where I explore new stuff and keep it so that I can come back to it if I forget and share it with the people all over the world.

Jenkins Toolset

I used to do a lot of automation with Jenkins. I worked for a software shop that had multiple instances of Jenkins for prod, stating and development with hundreds of jobs. It was not so easy to manage all of them without a tool, so I had decided to create a desktop software with WPF. I still have the project on GitHub today.

As the context menu indicates, it can do all of those things. It can list the jobs on the specified Jenkins instance and manage them. I made it downloadable on OCI’s Object Storage from here.

Here is the view of the builds of a job.

I would like to convert this project to a MAUI project eventually. I will write about the tool more later when I have some more time.