Day: August 6, 2019

Getting Jenkins Jobs Recursively in Python via API

I’ve wanted to come up with a code that can get the whole list of Jenkins jobs recursively. I was able to write it but I’m not really liking it… View it in GitHub.

    def get_jobs(self, recursive=False):
        r = None
        if self.base_url.endswith("/"):
            api_url = self.base_url + "api/json"
        else:
            api_url = self.base_url + "/api/json"

        try:
            urllib3.disable_warnings()
            headers = {'Content-Type': 'application/json'}
            r = requests.get(url=api_url, headers=headers, auth=(self.username, self.api_token), verify=ssl.CERT_NONE)
            jenkins = json.loads(r.text)

            result = {}

            if recursive:
                folders = []
                while True:
                    self.process_result(jenkins['jobs'], folders, result)
                    if len(folders) == 0:
                        break

                    for index, folder in enumerate(folders):
                        r = requests.get(url=f"{folder}api/json", headers=headers,
                                         auth=(self.username, self.api_token), verify=ssl.CERT_NONE)
                        jenkins = json.loads(r.text)
                        self.process_result(jenkins['jobs'], folders, result)
                        folders.pop(index)
                    if len(folders) == 0:
                        break
            else:
                for job in jenkins['jobs']:
                    if job['_class'] != 'com.cloudbees.hudson.plugins.folder.Folder':
                        result[job['url']] = job['name']

            return result
        finally:
            if r is not None:
                r.close()

    def process_result(self, jobs, folders, result):
        for job in jobs:
            if job['_class'] == 'com.cloudbees.hudson.plugins.folder.Folder':
                folders.append(job['url'])
            else:
                result[job['url']] = job['name']

The reason I’m not really liking it because I’m repeating process_result twice in the code. At least I was successful in what I wanted to do and I didn’t use recursive function for recursive result but there has to be a better way to accomplish it. I’m going to dig more into this.

Edit: I realized that the reason why I have repeat the process_result function twice in the code is because Python doesn’t have support do…while. I could be wrong but I’m thinking that’s the core of the issue but if anyone out there can suggest a better way, please let me know. 🙂

How to Get config.xml of a Jenkins Job with Python

Each Jenkins job is represented by XML. You can get the XML file using an URL like this.

https://your-server/job/your_job/config.xml

Actually by saving it, you can back up your Jenkins job as XML file. Here is a function that can return you the XML content.

import requests
import urllib3
import ssl

class jenkins_server:
    def __init__(self, base_url, username, api_token):
        self.base_url = base_url
        self.username = username
        self.api_token = api_token
        self.crumb = None

    def get_job_xml(self, job_path):
        r = None

        if self.base_url.endswith("/"):
            api_url = f"{self.base_url}{job_path}"
        else:
            api_url = f"{self.base_url}/{job_path}"

        try:
            urllib3.disable_warnings()
            headers = {'Content-Type': 'application/xml'}
            r = requests.get(url=api_url, headers=headers, auth=(self.username, self.api_token), verify=ssl.CERT_NONE)
            return r.text

        except Exception as e:
            print("Error occurred: ", str(e))
        finally:
            if r is not None:
                r.close()

Here is the test code.

    def test_get_job_xml(self):
        jenkins = jenkins_server(self.config['jenkins_server']['base_url'],
                                 self.config['jenkins_server']['username'],
                                 self.config['jenkins_server']['api_token'])

        xml = jenkins.get_job_xml("job/win-test/config.xml")

        print(xml)

Notice that I implemented a configuration file with the test so that I can reuse the data in different test cases.

The XML content looks like this.

<?xml version='1.1' encoding='UTF-8'?>
<project>
  <description></description>
  <keepDependencies>false</keepDependencies>
  <properties/>
  <scm class="hudson.scm.NullSCM"/>
  <assignedNode>win</assignedNode>
  <canRoam>false</canRoam>
  <disabled>false</disabled>
  <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding>
  <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding>
  <triggers/>
  <concurrentBuild>false</concurrentBuild>
  <builders>
    <hudson.tasks.BatchFile>
      <command>dir</command>
    </hudson.tasks.BatchFile>
  </builders>
  <publishers/>
  <buildWrappers/>
</project>

Racap

This technique looks kind of useless but it has a huge potential if you have 500 Jenkins legacy jobs that needs to be managed without DSL. Jenkins API allows us to update jobs by us uploading the XML files. It means you could make mass changes to the XML files and upload them.

The ideal way to manage many Jenkins jobs is to use Job DSL and there is no doubt about it, but for legacy Jenkins jobs, this technique will lead us to something really advantageous.