Bulk Create Jenkins Nodes

If permanent slaves are required and hundreds of them, it’s very manual and tedious tasks. I have created a PowerShell script to create Jenkins nodes in bulk. CloudBees’ version of Jenkins seems to have nice API to create nodes but open source/free version doesn’t seem to have one though Jenkins CLI could be utilized to accomplish it. Please follow my previous blog article to make sure your Jenkins CLI works before moving on with this article. Please note that this script does not install Jenkins slaves on separate machines but I am planning to create a script to do that in the future.

Prerequisites

  • Windows (The script I am going to introduce utilizes PowerShell, so Windows is the safest choice of OS.)
  • Java
  • Donwloaded jenkins-cli.jar

The PowerShell Script Execution

The PowerShell script that I wrote can generate nodes in bulk. It can be found here in GitHub.

For example, execute the script with the parameters like below.

.\Jenkins_BulkCreateNodes.ps1 -JenkinsServerUrl https://jenkins.linux-mint.local/ -Username myadminuser -ApiToken 1197adsfasdfadsfasdfasdfasdfasdf3a -NodeCount 100 -NodeNamePattern "winnode-###" -OverwriteNodeIfExists $True

The command above creates nodes in Jenkins master from winnode-001 to winnode-100. The red cross icon you see below only indicates that they are offline and no actual slaves is communicating with them yet.

You can change the naming pattern specifying -NodeNamePattern.

By specifying -OverwriteNodeIfExists parameter to $True, it can delete the node and recreate. It can be dangerous, so its default value is $False and it’s an optional parameter.

The PowerShell Code

I am pasting all the code I have written for this operation. But if you just want the file, please go to my GitHub repo. All the details of each parameters are documented in the code.

<#
    .SYNOPSIS
       Generates Jenkins nodes.
    .DESCRIPTION
        This script bulk generates Jenkins nodes on the master so that nodes will be ready for Jenkins slaves to talk to them. 
        It utilizes Jenkins CLI, so Java (https://java.com/en/download/) and downloaded jenkins-cli.jar file from Jenkins master is required.
    .PARAMETER JenkinsServerUrl
        Required. Base URL of the Jenkins master. e.g. https://myjenkins.com/
    .PARAMETER Username
        The username to use for this process.
    .PARAMETER ApiToken
        Generated API Token of the username.
    .PARAMETER NodeCount
        Specifies how many nodes to created.
    .PARAMETER NodeNamePattern
        Specify the pattern of node name here. If 20 nodes to be created and "foobar-###" is passed, foobar-001 to foobar-020 would be generated.
    .PARAMETER Description
        Optional. Specify the description of each node. Use this field to specify what kind of fleet you are creating.
    .PARAMETER RemoteRootDir
        Optional. Defaults to "C:\slave". Specify any directory where you expect slave bits to reside on each node.
    .PARAMETER Mode
        Optional. Defaults to NORMAL.
    .PARAMETER NumExecutors
        Optional. Defaults to 1.
    .PARAMETER Labels
        Optional. e.g. "WINDOWS 2019". Separate labels by space.
    .PARAMETER JenkinsCliJarFilePath
        Optional. When you download jenkins-cli.jar, the file is usually at ~/Download/jenkins-cli.jar. Downaload the file from https://your-jenkins-server/jnlpJars/jenkins-cli.jar
        There is a download link on https://your-jenkins-server/cli
    .PARAMETER OverwriteNodeIfExists
        Optional. If the node to be created already exists on the Jenkins master, it deletes it and recreates it if $True is passed. If $False, it is skipped.
    .PARAMETER KeepNodeXmlFiles
        Optional. Defaults to $False. If $True is passed, the XML files created for the nodes will be kept in the same directory as this script. This option could be used to reuse the XML files in different environments.
    
    
    
#>
[CmdletBinding()]
param 
(
    [Parameter(Mandatory=$True)] 
    [string]$JenkinsServerUrl,
    [Parameter(Mandatory=$True)]
    [string]$Username,
    [Parameter(Mandatory=$True)]
    [string]$ApiToken,
    [Parameter(Mandatory=$False)]
    [int]$NodeCount = 5,
    [Parameter(Mandatory=$False)]
    [string]$NodeNamePattern = "winnode-###",
    [Parameter(Mandatory=$False)]
    [string]$Description = "Bulk gen'ed node",
    [Parameter(Mandatory=$False)]
    [string]$RemoteRootDir = "C:\slave",
    [Parameter(Mandatory=$False)]
    [string]$Mode = "NORMAL",
    [Parameter(Mandatory=$False)]
    [int]$NumExecutors = 1,
    [Parameter(Mandatory=$False)]
    [string]$Labels = "",
    [Parameter(Mandatory=$False)]
    [string]$JenkinsCliJarFilePath = "$env:HOMEPATH\Downloads\jenkins-cli.jar",
    [Parameter(Mandatory=$False)]
    [bool]$OverwriteNodeIfExists = $False,
    [Parameter(Mandatory=$False)]
    [bool]$KeepNodeXmlFiles = $False
)

$digitCount = ($NodeNamePattern.ToCharArray() | Where-Object {$_ -eq '#'} | Measure-Object).Count

ForEach ($index In 1..$NodeCount)
{
    $number = "{0:D$digitCount}" -f $index
    $nodeName = $NodeNamePattern -replace "#{$digitCount}", $number 
    $xmlSettings = New-Object System.Xml.XmlWriterSettings
    $xmlSettings.Indent = $true
    $xmlSettings.Encoding = [System.Text.Encoding]::UTF8
    $xmlSettings.OmitXmlDeclaration = $True
    $xmlSettings.ConformanceLevel = "Document"

    $sw = New-Object System.IO.StringWriter
    $xw = [System.Xml.XmlWriter]::Create($sw, $xmlSettings)
    
    try 
    {
        $xw.WriteStartElement("slave")

        $xw.WriteElementString("name", $nodeName)
        $xw.WriteElementString("description", $Description)
        $xw.WriteElementString("remoteFS", $RemoteRootDir)
        $xw.WriteElementString("numExecutors", $NumExecutors)
        $xw.WriteElementString("mode", $Mode)

        $xw.WriteStartElement("retentionStrategy")
        $xw.WriteAttributeString("class", 'hudson.slaves.RetentionStrategy$Always')
        $xw.WriteEndElement() # retentionStrategy

        $xw.WriteStartElement("launcher")
        $xw.WriteAttributeString("class", "hudson.slaves.JNLPLauncher")

        $xw.WriteStartElement("workDirSettings")
        $xw.WriteElementString("disabled", "false")
        $xw.WriteElementString("internalDir", "remoting")
        $xw.WriteElementString("failIfWorkDirIsMissing", "false")
        $xw.WriteEndElement() # workDirSettings
        $xw.WriteElementString("websocket", "false")
        $xw.WriteEndElement() # launcher

        $xw.WriteElementString("label", $Labels)
        $xw.WriteElementString("nodeProperties", "")
        $xw.WriteEndElement() #slave
        $xw.WriteEndDocument()
        $xw.Flush()
    }
    catch [System.Exception]
    {
        Write-Host $_
        exit 1
    }
    finally 
    {
        $xw.Close()
    }

    # Write out the node XML to file to be used for standard input below.
    Set-Content -Path "$nodeName.xml" -Value $sw.ToString() -Force

    # Check the existence of the node.

    $processInfo = New-Object System.Diagnostics.ProcessStartInfo
    $processInfo.FileName = "java"
    $processInfo.RedirectStandardError = $True
    $processInfo.RedirectStandardOutput = $True
    $processInfo.RedirectStandardInput = $True
    $processInfo.Arguments = "-jar $JenkinsCliJarFilePath -s $JenkinsServerUrl -auth $Username`:$ApiToken get-node $nodeName"
    $processInfo.UseShellExecute = $False

    $process = New-Object System.Diagnostics.Process
    $process.StartInfo = $processInfo
    $process.Start() | Out-Null
    $process.WaitForExit()
    $stdo = $process.StandardOutput

    If ($process.ExitCode -ne 0)
    {
        Start-Process -FilePath java -NoNewWindow -Wait -ArgumentList "-jar $JenkinsCliJarFilePath","-s $JenkinsServerUrl","-auth $Username`:$ApiToken","create-node" -RedirectStandardInput "$nodeName.xml"
        Write-Host "$nodeName created ($(Get-Date))"
    }
    else 
    {
        If ($OverwriteNodeIfExists)
        {
            Write-Host "Deleting $nodeName..."
            Start-Process -FilePath java -NoNewWindow -Wait -ArgumentList "-jar $JenkinsCliJarFilePath","-s $JenkinsServerUrl","-auth $Username`:$ApiToken","delete-node $nodeName"

            Write-Host "Recreating $nodeName..."
            Start-Process -FilePath java -NoNewWindow -Wait -ArgumentList "-jar $JenkinsCliJarFilePath","-s $JenkinsServerUrl","-auth $Username`:$ApiToken","create-node" -RedirectStandardInput "$nodeName.xml"
        }
        else 
        {
            Write-Host "Node $nodeName exists."   
        }

        If (!$KeepNodeXmlFiles)
        {
            Remove-Item -Path "$nodeName.xml" -Force
        }
    }
}

Next Step

The script I created can bulk create the “receivers” for the actual slaves to talk to the Jenkins master. I will write another PowerShell to script to automatically install and communicates with the Jenkins master.

Recap

Pull Requests, any suggestion or even forking is welcome. This should make Jenkins permanent node/slave administration a little easier.

Author: admin

A software engineer in greater Seattle area

Leave a Reply

Your email address will not be published. Required fields are marked *