Integrating Puppet Enterprise with Active Directory

I have Active Directory running at home. I’ve been using it as a centralized credential store.

Puppet Enterprise can authenticate against Active Directory. I just successfully integrated them, so here are the parameters I’m actually using. I used this documentation for reference.

Directory name
homenet.iriumi.ad

Hostname
pdc01.homenet.iriumi.ad

Port
389

Lookup user (optional)
CN=svcldap,OU=UsersOU,DC=homenet,DC=iriumi,DC=ad

Lookup password (optional)
* Use the password for svcldap

Connection timeout (seconds)
60

Validate the hostname?
Yes

Allow wildcards in SSL certificate?
Yes

Base distinguished name
dc=homenet,dc=iriumi,dc=ad

User login attribute
sAMAccountName

User email address
mail

User relative distinguished name (optional)
* empty

Group object class
group

Group membership field
member

Group name attribute
name

Group lookup attribute
cn

Group relative distinguished name (optional)
cn=UsersOU

Turn off LDAP_MATCHING_RULE_IN_CHAIN?
No

Search nested groups?
Yes

One thing is that when you first login as an Active Directory user, you may get denied. Once you see it, logout and login as the local admin (root) user and then you will see your Active Directory user created in Puppet Enterprise. You can add the user to the Administrators group so that you can do administrative work as yourself.

The configurations I showed in this blog entry is actually working, so I hope this helps someone out there.

How to Force Code Deploy to Puppet Master

I previously wrote about how to setup Git integration with Puppet but with the way it’s set up, the code change you made is not deployed to Puppet master right away. We need to do something about this.

First of all, it’s quite easy to set up a webhook in GitLab to tell Puppet that “code is pushed, so deploy now”. Please see this documentation on how to set it up. This is the best way to go about deploying the code change to Puppet master.

What if your source control such as GitHub or GitLab cannot reach your instance of Puppet master because the Puppet master is behind a firewall or router or whatever? Here is a poor man’s solution to it.

Generate API Token

We will utilize Puppet’s API to get the code deployed. Let’s generate an API token.

  1. ssh into Puppet master.
  2. Execute the following command to generate the token. –lifetime 365d in the command below means the token will expire in 365 days.
    $ puppet-access login --lifetime 365d
  3. The API token is generated at the following location.
    /home/[user]/.puppetlabs/token

Deploy the Code

  1. Now execute the command to deploy the code from Git.
    $ puppet-code deploy --all --wait
  2. On any agent or on Puppet master, execute the following command to apply the change.
    $ sudo /opt/puppetlabs/bin/puppet agent -t
    or
    # puppet agent -t

Recap

Like I mentioned, the best way to get the code change to be deployed to Puppet is webhook. It basically means that Git server sends HTTPS request to Puppet master to let it know that there was a code change and Puppet does its job to sync the code. Otherwise, you could use the technique above. To take it a little further, bash script could be created to execute puppet-code deploy and puppet agent -t and get it to be run as a cron job. It’s really up to you.

How to Integrate Puppet Master with Git

Pretty much all of us engineers want to manage Puppet code in a source control for traceability and manageability. I’m going to write a step-by-step documentation on how to just do it based on this document, this documentation and some other documentation I found by Googling. I had a hard time finding a single document that takes me to where Puppet works out of Git source control, so here it is.

Create a Control Repo from the Puppet template

We will create a Git repo based on a template that Puppet offers on GitHub. Here is the picture of how it work.

Getting Puppet Master Ready to Sync

Using ssh-keygen, create public/private key pair.

First of all, ssh into the Puppet master you installed as root and generate an SSH private/public key pair.

# ssh-keygen -t rsa -b 2048 -C 'code_manager' -f /etc/puppetlabs/puppetserver/ssh/id-control_repo.rsa -q -N ''

It generates 2 files id-control_repo.rsa and id-control_repo.rsa.pub under /etc/puppetlabs/puppetserver/ssh. id-control_repo.rsa is the private key and id-control_repo.rsa.pub is the public key. Let’s print the content of the public key to get the text.

# cat /etc/puppetlabs/puppetserver/ssh/id-control_repo.rsa.pub

Next make sure that the user pe-puppet created by the puppet installation has the ownership on the directory /etc/puppetlabs/puppetserver/ssh Execute the following command.

# chown -R pe-puppet:pe-puppet /etc/puppetlabs/puppetserver/ssh

Next make sure that the pe-puppet account has rwx permissions for the files in SSH key directory.

# chmod 755 /etc/puppetlabs/puppetserver/ssh/

Getting a Git repo Ready

Puppet master (optionally) needs a Git repo to pull code from. In this example, I will use GitLab as my source. This process does not need to be performed on Puppet master. Please skip this section if you already have access to GitLab (or other Git) repository.

SSH Key to GitLab

  1. Open your terminal or console.
  2. Generate a SSH key.
    $ ssh-keygen -t ed25519
  3. Print the public key on your console.
    $ cat ~/.ssh/id_ed25519.pub
  4. Navigate to gitlab.com on your browser.
  5. Click on the icon at the upper right corner of the screen and select Settings.
  6. Click SSH Keys from the menu on the left.
  7. On the terminal, copy the printed public key in the previous section and paste it to the Key textarea and click Add key button.
  8. Now you are ready to access GitLab.

Create a New Repo on GitLab

In this section, we will create a repo on GitLab where Puppet master pull code from. The source control does not have to be GitLab. Any Git server will do as long as your Puppet master can reach it.

  1. Navigate to gitlab.com on your browser and login.
  2. Click New Project button.
  3. Enter control-repo in Project name (or whatever you like). Keep the project private if you don’t want to expose the code but I am making this one a public because this is just an example and I would like to share the code to public later on. Click Create project button.

Cloning control-repo from GitHub to GitLab

Puppet provides us with a template source in GitHub, so we will copy the repo to my own repo on GitLab. The following process can be done from your desktop. As long as you create a copied repo on GitLab, our mission is accomplished in this section.Puppet provides us with a template source in GitHub, so we will copy the repo to my own repo on GitLab. The following process can be done from your desktop. As long as you create a copied repo on GitLab, our mission is accomplished in this section.

  1. Open your terminal and navigate to the directory where you want to store the code to. (e.g. C:\Windows\Users\[myaccount]\Dev)
  2. Clone the source repo from GitHub.
    $ git clone git@github.com:puppetlabs/control-repo.git
  3. We don’t want to push any change to GitHub, so we will remove the origin.
    $ git remote remove origin
  4. Add the URL to the GitLab repo we created in the previous section. Please change the URL accordingly.
    $ git remote add origin git@gitlab.com:hiriumi/control-repo.git
  5. Push the code to the GitLab repo.
    $ git push --set-upstream origin production
  6. Alternatively, you can check the URL of the remote by executing the following command.
    $ git remote get-url origin
  7. When you open the GitLab UI, you can see production branch was created automatically because the original repo has production branch. All code must be pushed to the production branch for it to take effect.

Configure Puppet Master

Now we need to tell Puppet master where to pull the source code from. We will do this from the UI.

  1. Login to https://puppet (or wherever your Puppet is installed).
  2. Click Classification on the menu.
  3. Expand PE Infrastructure and click PE Master.
  4. Click Configuration tab.
  5. Navigate to Class:puppet_enterprise::profile::master and select r10k_remote from the dropdown list. Paste the SSH URL for the GitLab repo configured in the previous section. Click Add parameter.
  6. From the same dropdown list under Class:puppet_enterprise::profile::master, select r10k_private_key and enter /etc/puppetlabs/puppetserver/ssh/id-control_repo.rsa.
  7. From the same dropdown list, select code_manager_auto_configure and set the value to true. Click Commit 3 change button at the lower right corner of the screen.
  8. Let’s test the configuration by executing the following command.
    # puppet agent -t
  9. You will see an output like the following.
  10. Login to get a token to execute puppet-code.
    # puppet-access login --lifetime 2h
    This means the token to execute puppet-code will last for 2 hours.
  11. Next deploy the environment with puppet-code.
    # puppet-code deploy production
  12. The command above should show an output like this.

  13. Execute puppet agent -t again.
  14. All the code pulled from the source control can be located at /etc/puppetlabs/code/environments.

Recap

This document explained how to copy the existing template repo and apply it to your own environment. Puppet now can talk to the Git server. However, we need to understand how we can create groups and different environments for testing. I will talk more about it in my blog later.

Puppet Agent Installation on CentOS 7

I have explained how to install Puppet master in my previous blog. I will now explain how to install Puppet agent on a CentOS 7.

Let’s first open Puppet Enterprise UI. If you click Unsigned Certs tab, you will see the command that you can use to install Puppet agent.

ssh into your host where you plan to run your Puppet agent. And execute the command.

curl -k https://puppet:8140/packages/current/install.bash | sudo bash

Once the execution is complete, go back to Puppet Enterprise web UI. You will see number “1” next to Unsigned certs. This means that the agent you have just installed is waiting for the certificate request to be accepted so that it can start to really communicate with the Puppet master. Let’s just accept the certificate.

After a little while, if you click Nodes tab, you will see the node you just accepted the certificate for shows up in the list.

Create VirtualBox VM with Bash Script

When you need to create multiple VMs with slightly different settings, bash script comes in handy. I found myself constantly creating and destroying CentOS VMs on VirtualBox, so I decided to spend some time to come up with a bash script to create VirtualBox VM. Here is the script.

VMName="CentOS"
VMDiskRoot="/mnt/firecuda/VMs"
DiskDir="$VMDiskRoot/$VMName"
DiskSize=$((1024*200))
MemorySize=$((1024*9))
VRamSize=128
CPUCount=2
OSTypeID="RedHat_64"
NetworkInterface="wlx503eaa732ee3"
ISOFilePath=~/Downloads/CentOS-7-x86_64-Minimal-1810.iso

echo "Creating $VMName disk at $DiskDir. Its size is $DiskSize."
if [ ! -d "$DiskDir" ]; then
    mkdir -p $DiskDir
fi

#VBoxManage list vms
echo "Creating disk..."
VBoxManage createhd --filename "$DiskDir/$VMName.vdi" --size $DiskSize 

echo "Creating VM..."
VBoxManage createvm --name $VMName --ostype "$OSTypeID" --register

echo "Adding the created disk to the VM..."
VBoxManage storagectl $VMName --name "SATA Controller" --add sata --portcount 1 --controller IntelAHCI
VBoxManage storageattach $VMName --storagectl "SATA Controller" --port 0 --device 0 --type hdd --medium "$DiskDir/$VMName.vdi"

VBoxManage storagectl $VMName --name "IDE Controller" --add ide
VBoxManage storageattach $VMName --storagectl "IDE Controller" --port 0 --device 0 --type dvddrive --medium "$ISOFilePath"

echo "Setting memory..."
VBoxManage modifyvm $VMName --memory $MemorySize --vram $VRamSize --cpus $CPUCount

echo "Setting boot sequence..."
VBoxManage modifyvm $VMName --boot1 dvd --boot2 disk --boot3 none --boot4 none

echo "Setting network..."
VBoxManage modifyvm $VMName --nic1 bridged --bridgeadapter1 $NetworkInterface

echo "VM Creation completed."

If you would like to fork the code, it can be found here.

Puppet Enterprise Installation

Puppet is the key to a successful server management. In my own blog, I’m switching my gear to Puppet. I’m going to focus on Puppet Enterprise because Puppet allows 10 free agent complimentary license. This is more than enough for me to test things out.

Puppet makes all versions of Puppet Enterprise available on their website. I will try to use the latest LTS (Long Term Supported) release. As of June 5, 2019, 2018.1.8 is the latest LTS.

Let’s click Puppet Enterprise 2018.1.8 – Long Term Supported Release.

Right click the link to copy the link to the PE for the platform where you are installing your Puppet Enterprise.

Open Terminal and execute the following command to download the file.

$ curl -JLO https://pm.puppetlabs.com/puppet-enterprise/2018.1.8/puppet-enterprise-2018.1.8-el-7-x86_64.tar.gz

Extract the tar file by executing the following command.

$ tar -xvf puppet-enterprise-2018.1.8-el-7-x86_64.tar.gz 

Now open port 3000 for us to be able to use the GUI mode during the installation process. Whether you use text mode or GUI mode is up to you, but I like to use GUI mode.

We’ll also open 443 and 8140 because 443 is used for the Web UI and 8140 will be used for Puppet agent to communicate with the master. 8170 will be used for API call to Puppet master.

$ sudo firewall-cmd --add-port=3000/tcp --permanent --zone=public
$ sudo firewall-cmd --add-port=443/tcp --permanent --zone=public
$ sudo firewall-cmd --add-port=4433/tcp --permanent --zone=public
$ sudo firewall-cmd --add-port=8140/tcp --permanent --zone=public
$ sudo firewall-cmd --add-port=8170/tcp --permanent --zone=public
$ sudo firewall-cmd --reload

One more thing before we can execute the Puppet Enterprise installer. Let’s edit /etc/hosts file to mimic DNS entry. My test Puppet Enterprise machine has “pem” as its host name and its IP address is 192.168.1.24, so we need 2 entries in the hosts file like the following. (Use sudo vi /etc/hosts to edit it in vi) If you don’t know how to use vi, go over the basics here.

192.168.1.24 pem
192.168.1.24 puppet

Save the file by hitting ESC and :wq and then Enter key. On the side note, if you have DNS server in your network, it’s better to create the entry in DNS server instead of using the hosts file.

In the terminal, cd into the directory where Puppet Enterprise was extracted to and execute the installer.

$ cd puppet-enterprise-2018.1.8-el-7-x86_64
$ sudo ./puppet-enterprise-installer

When you see How to proceed?, enter 2 for Graphical-mode Install and hit enter. After a little while, it awaits you to hit https://yourpuppetserver:3000 to start the setup process. Click Let’s get started!

Enter a new admin password and click Continue.

Once the validation is clear, you will see a screen like the following screenshot. Click Deploy now button to start the whole process.

Once the deployment process is complete, you should able to hit https://yourpuppetserver and login as admin.

Success!

Finally close the port 3000 because it’s no longer necessary for it to be open.

$ sudo firewall-cmd --remove-port=3000/tcp --permanent --zone=public
$ sudo firewall-cmd --reload