Homelab Update - Automate Infrastructure

Overview

After tearing down and rebuilding my lab manually many times I finally decided to create a pipeline to automate the provisioning of my Vms. For this I chose to use Packer, Terraform and Ansible.

Packer

I used Packer to create a Proxmox VM template that I can then use Terraform and Ansbile on to create my different VMs. I followed Christian Lempa’s guide on YouTube. Create VMs on Proxmox in Seconds!. Most of his code still worked but I made a few tweaks.

Ubuntu Server Packer File

My template is using Ubuntu Server. Here is the pkr.hcl file that you need to get this running. This can also be changed to work with many other ISO images.

ubuntu-server-jammy.pkr.hcl

# Ubuntu Server jammy
# ---
# Packer Template to create an Ubuntu Server (jammy) on Proxmox

# Packer plugin for proxmox
packer {
  required_plugins {
    proxmox = {
      version = " >= 1.1.3"
      source  = "github.com/hashicorp/proxmox"
    }
  }
}

# Variable Definitions
variable "proxmox_api_url" {
    type = string
}

variable "proxmox_api_token_id" {
    type = string
}

variable "proxmox_api_token_secret" {
    type = string
    sensitive = true
}

# Resource Definition for the VM Template
source "proxmox-iso" "ubuntu-server-jammy" {

    # Proxmox Connection Settings
    proxmox_url = "${var.proxmox_api_url}"
    username = "${var.proxmox_api_token_id}"
    token = "${var.proxmox_api_token_secret}"
    # (Optional) Skip TLS Verification
    insecure_skip_tls_verify = true
    
    # VM General Settings
    node = "pve"
    vm_id = "8000"
    vm_name = "k3s-template"
    template_description = "Ubuntu Server jammy Image"

    # VM OS Settings
    iso_file = "local:iso/ubuntu-22.04.2-live-server-amd64.iso"
    iso_storage_pool = "local"
    unmount_iso = true

    # VM System Settings
    qemu_agent = true

    # VM Hard Disk Settings
    scsi_controller = "virtio-scsi-pci"

    disks {
        disk_size = "32G"
        format = "raw"
        storage_pool = "local-lvm"
        type = "virtio"
    }

    # VM CPU Settings
    cores = "4"
    
    # VM Memory Settings
    memory = "10240" 
    ballooning_minimum = "4096"

    # VM Network Settings
    network_adapters {
        model = "virtio"
        bridge = "vmbr0"
        firewall = "false"
    } 

    # VM Cloud-Init Settings
    cloud_init = true
    cloud_init_storage_pool = "local-lvm"

    # PACKER Boot Commands
    boot_command = [
        "<esc><wait>",
        "e<wait>",
        "<down><down><down><end>",
        "<bs><bs><bs><bs><wait>",
        "autoinstall ds=nocloud-net\\;s=http://:/ ---<wait>",
        "<f10><wait>"
    ]
    boot = "c"
    boot_wait = "5s"

    # PACKER Autoinstall Settings
    http_directory = "http" 

    ssh_username = "your username"
    # (Option 2) Add your Private SSH KEY file here
    ssh_private_key_file = "~/.ssh/path-to-ssh-key"

    # Raise the timeout, when installation takes longer
    ssh_timeout = "55m"
}

# Build Definition to create the VM Template
build {

    name = "ubuntu-server-jammy"
    sources = ["source.proxmox-iso.ubuntu-server-jammy"]

    # Provisioning the VM Template for Cloud-Init Integration in Proxmox #1
    provisioner "shell" {
        inline = [
            "while [ ! -f /var/lib/cloud/instance/boot-finished ]; do echo 'Waiting for cloud-init...'; sleep 1; done",
            "sudo rm /etc/ssh/ssh_host_*",
            "sudo truncate -s 0 /etc/machine-id",
            "sudo apt -y autoremove --purge",
            "sudo apt -y clean",
            "sudo apt -y autoclean",
            "sudo cloud-init clean",
            "sudo rm -f /etc/cloud/cloud.cfg.d/subiquity-disable-cloudinit-networking.cfg",
            "sudo rm -f /etc/netplan/00-installer-config.yaml",
            "sudo sync"
        ]
    }

    # Provisioning the VM Template for Cloud-Init Integration in Proxmox #2
    provisioner "file" {
        source = "files/99-pve.cfg"
        destination = "/tmp/99-pve.cfg"
    }

    # Provisioning the VM Template for Cloud-Init Integration in Proxmox #3
    provisioner "shell" {
        inline = [ "sudo cp /tmp/99-pve.cfg /etc/cloud/cloud.cfg.d/99-pve.cfg" ]
    }

    # Add additional provisioning scripts here

}

Most of this is just setting up the Proxmox VM itself to create a template from. The Variable Definitions are found in the credentials.pkr.hcl file.

credentials.pkr.hcl

proxmox_api_url = "https://proxmox-ip-address:8006/api2/json"  # Your Proxmox IP Address
proxmox_api_token_id = "API Token"  # API Token ID
proxmox_api_token_secret = "api secret"

These are the configs that Packer needs to be able to access Proxmox to create the template. In the Packer Boot Commands you can see that it is calling the Ubuntu Auto Installer. This allows you to define your Ubuntu installation without going through the GUI. You need to make a seperate folder called http to store your auto installer file. Packer will server this file on a port on the machine you are running packer on so that the Ubunut VM can pull it. In this folder create meta-data and user-data. The meta-data file will jsut be blank but needs to be in there for the web server to work. The user-data file will container your Auto Install configs.

user-data

#cloud-config
autoinstall:
  version: 1
  locale: en_US
  #keyboard:
    #layout: en
    #https://bugs.launchpad.net/subiquity/+bug/1621378
  ssh:
    install-server: true
    allow-pw: true
    disable_root: true
    ssh_quiet_keygen: true
    allow_public_ssh_keys: true
  packages:
    - qemu-guest-agent
    - sudo
  storage:
    layout:
      name: direct
    swap:
      size: 0
  user-data:
    package_upgrade: false
    timezone: America/Denver
    users:
      - name: Your Username
        groups: [adm, sudo]
        lock-passwd: false
        sudo: ALL=(ALL) NOPASSWD:ALL
        shell: /bin/bash
        ssh_authorized_keys:
          - ssh pub key

Lastly, you need to create a files folder and then 99-pve-cfgfile. This file is copied to the machine in Proxmomx Provisioning Step 3.

99-pve.cfg

datasource_list: [ConfigDrive, NoCloud]

now that your files are completed just run

packer init ubuntu-server-jammy.pkr.hcl

This will have packer install the Proxmox plugin that is defined at the top of the template file.

Then run

packer validate -var-file=credentials.pkr.hcl ubuntu-server-jammy.pkr.hcl 

to have Packer validate your template file. If that is good to go then run

packer build -var-file=credentials.pkr.hcl ubuntu-server-jammy.pkr.hcl

It could take about 5-10 minutes to complete. Jsut keep an eye on your Ubuntu machine as it is provisioning to check for error messages

Terraform

Ansible