Difference between revisions of "Creating Your Own ProxCP KVM Templates"

From ProxCP Documentation
Line 147: Line 147:
 
     devices: ["/"]
 
     devices: ["/"]
 
     ignore_growroot_disabled: false
 
     ignore_growroot_disabled: false
 +
 +
== OS Cleaning ==
 +
 +
At this point you should have a VM with a single user (root), a serial console, any packages you want included in the template, configured SSH, and configured cloud-init. Now we need to clean the OS for redistribution.
 +
 +
* Clean package manager <code>apt autoremove && apt autoclean && apt clean</code> or <code>yum clean all</code>
 +
* Stop logging <code>service rsyslog stop</code> or <code>service auditd stop</code>
 +
* Rotate log files <code>logrotate -f /etc/logrotate.conf && logrotate -f /etc/logrotate.d/*</code>
 +
* Remove things:
 +
** <code>rm -rf /etc/ssh/ssh_host_*</code>
 +
** <code>rm -rf /var/log/*.log.* /var/log/apt/* /var/log/btmp.* /var/log/dmesg.* /var/tmp/* /tmp/* /etc/udev/rules.d/70*</code>
 +
** <code>cat /dev/null > /var/log/btmp && cat /dev/null > /var/log/dmesg && cat /dev/null /var/log/lastlog</code>
 +
** Ensure all files in /var/log are deleted or emptied (0 bytes)
 +
* Clear network configuration:
 +
** Ubuntu/Debian w/ ifupdown: remove everything in /etc/network/interfaces except "lo loopback" device; remove all files in /etc/network/interfaces.d/
 +
** Ubuntu/Debian w/ netplan: remove all files in /etc/netplan/
 +
** CentOS/RHEL: remove UUID, HWADDR, NETMASK, GATEWAY, IPADDR, NAMESERVERS from /etc/sysconfig/network-scripts/ifcfg-eth0
 +
* Set random, long root password <code>passwd</code>
 +
* Remove history <code>rm ~/.bash_history ~/anaconda*</code>
 +
* <code>unset HISTFILE</code>
 +
* Shutdown <code>shutdown -h now</code>

Revision as of 01:54, 26 July 2020

This tutorial covers how to create a custom KVM template for use with ProxCP and Proxmox.

Required: at least 1 Proxmox node

Assumption: The desired operating system ISO is already downloaded onto your Proxmox node.

The commands covered here are relevant to Debian and RHEL-based operating systems however the process can be applied to any other operating system.

Create VM

  • Create a new VM in Proxmox with the following settings:
    • VMID: anything
    • Name: operating system description (i.e. centos-8-amd64)
    • OS: your chosen ISO image
    • Hard Disk: use VirtIO SCSI (scsi0) with 10GB storage and no caching
    • CPU & Memory: 1 kvm64 CPU core, 1GB memory
    • Network: VirtIO (virtio0), vmbr0 bridge, no firewall
  • After creation, go to the Hardware tab and delete the CDROM ISO. Add a cloud-init drive on ide2. Add the CDROM ISO back as ide0.

Base OS

  • Boot the new VM and complete the base operating system installation. Generally, templates should use most default settings and minimal packages. There are a few things to keep in mind during installation:
    • Partitions: create 1 ext4 root "/" partition (no LVM, no SWAP)
    • Networking: use a generic hostname (i.e. localhost or centos-8-amd64)
    • Packages: use minimal packages, however the OpenSSH server is required

OS Setup Stage 1

  • Once the base OS installation is complete, shutdown the VM
  • In Proxmox, go to Options and set the boot order to scsi0 only (1 option)
  • Go to the Hardware tab and remove the ide0 CDROM. Verify the network device is net0 using vmbr0 bridge
  • Power on the VM
  • Login as root
  • If you created a user during installation, remove that user with deluser [username] && rm -rf /home/[username]
  • Complete updates apt update && apt upgrade && apt install setserial or yum update && yum install setserial

Serial Console Setup

KVM templates have to be configured to use a serial console for noVNC access.

  • Edit the /etc/default/grub file:
    • Add or replace
GRUB_CMDLINE_LINUX='quiet console=tty0 console=ttyS0,115200'
GRUB_TERMINAL=serial
GRUB_SERIAL_COMMAND="serial --speed=115200 --unit=0 --word=8 --parity=no --stop=1"
  • Save the file
  • Run update-grub (debian/ubuntu) or grub2-mkconfig -o /boot/grub2/grub.cfg (centos/rhel)
  • Run cat /etc/securetty | grep ttyS0 and ensure ttyS0 is in /etc/securetty. This file may not exist which is okay.

If you are creating a template for a newer operating system using "systemd":

  • Run systemctl enable serial-getty@ttyS0.service && systemctl start serial-getty@ttyS0.service

Next:

  • Shutdown the VM shutdown -h now
  • In Proxmox go to the Hardware tab and add a Serial Port (serial0) then set the Display to Serial Terminal 0 (serial0)
  • Start the VM

OS Setup Stage 2

  • Login as root
  • Edit /etc/ssh/sshd_config and ensure Port is 22 and PermitRootLogin is yes
    • It is best to uncomment these lines to explicitly declare these settings
  • Install any other software you want to include in this template
    • Some good examples to include: htop bmon zip unzip wget nano curl ethtool net-tools
  • Confirm with fdisk -l that there are no other disk partitions after root "/"

Cloud-init

  • Install cloud-init:
    • Debian/Ubuntu: apt install cloud-init cloud-initramfs-growroot cloud-guest-utils xfsprogs gdisk
    • CentOS/RHEL: yum install cloud-init cloud-utils-growpart cloud-guest-utils xfsprogs libicu gdisk
  • Proxmox requires cloud-init 18.2 or newer cloud-init --version
  • Edit /etc/cloud/cloud.cfg as the following. Note the OS-dependent options
datasource_list: [ NoCloud, ConfigDrive ]
users:
   - default
disable_root: false
preserve_hostname: false
apt_preserve_sources_list: true Ubuntu/Debian-based OS only
cloud_init_modules:
 - migrator
 - seed_random
 - bootcmd
 - write-files
 - growpart
 - resizefs
 - disk_setup
 - mounts
 - set_hostname
 - update_hostname
 - update_etc_hosts
 - ca-certs
 - rsyslog / audit rsyslog for ubuntu/debian, audit for centos/rhel
 - users-groups
 - ssh
cloud_config_modules:
 - emit_upstart
 - ssh-import-id
 - locale
 - set-passwords
 - grub-dpkg
 - apt-pipelining Ubuntu/Debian-based OS only
 - apt-configure Ubuntu/Debian-based OS only
 - ntp
 - timezone
 - disable-ec2-metadata
 - runcmd
 - byobu
cloud_final_modules:
 - package-update-upgrade-install
 - fan
 - landscape
 - lxd Ubuntu/Debian-based OS only
 - puppet
 - chef
 - mcollective
 - salt-minion
 - rightscale_userdata
 - scripts-vendor
 - scripts-per-once
 - scripts-per-boot
 - scripts-per-instance
 - scripts-user
 - ssh-authkey-fingerprints
 - keys-to-console
 - phone-home
 - final-message
 - power-state-change
system_info:
   distro: debian / centos / ubuntu OS dependent
   default_user:
     name: debian This can be anything
     lock_passwd: True
     gecos: Debian Same as name above
     groups: [adm, audio, cdrom, dialout, dip, floppy, lxd, netdev, plugdev, sudo, video]
     sudo: ["ALL=(ALL) NOPASSWD:ALL"]
     shell: /bin/bash
   ntp_client: auto
   paths:
      cloud_dir: /var/lib/cloud/
      templates_dir: /etc/cloud/templates/
   ssh_svcname: ssh / sshd ssh for ubuntu/debian, sshd for centos/rhel
resize_rootfs: true
growpart:
    mode: auto
    devices: ["/"]
    ignore_growroot_disabled: false

OS Cleaning

At this point you should have a VM with a single user (root), a serial console, any packages you want included in the template, configured SSH, and configured cloud-init. Now we need to clean the OS for redistribution.

  • Clean package manager apt autoremove && apt autoclean && apt clean or yum clean all
  • Stop logging service rsyslog stop or service auditd stop
  • Rotate log files logrotate -f /etc/logrotate.conf && logrotate -f /etc/logrotate.d/*
  • Remove things:
    • rm -rf /etc/ssh/ssh_host_*
    • rm -rf /var/log/*.log.* /var/log/apt/* /var/log/btmp.* /var/log/dmesg.* /var/tmp/* /tmp/* /etc/udev/rules.d/70*
    • cat /dev/null > /var/log/btmp && cat /dev/null > /var/log/dmesg && cat /dev/null /var/log/lastlog
    • Ensure all files in /var/log are deleted or emptied (0 bytes)
  • Clear network configuration:
    • Ubuntu/Debian w/ ifupdown: remove everything in /etc/network/interfaces except "lo loopback" device; remove all files in /etc/network/interfaces.d/
    • Ubuntu/Debian w/ netplan: remove all files in /etc/netplan/
    • CentOS/RHEL: remove UUID, HWADDR, NETMASK, GATEWAY, IPADDR, NAMESERVERS from /etc/sysconfig/network-scripts/ifcfg-eth0
  • Set random, long root password passwd
  • Remove history rm ~/.bash_history ~/anaconda*
  • unset HISTFILE
  • Shutdown shutdown -h now