Vagrant OKD setup

Table of Contents

• 1. Introduction
• 2. Host Setup
  • 2.1. Requirements
  • 2.2. VMs DNS names and resolution
  • 2.3. Optional: Host Resolution
• 3. VMs deployment
  • 3.1. Storage setup
• 4. OKD Deployment

1. Introduction

Use this repo to setup a Vagrant based OpenShift Origin 3.11 cluster. It works with vagrant-libvirt, and needs a little bit of tuning to get the job done.

2. Host Setup

2.1. Requirements

• 64 Gigs of RAM (default setup will use 58 Gigs for the VMs)

• Fastest possible CPU

• Fedora 29 (may work on other distro / versions but this is the only tested setup)

• vagrant

• vagrant-libvirt

2.2. VMs DNS names and resolution

You need to set the DNS name of your vagrant-libvirt bridge to okd.local.

user@localhost $ sudo virsh net-edit vagrant-libvirt
## add
<domain name='{okd_local_dns}.local' localOnly='yes'/>

Then, restart the network with virsh net-restart vagrant-libvirt and you should be done.

2.3. Optional: Host Resolution

TODO: write the stuff for Fedora / NetworkManager / dnsmasq.d

3. VMs deployment

By default, we deploy a number of VMs:

• One deploy machine to run openshift-ansible

• One openshift load-balancer

• Three Openshift masters

• Three Openshift inftrastructure nodes, with additional storage for Persistant Volumes

• Three Openshift compute nodes.

3.1. Storage setup

By default, the Vagrantfile will deploy 3 infrastructure nodes with 3 extra disks of 150G, you can override these defaults via the OKD_INFRAS, OKD_MASTERS, OKD_NODES, OKD_EXTRA_DISKS_NUMBER, OKD_EXTRA_DISKS_SIZE environment variables.

For example:

user@host $ OKD_EXTRA_DISKS_NUMBER=2 OKD_EXTRA_DISKS_SIZE=10G vagrant up

Or export them:

user@host $ export OKD_NODES=2
user@host $ vagrant up

Since we will launch 11 virtual machines by default, vagrant can struggle to provision them all in parallel (my machine has no SSD), so the recommended way to deploy the VMs is to begin with the deploy machine and then use the provided recycle-nodes.sh script to bring the nodes one by one:

user@host $ vagrant up deploy
user@host $ ./scripts/recycle-nodes.sh

Note	Don’t forget to export the environment variables if you use any non default setting.

4. OKD Deployment

Everything is done on the deploy VM, it is defined as the primary VM for vagrant so you can use it easily with vagrant ssh without specifying the VM name.

openshift-ansible is cloned in $HOME/okd-ansible. The branch release-3.11 is already checked out and an inventory file is generated in $HOME/okd-ansibe/hosts.

All you have to do in go to that directory and run the playbooks:

[vagrant@deploy ~]$ cd okd-ansible
[vagrant@deploy okd-ansible]$ ansible-playbook -i hosts playbooks/prerequisites.yml
[ ...output snipped...]
[vagrant@deploy okd-ansible]$ ansible-playbook -i hosts playbooks/deploy_cluster.yml