VMware + Chef A VMworld Workshop

Agenda 1 What is Chef? 2 Benefits of Infrastructure as Code 3 Installing Chef DK 4 Building Blocks of Chef 5 Chef Tools and Testing 6 Chef VMware Integrations 7 Hands On with vCloud Air 8 Q&A CONFIDENTIAL

Introduction Curtis Stewart Consultant for the DevOps by VMware Professional Services group 4+ years working with DevOps practices and Chef Automation Enthusiast Open Source Contributor CONFIDENTIAL

About Us: VMware DevOps Consulting Services Developer friendly. Enterprise ready. Assessment Strategy development People, process & technology transformation for Continuous delivery Configuration management Cloud operations Security & resilience optimization Our team of dedicated DevOps experts provide: Build and deliver applications sooner Fuel innovation and accelerate time to market Transform your enterprise to support high velocity, modern application development Deploy an agile future-ready datacenter where any app can thrive For those not familiar with our DevOps servics – here’s a quick overview CONFIDENTIAL

Chef Chef is a configuration management tool Infrastructure as code Composed of various components Chef Server chef-client Nodes Cookbooks Resources Recipes Workstation We’ll get into more details of the components here in a bit… CONFIDENTIAL

Benefits – Infrastructure as Code Rebuild your entire system from a code repository, data backups, and compute resources Programmatically provision and configure components Limits the needs for full instance backups Provides the ability to keep base images lightweight Executable documentation Instance backups - Cloning instances can be very costly in terms of storage Lightweight base images - Golden images need to be rebuilt when changes occur Executable documentation – writing docs is a pain and most are outdated as soon as you’re “done” CONFIDENTIAL

Lab 1: Quick Tour of Chef In this lab you will: This will demonstrate: Install the Chef Development Kit Sign up for hosted Chef Verify connection to Chef server This will demonstrate: Setting up your workstation for Chef development Overview of core components of Chef - Because of our limited time, we’re going to jump right in to setting things up. As some of these items are being downloaded, we’ll continue to move forward and I’ll do more explaining of the components once we’re setup CONFIDENTIAL

Lab 1: Quick Tour of Chef - Script https://github.com/vmwaredevops/vmworld-chef-repo Go to this URL for a script of our workshop Also, please clone the repo if you already have Git installed, otherwise click the “Download Zip” button and unpackage to your local machine This also includes some helper files for moving along in the workshop CONFIDENTIAL

Lab 1: Quick Tour of Chef – ChefDK Installation Chef Downloads Page https://downloads.chef.io/chef-dk/ Chef Development Kit is easiest way to get started Most common Chef tools all bundled together Install latest version – v0.7.0 – for your platform Once package is downloaded, run the installer Binaries on USB sticks were added prior to v0.7.0 release. If network allows, go ahead and download latest, otherwise we’ll be OK with v0.6.2 In USB stick, under Chef/chefdk-0.6.2.dmg for macs, .msi for windows Once you’ve started the installation, please proceed to signup for hosted Chef CONFIDENTIAL

Lab 1: Quick Tour of Chef – Hosted Chef Signup Signup for a free trial of hosted Chef https://manage.chef.io/signup/ Click Create New Organization Enter full name and short name for your organization Click Create Organization Fill out sign up form Create new organization Names can be whatever you want (your name, company name, department, etc.) Short name must be unique If you already have an account and can’t remember your password, try ‘Forgot Password’; or use another email account. CONFIDENTIAL

Lab 1: Quick Tour of Chef – Chef User Key Navigate to the Administration panel and select Users from the left menu. Select your user, then click "Reset Key" from that Actions menu. Click "Reset Key" to download your key. Add downloaded user pem to: /vmworld-chef-repo/.chef/[username].pem Click Reset Key and save or copy your downloaded key file to the .chef directory under your vmware-chef-repo CONFIDENTIAL

Lab 1: Quick Tour of Chef – Knife Setup Copy /vmworld-chef- repo/.chef/knife.rb.example to: ./vmworld-chef-repo/.chef/knife.rb Replace the following values with your signup information: INSERT_CHEF_USERNAME INSERT_CHEF_ORG_SHORTNAME Go ahead and update your knife.rb (by copying from knife.rb.example and replacing the values) Verify by running knife client list, should just see you a single ORG-validator key Won’t go into much detail, but this is a special key for some bootstrapping workflows CONFIDENTIAL

Lab 1: Quick Tour of Chef – Cookbook Upload Upload your cookbook to the Chef server using knife This will also verify that out knife.rb file has been properly updated Let’s upload our hello_vworld cookbook to our Chef server CONFIDENTIAL

Lab 1: Quick Tour of Chef – ChefDK Verification Open Terminal (Mac) or PowerShell Console (Windows) Execute $ chef verify Tests the embedded ChefDK applications CONFIDENTIAL

Chef Components: Cookbook A cookbook is a collection of recipes Cookbooks are typically designed to be composable so you can pick and choose the pieces you need Cookbook - Container for everything you need for this specific piece of software, application, etc. CONFIDENTIAL

Chef Components: Resources and Recipes A resource is one of the most basic components of the Chef ecosystem Examples of a resource: package, file, service Resources are defined using attributes and actions Recipes are a compilation of resources Resources within a recipe are executed sequentially CONFIDENTIAL

Chef – Workstation / Server / Node / Chef-Client / Knife Your Chef workstation is where you’ll install the Chef DK to develop and test cookbooks as well as interact with the Chef server APIs The Chef Server is a centrally managed host for storing your configuration data and node objects All of this data is indexed and completely searchable via RESTful APIs Chef Server available in two flavors Hosted Chef – Centrally administered server hosted by Chef, Inc. Open Source – Installable version to run on local infrastructure A node is any physical, virtual, or cloud machine that is managed by the chef-client The chef-client is an agent that’s installed on all nodes that actually performs the configurations Knife is a command line tool that provides an interface between local chef components and the Chef server CONFIDENTIAL

Chef – Tools and Testing Test-Kitchen Testing framework Supports multiple drivers including vSphere, vCloud Air and many more Berkshelf Dependency manager for cookbooks Supports multiple sources including Chef Supermarket, repositories, local paths ChefSpec Unit testing framework Fast feedback Serverspec Integration testing framework Verifies end state of system is what we expected ***Check time*** Explain test kitchen, but skim the rest to help speed things up Let audience know we’re limited on time so we can dive too deep here Be sure to expand a bit on each tool – use cases, benefits, features, etc. Test-Kitchen Allows execution of cookbooks in isolation No Chef Server required CONFIDENTIAL

VMware + Chef Integration There are many products of VMware that Chef integrates with through various plugins vCloud Air knife-vcair kitchen-vcair VMware vSphere knife-vsphere chef-provisioning-vsphere vRealize Automation knife-vrealize kitchen-vra vRO Chef Server Workflows VMware Integrated OpenStack knife-openstack kitchen-openstack VMware Fusion / Workstation Vagrant Plugins VMware ESXi knife-esx CONFIDENTIAL

VMware + Chef: vCloud Air knife-vcair Create, bootstrap, and manage vCloud Air compute instances List networks and templates Supports OnDemand and Subscription services kitchen-vcair Test Kitchen driver for developing and testing cookbooks against vCloud Air Currently supports Subscription services CONFIDENTIAL

Lab 2: Chef + vCloud Air In this lab you will: This will demonstrate: Install the knife-vcair plugin Configure knife with your vCloud Air endpoint Create and bootstrap a vCloud Air VM using your cookbook This will demonstrate: Provisioning a VM in vCloud Air through Chef tools Bootstrapping your cookbook on your new VM CONFIDENTIAL

Lab 2: Chef + vCloud Air – knife-vcair $ chef exec gem install knife-vcair Let’s go back to our hello_vmworld cookbook and update our default recipe by adding a few resources CONFIDENTIAL

Lab 2: Chef + vCloud Air – knife.rb knife[:vcair_username] = “vi-admin@vmwareprompt.com” knife[:vcair_password] = “VMware123!” Update /vmworld-chef-repo/.chef/knife.rb to include vcair settings CONFIDENTIAL

Lab 2: Chef + vCloud Air – Create VM $ knife vcair server create \ --ssh-password vmworld2015 \ --image "CentOS64-64BIT" \ --node-name YOURNAME-chef-node \ --customization-script bootstrap/install-linux-vcair-example.sh \ --run-list 'recipe[hello_vmworld::default]' \ --ssh-gateway root@107.189.120.118 \ --fog-version 1.33.0 \ --vcair-net chef-routed-network \ --no-host-key-verify NOW LET’S RUN IT Once your node converges: got to http://GATEWAY_IP:800[last_digit_of_ip] to see your message NOTE: It’s important to use this specified password because of how we’ve setup our SSH gateway (to make it easier on everyone so we didn’t have to create any private keys) This provisioning and bootstrap will take between 10-15 minutes, so let’s continue on going over other Vmware Chef integrations as our web servers come up. If you have questions or issues, raise a hand and the proctors will come around to support. CONFIDENTIAL

Lab 2: Chef + vCloud Air ## This is sample kitchen-vagrant output….need to update with kitchen-vcair output ## Should see output similar to this CONFIDENTIAL

VMware + Chef: vSphere knife-vsphere chef-provisioning-vsphere Integrates with your existing vSphere vCenter installation List, clone, delete, snapshot VMs via knife List datastores, resource pools and clusters. Execute commands on running VMs When cloning a VM, can interact with a customization specs to customize vCPUs, vRAM, IPs, hostname, etc at VM creation. chef-provisioning-vsphere Use machine resources to declare the topology of your app – machines and recipes to run on them Includes Test Kitchen driver to test your cookbooks on your existing vSphere vCenter installation ***Careful with time here – skip through support and spend a couple minutes on chef-provisioning*** Knife-vsphere Uses knife & fog to interact with the vSphere API. Customization specs: Requires a VM template that can be customized like Ubuntu, Red Hat or Windows. Hint: when using CentOS, set the OS type to Red Hat not CentOS to take advantage of this! CONFIDENTIAL

VMware + Chef: Chef Provisioning Example Just wanted to spend a few extra minutes on chef-provisioning – true infrastructure as code! (VERY COOL) Harnesses the simplicity and power of Chef to go one step further: to describe and automate the whole cluster with Chef, soup to nuts, hardware to network to software CONFIDENTIAL

VMware + Chef: vRealize Suite kitchen-vra A driver to allow Test Kitchen to consume vRealize Automation (vRA) resources to perform testing knife-vrealize Knife plugin that interacts with vRA and vRealize Orchestrator (vRO) List catalogs, servers Display detailed server information Create servers from catalog blueprints Execute vRO workflows vRO Chef Server Workflows vRO plugin installed to vRO directly Interfaces directly with Chef Server REST API vRO Chef Server Workflows These workflows will allow you to create a self service application portal using vRA/vCAC to provision VM’s and Chef to install and manage your applications CONFIDENTIAL

VMware + Chef: VIO knife-openstack kitchen-openstack Create, bootstrap, and manage OpenStack compute instances List networks, floating IPs, security groups, images, snapshots, volumes, flavors kitchen-openstack Test Kitchen driver for testing your cookbooks on OpenStack cloud resources instead of your workstation Supports various configurations including block device mappings, network settings, security groups Knife openstack – Chef plugin that allows you to create, bootstrap, and manage OpenStack compute instances Kitchen-openstack – Test Kitchen plugin that helps you develop and test your cookbooks on your OpenStack cloud Importance of Test Kitchen: you’re testing on the exact same infrastructure that you will use to deploy to production CONFIDENTIAL

VMware + Chef: Fusion and Workstation Vagrant is a free and open-sorce tool used to quickly spin up & down new development environments, locally, on your workstation or laptop. Used for development to test automation or deployments of new software, or in Chef’s case: cookbooks. The vagrant plugin for VMware Fusion/Workstation is a pay for add on for the free vagrant Vagrant uses local hypervisor on your workstation (VirtualBox, etc.) Vagrant VMware Fusion/Workstation plugin allows Vagrant to use VMware products, which are faster and more stable VMware Vagrant plug-ins are a commercial offering from HashiCorp http://www.vagrantup.com Use Chef to manage development machines that run under VMware Fusion or VMware Workstation. Chef works with an open-source tool called Vagrant to quickly create and provision new development environments that run in VMs. For VMware Fusion, use the vagrant-vmware-fusion plugin. For VMware Workstation, use the vagrant-vmware-workstation plugin. CONFIDENTIAL

VMware + Chef: ESXi knife-esx Interacts with your solo, or non managed ESXi host. Uses knife & fog to interact with APIs presented on the ESXi host. Can list, delete and clone VMs on a single ESXi host. Installs (bootstraps) Chef & makes it part of a Chef server and, additionally, can be configured to add the new node to specific environments and/or roles on your Chef server. Community maintained & under active development @ https://github.com/maintux/knife-esx ***Skim through this slide*** Use Chef to automatically provision and manage virtual machines that are hosted by VMware vSphere ESXi CONFIDENTIAL

Questions? …comments…and thank you! Also, checkout our vmwaredevops/vmworld-chef-repo for more examples of VMware+Chef integrations! CONFIDENTIAL