1. Lightweight inventory management with Pallet Jack
Linux.conf.au 2017 sysadmin miniconf
Karl-Johan Karlsson

2. Problem statement Inventory management for system administrators
Store system information
IP addresses, locations, serial numbers, …
Generate service configuration
DHCP, DNS, configuration management, …
Search for static information
Location, service contracts, ...

3. Goals Single source of truth for system administration
Version controlled together with configuration management
Easily interoperable
Reasonably scaleable
Decomposable database
Single source: Easily extensible data model to store all kinds of data. Don't repeat yourself.
Version control: Need to be able to feature freeze individual branches and revert to previous state
Interoperable: Export data to create service configuration in many different forms
Reasonable: 1k machines, 10k objects
Decomposable: Vertical slice for delivery to customer, horizontal slice for separation of responsibilities

4. Data store “Warehouse” with “pallets” (objects)
Objects are directories
Keys and values stored in YAML files
Inheritance by symlinks and directory hierarchy
system
├── testvm
│ ├── architecture.yaml
│ ├── domain -> ../../domain/example.com/
│ ├── machine -> ../../machine/testvm/
│ ├── netinstall -> ../../netinstall/CentOS x86_64/Kickstart_vda/
│ └── role.yaml
Database is not very big (~10k objects), so a file system has sufficient performance
File system and Git gives us lots of tools:
Store files in the same Git repository as configuration management state, run export at Git push
Infrastructure as code, with code review!
Vertical decomposition: traverse the graph starting from affected systems, keep everything you visit, throw away the rest
Horizontal decomposition: logically separate information in separate files, merged by the system at load time
system:
role:
- kvm-server
- ssh-server

5. Transformations Synthesize new values from existing information
Value substitution and regular expressions
- net.dns.fqdn:
  - synthesize: "#[net.dns.name].#[net.dns.domain]"
- host.pxelinux.kernel:
  - synthesize: "/boot/#[system.os]/vmlinuz"
Don't repeat yourself!
Regular expressions to read parts of values.

6. Database tools
create_system --warehouse examples/warehouse/ --system vmhost1 --domain example.com --os CentOS x86_64
dump_pallet --position --warehouse examples/warehouse/ --type system vmhost1
[...] system: os: CentOS x86_64  # transforms.yaml                               # (line 175, column 15) role: - kvm-server  # system/vmhost1/role.yaml (line 3, column 4) - ssh-server  # system/vmhost1/role.yaml (line 4, column 0) architecture: x86_64  # system/vmhost1/architecture.yaml                         # (line 2, column 0) name: vmhost1  # transforms.yaml (line 165, column 15) [...]
create_*: Minimal helper wrappers around mkdir and friends for quickly creating empty template objects.
dump_pallet: Debugging tool, dumps all data for an object, optionally including its source, taking into account inheritance and transformations.

7. Data export tools
palletjack2kea --warehouse examples/warehouse/ --service example-com
## Automatically generated by palletjack2kea from
## Repository: /home/creideiki/src/palletjack/
## Branch: refs/heads/master
[...]
"subnet4": [ {
"subnet": " /24",
"reservations": [
"hw-address": "14:18:77:ab:cd:ef",
"ip-address": " ",
"hostname": "vmhost1.example.com",
Generates configuration in whatever format another service needs.
Intended to be run automatically as Git post-receive hooks.
Our setup generates configuration files into the Salt state repository, then Salt distributes them if they have changed.

8. Current integrations Configuration management: Salt
System installation: PXELINUX and Kickstart
DHCP server: Kea
Authoritative DNS: Knot
Recursing DNS: Unbound
(Directory service: FreeIPA)
FreeIPA integration under development.
We have only implemented what we need right now.
Integrating with other services is easy using the API.

9. API > require 'palletjack'
> jack = PalletJack.load('examples/warehouse')
> vmhost1 = jack.fetch(kind:'system', name:'vmhost1')
> vmhost1['system.role']
["kvm-server", "ssh-server"]
> jack.each(kind:'system') { |s| puts s['system.name'] }
vmhost1
testvm
Ruby library.
Example run in IRB.
Loading a warehouse does the heavy lifting, keeping everything in RAM.
Extract values from a single object or loop over all objects satisfying a filter.
Helper library for common tool functions, e.g. creating files.

10. gem install palletjack-tools
(or search GitHub for “palletjack”)
Published on RubyGems.
Development on GitHub.
“Palletjack” is unique on GitHub, so just search for it.
Patches welcome!
Thanks to our employer, SAAB, for letting us develop Pallet Jack as free software. I don't speak for them.