1. Recovery-Oriented Computing Discovering Correctness Constraints for Self-Management of System Configuration Emre Kıcıman and Yi-Min Wang emrek@cs.stanford.edu, ymwang@microsoft.com Software Infrastructures Group Microsoft Research Stanford University

2. 2 ROC Retreat, January 12, 2004 Emre Kıcıman Introduction Managing configuration systems is really hard Config errors largest category of operator mistakes [Oppenheimer] Network problems: BGP Routing, DSN overload [Mahajan,Brownlee] Windows registry: -> instability, unexpected behavior [Ganapathi] End-goal is “self-managing” configuration system Keep system from entering illegal state; fix problems that sneak past. Part of solution: Monitor for misconfigurations But what do we monitor for? - Configuration that matches known problems - Configuration that does not match known good state Glean: learning rules that describe good configuration state

3. 3 ROC Retreat, January 12, 2004 Emre Kıcıman Background: Windows Registry ● Windows registry is a configuration database ● Settings for hardware, operating system, installed applications, user account info, user preferences ● Even, temporary info for Internet cache files, etc. ● Hierarchical set of keys and values

4. 4 ROC Retreat, January 12, 2004 Emre Kıcıman Glean Approach Goal: Discover correctness constraints for monitoring sys. Analyze snapshots of “believed-good” registries 1. Find configuration class ● Def'n: repeated groups of settings that share common structure ● E.g., file type information, ActiveX registrations,... 2. Look for invariants on and dependencies on each class ● Size constraints ● Enumeration constraint -> ● Reference constraint -> Key ∈ {identifies instance of C }

5. 5 ROC Retreat, January 12, 2004 Emre Kıcıman Configuration Classes ● Def'n: Repeated groups of settings that share common structure ● Ex: Both \hklm\Software\classes\.jpeg and \hklm\Software\classes\.gif have the subkeys {perceivedtype, persistenthandler,content type, openwitprogids} ●... same structure is shared by other registered image types. ● Measure similarity based on substructure of a key ● Ignore hierarchical structure during class discover ● Allows finer-granularity config classes ● Allows discovery of classes across user accounts, backups,...

6. 6 ROC Retreat, January 12, 2004 Emre Kıcıman Discovering Config Classes 1. Filter out keys with little substructure ● They won't be useful during comparison 2. Use data clustering to group similar keys together ● Measure distance by # of common subkeys ● Stop when we hit some threshold 3. Each cluster with >N keys is one config class 4. Look at the common subkeys that define each cluster ● Use these common subkeys as the “name” of the class

7. 7 ROC Retreat, January 12, 2004 Emre Kıcıman Naming Instances of Classes Use differing strings in the hierarchy as identifiers \hklm\Software\classes\interface\{F08400BB-0960-47F4-9E12-591DBF370546} \hklm\Software\classes\interface\{D93A191C-525A-43BC-ACFD-7EF494143CF4} \hklm\Software\classes\interface\{BF955013-A875-439D-A4E7-A3BBDF12AA4F}... \hklm\Software\ classes\interface \*... can be more complicated than just a single id per key... \hku\*\software\microsoft\windows\*\bags\*\shell

8. 8 ROC Retreat, January 12, 2004 Emre Kıcıman Configuration Classes in a Registry ● Start with a typical registry snapshot ● 179,776 keys ● Filter out all keys with < 3 subkeys ● 24,907 keys ● Data clustering ● 1287 keys don't cluster at all, 657 clusters w/ 2 keys ● 909 Configuration Classes ● largest class has 5657 keys (\HKLM\Software\CLASSES\INTERFACE\*) ● mean size of class = 24 keys; median = 6;

9. 9 ROC Retreat, January 12, 2004 Emre Kıcıman Correctness Constraints ● Internal constraints describe structure within a config class ● External constraints describe dependencies between config classes ●... and between any registry key and a class

10. 10 ROC Retreat, January 12, 2004 Emre Kıcıman Internal Constraints ● Size constraint ● Template: sizeof( i.subkey ) = X | i ∈ C ● need to discover values for C, subkey and X ● Enumeration constraint ● Template: i.subkey ∈ {x 1,x 2,x 3,...} | i ∈ C ● need to discover values for C, subkey and {x 1,x 2,x 3,...} 1. For each constraint, we start w/a template rule. 2. Iterate through configuration classes 3. Fill in the template based on snapshot data -> hypothesis 4. Test hypothesis and either accept or reject.

11. 11 ROC Retreat, January 12, 2004 Emre Kıcıman Internal Constraints Summary Results ● 2646 SizeConstraint rules ● 212 size=0 rules ● Ex., sizeof(...\certificatetemplatecache\*\RENEWALOVERLAP) = 8 ● 2250 EnumConstraint rules ● Ex. \hklm\System\controlset*\services\* \TYPE ∈ {16, 32} ● Next step: Cross-registry validation 1. Generate rules from each of N registry snapshots 2. Reject or generalize rules which aren't true across snapshots

12. 12 ROC Retreat, January 12, 2004 Emre Kıcıman Reference Constraints ● We want to discover any keys that refer to an instance of a configuration class ● E.g., “default printer” keys must match “printer config” name ● Template: k ∈ ID(i) | i ∈ C 1. Take all values in registry and put into HashTable 2. for each configuration rule C 3. Pull out all values from HT that equal an ID of instance(C) (no summary results yet, but “default printer” ex. is real)

13. 13 ROC Retreat, January 12, 2004 Emre Kıcıman Summary ● Infer likely correctness constraints ●... based on good snapshots of registries ● Take advantage of extra structure of configuration classes ● Fill in simple template rules to generate likely constraints ● Easily generate 1000s of rules ● Much easier than writing by hand ● Spot checked rules make intuitive sense ● Next steps ● Validate rules across many registries ● Q: Will these rules help detect real problems? Analyze problem reports from Product Support.