LSST DC2 Post-Mortem February 5–6, DataProperty Generally works Uses: –FITS headers –Events –Persistence parameters (additionalData)‏ 1

LSST DC2 Post-Mortem February 5–6, DataProperty boost::any problems Must know type before extracting value Need functions to get an integer/real/string from anything compatible Need function to get a dynamically-typed Python object Added explicitly typed construction/extraction functions for Python 2

LSST DC2 Post-Mortem February 5–6, Added and excess functionality Added function to combine two DataProperties –For updating WCS parameters findUnique() used everywhere –Ability to have multiple identical keys not useful SupportFactory overdesigned for DC2 –None of functionality used; class itself not used sometimes 3

LSST DC2 Post-Mortem February 5–6, Policy Very successful Formats –JSON deprecated; should be removed –XML intended in original design; not necessary? File-include functionality not used yet Several policy files per pipeline and per run –Policies used only determinable from interpreting other Policies –Therefore need to persist all for provenance –Master Policy not currently saved (will be in ticket #315)‏ Has get()/getArray() functions for Python Does not try to support float, int64_t Should be merged with DataProperty –But marked read-only, since modifying Policies is undesirable 4

LSST DC2 Post-Mortem February 5–6, Utility Functions These functions are in various “utility” modules; they belong in DataProperty/Policy or a common location instead: –fw::getInt64FromAny() — Extract an int64_t value from a boost::any if it holds a signed integral type –dps.Utils.getDPValue() — Extract a value from a DataProperty into a Python dynamically-typed object –dps.Utils.dataPropertyToDict() — Convert a DataProperty into a Python dictionary –ap::extractRequired() — Extract a value from a DataProperty, throwing an exception if not found –fw::extractPolicyString(), ap::extractPolicyString() — Extract a value from a Policy if the Policy exists, otherwise return a default value 5

LSST DC2 Post-Mortem February 5–6, Policy & DataProperty: design differences DataProperty models a single name-value mapping –schizophrenic feature: is it a leaf or a node (or both)? –children implemented as list because name is locked to value is order important? shouldn't be if names are different inefficient access: list searched sequentially –a single name-value pair is not useful as a modeled object interface provides no functionality for handling a single pair in practice, if you have a lone piece of metadatum, you know what it represents –unclear semantics hierarchy suggests that meaning of named values depends on context findAll() suggests absolute meaning of names regardless of context

LSST DC2 Post-Mortem February 5–6, Policy & DataProperty: design differences Policy models a set of name-value mappings –like Java's Properties and C++'s Map –clean distinction between node & leaf: Policy=node, other types=leaves –efficient look-up of name to one or more values order is only important for array values –semantics: meaning is always context-specific outer context is assumed by user of Policy

LSST DC2 Post-Mortem February 5–6, Policy capabilities not fully realized Automatic loading/importing of policies across multiple files Dictionary-based validation Request: ability to load default policies from package directory –policy contents can be loaded from another file (include)‏ –implemented but not being used Request: ability to easily share values at a high level in hierarchy with lower levels –designed but not implemented PolicyRule –a way parameterize policy values –values may change due to changing conditions