Managing Extended Attributes With an Enterprise Guide Add-In Larry Hoyle, Institute for Policy & Social Research, University of Kansas

2 SAS 9.4 – Extended Attributes (name,value) pairs data mySasData; fee=1; run; proc datasets lib=work nolist ;modify mySasData; XATTR SET VAR fee (Question='How much did you pay?') ; XATTR SET VAR fee (MeasurementUnits='Total payment in Euros.') ; XATTR SET DS Abstract='dataset abstract'; Name Value

3 PROC Contents data=mySasData; Dataset Variable

4 Metadata in the Dataset  Extended attributes (name,value) pairs useful for metadata about  The dataset  Each variable  Recording metadata early is desirable  If you don’t have a repository, how about in the dataset?  Using standard names is desirable  Searching, comparison  Prompts could be useful (one less vocabulary to learn)

5 Enterprise Guide Addins .Net applications integrated into EG  Reference: Hemedinger, Chris. Custom Tasks for SAS® Enterprise Guide® Using Microsoft.NET

6 Addin “Extended Attributes”

7 Edit Settings and Extended Attributes Multiple tab interface Here the “Edit” tab is the tab for entering the extended attributes

8 Global Settings These global settings apply to all output In detail they are ….

9 Agency, Version, Dataset Name DDI IDs unique within agency Default metadata version Name for the updated data

10 Example: Set a Title for the Dataset When “{DATASET_Attribute}” Is selected, the Key,Value pair applies to the whole dataset

11 Set a MeasurementUnits for the Variable “fee”

12 Extended Attribute Names from a Vocabulary A combo box allows for: selection from a list or entry of a new term Promotes consistency

13 Make Definitions of Provided Terms Available

14 Vocabulary Based on a Metadata Standard (DDI) Metadata Can be Output in XML

15 Outputs DDI Lifecycle 3.2 – Data Documentation Initiative Metadata  A major standard for the social sciences  See:  Could be adapted for other standards (e.g. CDISC? EML? Open Data?)  Would need code to write the XML, JSON, etc.

16 Recognized Terms Generate Valid DDI Structure fee Fee in Euros Euros r:MeasurementUnit defined in DDI 3.2 schema

17 SAS Built-in Metadata into Valid XML Structure fee Fee in Euros Euros From SAS variable name From SAS variable label Entered by user

18 Other Terms in DDI as Name,Value Pairs RockChalk Jayhawk MeasurementUnits Euro

19 Outputs a Codebook Note: A codebook could also be generated with an XSLT from the DDI

20.Net Code Writes SAS Code (PROC Datasets)

21 Saves, and Runs it

22 Extended Attributes in a Copy of the Dataset  The extended attributes become part of the dataset  In Windows XATTRS are in a.sas7bxat file

23 How do you Build a Custom Task?

24 GUI Tool Helps Lay Out the Interface Form Designed in Visual Studio DLL called by Enterprise Guide

25.Net Code Handles the Interface Controls Generate Events private void btnEnterAttribute_Click(object sender, System.EventArgs e) { try { // composite key: var name and attribute string key = Settings.PackVarAttrKey(cmbVariable.Text, cmbExVarAttribute.Text); // Look up the key in the hash. If it is there then replace the text in the hash. If it is not then add a new entry to the hash if (Settings.hashVarAttrs.ContainsKey(key)) { Settings.hashVarAttrs.Remove(key); } Settings.hashVarAttrs.Add(key, txtVarAttributeValue.Text); rtbXmlProperties.Text = Settings.ToXml(); Events Trigger Event Handler Code Click

26 Embedded SAS Code Scrapes Dataset for Metadata /* capture the current user formats in _MyFormats */ proc format cntlout=work._MyFormats0; run; /* ods select nlevels; */ ods output nlevels=_VariableLevelsNF; proc freq data=&lib..&dataset nlevels ; tables _all_; format _all_; run; Lots more SAS code ….

27 C# Code Writes SAS Code sb.Append("title 'Contents of the Revised Dataset';"); sb.AppendFormat("proc datasets lib={0} nolist ;\n", ActiveLibrary); // NOTE: *************** hard wired segment length here ***************************** sb.AppendFormat("\tmodify {0} ;\n xattr options seglen = 4000;\n", outMember // go through the hash table and generate the appropriate xattr commands

28 C# Program Can Run SAS Code // submit program and wait for completion submitter.SubmitSasProgramAndWait(rtbSASProgram.Text, out log); Submits the SAS Code in this window

29 On Closing EG Calls GetSasCode()  EG submits SAS code to run in a SAS session after the.NET program has terminated  Important user settings can be saved and retrieved for this session  In this case the user choices for XATTRS are needed for a PROC Datasets  These can be saved in an XML structure with ToXML()

30 Saving State, XML Produced by ToXML() Preserved: Valid attribute names ( Name,value) pairs

31 An Ad Hoc XML Schema for Saving Settings XDocument doc = new XDocument( new XDeclaration("1.0", "utf-8", string.Empty), new XElement("ExtendedAttributesTask", new XElement("Version", XmlVersion), new XElement("OutMember", outMember), datasetAttributeElements, variableAttributeElements, xattrs) 1.0

32 Settings Retrieved by FromXML() public void FromXml(string xml) { XDocument doc = XDocument.Parse(xml); XElement elVersion = doc.Element("ExtendedAttributesTask").Element("Version"); XmlVersion = elVersion.Value; 1.0

33 Potential Uses

34 Codebook - Dataset

35 Codebook – Variables, Native to SAS Sorry, Couldn’t avoid the pun

36 Codebook – Variables, Native to SAS Categorical Variables Continuous Variables

37 Variables – Extended Attributes

38  Just some kinds of metadata  Attribution (e.g. Creator, Contributor)  Variable relationships (e.g. derived from)  Universe (e.g. persons over 65)  Survey question  Code snippets

39 Issues

40 Representing a Hierarchy as Name,Value Pairs  Sometimes more structure is needed than simple values  Example:  Contributor* Name ORCID (Role, Degree of Contribution)* Use XML for values? Several Contributors Each with multiple roles

41 Future Directions

42 Possibilities  Process-  Capture EG process flow  Allow structure in attributes (XML?)  Example: multiple roles and degree of contribution for contributors

43 Other Approaches  Adapt “literate programming” methods to incorporate structured metadata  Examples SASWeave, StatRep  See:    G-2012-PH09.pdf G-2012-PH09.pdf

44 Suggestions for SAS

45 Allow Extended Attributes to Follow Variables  “regular” attributes follow variables into other datasets  E.g. DATA step set statement, or SQL select statement.  Shouldn’t extended attributes do the same?

46 Imagine  Imagine if when a variable was created it was assigned a UUID  This would not change if the variable was renamed?  (e.g. data dataB(rename=(sex=gender)); )  Or it could be used for provenance chains  BMI = Weight/Height**2;  “BasedOn” attribute for BMI gets UUIDS for Weight and Height  This could be harvested for documentation

47 Acknowledgements  Thanks to Chris Hemedinger for his very useful book and his response to questions.

48 Session ID #2481 Thank you. Contact Information: Complete code for this project has been archived at: