1. Large Effects With Small Efforts: What Does it Take to Leverage User Data? Joe Zucca, Director for Planning and Organizational Intelligence, Penn Libraries SLA, Philadelphia, April 2, 2014

2. Prologue Producing Large Effects With Small Effort?

3. Count of User TypeColumn Labels Row LabelsFACOtherStaffStudentGrand Total Canvas Features21725316288 Access20025869329 Site Request103 41 144 Administration817 16 Account Request11 2 Other1 4 5 Grand Total530616385784 Courseware Problem Incidents: Frequency by Incident & User Group How Well Are They Managing Complexity (in the LMS)? Migrate content41 Settings29 Course content27 Assignments21 Copy content17 Other (Features)15 Announcements14 Conversations/messa ges13 Multimedia11 LTI Tools (other)8 Quizzes/surveys6 Gradebook6 Modules4 Discussions3 Turnitin1 Learning Objects1

4. Average of lag Column Labels Row LabelsFACStaffStudentOtherGrand Total Administration3.05.4 5.03.9 Canvas Features1.94.00.627.52.4 Site Request1.41.8 1.5 Access1.30.41.01.51.1 Account Request0.0 7.03.5 Other0.03.3 3.2 Grand Total1.62.20.911.71.8 Courseware Problem Incidents: AVG resolution time by Incident & User Group How Well Are We Managing Complexity (in the LMS)? Quizzes/surveys9.2 Multimedia4.0 Gradebook3.8 Assignments2.4 Migrate content2.0 Copy content1.8 Course content1.6 Settings1.6 Turnitin1.0 Modules0.5 LTI Tools (other)0.4 Other (Features)0.4 Announcements0.4 Conversations/mess ages0.3 Discussions0.0 E-reserves0.0 Learning Objects0.0

5. The Challenge: The Scope & Complexity of Service

6. What products and services does your library offer or directly support for the university?  Acquisition and licensing services  Access to massive e-content  Discovery service  Supply chain services [local circulation & third party - ILL]  Inventory control  Research | Instructional services  Courseware service (e.g., Blackboard)  Learning environments (e.g. MOOCs)  Author service  Copyright compliance service  Technology support services (e.g., desktops, digital media, scanning)  Digitization services (image production, description, curation)  Metadata service  Data services  Repository services (internal and public-facing digital archives)  Researcher intelligence services (e.g., VIVO)

7. COUNTER SFX SOLR log ILLIAD VIVO Canvas LMS BePress FootPrints Fedora Repo Banner IDM Relais Google Analytics ARES HR db STAFF SpringShare AEON ILS APACHE log EZPROXY log Analytics Repository

8. A B.I. Framework

9. Analytics Repository Extract TrnsfrmLoad Data MartAnalysis Enterprise Products &Services Resolvers

10. a framework for: Extracting activity data from systems Transforming those data into readable, normalized formats Loading transformed/normalized payload into a repository Supporting analysis through local and collaborative dissemination channels. https://github.com/upenn-libraries/metridoc-app

11. Abstracts 4 key functions, exposes interfaces for interoperability Target Source, e.g. Relais, Illiad, ILS Ingest Log Parse Format Refined output 1. Extract Resolution Sources e.g. IdM, WorldCat Refined output Resolve Codes & IDs Normalize 2. Transform Query Srvc Data Repo 3. Load MetriDoc Tiered Architecture User Interface Local Data Stores Results Document Query Document 4. Query

13. Economic failure of German colonies in Africa Using Google Fusion Tables Papers on reaction kinetics on ethylene oxide hydration History of marketing listerine monetization of contemporary art especially digital art The folklore of the supernatural in the media Arab nationalism at the beginning of the 20th century, and relations with the Ottoman Empire. Medicaid eligibility rules by state data Using GIS for studying health in neighborhoods Researching Democratic party's views on charter schools and high-stakes testing. Health of pregnant women in prison Locate the full list of proscribed authors under the Yuanyou ban 20thh-century China. Pediatric Tuberculosis Caregivers Attitudes history of Chinatown developments in the U.S./Canada How the Illiad reflects political realities in Bronze Age Greece. Economic effects Sykes-Picot agreement in Syria. The effect of the Nike settlement on labor relations in the footwear industry around the world. educational linguistics, semiotics, reflectivity and meta-language type 1 diabetes and telecommunications Native American slaveholders (specifically the Cherokee) Import substitution in Brazil under Vargas household interventions for asthma prevention in urban setting data sets on toilets in Mozambique The changing Confucian-influenced perceptions among Song-dynasty writers of the Kitans. Familial lineages in Commedia del Arte Walter Mischel and original marshmallow, gratification studies Research on democratically elected Presidents of Brazil since the 1980s. Why Employment Non-Discrimination Act is taking so long to pass. Economic consequences of French colonialism in North Africa. Raster creation in arc gis Possible use of Shoah VHA in class about the Holocaust in Italy. Relation between U.S. intervention in Philippines and Cuban independence. glycogen storage disease and mortality Putting together an annotated bibliography on the Cristero rebellion. Researching medieval pilgrimages to Hebron Lliterature by Christian missionaries in the Middle East during the late 19th and early 20th centuries.

14. Epilogue Raising the Organizational I.Q.

15. The O.I.Q. NOT about Software, IT Systems, Network Plumbing, Assessment Plans, or Culture… …it IS about creating an infrastructure for learning. Analytics Repository ServicesAnalytics Organizational Knowledge

16. Understand how users work, what they want, need, and expect, and what’s expected of them And What Is It We’re Learning to Do? Guide the library through the morass of strategic choices… Improve our response to change and the value of our services Heighten the observational power of our organizations.

17. “…the key requirement for institutional success is to move from scalable efficiency to scalable learning. Said differently, the rate of learning, innovation, and performance improvement within the institution must match (or exceed) that of the surrounding environment if the institution is to survive (or thrive).” -- John Hagel III and John Seely Brown The New Organization Model: Learning at Scale HBR Blog Network

18. Joe Zucca | zucca@upenn.edu PENN Libraries With the support of Producing Large Effects With Small Efforts MetriDoc: https://github.com/upenn-libraries/metridoc-app