Presentation on theme: "The Avalon Media System"— Presentation transcript:
1The Avalon Media System Yale UniversityNovember 24, 2014Jon DunnIndiana UniversityMark NotessIndiana UniversityJulie RudderNorthwestern University
2AgendaIntroduction & Overview – Jon Dunn, Indiana Demo – Julie Rudder, Northwestern Technical architecture, development process, release plans – Mark Notess, Indiana NU implementation and services - Julie IU implementations and services – Mark Future directions – Jon Q & A - All
3Avalon Project Objectives Goal: Create an open source system to enable libraries and archives to provide online access to video and audio collectionsDigital audio/video management and delivery system, focused on needs of libraries and archivesFollow an agile, open source development modelLeverage existing technologies, where feasibleCommunicate and market the project broadly to increase awareness and grow the community of users and developers
4Project OverviewFunded in part by a National Leadership Grant from the Institute of Museum and Library ServicesOriginal name: Variations on VideoPlanning grant:August 2010 – July 2011Implementation grant:October 2011 – September 2014 September 2015Partnership between libraries at Indiana University and Northwestern UniversityDiscuss motivations
8Why Avalon?Existing repository systems don’t work well for time-based mediaDSpace, Fedora, Digital Commons, ContentDM, …Issues: integration with streaming servers; hard to support time-based navigation; difficulties in integrating access controlExisting media systems don’t work well with repositoriesKaltura, Brightcove, Ensemble, ShareStream, …Issues: focus on short-term access for teaching and learning; limited metadata and access control capabilities; integration with preservation workflowsStrong desire for community-based solution
10Media Player (mediaelement.js) Avalon ArchitectureIntegrationsUsersAvalon Media SystemILSDesktop, MobileHydrant Rails AppMedia Player (mediaelement.js)LMS, websitesSearchBrowseViewOmni-AuthAll UsersAuthentic-ationHydra HeadRuby- hornCan-CanBlack- lightAuthoriz-ationBrowser,Drop boxIngestDescribeManageArchival StorageCollection ManagersFedoraSolrMatterhornMedia Server Avalon code – consists of Hydrant Rails App and the Rubyhorn gem. Other pieces are leveraged.
11Release History 0.1 – July 2012 0.5 – October 2012 1.0 – May 2013 3.2 – December 2014 (expected)
12Development Team Julie Hardesty Jon Dunn Chris Colvard Mark Notess metadata specialistJon Dunnproject directorChris Colvardlead developerMark Notessproduct ownerCynthia Ramlolead user experience designerLeah LeedeveloperBrian KeesedeveloperAriadne Rehbeinproject assistantDevelopment TeamMichael Kleinlead developerKaren Millermetadata specialistStuStefan Elnabliaudiovisual specialistDan Zellnertesting and production supportClaire Stewartco-project directorJulie Rudderproduct owner
13Current development process A single, blended teamAgile Scrum processCode in public GitHubFace-to-face meetings at least 2x/yearDaily standups, IRC
140 – 3.1 Release functionality End-user functionalityFaceted discovery for search and browseVideo and audio playback in browsers or on mobile devicesStream-level securityAbility to embed media player in other websitesPersistent URLsRTMP and HTTP streaming, Flash and HTML5-based playerCollection managementInteractive or batch uploading of media files and MODS metadataPrivate collection dropboxesA hierarchical model for permissions that supports a flexible approach to batch collections-based content managementCustomized thumbnails by taking a “snapshot” or specifying a timepointDirect import of previously transcoded derivativesAvalon transcoding for multiple quality derivativesClaire?
150 – 3.1 Release functionality Integration with other enterprise systemsRed5 and Adobe media serversLocal authentication services (CAS, LDAP and others, using OmniAuth)Learning management systems, via the Learning Tools Interoperability (LTI) standardSupport for LDAP group access controlSupport for persistent URL systems (PURL, Handle)Master files: leave, delete, or rename/move after transcodingInstallationEasy installation and configuration via a virtual machine image and other methodsFully-transparent integration testing using travis-ci.org - every commit and pull request triggers a buildConfiguration approach for easier customization where possibleClaire?
17Release 4 (Spring 2015) Structural metadata Import and store structural metadataAllow users to navigate using media sectionsPrioritize media in the transcoding queueBulk update of metadataFedora 4 (testing?)Mark
18Beyond Release 4 Reporting and metrics Time limits for access IP restricted accessAPIs for ingest, analytics, metadata, etc.Fedora 4 migrationAccessibility (navigability, closed captioning)Transcripts/Auxiliary itemsImproved metadataPlaylists & clip makingIntegration with Wowza and Amazon Cloud Front Publish to YouTube or other systemsIntegration with Spotlight exhibits toolInternationalizationMark
19Follow our progress For sprint summaries, read our blog For more details and recordings, visit our wiki: wiki.dlib.indiana.edu/display/VarVideo/Avalon+Media+SystemAvalon Current Sprint pagewiki.dlib.indiana.edu/display/VarVideo/Avalon+Current+SprintPrevious Sprints pagewiki.dlib.indiana.edu/display/VarVideo/Previous+SprintsSprint Demoswiki.dlib.indiana.edu/display/VarVideo/Sprint+DemosMark
20Community and future directions Multiple implementations currently underwayStanford, UVa, UToronto, UT Austin, …Working on new features, including:Structural metadataTranscripts and captioningStarting to deal with code contributionsWorking with Hydra communityLeverage common features across “heads”HydraDAM and Avalon, other preservation-specific integrationsSustainability/governance/business modelExploring hosted options
21Sustainability and Governance Focus in 2015 with potential funder supportWant to grow the number of implementersOngoing costs of software support and developmentPotential revenue streamsMembership/sponsorshipIncome from hostingIn-kind developer contributionsGovernance and management structurePotential alignment with existing 501(c)(3) organization in higher ed open source community
22Julie Rudder j-rudder@northwestern Julie Rudder Digital Initiatives Project ManagerDLF Forum 2014
23Avalon replaces this system and does more Avalon replaces this system and does more. We launched a pilot in Summer 2013, support a few course and we went live.
24Use Cases Course Reserves – restricted access Archival Collections MMLC – video management by unitsoutside the library
25Future Additional units outside the library? Integration with other campus media solutions?Better integration with Ares Course Reserves?
27Production Environment Existing Production SystemsLoad BalancerVMStreaming - FMS 2Streaming - FMS 1Avalon Web AppShared SOLR4 TB dedicated HardwareDedicated SOLR CoreTranscodingShared Production FedoraFedoraProduction Environment
28Total Audio-Video Assets Served: 34,146 Background on Streaming at NULIntegrationsCanvas and BlackboardExisting handle serverLDAPAdobe Media Server (secure streaming)Related -- Ares Course Reserves (request mechanism)Streaming Media since 2001Total Audio-Video Assets Served: 34,146
29Implementation Pilot – Summer 2013 Full production – Fall 2014 A few classes supportTwo public collectionsLocal VM infrastructureFull production – Fall 2014All new requests via Avalon (no migration)MMLC launches UnitNUIT (central data center) infrastructure
30Retire current infrastructure by summer 2015 AVALONAll new digitizationSummer 2014Summer 20152004 – 30,000+ assets servedRetire legacy systems
31Staffing 1 DevOps (install, configure, support) 1 Developer (app customization, support)1 Project Manager (app oversight, integrations oversight)1 Service Owner (liaise with units & users, workflow, support)The DevOps time and my times was a lot of upfront work, the other two continue to have ongoing work.
32Additional support – 1st tier Business hoursLibrary Digital Collection Staff24 hour -Canvas helpdesk support24 HOURCentral Campus IT helpdesk24 HOURCanvas helpdeskCommon problems include, trying to direct link, navigation,
36IU Deployment Environment University IT ServicesLibrary serversRedundant Archive Storage(HPSS)Master Files4 core, 1GB RAM VM1GB Disk (VMware)Avalon Web AppSolr, FedoraShared SANDerivatives12 core, 120GB RAM,8TB dedicated hardwareTranscodingIU Enterprise StreamingAdobe Media Server
37Integrations CAS authentication Microsoft Active Directory (via LDAP) PURL resolverCanvas and Sakai (via LTI)Scholarly Data Archive (HPSS master file storage)Adobe Media Server (secure streaming)
38IU Production Instance Pilots began Fall 2013Moved to production, October 2014
44Specialized Avalon Instance: Kelley School of Business
45Future Migrations Video e-reserves (in pilot) LTI integration with Sakai, CanvasLicensed educational videoLDAP for campus restrictionVariations: music audioPedagogical, annotation toolsMedia Digitization and Preservation Initiative
46Production Staffing10% of a systems administrator (server management; higher initially)20% of a developer (install, configure, support; higher initially)20% of project/service manager (app oversight, workflow planning, work with units and users, support)Does not include collection management or central IT staffing of shared services
47IU Media Digitization and Preservation Initiative mdpi.iu.edu
48Avalon in the IU Media Digitization and Preservation Initiative (MDPI) 8.4 petabytes of audio and video data over 5 yearsAvalon serving as key access componentIntegrations:Metadata: MARC/ILS, EAD/XTFAuthentication: CASAuthorization: Active DirectoryLearning Management: Canvas, SakaiMedia streaming: Adobe Media ServerSide-by-side with trusted repository for preservationBit storage in existing hierarchical storage management systemHope to leverage HydraDAM workExploring out-of-region options
49Questions? For more about Avalon Website:Avalon list: to firstname.lastname@example.org/avalonmediasys