Presentation on theme: "B2: Storage and Compatibility John Knight Blackboard, Inc."— Presentation transcript:
B2: Storage and Compatibility John Knight Blackboard, Inc.
And now a word from our lawyers. Any statements in this presentation about future expectations, plans and prospects for the Company, including statements about the Company, the Building Blocks Program and other statements containing the words “believes,” “anticipates,” “plans,” “expects,” “will,” and similar expressions, constitute forward-looking statements within the meaning of The Private Securities Litigation Reform Act of Actual results may differ materially from those indicated by such forward-looking statements as a result of various important factors, including: product development, and other factors discussed in our Registration Statement filed on Form S-1 with the SEC. In addition, the forward- looking statements included in this press release represent the Company’s views as of July 26, The Company anticipates that subsequent events and developments will cause the Company’s views to change. However, while the Company may elect to update these forward-looking statements at some point in the future, the Company specifically disclaims any obligation to do so. These forward-looking statements should not be relied upon as representing the Company’s views as of any date subsequent to July 26, 2004.
Topics Overview Limitations Solutions –Portability –Structured data
B2 Storage Overview Building Block data types –Private data Configuration data Operational data (encryption keys, etc.) –Content data –Tool data –Extended attributes
B2 Storage Problems Private data –Data that has no relationship to Blackboard entities Custom attributes attached to Blackboard objects Reference data –References to Blackboard objects –Custom attributes referencing an external object
Private B2 Storage Stored in a directory specific to the Building Block –E.g., configuration data –Not visible to other Building Blocks –Data not related to Blackboard entities blackboard.platform.plugin.PlugInConfig
Extended Data User and Course support arbitrary name/value pairs, called “registries” Package blackboard.data.registry Loader/persister entities Values are portable (i.e., moved in Content Exchange and copy) –Copy settings may affect the outcome Values are “public” – visible to other Building Blocks
Extended User Data User user = ctx.getUser(); //load user registry UserRegistryEntry entry = UserRegistryEntryDbLoader.Default.getInstance().loadByKeyAndUserId( “key”, user.getId() ); String value = entry.getValue(); entry.setValue( “value” ); //store registry value entry.persist();
Extended Course Data Course course = ctx.getCourse(); //load user registry CourseRegistryEntry entry = CourseRegistryEntryDbLoader.Default.getInstance().loadByKeyAndCourseId( “key”, course.getId() ); String value = entry.getValue(); entry.setValue( “value” ); //store registry value entry.persist();
Content Storage Centralized storage for the bulk of the data that makes up a course Organized by course, then content Database component –Single row, fixed fields –Hierarchical File system component –Private area per content item
Content Storage – B2 View No “registry” of arbitrary attributes No indexed, semi-structured storage (e.g., XML data type) File system “blindly” processed by Content Exchange, Course Copy –E.g., SCORM support is implemented as a Building Block. The manifest, etc., are stored in the Content item’s private directory. –“Attached” files also get a database record and are “rendered” by the content list page –Not all files are “attached”
Content Storage Database Record“Attached” File
Content Storage Attached files are files with ContentFile records in the database To display a “private” file, a ContentFile record is not required –Just the URL –Useful for creating data files that are referenced by client-side viewers (e.g., applets)
Content Storage – CX and Copy By storing “custom” data in the file system component, the data will get picked up by Content Exchange and Course Copy Data moved as-is, with no transformations As a rule, avoid hard-coding references –Use template variables
Template Variables Portable placeholders for referencing Blackboard object data Dynamically expand at render time, using Context information No copy/export processing required
Template Syntax The object/attribute names are taken from the Perl syntax, since that’s where the bulk of the variable expansion originally occurred.
Referencing an Uploaded File For example: – Expands to: –
Must Have Templates content.url – expands to private directory for a particular content item course.url – expands to root course directory user.user_id – expands to the login name for the current user course.course_id – expands to the unique, string identifier for the current course
Template Expansion Or expanded programmatically… BbSession sess = BbServiceManager.getSessionManager().getSession( request ); String encoded = sess.encodeTemplateUrl( request, ); Can be expanded at render time…
Tool Storage No pre-defined storage location –Ad hoc; no central “repository” –Custom per-tool Some db storage, some file storage Example: Discussion Board Without B2 database extensibility mechanism, nowhere to put tool data Custom storage required
Storage Options Property files XML Data Third party storage engines
Property Files Easy to read; easy to write –Built into the Java platform –User editable as a fallback Drawbacks –Name/value pairs can be limiting –I18N limitations (hard-coded ISO )
XML Files Reasonably Complex Structure Available libraries Additional complexity –JAXB? –W3 DOM? –SAX? –DOM4J?
Third Party Storage Engines Arbitrary capabilities and flexibility Some very easy to use Some, not so much Possible gotchas (system permissions, etc.)
Third Party Storage Examples Sleepycat Software Berkeley DB Java Edition –Works in web application –Requires file system access Lucene –Text indexing –Unstructured data
Berkeley DB Example Java implementation of Berkeley DB Name/value pairs, but with primary and secondary indexes –Create reasonably complex data structures File-based storage –When written into B2 private storage, no custom permissions required
Berkeley DB Example //create environment on first access EnvironmentConfig envConfig.setAllowCreate( true ); Environment environment = new Environment( plugInConfig.getConfigDirectory(), envConfig ); //create database on first access DatabaseConfig dbConfig.setAllowCreate( true ); //we want multiple values for each record dbConfig.setAllowDuplicates( true ); Database db = environment.openDatabase( null, dbId, _dbConfig );
Berkeley DB Example User user = ctx.getUser(); DatabaseEntry keyEntry = new DatabaseEntry( user.getUserName().getBytes() ); DatabaseEntry valEntry = new DatabaseEntry(); //create new record valEntry.setData( value.getBytes() ); db.put( null, keyEntry, valEntry );
Outstanding Storage Issues Data reconciliation on copy/move –Updating an embedded Id –Updating external references No hooks for integration yet –Course Copy hooks—reference updates –Content Exchange hooks—transformations to and from packaged data
Migration For most plugins, not an issue Subset of R5 to R6 data is migrated (content only) Building Blocks required substantial modifications anyway
Course Copy Processes registry entries –Course –User Copies all content files, regardless of whether they’re registered as “attachments” Identical rules in Content Exchange
Looking Ahead Course Copy integration –Post-processing handlers (See System Admin Logs System Tasks Status) –Coarse-grained notification of a course copy event Content Exchange integration –Transform data dynamically during the packaging/unpackaging operation
Common Practice Per-User data, attach via the Registry Keep private data private –B2 data, store in private file store –Encrypt sensitive registry values (e.g., external system password), keep the key in B2 file store
Common Practice Tool data –Use a storage engine for complex data structures –Existing, external databases can be used Requires careful investigation of security issues Difficult to implement in a portable way Content data –Use portable references where possible General –Store “reference” data for sanity checks, e.g., store the original course id with a record. You can check the current course to know if the data has been copied, etc., and take appropriate action
Conclusion Know the limitations Understand where and how data can be read and stored (not just by your code) There are good solutions that work within the (current) confines of the system –Better ones to come!