OCAP RI Training CableLabs Winter Conference Philadelphia March 15-16, 2012 1

Schedule 2 Thu Mar 15 – 8:30am to 4:30 pm Fri Mar 16- 8:30am to 2:30 Project Overview and History - DaveH  Build system - Chris Logging and debugging - ScottD Software architecture overview - MarcinK  Profiling applications with the RI’s JVM – SteveA/ScottD [lunch] System Information – Prasanna/Craig Service Selection and DVR subsystem - Craig PC Platform – Marcin/SteveM/Neha HN subsystem – Doug/Craig/Marcin/Lori Fri Mar 16- 8:30am to 2:30 Graphics - SteveA Testing – ScottA/Nicolas Issue reporting and handling - ScottA Stabilization/Robustness project – Marcin/Chris Miscellaneous topics - TBD 2

Project Overview and History March 15-16, 2012 3

Project started June 2008 Core OCAP I16+ compliant DVR I03 extension Several ECNs short of 1.0.0 compliance DVR I03 extension Missing ECNs 897, 994, 1017 Front Panel I02 extension Approx 200 bugs carried over from legacy code base Vision Workbench/Clientsim SDK Closed code base 4

Current Status 5 PC Platform SDK This is an open-source project Feb 2012 (1.2.2 Bundle Release) Core OCAP 1.2.2 DVR Extension I08 Front Panel Extension I05 Device Settings Extension I05 Home Networking Extension I08 Home Networking Protocol I07 JVM Based on Sun open source PhoneME advanced MR2 PC Platform Windows XP initially; now Windows & Linux (several versions) SDK Rel-6 released Nov 2011 – no further versions planned This is an open-source project http://java.net/projects/ocap-ri/ 5

2012 Schedule 6 1.2.1 Bundle released on 01/12/12 1.2.2 Bundle (1.2.2 Rel-A RI) released on 02/24/12 Included VPOP as primary feature Also MIBObject.getValue Completed implementation of ServiceResolutionHandler Completed implementation of DTCP/IP on PC platform Maintenance against 1.2.1 RI 1.2.2 Rel-B planned for 04/05/12 No feature development Maintenance against 1.2.2 RI Fixes will be taken off the current trunk 1.2.2 will not be maintained as a separate branch 1.2.3 Bundle planned for 05/17/12 HN features to align with CVP-2 compliance Backlog items Add IPv6 and https support to the PC platform Stack housekeeping tasks 6

Other DTCP platform port for Windows and Linux MPEOS API changes were included with 1.2.1 RI release RI Implementation now available for internal testing at CableLabs Phasing out support from Windows XP to Windows7 Some performance issues with GStreamer plugin on Win7/newer Linux platforms are being investigated SDK Rel-6 released on 11/17/11 No current plans for a future release Upgrade of our DirectFB library we use (low priority) Port RI to the MAC (very low priority) 7

Project site walk through Main Project Page Bug Databases Forums Wikis Contribution Process Bug fix cutoffs Release notes Coding standards 8 8 8

Why an RI? 9 OCAP Specifications Stubs & DTDs Release Bundle So Vang Clarify Release Bundle Test So Vang OCAP Reference Implementation OCAP Tests (CTP) Test 9

OCAP Release Bundle Components of a bundle are Specs Stubs DTDs RI implementation, including integration tests CTP conformance tests 10

OCAP RI Requirements 11 RI runs on a PC Windows initially – now Linux RI and PC IDE must be available on open-source terms RI and PC IDE must only include components with licenses compatible with the ODL dual-license plans Components available only under GPL are not OK Licenses for all third-party RI components must be reviewed by both CableLabs and the ODL legal teams RI works with existing CableLabs ATE/CTP tests RI adheres to current and future OCAP core specs RI adheres to current and future OCAP extensions specs To ensure backwards compatibility of the spec,MSO guides must run on the RI To ensure backwards compatibility of stack ports of the RI, any changes to the MPEOS porting layer must be approved by the RI steering committee 11

Licensing Models 12 CableLabs OpenCable Project GPL License on java.net CableLabs OpenCable Project OCAP Stack, PC Platform, Head-end Emulator Sun PhoneME Project - JVM Commercial License CableLabs Commercial License Also free Stack, platform and emulator RAND IPR commitment Bug fixes in stack contributed back Sun or other JVM vendor Commercial CDC/PBP 1.1 JVMPhoneME JVM Phil 12

OCAP RI Branching Strategy Three principal branches Mainline/Development Branch Code implemented by internal RI Dev Team Code from open source contributors that are vetted by RI Tech Leads Other working branches get merged back to Mainline periodically Branded Branch (eg, “1.1.4”) Fixes and enhancements that are tied to the spec and which have been verified by the CTP Branded branch is maintained separately from mainline Changes from branded branch eventually migrate back to mainline development One branded branch per spec release Experimental Branch Open source contributors have write access to this directory No other restrictions Merging to Mainline on a case-by-case basis 13

Bug Tracking Two Bug Tracking Databases Internal (private) JIRA db (OCORI) at CableLabs, tied to CableLabs CTP bug db External (public) JIRA db on java.net (IT); hides details of CTP-related issues 14

RI Build System March 15-16, 2012 15

Building the RI – The Easy Way See https://community.cablelabs.com/wiki/display/OCORI/Quick+Start for detailed instructions. Setup development environment Cygwin + JDK + Ant for Windows A little more required for Linux (see Wiki) Get checkout_dev_env.sh (from svn) Use checkout_dev_env.sh to get source, create setEnv file Execute ant in appropriate directory. Builds Platform and Stack. See Wiki for detailed instructions. 16

Build System – Environment Variables Easy to work in several different RI code bases at the same time. OCAPROOT The absolute path to the OCAP-1.0 directory. Required for compilation/execution Example: E:/Cablelabs/svn/OCAPRI/trunk/ri/RI_Stack OCAPHOST Defines the host build environment Build system reads host environment configuration files from ($OCAPROOT/hostconfig/$OCAPHOST) Required for compilation only Example: Win32-Cygwin 17

Build System – Environment Variables OCAPTC The Target Configuration for the build. Basically the port you are working on. Defines a subdirectory hierarchy where: build configuration files are found ($OCAPROOT/target/$OCAPTC) binary intermediate products are built $(OCAPROOT/gen/$OCAPTC) final binary products are installed and runtime configuration files are kept ($OCAPROOT/bin/$OCAPTC) Suggested format is: <org>/<platform>/<os>/[debug|release] Example: CableLabs/simulator/Win32/debug Required for compilation/execution 18

Build tools Make Ant JDK (1.4 or higher) Compiles JNI, MPE, MPEOS, and thirdparty native libraries Ant Coordinates the entire build system Wiki contains a list of top-level build targets JDK (1.4 or higher) Used to compile stack and test application sources 19

Win32 Port Host environment is Cygwin See Wiki for a full list of Cygwin libraries required to build the RI Stack and Platform Cross-compile to MinGW (no Cygwin DLL) Lots of work (including JVM patches) to deal with POSIX vs. Win32-style paths POSIX for gcc Win32 for javac, javah, etc.. VERY SLOW (compared to Linux) JVM binaries pre-built and checked-in to save compilation time since most won’t be modifying the JVM WindowsXP, Vista 20

Linux Port Known working distros/versions: Fedora 10/12/13/15 Ubuntu 10.04/10.10/11.04 Ubuntu 11.04 Much faster than Win32 on the same hardware. See Wiki for detailed instructions. 21

Logging and Debugging March 15-16, 2012 22

Stack logging log4j APIs included in the spec for use by applications Additional Logger methods avoid String concatenation overhead in most cases Monitor applications configure logging through DOMConfigurator or PropertyConfigurator Groups Multiple loggers can share a common group name, which can be used during configuration 23 23

Stack logging continued New appenders MIB Appender AsyncAppender uses an additional thread and a queue to offload writing to the target appender off of the caller thread New configuration capabilities Configure at the ‘group’ level or the logger level Filter support, including ExpressionFilter (ability to use regular expressions for fine- grained control over logging verbosity) 24 24

Stack logging continued Additional information available from the Wiki https://community.cablelabs.com/wiki/display/ OCORI/Configuring+Java+stack+logging 25 25

Platform logging Platform code uses log4c to manage logging Configuration found in $PLATFORMROOT/log4crc Additional information available from the Wiki https://community.cablelabs.com/wiki/display/ OCORI/RI+PC+Platform+Logging 26 26

Logging and IssueTracker When attaching a log to IssueTracker Ensure the log contains timestamps Helpful if Java debug logging is enabled 27 27

Chainsaw screenshot 28 28

Java Stack debugging Possible to step through breakpoints in Java stack code, generate stack traces and thread dumps Stack trace, thread dumps available via jdb (included with the Sun JDK) To enable Java debugging, un-comment VMOPT19 & 20 in mpeenv.ini and start debugger or jdb Re-comment VMOPT 19 & 20 when done.. 29 29

Platform debugging gdb can be used to generate a trace if the Platform terminates unexpectedly ./runRI.sh -gdb 30 30

RI Software Architecture March 15-16, 2012 31

Host Operating System 32

Platform Support Libraries 33

Platform Implementation 34

Platform API 35

Platform Summary Full software emulation of STB media decoding and presentation hardware. Majority of the code is 3rd party support libraries. Leverages existing frameworks: GLib – utility library. Gstreamer / HDHomerun / VLC– tuner control. GStreamer – media decoding and presentation. wxWidgets – user interface. Net-SNMP – Master Agent. No OS abstraction APIs. 36

OCAP Porting API 37

OCAP Native Library 38

OCAP JVM 39

OCAP Java Implementation 40

OCAP API 41

OCAP Summary Ported to Platform APIs for STB/video-related functionality. OS functionality MPEOS port maintained together with the stack code. MPE contains platform-independent / portable C code. Integrates Advanced phoneME JVM. OCAP Functionality implemented via pluggable Java Manager implementations. 42

Profiling applications with the RI’s JVM March 15-16, 2012 43

phoneME JVM RI uses the open source (GPL2) phoneME Advanced JVM from Sun/Oracle Closely based on J2SE 1.4 Adding JSSE (secure sockets) support in an upcoming release OCAP requires a JVM compliant with the latest maintenance releases of: Personal Basis Profile 1.1 Foundation Profile 1.1 Connected Device Configuration 1.1 Last update: October 24, 2009 Patch common source to fix bugs and build problems Win32/Cygwin/Make filesystem issues JDWP (VM debugging) thread sync issues PNG bug GIF bug? 44 44 44

JVM Build All JVM-related files located in $OCAPROOT/jvm Build disabled by default for RI Win32 – pre-built binaries checked in to SVN Enable/Disable building the VM with “build.jvm.exclude” entry in $OCAPROOT/target/$OCAPTC/buildrules.properties 45 45 45

ocap_vmlib Interfaces and classes to assist in integrating a VM with the RI Stack Includes a full AWT implementation ported to MPE graphics APIs (DirectFB) Documentation $OCAPROOT/docs/JVMPortingGuide.doc 46 46 46

Profiling and analysis tools There are a number of tools available for investigating issues in the Java stack code NetBeans Profiler CVM Inspector HPROF jdb 47 47 47

Profiling with NetBeans 6.8 JVMTI-based Supports profiling of CPU, Memory and high level JVM stats (GC, thread activity) Used to identify CPU and memory hot spots Does not support creation or comparison of heap dumps https://community.cablelabs.com/wiki/display/OCO RI/Profiling+the+RI%27s+JVM+using+NetBeans+6. 8 48 48 48

CVM Inspector Set of utility functions and a shell script which can assist in inspecting the state of the JVM Available by setting a flag in the JVM build Used to generate and compare heap dumps Runs in either client-server mode (standard JVM client connects via a socket to the RI) or standalone mode (GDB) https://community.cablelabs.com/wiki/display/OCO RI/Generating+heap+dumps+on+the+RI%27s+JVM +using+CVM+Inspector 49 49 49

HPROF A command-line profiling tool Used to generate detailed monitor statistics https://community.cablelabs.com/wiki/display/OCO RI/Generating+thread+monitor+stats+on+the+RI%2 7s+JVM+using+hprof 50 50 50

JDB The Java debugger command line interface Used to generate lock and thread stack information https://community.cablelabs.com/wiki/display/OCO RI/Generating+Java+thread+dumps+and+monitor+i nformation 51 51 51

Service Information (SI) March 15-16, 2012 52

OCAP SI OCAP SI access API Provides information about available services in an interactive broadcast environment OCAP uses SCTE-65 SI model (standard for SI delivered out-of-band for Cable networks) SI tables are acquired from out-of-band channel (ex: legacy OOB/DAVIC or DSG broadcast tunnel) Incorporates JavaTV SI API When CableCARD is present, all applications (service bound/unbound), have access to SI delivered on the OOB channel. Service Information API - provides an application with a mechanism for finding out information about available services in an interactive broadcast environment Service Selection API - provides an interface for selecting these services for presentation The JavaTV APIs are sufficiently abstract to support the variations of Service Information including DVB, and OCAP. The additional OCAP SI APIs are necessary to address features unique to the Open Cable environment. When the CableCARD device is absent, any unbound applications that run, SHALL not have access to service information in the out-of-band channel, as defined in [SCTE 65]. OCAP supports Service Information as defined in [SCTE 65] and [SCTE 54] instead of DVB Service Information. In an OpenCable Host, [SCTE 54] is used to provide Service Information in-band. In an OpenCable Host with a CableCARD device inserted, [SCTE 65] is used to provide Service Information out-of-band.[SCTE 65] defines Service Information for application-type virtual channels and allows Host platforms to optionally support these channels. OCAP platforms SHALL NOT expose information relating to application-type virtual channels through the Service Information interfaces described in Annex T. OCAP platforms MAY discard Service Information relating to application-type virtual channels as defined [SCTE 65] (primarily for delivery through the Virtual Channel Map and Source Name Sub Table) and not allow this information to be retrieved by an OCAP-J application. 53 53

SI profiles SCTE-65 defines 6 profiles for OOB SI RI stack supports Profiles 1-3 Includes tables: NIT-CDS, NIT-MMS, SVCT-DCM, SVCT-VCM, NTT-SNS and STT (optional) SI Profile Tables 1 - Baseline SVCT, NIT, NTT 2 – Revision Detection Versioning enabled for NIT, NTT, SVCT 3 – Parental Advisory Profile 2 plus RRT 4 – Standard EPG Data Profile 3 plus AEIT, AETT 5 - Combination LVCT, MGT. Backward compatible (1-4) 6 – PSIP only LVCT, AEIT, Optional AETT SCTE-65 defines a standard for SI delivered out-of-band for Cable. We have seen mostly Profile-1 with some Profile-2 tables. 54 54

Stack Components

OCAP SI javax.tv.locator javax.tv.service.transport.Transport Incorporates following Java TV SI packages javax.tv.locator javax.tv.service.transport.Transport javax.tv.service.transport.Network javax.tv.service.transport.TransportStream javax.tv.service.transport.ServiceDetails 56 56

OCAP PSI Incorporates following packages Supports Object Carousel javax.tv.service.navigation.ServiceComponent org.ocap.si.ProgramAssociationTable org.ocap.si.ProgramMapTable Supports Object Carousel Carousel ID resolution Deferred association tag resolution NSAP resolution 57 57

Java SIDatabase/SICache Access to native SIDB via Java SIDatabase SI data/events processed and dispatched to registered listeners Most recent SI is cached in the SICache to avoid trips across JNI layer to MPE SI Database SICache is flushed periodically (flush interval configurable) Discrete SI request types to access various SI objects SI requests asynchronously satisfied or timeout (timeout value configurable) after time elapses 58 58

MPE SI When does it start? Gated by POD_READY indication Event-driven acquisition and parsing of OOB table sections via MPEOS section filtering API Dedicated thread for OOB SI At start-up sets filters for all five table types NIT- CDS, NIT-MMS, SVCT-DCM, SVCT-VCM, NTT- SNS Individual table timeout values configured via mpeenv.ini SIDB populated in MPE layer Table acquisition logic: SVCT-VCM RDD Heuristic (section-seen-count, CRC) All defined channels acquired VCM timer (4 minutes) SVCT-DCM DCM timer (2.5 minutes) 59 59

SITP SI State diagram 60 60

MPE SI (contd.) Profile-1 SI and CRC processing Section count unknown. MPE SI engine employs a heuristic based on unique section CRC values and CRC match count for deterministic... Section CRC match counts are configurable for individual tables via mpeenv.ini Table acquisition timeout values also aid in faster acquisition SVCT DCM scoreboard used to accelerate VCM acquisition On slower networks with infrequent section repeat cycle SI acquisition can be problematic Table acquisition logic: SVCT-VCM RDD Heuristic (section-seen-count, CRC) All defined channels acquired VCM timer (4 minutes) SVCT-DCM DCM timer (2.5 minutes) 61 61

MPE SI (contd.) Profile-2 and above rely on the Revision Detection Descriptor (RDD) for table section count Table acquisition logic: SVCT-VCM RDD Heuristic (section-seen-count, CRC) All defined channels acquired VCM timer (4 minutes) SVCT-DCM DCM timer (2.5 minutes) 62 62

MPE SI Cache Enable/Disable by mpeenv.ini configuration (Disabled in RI by default) Speeds up stack start-up on slower networks by using cached SI SI data is written to persistent memory SI data read from file at start-up if cache is enabled Normal OOB SI acquisition also continues Updates to SI are reflected in the cache For testing only (Not intended for deployment) Table acquisition logic: SVCT-VCM RDD Heuristic (section-seen-count, CRC) All defined channels acquired VCM timer (4 minutes) SVCT-DCM DCM timer (2.5 minutes) 63 63

MPE PSI Subsystem PSI: PAT & PMT acquisition using the MPE Section Filter subsystem Manages PAT/PMT acquisition and parsing from OOB (DAVIC/DSG Broadcast), DSG application tunnels, tuners, and HN streams SCTE-65 defines a standard for SI delivered out-of-band for Cable. We have seen mostly Profile-1 with some Profile-2 tables. 64 64

MPE PSI Subsystem SITP uses 6 different filter classes on each transport stream: Initial PAT Initial selected PMT Initial secondary PMT(s) Revision PAT Revision selected PMT Revision secondary PMT(s) Fixed resources (local tuners & legacy out-of-band) are assigned filter groups at startup according to mode Dynamic resources (DSG app tunnels, HN streams, remote tuners) are assigned filter groups when the session is started SITP acquisition logic then works in terms of classes and filter priorities without concern for class-to-group associations SCTE-65 defines a standard for SI delivered out-of-band for Cable. We have seen mostly Profile-1 with some Profile-2 tables. 65 65

MPE PSI State machine: SCTE-65 defines a standard for SI delivered out-of-band for Cable. We have seen mostly Profile-1 with some Profile-2 tables. 66 66

MPE PSI Subsystem Has 6 defined acquisition modes for tuner-based section acquisition, to tailor section filter resource usage to the platform’s filtering capabilities. Mode 1: Legacy single-filter sharing Mode 2: Dedicated filter per tuner Mode 3: Dedicated 2 filters per tuner without secondary acquisition Mode 4: Dedicated per-filter tuner for PAT and selected PMT with “wandering” PSI pre-fetch filter Mode 5: Mode 3 +1 filter that picks up all secondary PMTs across all tuners Mode 6: No filter sharing (every section request uses its own filter) SCTE-65 defines a standard for SI delivered out-of-band for Cable. We have seen mostly Profile-1 with some Profile-2 tables. 67 67

SIDB MPE SIDB contains following (SI/PSI) objects Transports Networks Transport Streams Services Programs Elementary Streams Provides API to access the various SI objects 68 68

SIDB API Lock/unlock SI DB for read/write access Access provided using opaque SI handles Service resolution methods mpe_siGetServiceHandleBySourceId() mpe_siGetServiceHandleByFPQ() mpe_siGetServiceHandleByServiceName() Etc. SI enumeration methods mpe_siGetAllNetworks() mpe_siGetAllServicesForTransportStream() mpe_siGetServiceComponentsForServiceHandle() Approximately 125+ API in SI DB 69 69

SIDB API (contd.) Object Carousel support methods CA support methods Look-up PID given CarouselID/Component Tag Find Program Number given Deferred Association Tag CA support methods Enumerate CA descriptors Lookup ECM PID 70 70

System Time RI parses out-of-band System Time Table (STT) to extract system time in UTC STT section filtering and parsing done on a dedicated thread in MPE SITP layer Can be disabled for platforms which directly process STT or use alternate mechanism Maintains a difference between system time and network time Employs a smoothing algorithm to avoid jumps 71 71

Service Selection & DVR Subsystem March 15-16, 2012 72

ServiceContext JavaTV ServiceContext is the primary entity for Service presentation/selection AV Services can be: “Broadcast” Service. Service object retrieved from SIManager or OCAPLocator representing a frequency/program/modulation. RecordedService. A recording accessed via a RecordingRequest. RemoteService. A Service representing an HN- streamable ContentItem retrieved from the ContentServerNetModule 73 73

ServiceContext The RI’s ServiceContext implementation utilizes delegates for each type of Service that can be presented: BroadcastServiceContextDelegate: Used for presenting Broadcast Services when the DVR extension is not present RecordedServiceContextDelegate: Used for presenting RecordedService (DVR-only) RemoteServiceContextDelegate: Used for presenting RemoteServices (HN-only) DVRBroadcastServiceContextDelegate: Used for presenting Broadcast Services when the DVR extension is present. Supports the TimeShiftProperties interface which allows the app to enable/disable timeshift and specify timeshift duration. AbstractServiceContextDelegate: Used for selecting AbstractServices – Services which represent unbound xlets 74 74

ServiceContext Class relationship: 75 75

ServiceContext OCAP requires that the ServiceContext provide a JMF Player to the application for media control On the RI, the Player and the Player-associated Controls provide rendering control, but actual rendering is done by the platform. The ServiceContextDelegate acquires tuner/TSB resources and the RI’s Player infrastructure manages everything else, including: MediaAccessAuthorization (parental control, etc) Conditional Access PSI/ServiceComponent retrieval NetworkInterface/component validation CCI enforcement TSB/Segmented Time-shift attachment (for DVRBroadcastServiceContextDelegate) SegmentedRecordedService navigation (for RecordedServiceContextDelegate) SCTE-65 defines a standard for SI delivered out-of-band for Cable. We have seen mostly Profile-1 with some Profile-2 tables. 76 76

ServiceContext Resources required for presentation:

ServiceContext Basics Delegate/Player relationship:

ServiceContext Basic Broadcast Service “live” playback example: 79 79 SCTE-65 defines a standard for SI delivered out-of-band for Cable. We have seen mostly Profile-1 with some Profile-2 tables. 79 79

ServiceContext Basic Broadcast Service time-shift playback example: 80 SCTE-65 defines a standard for SI delivered out-of-band for Cable. We have seen mostly Profile-1 with some Profile-2 tables. 80 80

ServiceContext SC State machine: 81 81 SCTE-65 defines a standard for SI delivered out-of-band for Cable. We have seen mostly Profile-1 with some Profile-2 tables. 81 81

DVR TimeShiftManager 82

TimeShiftManager Basics Internal Manager only (not exposed to OCAP Applications) Provides both tuning and TSB management for multiple parts of the DVR-enabled RI stack ServiceContext implementation uses TSM to: Tune Enable Service buffering (via TimeShiftProperties) Discover already-tuned/buffering Services (and NetworkInterfaces) Acquire/enumerate TimeShiftBuffers (TSBs) for playback. RecordingManager uses TSM to: Enable Service Buffering (via BufferingRequests) Discover already-tuned/buffering/buffered Services (and NetworkInterfaces) Convert previously-buffered and currently-buffering TSBs into RecordedServices 83

TimeShiftManager Basics HN TSBChannelStream uses TSM to: Tune (ChannelContentItem Tuning Locator) Discover already-tuned/buffering Services Initiate buffering Acquire/enumerate TSBs for streaming 84

TimeShiftManager 85

TimeShiftManager Responsibilities Base Service acquisition mechanism for DVR-enabled RI stack. Manages the pool of MPE time-shift buffers (the number of TSBs and their size) Embodies the knowledge of how Network Interfaces are shared and creates SharedResourceUsages Represents the use of the NI in ResourceContention and manages the coordinated shutdown of NI-based operations when/if the NI is lost or necessary components are lost, including: Transport stream synchronization Service component availability Conditional Access (CA) Switch digital video transitions Manages the TimeBases for all TSBs to allow for: Proper navigation of TSBs by JMF Discovery of TSB/TSBs for retroactive recording (conversion of already-buffered content into RecordedService(s)) 86

TimeShiftManager Internals 87

TimeShiftManager Internals Static/Sequence Diagrams 88

DVR RecordingManager 89

RecordingManager Basics Defined as part of OCAP DVR I08 and DVB GEM DVR org.ocap.shared.dvr.RecordingManager is defined by DVB GEM – meaning its interface is used by specifications other than OCAP org.ocap.dvr.OCAPRecordingManager extends the GEM RecordingManager – adding OCAP-defined functionality Has to work with/enable HomeNetworking extension when present 90

RecordingManager Responsibilities RecordingManager-provided functionality: Is responsible for starting and stopping RecordingRequests at particular times and managing conditions which can interrupt/resume recordings (so RecordingManager is always running) Manages and attempts to honor BufferingRequests Manages RecordedServices (“Recordings”) Persistently saves RecordingRequests and associated application-supplied metadata (so RRs survive reboots) Manages the space allocation/prioritization for RecordingRequests Provides warnings when NetworkInterfaces are about to be utilized for RecordingRequests (RecordingAlertListers) 91

RecordingManager API Diagrams 92

RecordingManager Implementation RecordingManager Class Diagram 93

RecordingInfo Metadata 94

RecordingRequest States The “blue sky” path for a (Leaf) RecordingRequest takes it through these states: PENDING_NO_CONFLICT: Recording is ready to record and there is no contention for tuners with any other RecordingRequest IN_PROGRESS: Recording is on-going (tuner is tuned and data is being written to disk) COMPLETE: Recording has completed and content is present in its entirety DELETED: Recording has been deleted (RecordedService.delete() has been called) 95

RecordingImpl states Many things may not go as planned during the recording process which put the RR into IN_PROGRESS_WITH_ERROR: The RecordingRequest may not acquire an NI (tuner) at the time it’s supposed to start The NI may be lost after the RR has started Sync may be lost on the NI PSI (PAT/PMT) may not be acquired or lost Conditional Access may be denied RemovableStorageVolume is disconnected MediaStorageVolume (disk) becomes full Service may be re-mapped via SPI (SDV) Any of these conditions may remedy themselves 96

RecordingRequest States 97

RecordingImpl States Each RecordingImpl has an external state and an internal state (IState) 98

Recording Process 1 99

More RM Detailed Diagrams More detailed recording scenarios 100

RI PC Platform March 15-16, 2012 101

Platform Components Support Libraries Implementation API 102 3rd party libraries supplying foundation for providing TV/Media functionality in the Platform. Meet the RI dual licensing requirements. Implementation Leverages features provided by the support libraries. Adds extensions and customizations. Glues everything together into a coherent implementation. API Hides the underlying technology-specific terminology. Uses STB concepts to simplify command and control of the Platform features. 102

Support Libraries Overview 103

Support Libraries Overview iconv & gettext provide internationalization support and are required by glib xml2 library requires zlib and is used by both gstreamer and clinkc pthreads implementation wraps Win32 thread support in POSIX APIs and is required by clinkc gstreamer requires glib gst-plugins-base need gstreamer and oil – optimized low-level routine library (i.e. fast memcpy using MMX or SSE extensions) gst-plugins-good requires gst-plugins-base ffmpeg supplies A/V decoding support; at present, only MPEG-1/2 video decoding capability is used wxWidgets library provides multi-platform GUI framework and requires gtk+ library on Linux log4c is a native implementation of the Apache logger 104

Implementation Overview 105

Recent updates Trying to stay up-to-date on 3rd party library revisions as much as possible. Glib Updated from 2.22.4 to 2.28.7 Should be available in the 1.2.2 Rel-B and future releases. Gstreamer Currently in-progress. gstreamer : 0.10.22 to 0.10.35 gst-plugins-base : 0.10.23 to 0.10.35 gst-plugins-good : 0.10.10 to 0.10.30

Supporting Infrastructure Logging – uses log4c, available to all platform modules. Config – human-readable name-value keystore: # This is a comment RI.Launch.App.0 = ocap_stack RI.Launch.App.0.cwd = $(OCAPROOT)/bin/$(OCAPTC)/env Launcher - .EXE that: Starts the platform followed by all platform client DLLs. Enters that main loop and waits for reset/shutdown signal. 107

GStreamer Overview 108

GStreamer Summary Framework and building blocks for the MPEG-2 transport stream processing. Supplies in-band and OOB section filtering. Supplies MPEG-1 & 2 video decoding capability. Provides Time-Shift Buffer recording/playback. Supplies video and graphics plane display capability (in conjunction with UI module). Supplies VPOP output stream packet source 109

User Interface Overview 110

User Interface Summary Emulates the final TV display output – provides a physical view of the rendered frame buffer. Relies on OpenGL technology for graphics- intensive operations (scaling, compositing, rendering). Abstracts out window-specific operations into an API to permit the usage of any UI/Widget Toolkit/Framework. Currently supports wxWidgets framework as well as native OS targets (Win32/Linux). 111

Platform API Object-based. Attempts to be functionally equivalent with an STB driver API. Supports multiple instantiation of each object/API. Possible to supply multiple implementations of each API (like Java interfaces). Removes dependencies on any technology-specific terminology (abstraction layer). Some objects support multiple clients (e.g. display module for iDCR). 112

Home Networking Subsystem March 15-16, 2012 113

HN Topics Overview Public Class Hierarchy Streaming Mapping to Platform Gstreamer pipeline for serving, playing back Porting DTCP-IP Current Status Limitations 114 114

HN Overview The Home Networking Extension provides support for an ever increasing set of use cases surrounding the discovery and delivery of content over the customer’s home network. Use cases have evolved from an adjunct option for content delivery to a primary one. Multi-room DVR (1.1.3 – 1.1.5) 3rd Party DLNA Device compatibility (1.2 – 1.2.2) CVP-2 and Cloud based scenarios (1.2.3 – later) 115 115

HN Overview OCAP 1.1.3 : Multi-room DVR Initial Release of HN Extension Focus on CTP test conformance Limited streaming capability OCAP 1.1.4 – 1.1.5 : Multi-room DVR cont. Redesigned Content Directory Service Continued CTP test conformance Improved streaming capability 116 116

HN Overview OCAP 1.2 - 1.2.2 : 3rd Party Device Compatibility Lower level UPnP Diagnostics API Improved UPnP/DLNA compatibility Live channels, Hidden items, Video Primary Output Port, Alternate URI OCAP 1.2.3 : CVP-2 Support RUI Discovery and Authentication 117 117

HN Overview Java abstraction of: Based upon, but abstracted from UPnP Network device discovery Content publishing Recordings, Live streaming, personal content Content discovery Content playback/display Remotely scheduling recordings Based upon, but abstracted from UPnP 2 specs HNEXT I08 Spec (Java APIs) HNP2.0 I07 Spec (Mapping to UPnP) 118 118

Baseline HN Technology UPnP Device Architecture 1.0 Media Server 2.0 Content Directory Service 3.0 Connection Manager Service 2.0 Scheduled Recording Service 1.0 DLNA DLNA Guidelines, August 2009 119 119

HN Stack Components 120 120

HN Stack Architecture 121 121 Java Stack org.ocap.hn org.ocap.dvr upnp media server UPnP Diagnostics API CyberLink For Java mpe mpeos Platform HN Gstreamer Pipeline 121 121

Design Choices UPnP Diagnostics API implemented using a 3rd party UPnP stack Almost all code at Java level Content served/consumed at platform layer Rather than pass data up/down from Java Sockets passed between layers 122 122

UPnP Diagnostics Classes/Interfaces Singleton UPnPControlPoint Discovers UPnP devices on the network Register low level message handlers UPnPClientDevice Maps to each UPnP device on a specific network UPnPClientService Maps to each UPnP service on a device on a specific network Invoke UPnP actions on a network service UPnPActionInvocation 123 123

Basic Public Classes/Interfaces Singleton NetManager Get list of discovered Devices on the home network Get aggregated list of NetModules NetModule is java abstraction of UPnP Service Device Straightforward mapping of UPnP device NetModules Loosely based on UPnP Services ContentServerNetModule RecordingNetModule Event driven listeners for discovery and activity DeviceEventListener/NetModuleEventListener/Conten tServerListener 124 124

Basic Client Operation NetManager.addNetModuleEventListener(…) NetManager.getNetModuleList(…) ContentServerNetModule.requestRootContainer ContentServerNetModule.requestBrowseEntries ContentServerNetModule.requestSearchEntries Get back ContentList of Items/Containers Render Item(s) 125 125

Content Public Classes/Interfaces 126 126

Content package Natural 1:1 mapping to UPnP ContentDirectory ContentContainer = <container> ContentItem, ChannelContentItem = <item> ContentResource = <res> MetadataNode ≈ <property attribute=blah> requestBrowseEntries = Browse action requestSearchEntries = Search action 127 127

Recording Content Interfaces 128 128

Client Recording Content Not so simple UPnP Mapping RecordingContentItem ≈ <item> in CDS NetRecordingEntry ≈ <item> in CDS But also: RecordingContentItem ≈ recordTask in SRS NetRecordingEntry ≈ recordSchedule in SRS Refer to HNP2.0 Appendix I Singleton NetManager gives access to RecordingNetModules RecordingNetModule represents a DVR device requestSchedule ≈ CreateRecordSchedule SRS Also requestReschedule, etc. 129 129

Recording Scheduling Interfaces Implemented by Stack Class Implemented by Application Class 130 130

Server Recording Content Server stack per se does not initiate recording Application (EPG) gets RecordingNetModule and registers request handler RecordingNetModule is a NetRecordingRequestManager UPnP SRS actions result in handler invocation Handler responsible for calling DVR APIs and publishing resulting recordings OcapRecordingRequests are RecordingContentItems Note: RI HN extension currently requires DVR extension In future, no DVR APIs will be referenced in HN Allowing HN-Only client or server 131 131

HN Streaming Once content has been published on the network by the DMS, and clients (DMCs) have discovered ContentItems, streaming can be initiated. Remote client (DMC) initiates HTTP-based streaming from the DMS using either the “res” URI from the ContentDirectoryService (CDS) or the “alternateURI” registered by the guide application on the ContentItem RI uses CyberGarage handle connection initiation to the HTTP port Streaming initiation requires authorization by the NetAuthorizationHandler, if registered. For streaming requests, the NAH is passed the requested URI and source IP The NAH2 is additionally passed the entire request line and request headers 132 132

HN Streaming Incoming streaming requests are delegated to one of 5 interceptors: IconInterceptor: Provides UPnPManagedDeviceIcon support (URI path /ocaphn/icon) RecordedServiceInterceptor: Provides RecordedService streaming support (URI path /ocaphn/recording) ChannelRequestInterceptor: Provides ChannelContentItem streaming support (URI path /ocaphn/service) VPOPInterceptor: Provides VPOP streaming support (URI path /ocaphn/vpop) PersonalContentInterceptor: Intended to provide content to local files (currently unsupported) 133 133

HN Streaming: Recorded Service Multiple HTTP range requests for a RecordingContentItem 134 134

HN Streaming: Recorded Service SegmentedRecordedService playback (2 segments, 1x playback) 135 135

HN Streaming: Live Streaming Streaming requests for ChannelContentItems also are subject to additional processing steps: The “Channel Locator” is subject to resolution via the ServiceResolutionHandler to establish the “Tuning locator” NetworkInterface has to be tuned to the program specified in the Channel Locator (if the program is not already tuned) In attempting to acquire a tuner for streaming, the ResourceContentionHandler may be invoked – with a NetResourceUsage representing the network-initiated tuner acquisition When the DVR extension is present, time-shift buffering also needs to be started (if the program is not already buffering) 136 136

HN Streaming: Live Streaming ChannelContentItem playback 137 137

HN Streaming: VPOP Virtual Primary Output Port: Allows for streaming of whatever content is playing back on the DMS 138 138

HN Streaming: RemoteService playback OCAP RI DMC can playback a MPEG single-program transport stream (modeled on the DMC as a RemoteService) 139 139

HN Platform Functionality Two levels of RI HN Platform Code MPEOS C code GStreamer pipeline C code MPEOS Level – Player Performs HTTP request & response handling Receives content via socket Transfers to HN player pipeline for decoding MPEOS Level – Server Sends content retrieved from platform via socket NOTE: HTTP handling is performed in RI Stack GStreamer HN Pipelines Server pipeline supplies content Player pipeline decodes content 140 140

MPEOS HN Changes Defines the porting interface for HN - mpeos_hn.h Methods & data structures used in platform for both Server & Player roles Summary of recent changes Non-platform specific Server side processing of HTTP requests moved to RI Stack level Clearer separation of methods b/w player & server Enhanced “javadoc” type documentation updates Added DTCP support Need to be sensitive to changes Impacts ALL platforms not just RI PC Platform All changes are posted to forum 141 141

GStreamer HN Server Pipeline Native File representation of Recorded Content GStreamer Trick Play File src Same Element which is used for local playback GStreamer AppSink MPEOS HN Player & Socket Sends HTTP packets out on network Native File Representation Of Non-Recorded Content Gstreamer FileSrc 142

GStreamer HN Client Pipeline MPEOS HN Player & Socket Gstreamer AppSrc Decode Bin HTTP Packets received across nework Same Decode bin used by other pipelines 143

DTCP/IP RI Stack support complete with 1.2.1. DTCP_ profile ID signalling. application/x-dtcp1 mime types. Support for Range.dtcp.com in 1.2.2-B. RI Platform integration complete with 1.2.1. Integrated Intel SIK v4.02. Library presence dynamically detected and loaded at RI start-up. Fallback to No-OP version. MPE configuration options: MPEOS.HN.DTCTPIP.DLL=/f/encrypted/dtcpip.dll MPEOS.HN.DTCPIP.STORAGE=/f/encrypted/keys/ 144 144

DTCP/IP (cont’d) 145 145

DTCP/IP Testing Successful sink/source interoperability. Verified support/interoperability with both facsimile and production keys. Successful streaming to a production DMP device. Currently working through issues with DLNA LPTT. 146 146

Limitations Network interface mapping Playback formats MoCA support Supporting multiple interfaces Bridging/loop mitigation Playback formats No Audio playback support Only standard MPEG transport stream DLNA “ISO” ts formats, not tts (zero or otherwise) No MPEG PS support (adding in July) 147 147

Graphics March 15-16, 2012 148

Setting up Graphics Resolution Platform Window size from display.window.width display.window.height (platform.cfg) TV Screen size from DISP.DEFAULT.CONFIG (mpeenv.ini) Video scaled according to incoming video size and applicable DFC rule 149 149 149

Coherent Configurations DISP.DEFAULT.CONFIG Configuration 640x480 1:1 graphics, 720x480 8:9 video, 640x480 1:1 background 1 640x480 1:1 graphics, 720x480 8:9 video, 720x480 8:9 background 2 960x540 3:4 graphics, 720x480 8:9 video, 640x480 1:1 background 3 960x540 3:4 graphics, 720x480 8:9 video, 720x480 8:9 background 4 640x480 4:3 graphics, 1920x1080 1:1 video, 1920x1080 1:1 background (with I-frame support) 5 960x540 1:1 graphics, 1920x1080 1:1 video, 1920x1080 1:1 background (with I-frame support) 150 150 150

Stack Graphics Overview 151 151 151

MPE Java Classes (partial list) MPEGraphics MPEGraphicsConfiguration MPEGraphicsDevice MPEToolkit MPESurface MPEImage 152 152 152

MPEOS Graphics Object: mpeos_GfxScreen typedef struct mpeos_GfxScreen { int32_t x; int32_t y; int32_t width; int32_t height; int32_t widthbytes; mpe_GfxColorFormat colorFormat; mpe_GfxBitDepth bitdepth; mpeos_GfxSurface *surf; os_GfxScreen osScr; } mpeos_GfxScreen; 153 153 153

MPEOS Graphics Object: mpeos_GfxSurface typedef struct mpeos_GfxSurface { os_Mutex mutex; /**< surface is thread safe */ int32_t width; /**< width of surface in pixels */ int32_t height; /**< height of surface in pixels */ int32_t bpl; /**< bytes per line */ mpe_GfxBitDepth bpp; /**< bit depth (bits per pixel) */ mpe_GfxColorFormat colorFormat; /**< color format */ void* pixel_data; /**< pixel data */ mpe_Bool primary; /**< true if on-screen surface */ mpe_GfxPalette clut; /**< color palette used (if colorFormat == MPE_GFX_CLUT8) */ os_GfxSurface os_data; /**< os-specific surface info */ } mpeos_GfxSurface; 154 154 154

MPEOS Graphics Object: mpeos_GfxContext typedef struct mpeos_GfxContext { mpe_GfxColor color; mpe_GfxFont font; mpe_GfxPoint orig; mpe_GfxRectangle cliprect; mpe_GfxPaintMode paintmode; uint32_t modedata; mpeos_GfxSurface *surf; os_GfxContext os_ctx; } mpeos_GfxContext; 155 155 155

Third-party packages DirectFB -- Used for alpha blending onto graphics buffer FreeType – Font support 156 156 156

Relationship with Platform Graphics Buffer Allocated by Platform Pointer passed to Stack via get_graphics_buffer (display.c) Drawn upon request via draw_graphics_buffer (display.c) Graphics resolution Changed by a call from Stack to Platform: update_configuration (display.c) 157 157 157

Use Case: FillRect() in Paint() MPEToolkit creates MPEGraphics and wraps it in DVBGraphicsImpl2 object, which is passed into paint. FillRect called on DVBGraphicsImpl2 passes to inner MPEGraphics object. Call passes to gfxRectangle in mpeos_draw.c, with native graphics context passed in. gfxRectangle calls FillRectangle on IDirectFBSurface in native graphics context DirectFB paints to graphics buffer, then calls Platform to redraw 158 158 158

RI Graphics UML - 1 159 159 UML Diagrams https://community.cablelabs.com/svn/OCAPRI/trunk/ri/RI_Stack/docs/design/Graphi cs/OCAP_graphics_PhoneME.vsd Free Microsoft Visio 2003 Viewer enableTv Contribution (Russell Greenlee) 9 Structure Diagrams RI Launcher & Display MPE/MPEOS DirectFB Java HAVi AWT 23 Sequence Diagrams RI Emulator Startup MPE/MPEOS Initialization DirectFB Initialization OCAP Main/AWT Toolkit Initialization MPE Graphics Initialization HAVi Initialization AWT Rendering 159 159

RI Graphics UML - 2 160 160

Graphics vs VideoOutputPorts VideoOutputPorts: physical “spigots” on back of OCAP box. Each HScreen has a main VideoOutputPort. VideoOutputPorts AND CoherentConfigs control video plane resolution 161 161

CoherentConfig vs VideoOutputPort Coherent Config and Video Output Config BOTH control video resolution. On RI startup, persisted Video Output Port Config is read and supercedes CoherentConfig When OCAP app change coherent config, Video Output Config is changed When OCAP app changes Video Output Config, CoherentConfig is changed 162 162

Testing March 15-16, 2012 163

Different Tests Play Different Roles Smoke Tests Standards Compliance Integration Demonstrations See: https://community.cablelabs.com/wiki/display/OC ORI/Testing+the+RI 164

Smoke Tests - Overview Philosophy of daily smoke tests Current smoke tests For more information Recommendations for smoke tests 165

Smoke Tests - Philosophy Discover unexpected consequences ASAP Documented procedures for how and when to run the tests Published reports from running guide tests Routine, periodic execution Executed by all engineers, verified by the build. Maximum coverage for short period of time Focus on troublesome areas of the code Manual execution < 30 minutes effort for MSO guides Manual execution < a few minutes effort for CTP tests 166

Smoke Tests – Current Procedures Manual tests, not fully automated For every code change CTP test suite TuneTest Other tests, as appropriate Daily, at least once Aspen Guide, dual tuner configuration ODN, will begin soon 167

Smoke Tests - Guides 168

Smoke Tests – For more information Wiki page: https://devzone.cablelabs.com/web/oc/9/- /wiki/OCAP%20RI/Smoke+Testing 169

Smoke Tests - Recommendations Suggestions for additional tests? Recommendations for alternative procedures? 170

Smoke Tests - Summary Smoke tests are manual, quickly-executed, formal tests to gauge the health of a build All engineers on the RI project are running smoke tests Please contribute on the forum your thoughts about smoke tests. 171

CTP Testing against RI Full suite of unattended CTP tests (approx 5500) are run against tip of RI trunk every week. CTP test are run using a “real ATE” – Automated Test Environment: Dedicated machine that executes ATE software, generates streams in real time and sends them via RF channel to RI running on Windows and Linux. Priority given to testing full configuration of RI. The other three configuration are tested monthly. 172 172

CTP Testing against RI Failure Results: RI QA does initial analysis. File bugs in RI JIRA db at CableLabs which is tied to CableLabs CTP bug db. If the failure is determined to be due to a test bug, an issue is entered into the CL CTP bugs database and the 2 issues are linked. Weekly results and corresponding Jira issues are captured in a spreadsheet for easy comparison of RI status from week to week. 173 173

Attended CTP Test Automation Existing Scenario Problems faced Tool used for Analysis/Report Generation. 174

Attended CTP - Existing Scenarios 2 Groups of Attended Test Interactive Test Visual Inspection Test Automation is done only for the second group. This is to increase the efficiency of the tester. Reduce the wait time spent for running tests. Minimal changes were made to ATE emulator Script changes to avoid pop up of a question. Take a screenshot of RI screen. 175

Tools Used for Test Run/ Test Analysis Screenshot tool: (Test Run) Tool written in Java to enable screenshot of RI screen. Spreadsheet creation tool: After the Test run, to collect all the images and questions and organize them to a spreadsheet. Test Result Update tool(TestResultMux): To view the images and questions through an interface and create a test result. 176

Advantages of using the tool Usability : More user friendly so that it takes less time for a new person using this tool. Efficiency : Reduces the time taken for test result analysis and creating a test report. Robust : Avoids human errors while executing/updating test analysis reports by showing all the mandatory parameters on the screen. 177

Integration Tests $OCAPROOT/apps/qa/org/cablelabs/xlets Each test application should have a readme.txt in its directory. Ongoing clean-up and rework Latest status is found in $OCAPROOT/apps/qa/docs/IntegrationTestInfo.doc New test applications will be added going forward for integration testing of new features. 178 178

Integration Test Automation AutoXlet Automated xlet examples ($OCAPROOT/apps/qa/org/cablelabs/xlet/): TuneTest PermissionTest FileAccessPermissionTest PropertiesTest DvrTest 179 179

Home Networking Test Automation Home Networking tests are challenging Video comparison Network issues Rx Project RiScriptlet RiExerciser Pragmatic automation of HN Integration tests Overnight smoke tests 180

MSO Guide Testing - Approach We want to run the same tests against the RI that are run against any STB RI/QA is always looking for more tests “Canary” boxes All non-blocked suites are run before a release, or as needed. TestLink manages the test procedures, execution and results. 181

MSO Guide Testing - Suites Tuning, live streaming, VPOP DVR and Trick Mode viewing Locks and PINS Various ways to find content – Guides and HN content searches All buttons on the simulator remote control All “feature menus” 182

MSO Guide Testing – Tests NOT Run Environment at Cablelabs determines tests to be run. Some tests not run due to: Specialized headend support at CableLabs (Upsell, CallerID, etc.) Headend triggered features VOD and CA support Platform features – audio, analog channels What tests do you think should be run, and how frequently? 183

TestLink – Manage MSO Guide Tests Web Based Test Management http://www.teamst.org/ GNU GPL, version 2 Architecture – Apache web server, MySQL, PHP http://www.apachefriends.org/en/xampp.html 184

TestLink – Test Cases 185

TestLink - Reports 186

MSO Guide Testing - TestLink We want to run the same tests against the RI that are run against any STB RI/QA is always looking for more tests RI/QA is always looking for more canary boxes Not all tests can be run here – however, if YOU can run them at your site….. TestLink is our Web Based Test Management tool 187

Unit Tests – Design and Expectation Historical JUnit Tests $OCAPROOT/java/test Out of date when received by RI Project Contain useful coding examples Design of new tests for fixes and contributions Fail against old code, pass on new code Expectation of new tests Coverage for all lines of code for contributions Your experience with JUnit tests? 188 188

Rx - Ri eXerciser March 15-16, 2012 189

Goals and Benefits of RX Rx Project Overview: The RxProject is an effort to take the current DvrExerciser OCAP Xlet and refactor it into a frame work which provides the following: A library of functionality, called OcapAppDriver, that aggregates OCAP API calls into functional building blocks provided by RI Stack & PC Platform.  A scripting interface, RiScriptlet Xlet, that allows automation of testing of these functional building blocks. A "Guide" Type OCAP Xlet, which is referred to as the RiExerciser, that supports functional/application level testing and development through a GUI. 190

Another testing framework… Really? This framework allows for automation without much additional effort.   RxProject will be part of the open source so outside contributors will be able to write bean shell scripts to illustrate issues and also demonstrate fixes Currently have a multiple of Xlets in the QA directory, and there is much redundant code. The Rx framework allows Xlets to utilize common utilities already found in OcapAppDriver or add to the common functionality For HN, we need a framework where tests can be run to verify robustness in a network environment with many devices. HN CTP tests assume an isolated network. 191

OcapAppDriver A library of commonly used functions (e.g. tune, record, etc) that can be used by xlets or scripts.  The idea of this library is to be easier to use than the raw OCAP calls so that a script-writer in particular does not need detailed OCAP API knowledge to write a script.  OcapAppDriver lib calls need to clearly assert whether they are synchronous or asynchronous.  In general, synchronous calls will be favored over asynchronous. OcapAppDriver lib calls will pass/return only simple Java primitives (String, int, etc) in keeping with the philosophy that the API should require minimal OCAP API knowledge to use. 192

Asynchronous vs Synchronous example As previously mentioned, OCAP API calls will in general be synchronous. However, there are asynchronous OCAP API calls such as tuning and recording For any asynchronous API calls, a synchronous counterpart will exist. For example, waitForTuningState() is a synchronous counterpart to the asynchronous serviceSelectByIndex() method. In general, synchronous counterparts should include “waitFor” in the method name 193

Rx Architecture 194

RiScriptlet RiScritplet is an xlet that executes scripts Scripts Written in "Java" and executed by BeanShell interpreter Specified in hostapp.properties or via telnet Can have subroutines for code org Results file written when script is complete 195

Example Script 196 // Delete All Recordings rxLog.info ("Deleting all recordings..."); rxReturn = true; if(!rxDrvr.deleteAllRecordings()) { rxReturn = false; rxReturnString = "deleteAllRecordings failed" } rxLog.info ("Done deleting all recordings..."); 196

Synchronizing Scripts Can run multiple scripts in parallel Can sync scripts via sync points on one or more RI instances One RiScriptlet acts as sync server to coordinate sync points Syncing is TCP-based 197

Synchronization Architecture Script #1 RiScriptlet (Sync Master) Script #2 TCP (register, sync, unregister)   198

Sync Example Scripts 199 // Script #0 rxSyncClient.register ((byte)0 /* clientId */, (byte)2 /* expected num clients */, 3000 /* timeout */); Thread.currentThread().sleep(5000); rxSyncClient.sync ((byte)0 /* syncId */, 10000 /* timeout */); rxSyncClient.unregister(3000 /* timeout */); // Script #1 rxSyncClient.register ((byte)1 /* clientId */, (byte)2 /* expected num clients */, 3000 /* timeout */); Thread.currentThread().sleep(1000); 199

Home Networking Integration Testing Benefits Designing tests Building and running HN test Handy tools Inventory of current tests Continuous integration scripts Where to find more information 200

Benefits - HN Integration Testing Testing of HN features without needing a headend or guide RI can be server – third party devices RI can be a player for your STB port Advantages of PC tools for network debugging API code examples for new ECs 201

Design – HN Integration Tests Use Cases from specifications Narrative fro the beginning of specifications Common sense application of a feature Only happy path tests Limited scope - not exhaustive combinations Mind Maps 202

Manual HN Integration Tests 203

Inventory – HN Integration Tests Streaming, recording and trick modes VPOP Hidden content Resource contention handling New test suites for new ECs 204

Tools – HN Integration tests RiExerciser Intel’s UPnP Device Spy Third party devices – PS3, Samsung SmartTv 205

HN – Continuous Integration Regression – all tests expected to pass Results summarized and archived Optimized for our HN test lab …/OCAPRI/trunk/ri/hnTestScripts/buildRx.xml 206

More information – HN Tests How we test the RI https://community.cablelabs.com/wiki/display/OC ORI/Testing+the+RI Location of integration spreadsheets OCAPRI/trunk/ri/RI_Stack/apps/qa/hn/integration 207

Contributions – HN Integration Tests Designs for integration tests Additions to OcapAppDriver Contribute Scriptlets 208

Summary – HN Integration Tests Demonstrations of functionality – end-to-end Do not need: Guides Headends Learn about new APIs Designed from specs and common sense Manual test procedures in spreadsheets Automated with scriptlets 209

Issue Reporting and Handling March 15-16, 2012 210

Issues – information to report Basic journalism In what environment (RI Rev level, OS, etc.)? What did you do (sample code is best)? What happened (observations, logs, screen shots)? Priority – what should it be? Lastly, why do you believe this is an issue? Include sample code to demonstrate the behavior 211 211

http://java.net/jira/browse/OCAP_RI

http://java.net/jira/browse/OCAP_RI

http://java.net/jira/browse/OCAP_RI

Issues – Life Cycle Filed JIRA counterpart Comments, comments, comments Resolution Closed with the filer’s assent 215 215

Issues – Status Unresolved Resolved NEW – Initial state STARTED – Assigned and work begun REOPENED – Once resolved, but more work needed Resolved FIXED - A fix is checked in INVALID – The problem described is not an issue WONTFIX – The issue will never be fixed DUPLICATE – Duplicates an existing issues WORKSFORME – Attempts to reproduce were futile 216 216

Issues – Guidelines for Priority Blocker- Most important. Catastrophic consequences Potentially effects many applications No workaround by changes to the app, etc. Critical More limited scope or consequences Major Workaround available, limited scope Minor and Trivial – Least important. 217 217

Issues - Process Every release has a published cutoff date for considering issues to be resolved for that release. Three weeks prior to release. Higher priority, well documented issues are addressed sooner All correspondence should be through comments in the issue 218 218

Issues - Summary All correspondence and information about issues must be archived in the Issue Tracker system Report the required information for the most efficient response Recommendations and observations? 219 219

Issues - http://xkcd.com/627 220 220

Stabilization/Robustness Project March 15-16, 2012 221

Stabilization (aka Brownian Motion) Rationale Some tests are not predictably repeatable – sometimes they pass, sometimes they fail This is true for a wide range of test approaches – CTP, TDK, etc. Approach Need to bound the population of Brownian tests Approximately 114 of 5600+ CTP tests (all – not just HN) Approximately 5 of 400+ TDK tests (all “good” tests) HN is a focus area since there are many potential factors that can affect test results (eg, # of interfaces, fluctuating network conditions) Resolution Determine if the cause of the uncertainty is the network, the RI or the test itself Eliminate the test, fix the test, or fix the RI to accommodate a fluctuating network environment Will rely on RiScriplet for test automation 222 222

Robustness Rationale We have done very little stress testing on the RI We would like to begin addressing robustness issues now Approach Identify potential areas to research Focus on a few of those initially 223 223

Possible Approaches - 1 224 224 Brownian Motion CTP investigation FindBugs (a static analysis tool) Drill down into code/walk throughs Identify functional areas Socket timeouts Run in context Monitor/locks Lack of synchronization in CyberGarage Design review of Cybergarage to gain better understanding Address individual OCORIs which are categorized Stop new feature development and focus on fixing issues Setup “Typical/Real world” Home Network testing environment Perform STB type testing Gather stability metrics Expand Rx Framework testing 224 224

Possible Approaches - 2 Run CTP tests (either all or a subset) without consecutive reboots More Guide Type testing by developers Identify scenarios with guides to be run Start fixing known issues rather than looking for more issues Perform deployment type testing Run CTP tests on a busy network Investigate isolation of jvm/platform /stack to do targeted testing Run surfer tests Prioritize guide issues Investigate TDK Brownian motion Form a focus team to address discovery issue in simplified environment (local loopback) Re-architect problem areas 225 225

Miscellaneous Topics March 15-16, 2012 226

MPEOS Review Low priority task to review and correct MPEOS header file comments, remove unused items, etc Comments in Doxygen format Will expose this info as HTML pages Will not update MPEOS Porting Guide Initial file to be refactored was mpeos_dvr.h Was integrated into the 1.2.1 Release Next up are mpeos_media.h and mpeos_hn.h Targeting 1.2.2 Rel-B 227 227

RI Conditional Access RI provides support for Conditional Access. Portable implementation Compliant with CCIF 2.0 CA Support (Section 9.7) MPE and Java-level decrypt session constructs RI MPE CA management can be disabled via ini configuration (When disabled the CA management is done in the MPEOS porting layer)

MPE POD Manager Responsibilities Methods to initiate/terminate decrypt sessions CAS session opened when POD_READY indication is received All CA APDUs processed by the MPE POD Manager CA_PMT constructed for the selected Service when MPE decrypt session is started Explicit LTSID assignment. MPE POD Manager assigns a randomly generated LTSID when initiating decrypt session LTSID is included in the CA session event and passed around to all layers to maintain tuner to LTSID association All CA sessions on a tuner share the same LTSID CA sessions between clients are shared when possible (e.g. decode and buffering of the same program on a tuner)

MPE POD Manager Responsibilities CableCARD decrypt resource capabilities queried at start-up POD Manager internally keeps track of resources being used High priority decrypt requests will pre-empt lower priority requests (e.g. ServiceContext select is higher priority than DSMCC ServiceDomain attach) CP session is opened when CA session is successfully started Terminate CP session when CA session is stopped CCI support (can be signaled via java/MPE decrypt session)

RI Conditional Access CA session supported for JMF BroadcastSession (initiate decrypt session prior to calling mpe_mediaDecode()) DVR (initiate decrypt session prior to mpe_dvrTsbBufferingStart()) Object Carousel (initiate decrypt session before attaching a DSMCC service domain) Section Filter (initiate decrypt session before setting native section filter)

RI Conditional Access