© 2006 by IBM and Wind River Systems; made available under the EPL v1.0 | 22-March-2006 Martin Oberhuber, Wind River Systems David Dykstal, IBM WebSphere DSDP/TM Project Using and Extending the DSDP Target Management Framework

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v1.0 2 So what is Target Management all about? Remote Computer Systems…  Targets (Locally connected, shared, fielded)  Hosts (Grids, farms, nodes) … and developing software on them  Build, connect, get status  Download, run, debug, test  Upload An Eclipse Project under DSDP since June 2005.

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v1.0 3 The Ugly Truth: Reality Now  Type in IP addresses and config options  Again and again in each distinct tool  Manually configure JTAG scan chains  Hassle with Register settings and reset options  ftp, telnet, rsh, ssh  Work in two separate environments  With insufficient tooling on the remote system  No tool integrations from different vendors

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v1.0 4 The Vision You’ve got that new board from Yoyodyne ACME Plug it in… … and it just works like your local desktop computer

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v1.0 5 The Mission To become the standard tool environment for interacting with remote systems for software development

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v1.0 6 How do we get there?  Create a Framework for interoperability  Focus on tools for remote (potentially embedded) systems  Data models: standardized hardware descriptions, e.g. multi-core configurations, JTAG scan chain setup, memory layout  Configure and Manage: e.g. target groups, board labs, access restrictions, connection configurations shared in a team  Connection plug-ins: TCP/IP, Tunneling, Serial, JTAG connections – pluggable with fine granularity  Pluggable Actions and Services (Download, Launch, Query status, Auto- detect services, Reboot, Flash utility, Remote console, …)   Overview, Scope and Use-Cases Document

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v1.0 7 And who are we, by the way?  CDT and Derivatives: Launches for remote systems  Want to share common target properties between Launches  Bugzilla 65471: Remote System Framework (RSF) code and presentation from EclipseCon 2005  Commercial Debuggers migrating to Eclipse  Want a Target Manager again  Hardware Vendors  Want minimal open plug-ins for their hardware  Eclipse-based commercial Target Managers  Want open interfaces for interoperability  And a Community to grow Logos and trademarks are copyright by their respective owners.

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v1.0 8 Remote System Explorer  A component that has been shipping for several years in various IBM products  Used for accessing IBM server systems  RSE “8.0” submitted to Eclipse in January 2006  Will become RSE 1.0 after EMO approval  Will form the base of the first DSDP/TM Release in September

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v1.0 9 Remote Systems Perspective

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v Pluggable Target System Types

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v Target System Instances

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v Subsystems manage resources of a particular kind

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v Filters select resources dynamically

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v Resources exposed by the filter

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v Properties of the selected resource

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v Table provides alternative view

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v File Resources can be directly edited

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v Remote File Systems can be searched

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v What’s in the box?  Resource Access  Remote files  “dstore” based or FTP  Remote search & archive support  Processes  “dstore” only  Commands & results  “dstore” based or Telnet  Access to Linux, Unix, and Local systems  All within a consistent, extendible framework

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v1.0 20

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v Structure - dstore

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v Extending RSE  Add New System Types  IBM has extended for their systems  Connection properties, wizards, connection schemes, character set conversions, …  Add New Subsystems  For accessing new types of remote resource (loads, memory regions, …)  Add New Actions and Properties for existing remote objects  By system type and/or file type

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v A New System Type – What’s Involved  Extension points  System Type  Subsystem  Subsystem implementations are assigned to System Types  These can be reused across System Types  System Types are just collections of services supplied by subsystems Example: Using only some of the existing services for a target

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v <type description="An XYZ System" icon="icons/system-xyz.gif" iconlive="icons/system-xyz-live.gif" id="org.eclipse.rse.examples.xyz" name="XYZ"/>

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v Causes a new system type to appear Which, in turn, allows it to be instantiated later

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v <configuration systemtypes="XYZ" name="Files (FTP)" description="Manager for XYZ file system resources." icon="icons/system-files.gif" iconlive="icons/system-files-live.gif" category="files" class="org.eclipse.rse.subsystems.files.ftp.FTPFileSubSystemConfiguration" vendor="XYZ MegaCorp" id="ftp.files"> This class exists, just reuse

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v Causes this to appear for XYZ systems

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v This Linux target system shows up with 3 subsystems – using both dstore and FTP Our new XYZ target system shows up with 1 subsystem – using FTP

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v Subsystems – access to resources  Have a new kind of resource? Add a subsystem!  Extension Point – The subsystem configuration  Subsystems access resources through a service layer  Not required, but a suggested implementation  Useful if you have resources accessible by multiple protocols  Lifecycle  Connecting  Applying filters to resources  Definition persistence and sharing  Subsystem Properties, Filter Definitions, Saved Commands  Done through the SystemRegistry  Pluggable Persistence Managers

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v The Service “Layer”  Do you have an alternative way to access a resource (like a file) that already has a subsystem?  Just plug in a new service (well, sort of)  No Extension Point for services, instead…  Define a new subsystem configuration to act as a factory for the standard subsystem but connects that subsystem to the new service.  The new subsystem configuration contributed in the usual way  Standard subsystem defines the subsystem/service API  File, Shell, and Process subsystems defined in this manner

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v Current Packaging

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v A new implementation – FTP – What did we add?  Plugins  a service  a subsystem  Extension Points  a subsystemConfiguration  Resources and Adapters  A File Resource for FTP Files  An Adapter for this resource  Protocol  Wraps the FTP client library

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v <configuration systemtypes="Linux;Unix;AIX" name="%Files" description="%FilesDescription" iconlive="icons/full/obj16/systemfileslive_obj.gif" icon="icons/full/obj16/systemfiles_obj.gif" category="files" class="org.eclipse.rse.subsystems.files.ftp.FTPFileSubSystemConfiguration" vendor="IBM Corporation" id="ftp.files"> We reused this before, this time we wrote it

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v The protocol wrapper

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v These are the methods that FTPService defines. It implements the IFileService API. This is the largest class of the implementation and wraps the FTP library client to deliver FTPHostFile objects.

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v The new “factory” for FTP Subsystems

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v Connects the FileServiceSubSystem to a particular instance of an FTPService

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v Adapts FTPHostFile to have appropriate UI properties for the RSE Views

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v Speaking of Adapters…You can reuse RSE Adapters

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v Directly…

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v Or Indirectly

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v RSE for the Device world – what’s different?  Different Resource Types, e.g. Kernel Modules, Semaphores, Message Queues  Register a new Subsystem!  Different communication protocols, e.g. minimal debug agents, hardware debuggers, standard protocols like ssh  Register a new Service for existing Subsystems!  Slow connections  RSE model objects can use Jobs for delayed updates. Each node has control over its update policy! The RSE Model is extremely flexible thanks to Adapters. RSE APIs will be evolved as we go along.

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v So what’s still missing?  Have a Framework for interoperability  Focus on tools for remote (potentially embedded) systems  Data models: basic PropertySetContainers now, need more structured models, standardized hardware descriptions  Configure and Manage: connection configurations shared in a team, target groups, board labs, access restrictions  Connection plug-ins: TCP/IP, Tunneling, Serial, JTAG connections – pluggable with fine granularity  Pluggable Actions and Services (Download, Query status, Launch, Auto-detect services, Reboot, Flash utility, Remote console, …)

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v Next Steps  Add an RSE Service layer for the ssh protocol  Files Subsystem and Command subsystem will work immediately  Eclipse Committers can access dev.eclipse.org directly from within Eclipse (File Transfer, Remote Shell)  Provides a nice example for how to add more protocol support  Add a CDT Launch Integration  Polish the APIs

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v CDT Launch Integration  Will be provided as an RSE Example  Sample will use gdbserver; vendors will need to adapt to their specific debugger communication properties

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v CDT Launch Integration - API  Fill the Combo:  IHost[] hosts = SystemPlugin.getTheSystemRegistry().getHosts();  Filter by System type or connected status:  hosts[i].getSystemType();  hosts[i].isOffline();  Get or Set Properties:  Every RSE Model Object is a PropertySetContainer  hosts[i].getPropertySet(“gdbserver”).getProperty(“port”);  Create a new connection (lauches the Wizard)  new SystemNewConnectionAction(shell, false, myCombo).run()

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v CDT Launch Integration – Attach Action

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v Future: Component-Based Launching (CBL)  A Launch is really a sequence of Actions  Connect, Download, Reset, Run Script  The “Build before Launch” can also be seen as an Action  Allow to contribute actions and compose a Launching “script”  Launch “Scripts” can operate on one or multiple targets  Pre-configured Launches with an UI as we know  Flexible Launches configurable by the user

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v An FAQ: Where is the border between Target Management and Debugging?  TM provides a framework and “broker” for services to be registered.  Especially on Hardware Debug Connections, a debugger may register services as TM extension (TM on top of debugger)  Or, TM may provide communication channels for debuggers to use (debugger on top of TM; inter-debugger communications)  TM shows the entire target (e.g. all processes), debugger focuses on items of interest  TM does everything before a debug context is available:  Select proper target, check resource availability, reset, download, launch agent, …

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v Flexible Target Connection Adaptors (Connectors) Link to Processor 1 / Core X using connector channel 1 Link to Processor 1 / Core Y using connector channel 2 Link to Processor 2 using connector channel 3 Supply an interface to communicate with the target system Users are the debug model an utility application like FLASH programmer tool to view OS events … Target System Tool A Tool B Connector plug-in Eclipse IDE Connection Device

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v  Already started  Component-Based Launching  Standardized Hardware Descriptions - SPIRITSPIRIT  Flexible Connector Chains (Target Connection Adapters)  Middle-term goals  Associations between Connection – Project – Build  Auto-detect protocols and services on remote systems  Inter-Debugger Communications  Connection Groups  Shared Board Labs and pluggable access control For Details, see TM Initiatives Logos and trademarks are copyright by their respective owners.

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v Target Management Base Architecture Data Models Connectivity Services Remote File system Predefined Elements Remote Process List Actions (Download, Reset, Debug, Flash, …) FiltersViews Target Connection Adaptors (aka Connectors) PersistencyProfilesTeam Sharing Property PagesWizardsAutodetect Remote Shell Launch Sequencer (CBL) Connection Configuration (Plumbing) Predefined connection types (connector chains) Target DefinitionsConnection GroupsAccess Control Reusable Widgets Legend: Existing components in RSE, Design TBD, Analysis TBD, Extension Point, Fixed implementation

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v TM and other Eclipse Projects  Eclipse Communication Framework (ECF)  Investigating for plugging in new protocols  Discovery / Autodetect  We will provide an RSE Example for ECF-based file sharing  Projects that do some sort of connection lists  Test and Performance Tools Platform (TPTP)  Web Tools Platform (WTP)  Platform Team Extras (FTP, WebDAV)  …

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v Summary: Plans and Timings First Release (September 2006, on Eclipse 3.2)  RSE – Release 1.0 (Milestones available earlier)  CDT Remote Launch Integration – Release 1.0  Target Definition & Selection, “Remote Debug” Launch configuration Milestones  Standards based target connections (FTP, Telnet, ssh, ECF)  Component-Based Launching (CBL)  “Download” and “Run” Launch actions based on above services  Open Connectors for HW Debuggers (Preconfigured Connection Types)  Connection Model for HW Debugging (complex connector setup) Future  Connector plumbing algorithm  Framework for autodetect  Adapters for Target access control (shared board labs)  SSH tunneling connector

Using and Extending the DSDP Target Management Framework | © 2006 by IBM and Wind River Systems all content except logos and trademarks is made available under the EPL v Getting Involved  RSE is very flexible, with good basic abstractions  An option for your Project?  RSE is a well established product  Still, a lot of community feedback will be needed as we are freezing public APIs (Milestone 3: June 30 – API freeze)  Join the dsdp-tm-dev mailing list -  Q & A – Join the DSDP BOF session