Presentation is loading. Please wait.

Presentation is loading. Please wait.

Welcome. OpenInterface Project FP6-35182 1 sept 2006 - 33 months Today = M9 Coordinator: Laurence Nigay, University of Grenoble 1 - Joseph Fourier Glasgow,

Similar presentations


Presentation on theme: "Welcome. OpenInterface Project FP6-35182 1 sept 2006 - 33 months Today = M9 Coordinator: Laurence Nigay, University of Grenoble 1 - Joseph Fourier Glasgow,"— Presentation transcript:

1 Welcome

2 OpenInterface Project FP6-35182 1 sept 2006 - 33 months Today = M9 Coordinator: Laurence Nigay, University of Grenoble 1 - Joseph Fourier Glasgow, May 24, 2007

3 3 Outlines Overview OI Platform Component based platform Runtime Kernel Repository of modalities Graphical editor

4 Overview

5 5 Consortium

6 6 Complementary expertise Signal Processing UCL MULT Multimodal Interaction ALL Hardware UGLA SWIN Software Engineering UJF UGLA FIT ARC PCLI TXT Application domain: Large information space FT UJF UGLA Application domain: Game ARC FT UJF UGLA Ergonomics Usability FIT FT

7 7 OpenInterface: objectives Scientific understanding of multimodality Testbeds and Validators Open source platform Extension of standards

8 8 Multiple modalities Interaction Continuity The 2 action lines Two complementary goals: Extend the sensory-motor capabilities of mobile devices and environments

9 9 Modalities + The 2 action lines Two complementary goals: Extend the sensory-motor capabilities of mobile devices and environments

10 10 Standards Validation R&D Ideas Concept Pilots Multimodal Interaction within a pervasive environment Multimodal Interaction on a mobile device Market Concept OpenInterface platform for multimodality Existing standards for multimodality Why 2 action lines?

11 11 Standards Validation R&D Ideas Concept Pilots Multimodal Interaction within a pervasive environment Multimodal Interaction on a mobile device Market Concept VALIDATORS Why 2 action lines? TESTBEDS

12 12 Testbed Large Information Space

13 13 Testbed Game Increasing the intensity of the vibration depending on how damaged the helicopter is Speech command for the speed Accelerometer or speech

14 14 Testbeds and validators Software prototype Requirements Ergonomic evaluation Design Software prototype Design Software prototype Design Software prototype Ergonomic evaluation TESTBEDS OpenInterface Platform Unit test Component test Acceptance test Code Final usable product VALIDATORS

15 OI Platform

16 16 OI Platform Open source platform To make it easier to develop effective interactive systems that exploit multimodal interaction techniques in modern application settings (mobile, ubiquitous, context sensitive)

17 17 OI Platform Open source platform for developing multimodal interfaces that enables quick replication, that enables iterative user-centred design, that contains a rich and extensible set of modalities, that enables a focus on innovation (new modalities or forms of multimodality), that supports dynamic selection and combination of modalities to fit the ongoing context of use.

18 18 OI Platform - Repository of modalities (components) - Repository of Fusion/Fission Mechanisms (components) - Runtime Kernel Tool for the application designer/developer Using/Assembling the components Tool for the provider of components Adding new components

19 19 OI platform Who are the users? Three types of users 1. Programmers Providing new components (populating the platform) New device New modality New fusion/fission mechanism 2. Multimodal Application Designers Assembling components to define multimodal interaction 3. End-users of multimodal applications Interacting with the multimodal application Adaptation at run-time Run- time Design time

20 OI platform Runtime kernel

21 21 Outlines Kernel Overview OI Components CIDL, PDCL Current Features Next Features

22 22 Overview Useful high level programming language support: C/C++, Java, Matlab, Python Plug n Play behavior. Lightweight skeleton platform.

23 23 Components Platform does not specify/define a Component Model (!!) Platform sees the external software as components components are assumed to behave like objects. Software/Component delivered as reusable independent unit with exported/imported I/O interfaces. Platform Kernel aims for the interaction of heterogeneous components

24 24 Component: OI point of view Facets can be seen as services provided by a component A Component is a set of facets This is how any external software is seen by the Kernel

25 25 Reusable Independent Unit Software With API With installation/configuration doc Without explicit dependencies with other components Tested and validated Much likely a library (!!) Impossible to test new modalities with an exe, for instance.

26 26 Exported/Imported IO Components only declare communication interface Enforce the requirement for independence Export IO (functions, etc.) when providing a functionality  Display an image Import IO for requesting a functionality  An event handler

27 27 OpenInterface CIDL Designed to describe Independent and Reusable component. (Intentionally) Not as expressive as WSDL (etc.) Restricted on purpose to cover common attributes of various programming languages Describes component interface using a programming language independent description (XML). Not intended to be written by human Generation tools are not yet available though (being developed)

28 28 I/O Interface Functions provided by a component facet 3 types of functions Sink  With no return argument Source  With return argument  Callback  With no return argument, for importing external functions

29 29 I/O interface: types Only supported languages common types Common primitives types Bool, integer(8,16,32,64bit), float, double, String Array of primitives types Complex types Objects

30 30 Runtime, Pipeline Pipeline description language to setup a running application initial connections.

31 31 Pipeline configuration Components Remote or local Connections Simple direct link Multicast (Publisher/Subscriber) (Temporal) Fusion Execution Pin initialization values Threaded execution

32 32 Current Features Supported Platform Linux and Windows Supported Languages (with minor restrictions) Java C/C++ Matlab Description languages CIDL, interfaces description PDCL, static pipeline description Temporal Fusion components Simple components distribution Specification on how to write a language plug-in Specification on how to write custom connectors

33 33 Next Features Installation Packages Windows installer, debian package Pipeline Kernel API component for controlling application dynamic connections Description languages tools Semi-Automatic CIDL generator Graphical tool for pipeline composition Supported Platform Other OS Supported Languages.NET framework Connectors More connectors plugins

34 OI platform Repository of modalities

35 35 Summary Device Components Filter/Translator Components Debugger Components (Development) Connector Components Future Work

36 Device components

37 37 Device Components Component NameSource ProviderIntegratorOSState Win32MouseDriverEmulUCL WinStable XWindowMouseDriverEmulUCL LinuxStable Win32KeyboardDriverEmulUCL WinStable IEEEFrameGrabberUCL LinuxStable ARToolkitUCL Win/linuxStable SpeechRecUCL Win/LinuxStable ShakeUGLAUJFWinBeta SpyTagClientMULTUCLWin/LinuxBeta MultitelSpeechRecognitionMULTUCLWinBeta FingerTrackerUJF WinBeta

38 Filter/Debugger/Connector Components

39 39 Filter/Translator Components Component NameSource ProviderIntegratorOSState Type&ValueFilterUJF WinStable EventsGeneratorUJF WinStable Example: Int 1 string go Example: Int 1 string zoom 10 string in

40 40 Debugger Components Component NameSource ProviderIntegratorOSState DataViewerUJF WinBeta OsciloscopeUGLA WinBeta Logger (data in a file)UGLA Win/LinuxBeta

41 41 Connector Components MulticastUCL OIStable Complementarity FusionUCL OIStable RedundanceEquivalence FusionUCL OIBeta Component NameSource ProviderIntegratorOSState

42 42 Components in Progress (UGLA) Abstract Devices Gesture Language Technique Spatial interaction device Pointing / panning / zooming Lots of variants Earcon device Tacton device Wrappers Phidget OI wrapper ARToolkit OI wrapper

43 43 Futur work (UGLA) Generic GPS device Smell device Physiological sensors Context-sensing device Audio recognisers ambient noise level detector Vibration recogniser Light level recogniser Proximity sensor [...]

44 OI platform Graphical Editor

45 45 Content Introduction Demo Roadmap Conclusion

46 46 Introduction What? : Graphically create an assembly of (OI) components At first: Technical (PDCL) Description Objective: OI Interface Not only a graphical editor but an IDE Actually: static assembly of components Next version: dynamic assembly of components Why?: Fast prototyping Runtime Testing and Monitoring

47 47 Demo Fast prototyping Create assembly Run the assembly Change Multimodal Interaction

48 48 Roadmap Introspection Runtime connection Monitoring of the components execution Usage data capture Link with Conceptual Model Multimodal interaction Multiple OS support

49 Thank you!


Download ppt "Welcome. OpenInterface Project FP6-35182 1 sept 2006 - 33 months Today = M9 Coordinator: Laurence Nigay, University of Grenoble 1 - Joseph Fourier Glasgow,"

Similar presentations


Ads by Google