Download presentation
Presentation is loading. Please wait.
Published byEugenia Barber Modified over 9 years ago
1
Building Great Devices Using Windows CE.NET Multimedia
2
Agenda Vision and Goal Vision and Goal Windows CE Multimedia Windows CE Multimedia Talisker Multimedia Features Talisker Multimedia Features WM Audio/Video streaming and codecs WM Audio/Video streaming and codecs Direct3D Direct3D DVD-Video DVD-Video Unified Audio Model Unified Audio Model
3
Vision “Enabling a connected world through a web of Windows Powered devices” “Enabling a connected world through a web of Windows Powered devices” Multimedia Multimedia It’s one reason why end users want to have a web of connected devices… …for entertainment, communication, advertising, training, industrial control, medical, etc. …
4
Goal Supply Windows CE devices with best of breed multimedia capabilities Supply Windows CE devices with best of breed multimedia capabilities Scalable, fully featured, small footprint (customizable), multi-CPU support Simple, one-stop shopping for advanced multimedia on all devices Easy for OEMs to use Standard Windows graphics, audio, video, multimedia APIs, and WM codecs Simple DirectShow playback Wireless A/V streaming for mobile devices VR, game support
5
Challenges Provide the latest and greatest WM, DX, and multimedia features Provide the latest and greatest WM, DX, and multimedia features Support multiple CPUs/ISAs Maximum configuration flexibility Minimum code footprint Minimum system resource requirements Drive the future Drive the future Hardware acceleration, SoC DRM, Security Mobile QoS
6
Agenda Vision and Goal Vision and Goal Windows CE Multimedia Windows CE Multimedia WM Audio/Video streaming and codecs WM Audio/Video streaming and codecs Direct3D Direct3D DVD-Video DVD-Video Unified Audio Model Unified Audio Model
7
Windows CE Multimedia Technologies Technologies Audio 2D and 3D Graphics Video Codecs Streaming DVD Digital rights management and security Communication Quality of Service Products Products MSTV client MSN Web Companion Car.Net WBT Digital music systems Multimedia internet appliances Portable audio/video players
8
Technologies DirectDraw DirectDraw Drawing surfaces, blting, flipping, overlays, alpha blend, video ports Direct3D Direct3D Real-time 3D polygon rendering Wave/DirectSound Wave/DirectSound Buffers, mixing, sample conversion, capture DirectMusic DirectMusic MIDI, composition DirectShow DirectShow Media playback DVD-Video DVD-Video DVD navigation and playback DVD player app Windows Media Windows Media Audio/video/speech screen codecs Streaming WM Player OCX WM Player app
9
Multimedia Architecture Graphics, Audio, DVD Drivers DirectShow Filter Graph WM Player ActiveX Control Application DirectDraw/GDI DirectSound/WaveDev Hardware Sample DX Drivers Content/HTML Page Comm Drivers IE browser Filters Codecs Internet or local file Filter Graph Manager Control Interfaces Windows Media Components DVD D3D DirectX Components COM Interfaces
10
File sys Filter Graph Manager Transformfilters RefClock Rendererfilters internet Sourcefilter Media Playback - DirectShow Display / Speakers Filter Graph WM Player ActiveX Control (Run, Pause, Stop, …) DirectDraw/GDI DirectSound/WaveDev Hardware Comm Commands Meta data Codecs
11
What We Have Shipped DXPAK 1.0, RTM 12-15-99 DXPAK 1.0, RTM 12-15-99 DXPAK 1.1, RTM 5-10-00 DXPAK 1.1, RTM 5-10-00 Add-ons to PB 2.12 DXPAK 1.0 - DirectDraw 6,1, DirectSound 6.1, DirectShow 6.1 DXPAK 1.1 - added WMP 6.4 Control, WMT 4.1 CPU’s - X86, MIPS R4300, SH-4 Sample video drivers IGST Cyberpro 5000 S3 Virge Sample audio drivers IGST Cyberpro 5000 Ensoniq 1371 Cedar Add-on Pack, RTM 9-15-00 Cedar Add-on Pack, RTM 9-15-00 DXPAK 1.1 features for PB 3.0 Added new CPUs - StrongArm (SA 1100), Integer MIPS (MIPS R4100) Added new WMT video codecs Video 7 and ISO MPEG-4 Video 7 and ISO MPEG-4 Sample WM Player application Localized for all PB 3.0 countries Talisker Beta 1 and Beta 2 Talisker Beta 1 and Beta 2
12
Agenda Vision and Goal Vision and Goal Windows CE Multimedia Windows CE Multimedia Talisker Multimedia Features Talisker Multimedia Features WM Audio/Video streaming and codecs WM Audio/Video streaming and codecs Direct3D Direct3D DVD-Video DVD-Video Unified Audio Model Unified Audio Model
13
Talisker New features New features Full PB 4.0 CPU support Real-time 3D graphics (DX8 D3D) Digital rights management (v7 & PD DRM) DVD navigation and playback DirectDraw and audio stack enhancements New optimized codecs (WM v8 and MP3) Enhanced WM player and DVD sample apps
14
CPU Support Support all Talisker CPU/BSP Platforms Support all Talisker CPU/BSP Platforms Optimized WM and MP3 codecs Direct3D only on FP-enabled CPU’s (x86, MIPS R4300, SH4, PPC603) No ASM, coprocessor support DX/Media playback performance will vary according to CPU and system!
15
Agenda Vision and Goal Vision and Goal Windows CE Multimedia Windows CE Multimedia Talisker Multimedia Features Talisker Multimedia Features WM Audio/Video streaming and codecs WM Audio/Video streaming and codecs Direct3D Direct3D DVD-Video DVD-Video Unified Audio Model Unified Audio Model
16
Overview – Windows Media on CE Windows Media functionality in Windows CE provides the ability to playback compressed audio and video content for a variety of scenarios: Windows Media functionality in Windows CE provides the ability to playback compressed audio and video content for a variety of scenarios: Rich content experience for “Windows Powered” Internet Appliances and mobile devices Consumer Electronics (DVD players, Internet Radio, WMA/MP3 Jukeboxes, Set-Top Boxes) Training materials, how-to clips Digital Rights Management enables compelling digital media commerce scenarios Digital Rights Management enables compelling digital media commerce scenarios Over 50% of Internet content is now in Windows Media format Over 50% of Internet content is now in Windows Media format
17
Codecs Video: Video: Microsoft MPEG-4 v2, v3 Microsoft MPEG-4 ISO v1 Windows Media Video v7, v8 Audio: Audio: WMA v1, v2, v7, v8 MP3 Note: All supported codecs must be encapsulated in ASF container for streaming Note: All supported codecs must be encapsulated in ASF container for streaming
18
Streaming Protocols Multicast Multicast Each packet is broadcast to multiple clients Requires use of Windows Media Station (previously NetShow Channel).NSC files Microsoft Media Stream (MMS) Microsoft Media Stream (MMS) TCP connection for commands between client and server (defaults to port 0) UDP (defaults to port 1755) or TCP for media content Automatic protocol roll-over (UDP, TCP) HTTP HTTP Will automatically come through corporate firewalls Will use Internet Explorer’s proxy-settings, or will use own settings File-based (local or network redirector) File-based (local or network redirector)
19
APIs A variety of APIs for controlling Windows Media playback: A variety of APIs for controlling Windows Media playback: DirectShow WMP 6.4 OCX hosted in IE4, via JScript or VBScript WMP 6.4 OCX hosted in VisualBasic, VisualC++ application
20
Windows Media through DirectShow IGraphBuilder *pGraph; IMediaControl *pMediaControl; // Create the filter graph manager CoCreateInstance(CLSID_FilterGraph, NULL, CLSCTX_INPROC, IID_IGraphBuilder, (void **)&pGraph); // Query for IMediaControl (not shown) // Construct the filter graph pGraph->RenderFile(L"\\VideoFile.wmv", NULL); pGraph->RenderFile(L"\\VideoFile.wmv", NULL); // Play the File pMediaControl->Run(); pMediaControl->Run();
21
ID="MediaPlayer" ID="MediaPlayer" CLASSID="CLSID:22d6f312-b0f6-11d0-94ab-0080c74c7e95" CLASSID="CLSID:22d6f312-b0f6-11d0-94ab-0080c74c7e95" TYPE="application/x-oleobject" TYPE="application/x-oleobject" WIDTH="320" WIDTH="320" HEIGHT="240" HEIGHT="240" STYLE="position:absolute; STYLE="position:absolute; left:50px; left:50px; top:50px;“ top:50px;“> </Object> No CODEBASE=url attribute – WMP control must be part of CE device image Make controls visible WMP 6.4 OCX via HTML
22
Digital Rights Management Prevents unauthorized distribution and protects content owners’ rights Prevents unauthorized distribution and protects content owners’ rights Issue/Acquire/Enforce/Manage rights Issue/Acquire/Enforce/Manage rights Allows copyright owners to encrypt ASF content Allows copyright owners to encrypt ASF content Users must acquire “license” to play Compressed content cannot be easily emailed, transferred, or copied with associated license Core DRM is “Media Agnostic” Core DRM is “Media Agnostic” Works with ASF v1 supported codecs Works with ASF v1 supported codecs Support for DRM v7 Support for DRM v7
23
Windows CE DRM System DRM system enhancements for CE DRM system enhancements for CE Secure OS Loader Disables kernel and application debuggers when DRM module is loaded Uses trusted model to verify the DRM module is signed DirectShow security enhancements All DRM filters are verified, non-DRM filters cannot attach to the DRM filter graph
24
Optimizing Platform Performance Display Drivers Display Drivers Color-space conversion accounts for 15% of MPEG4 decode, when displaying content in RGB16 format. Having native YUV (UYVY, YUY2) Overlay Surfaces exposed via a DirectDraw- enabled Display Driver will allow for the MPEG4 / Windows Media Video codec to bypass the YUV -> RGB conversion stage.
25
Optimizing Platform Performance Audio Drivers Audio Drivers Do not perform volume adjustments for every PCM sample Use underlying h/w support for attenuation
26
Optimizing for Footprint Windows Media has been componentized for CE, allowing for maximum flexibility during platform configuration. Windows Media has been componentized for CE, allowing for maximum flexibility during platform configuration. Componentized architecture allows selection of only the required components: Codecs, DirectShow, OCX, Player application Componentized architecture allows selection of only the required components: Codecs, DirectShow, OCX, Player application Other sub-components include: Other sub-components include: Independently selectable streaming formats (MMS, HTTP, multicast, multi-bitrate, and file-based streaming) Independently selectable codecs (WMA, WMV, MP3)
27
Optimizing for Footprint (cont.) DirectX Media ASF (DXMASF) (x86): DirectX Media ASF (DXMASF) (x86): ASF, ASX parsing - 135K File-based streaming - 40K Network-based streaming - 125K Codec footprint (x86): Codec footprint (x86): WMV/MPEG4 – 190K WMA – 90K MP3 – 50K WM Player (x86): WM Player (x86): OCX – 275K App – 155K
28
Unsupported Features ASX v3 format on CE does not include the PREVIEWDURATION, BANNER, and LOGO elements ASX v3 format on CE does not include the PREVIEWDURATION, BANNER, and LOGO elements MPEG-4 v1 (obsolete codec format) MPEG-4 v1 (obsolete codec format) PowerPoint support PowerPoint support JPEG/DJPEG support JPEG/DJPEG support Hotspot ASF authoring Hotspot ASF authoring Encoding Encoding ACELP.net speech codec ACELP.net speech codec
29
Sample WMP Application The CEPlayer.exe sample-app is included in Windows CE.NET The CEPlayer.exe sample-app is included in Windows CE.NET Simple host for the underlying WMP 6.4 OCX Simple host for the underlying WMP 6.4 OCX Easily expandable by OEM for customized look-and-feel, branding, service-based hooks, etc. Easily expandable by OEM for customized look-and-feel, branding, service-based hooks, etc. Provides rich playback experience to end-user, including support for media searching, playlists, and favorites Provides rich playback experience to end-user, including support for media searching, playlists, and favorites
30
CEPlayer Marcus Ash Program Manager Windows CE Multimedia Microsoft Corporation
31
Agenda Vision and Goal Vision and Goal Windows CE Multimedia Windows CE Multimedia Talisker Multimedia Features Talisker Multimedia Features WM Audio/Video streaming and codecs WM Audio/Video streaming and codecs Direct3D Direct3D DVD-Video DVD-Video Unified Audio Model Unified Audio Model
32
Real-time 3D Graphics DX 8 Direct3D API DX 8 Direct3D API Basic primitive rendering DrawPrimitive() DrawPrimitive() Lit, textured, Gouraud shaded polygon primitives Lit, textured, Gouraud shaded polygon primitives No pixel or vertex shader commands No pixel or vertex shader commands Hardware acceleration OEM can optimize driver Reference Rasterizer Image compliance
33
D3D Real-time 3D Graphics Fixed function rendering pipeline Fixed function rendering pipeline Vertex-state machine Immediate mode commands - no retained scene description Primitive (triangle) vertex lists and properties Pipeline Stages Application Initialization of display and rendering pipe Scene traversal Set rendering state, issue draw commands Vertex Processing Vertex transform and lighting (T&L) Gouraud and fixed shading lighting models Pixel Processing Pixel fill Texture Hidden surface removal Fog, Stencil, … Blending
34
Primitive Ops D3D Fixed Function Rendering Vertex Data Pixel Processing Tex Sample Data D3D Texture Surfaces Vertex Buffers X,YRGBU,V 1U,V 2 Output pixels Tex1 Tex0 Tex2 Vertex Transform and Lighting Vec0Vec1Vec2Vec3 PrimVectorData Multi-pass processing
35
Application Tasks Model scene description Model scene description Polygon (triangle) primitives and properties Allocate resources Allocate resources Texture surfaces Vertex buffers Set up vertex state machine Set up vertex state machine Vertex processing Pixel processing Traverse scene description Traverse scene description Issue draw commands Issue draw commands
36
Vertex Pipeline Tasks Transform and lighting (T&L) done in driver/hardware or CPU Transform and lighting (T&L) done in driver/hardware or CPU Vertex processing Vertex processing Vertex assembly Vertex lighting calculation Lighting equation Primitive assembly View frustum clipping Backface cull Perspective division Viewport transform
37
Raster Pipeline Tasks Hardware or software rasterization in driver Hardware or software rasterization in driver Pixel processing Pixel processing Triangle setup Pixel fill Iterate colors, texture coordinates, etc Sample/interpolate textures Texture/color blend Stencil buffer compare Hidden surface elimination Depth buffer compare (z or w) Fog blend Frame-buffer blend
38
D3D Interfaces Interfaces Interfaces IDirect3D8 – D3D object state IDirect3DDevice8 – device rendering state, display state IDirect3DResource8 – create texture and buffer resources IDirect3DTexture8 – manage texture storage IDirect3DSurface8 – manage drawing surface IDirect3DSwapChain8 – manipulate flipping chain IDirect3DVertexBuffer8 – manage vertex buffer IDirect3DIndexBuffer8 – manage index buffer
39
D3DX Functions Math classes Math classes D3DXMATRIX D3DXVECTOR3 D3DXVECTOR4 Math functions Math functions D3DXMartrix* D3DVec3* D3DVec4* Texture Texture D3DXCreateTexture* D3DXLoadSurface* D3DXFilterTexture D3DCheckTextureRequirements Macros Macros D3DXToRadians D3DXTo Degrees
40
D3D Driver 3D Labs Permedia 3 sample driver 3D Labs Permedia 3 sample driver Support for GDI, DirectDraw, Direct3D Uses desktop driver interfaces Non-DDGPE driver Important optimizations Important optimizations Do not handle individual vertices Process complete vertex buffer Store buffers in video memory Use DMA whenever possible D3D requires perspectivelly-correct texture mapping D3D requires perspectivelly-correct texture mapping
41
D3D - Not Supported Excludes Excludes IDirect3DVolumeTexture8 IDirect3DVolume8 IDirect3DCubeTexture8 Vertex and Pixel shaders Exotic texture and processing modes
42
Direct3D Rod Deyo Program Manager Windows CE Multimedia Microsoft Corporation
43
Agenda Vision and Goal Vision and Goal Windows CE Multimedia Windows CE Multimedia Talisker Multimedia Features Talisker Multimedia Features WM Audio/Video streaming and codecs WM Audio/Video streaming and codecs Direct3D Direct3D DVD-Video DVD-Video Unified Audio Model Unified Audio Model
44
DVD Navigation / Playback Supports DVD-V video disc standard Supports DVD-V video disc standard DVD navigation manager Controls presentation engine Uses playback control info in navigation data stream Responds to user operations (“Annex J”) Determines playback conditions, UI, search, menus/buttons, parental control, etc. - VM w/ 16- bit op code DVD presentation engine Control of what is displayed – based on Nav data and playback conditions Uses audio, video, subpicture, etc. info in presentation data stream MPEG-2/AC-3 video and audio decode and playback
45
DVD System Model Navigation Data Presentation Data DVD Disc Navigation Manager Presentation Engine DVD Player User operations Display
46
Video Title Set Individual Title Program Chain Program VOBU Video Object Unit (VOBU): List of data packs DVD Presentation Data Collection of cells Nav Pack Cell: List of VOBUs Data Packs MPEG-2 Video, Audio, and Subpicture presentation data Packs of data packets Lists of pointers to cells Control, UI, Nav, Search Cmds Information and presentation data
47
DVD Application DVDNavigator Object MetaDriver DataPort Objects Digital Media Driver DVDData Object OEM Extensions IxxxxExtensions : OSD, Zoom, Digest, etc., QI: IDVDRenderer IDVDDecoderCSS IDataPort IBitstreamProcessor IDMAChannel IUnknown IDVDUserOperations, etc. IDVDData, IDVDVolume, etc. IDVDRenderer Registry DVD Disk User Input DVD Data
48
DVD-Video DVD-Video key abstractions DVD-Video key abstractions DVDData object– base abstraction of DVD nav and presentation data Exports IDVDData, IDVDVolume, etc. Exports IDVDData, IDVDVolume, etc. DVDNavigator object – Navigation manager Exports IDVDUserOperations (Annex J) Exports IDVDUserOperations (Annex J) Data Port object – DMA streaming interface Data Port object – DMA streaming interface Exports IDataPort Exports IDataPort Digital Media driver – Presentation engine Exports IDVDRenderer, IBitstreamProcessor, IDMAChannel, IDVDDecoderCSS, OEM extensions Exports IDVDRenderer, IBitstreamProcessor, IDMAChannel, IDVDDecoderCSS, OEM extensions Meta driver – abstracts Digital Media driver Driver for the Navigator object Acquires correct IDVDRenderer from aggregation
49
DVD Driver Model Flexible driver model Flexible driver model Media APIs extensible by OEM for differentiation Mode independent - supports both hardware and software implementation Optimized for multifunctional hardware COM-based, with late binding through Registry (based on function, not media type)
50
Getting Started with DVD // Create the Data Object and bind to the DVDROM CoCreateInstance(CLSID_DVDData, NULL, CLSCTX_INPROC_SERVER,IID_IDVDROM, (LPVOID *)&pDisk); pDisk->Bind(TEXT("\\CDROM Drive")); // Get the VideoVolume interface and init the volume with the disk data pDisk->QueryInterface(IID_IDVDVideoVolume, (LPVOID *) &pVolume); pVolume->InitData(pDisk); // Create Navigation Manager Object and initialize to the video volume data CoCreateInstance(CLSID_DVDNavigationManager, NULL, CLSCTX_INPROC_SERVER,IID_IDVDNavigationManager, (LPVOID *)&pNavMan); pNavMan->SetVolume(pVolume); // Get Annex J interface, then play title 0 (first play) // NOW YOUR PLAYING DVD pNavMan->QueryInterface(IID_IDVDUserOperation, (LPVOID *) &pAnnexJ); pAnnexJ->TitlePlay(0);
51
DVD Richard Russell Program Manager Windows CE Multimedia Microsoft Corporation
52
Agenda Vision and Goal Vision and Goal Windows CE Multimedia Windows CE Multimedia Talisker Multimedia Features Talisker Multimedia Features WM Audio/Video streaming and codecs WM Audio/Video streaming and codecs Direct3D Direct3D DVD-Video DVD-Video Unified Audio Model Unified Audio Model
53
Audio Features Handles PCM Data Handles PCM Data Capture, playback Capture, playback Playback - static, looping, streaming, mixing, conversion Playback - static, looping, streaming, mixing, conversion WaveIn, WaveOut WaveIn, WaveOut Audio Compression Manager (ACM) Audio Compression Manager (ACM) DirectSound, DirectSoundCapture DirectSound, DirectSoundCapture Mixer API Mixer API DirectMusic DirectMusic
54
Audio Enhancements Unified audio driver model (Wav, Mix, DSHAL) Unified audio driver model (Wav, Mix, DSHAL) New sample audio drivers New sample audio drivers
55
CE 3.0 Audio Wave API (WaveDev) Wave API (WaveDev) WaveIn, WaveOut Single stream Standard Win32 audio API for system and apps DirectSound (DSHAL) DirectSound (DSHAL) Playback and capture Multiple streams Best performance Mixer API (MixDrv) Mixer API (MixDrv) Connects a UI to “analog” controls System has no visibility of intermediate data For volume control
56
CE 2.12 / 3.0 Audio Stack Drivers Device.exe App.exe Audio Hardware DSHAL WaveDev IdsWave WaveMan WaveAPI.lib Coredll WaveBuf/ DSound Client DirectSound Server SW BufferHW Buffer SW Mixer DirectSound Client Audio Application Mixer API Lib Mixer
57
Unified Audio Model Unified Audio Model (UAM) Unified Audio Model (UAM) New unified audio middle-layer architecture Unified Audio Driver Model (UADM) Unified Audio Driver Model (UADM) Supports both DSound and Wave with single driver model Power Management for DirectSound Simpler audio capture for DirectSound Multiple drivers/devices/streams Software mixing and sample rate conversion for Waveform Audio
58
CE 4.0 Audio Stack Drivers Device.exe App.exe Audio Hardware Standard WaveDev Unified SW Mixer WaveMan WaveAPI (Coredll) Audio Device Manager DirectSound Server SW BufferHW Buffer DirectSound Client Audio Application Mixer API (Coredll) DSound ExtensionsMixer Extensions
59
Summary/Call to Action Design multimedia features into your product with an eye towards system capabilities Design multimedia features into your product with an eye towards system capabilities Add multimedia hardware acceleration whenever possible! Add multimedia hardware acceleration whenever possible! Optimize your multimedia drivers!! Optimize your multimedia drivers!! Ask questions on the public/beta newsgroups Ask questions on the public/beta newsgroups Let us know how we can be of help Let us know how we can be of help
60
For more information: www.microsoft.com/windows/ embedded/community Got Feedback? edevfdbk@microsoft.com
61
© 2001 Microsoft Corporation. All rights reserved.
Similar presentations
© 2025 SlidePlayer.com Inc.
All rights reserved.