Download presentation
2
Cortana and the Speech Platform
WinHEC 2015 12/27/2017 6:26 AM Cortana and the Speech Platform May Ji 吉晓茜 Principal Program Manager © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
3
Introduction and agenda
Tech Ready 15 12/27/2017 Introduction and agenda Session introduction Learn about the new, exciting features in Cortana and the speech platform including far- field and Wake on Voice (WoV) from Modern Standby, Cortana’s availability on Windows 10 IoT Core, and how to build high-quality innovative devices to light them up. What is Cortana Cortana… what can she do? The Audio and Speech Platform Call to Action © 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
4
Cortana 小娜 WinHEC 2015 12/27/2017 6:26 AM
© 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
5
What is Cortana? Cortana is your truly personal digital assistant,
12/27/2017 6:26 AM What is Cortana? Cortana is your truly personal digital assistant, there for you whenever and wherever you need, to make sure nothing slips through the cracks. © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
6
12/27/2017 6:26 AM Cortana Experience Your truly personal digital assistant, there for you to make sure nothing slips through the cracks. Natural and easy to interact with Provides proactive and personalized assistance Cortana remembers, so you don’t have to Works with apps and services to help you get things done © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
7
Cortana Momentum +400M 1M >140M #1 +13B 1000+
Ask me anything +400M Devices running Windows 10— and growing every day 1M Voice Queries Daily >140M MAU use the Cortana Search Box Increase in revenue per device, as compared to Windows 8. #1 Most appealing feature in consumer research +13B # of questions consumers have asked Cortana 1000+ Skills
8
Cortana Market Availability
Team: Microsoft Search Template: Search Product Marketing 12/27/2017 6:26 AM Cortana Market Availability FRANCE French CANADA English, French GERMANY German UK English ITALY Italian JAPAN Japanese USA English SPAIN Spanish CHINA Mandarin INDIA English MEXICO Spanish BRAZIL Portuguese AUSTRALIA English © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
9
Cortana … what can she do
10
Immersive Cortana Experience
Immersive Cortana Experience Cortana can help you with all kinds of tasks on your lock screen, even when the screen is turned off!
11
Cortana on Windows 10 devices
WinHEC 2015 12/27/2017 6:26 AM Cortana on Windows 10 devices Because Cortana is available across device types, she’s able to help you even more effectively Cortana was first available on Windows phones Then Cortana came to Windows 10 PCs, tablets, Windows Holographic and Xbox Windows devices With upcoming Windows 10 Creators Update, Cortana will be available on Windows 10 IoT Core devices with displays © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
12
Powered by the Audio and Speech Platform
WinHEC 2015 12/27/2017 6:26 AM Microsoft is committed to reinventing the PC and inspiring ecosystem growth with Better Experience Cortana with far-field Voice and Wake On Voice (WoV) from Modern Standby Better Marketing will be differentiated features for 2017 Better Device …on innovative devices to drive excitement in the market. Powered by the Audio and Speech Platform © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
13
The Audio and Speech Platform 音频和语音平台
WinHEC 2015 12/27/2017 6:26 AM The Audio and Speech Platform 音频和语音平台 © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
14
Audio and Speech Platform Stack
WinHEC 2015 12/27/2017 6:26 AM Audio and Speech Platform Stack Cortana, Shell, Bing, Office, Skype, Microsoft’s and 3rd party’s Apps and Services Audio Pipeline, Keyword Spotter, Speech Recognition, text-to-speech Microsoft and 3rd party PC and IoT devices with displays Experience Speech Platform Hardware High quality hardware Performant Speech Platform Great Cortana Experience © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
15
Wake on Voice (WoV) from Modern Standby
WinHEC 2015 12/27/2017 6:26 AM Far-field Interact with Cortana from 4 meters away with great user experience, even with noises and music playing Requires high quality microphone design that meets Microsoft speech spec 2.0 Wake on Voice (WoV) from Modern Standby Wake up a device from a screen-off state to a screen-on, user-interactive state, by saying “Hey Cortana” Designed for Modern Standby Far-field Wake on Voice (WoV) from Modern Standby Differentiation © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
16
VOICE ACTIVATION Voice Activation
WinHEC 2015 12/27/2017 6:26 AM Voice Activation The scenario of providing keyword detection of a predefined activation key-phrase for an application. For example, "Hey Cortana" is the hero Microsoft Voice Activation scenario for Cortana VOICE ACTIVATION © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
17
WAKE ON VOICE Wake on Voice (WoV)
WinHEC 2015 12/27/2017 6:26 AM Wake on Voice (WoV) Any technology that enables Voice Activation from a screen-off state (lower power) to a screen-on full power (S0) state. SCREEN ON SCREEN OFF WAKE ON VOICE © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
18
CORTANA AND IMMERSIVE CORTANA
LEGACY SLEEP (S3) SUPPORT MODERN STANDBY SUPPORT WinHEC 2015 12/27/2017 6:26 AM The Windows 10 Anniversary Update provides a checkbox to entice users to prevent automatic system sleep, such that "Hey Cortana" can listen even when the screen is off. This is our first step on the WoV journey and provides a WoV- like experience SCREEN ON CORTANA AND IMMERSIVE CORTANA SCREEN OFF WOV-LIKE EXPERIENCE © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
19
CORTANA AND IMMERSIVE CORTANA
LEGACY SLEEP (S3) SUPPORT MODERN STANDBY SUPPORT WinHEC 2015 12/27/2017 6:26 AM Wake on Voice from Modern Standby WoV from a Modern Standby (S0ix) screen-off state to a screen-on full power (S0) state SCREEN ON CORTANA AND IMMERSIVE CORTANA SCREEN OFF WOV-LIKE EXPERIENCE WAKE ON VOICE FROM MODERN STANDBY © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
20
CORTANA AND IMMERSIVE CORTANA
LEGACY SLEEP (S3) SUPPORT MODERN STANDBY SUPPORT WinHEC 2015 12/27/2017 6:26 AM Wake on Voice with SW KWS from Modern Standby WoV using SW KWS from a Modern Standby (S0ix) screen off state to a screen on full power (S0) state. SCREEN ON CORTANA AND IMMERSIVE CORTANA SCREEN OFF WOV-LIKE EXPERIENCE WOV WITH SW KWS FROM MODERN STANDBY WINDOWS 10 CREATORS UPDATE S0 SCREEN OFF, OR LOW POWER AUDIO “DEEP SLEEP” S3, OR DRIPS © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
21
CORTANA AND IMMERSIVE CORTANA
LEGACY SLEEP (S3) SUPPORT MODERN STANDBY SUPPORT WinHEC 2015 12/27/2017 6:26 AM Wake on Voice with HW KWS from Modern Standby WoV using a HW KWS from a Modern Standby (S0ix) screen-off state to a screen- on full power (S0) state. Key Prerequisites: DSP hardware-offload SCREEN ON CORTANA AND IMMERSIVE CORTANA SCREEN OFF WOV-LIKE EXPERIENCE WOV WITH SW KWS FROM MODERN STANDBY WINDOWS 10 CREATORS UPDATE S0 SCREEN OFF, OR LOW POWER AUDIO WOV WITH HW KWS FROM MODERN STANDBY FUTURE “DEEP SLEEP” S3, OR DRIPS © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
22
Software Keyword Spotter (SW KWS) WoV from Modern Standby
WinHEC 2015 12/27/2017 6:26 AM Software Keyword Spotter (SW KWS) WoV from Modern Standby Algorithm Provider Runs on Modern Standby Big Buffer Capture AC Power Only Requirements or 3rd party APO Audio Processing To reduce power consumption, the audio driver is required to implement a big audio capture system buffer Premium devices should support 100ms to 200ms audio capture buffer size This is done by specifying MaxPacketSizeInBytes in the DEVPKEY_KsAudio_PacketSize_Constraints2 device property KSAUDIO_PACKETSIZE_PROCESSINGMODE_CONSTRAINT structure and KSAUDIO_PACKETSIZE_CONSTRAINTS2 structure on MSDN © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
23
Standard vs. Premium Near-Field Near-Field Far-Field
WinHEC 2016 12/27/2017 6:26 AM Standard vs. Premium Near-Field Near-Field Far-Field Works well in ambient conditions at the typical 0.5m distance. Works well from arms-length (leaning back) 0.8m in challenging environments like a busy kitchen or a family room. Works well from further away, up to 4m, in challenging environments like a busy kitchen or a family room Hey Cortana, What is the status of my flight Hey Cortana, will the Seahawks win this weekend? Hey Cortana… Hey Cortana, will the Seahawks win this weekend? © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
24
“Hey Cortana” Hola Cortana 你好小娜 Ehi Cortana Hé Cortana コルタナさん
WinHEC 2015 12/27/2017 6:26 AM “Hey Cortana” Hola Cortana OS Release Locale (BCP 47) Selected Key-phrase Win 10 en-US Hey Cortana en-GB fr-FR it-IT Ehi Cortana de-DE es-ES Hola Cortana zh-CN 你好小娜 (Nǐ hǎo xiǎo nà) Nov. (1511) Update ja-JP コルタナさん (Korutana-san) en-IN en-CA en-AU Anniversary Update pt-BR Ei Cortana fr-CA Hé Cortana es-MX Hé Cortana コルタナさん 你好小娜 Ehi Cortana The Voice Activation (VA) key-phrase for Cortana to respond is “你好小娜” in Chinese Currently supported in 14 locales for near-field Will be supported in English (US and GB) for far-field in Windows Creators Update To ensure a first-class, consistent experience across all Cortana devices “Hey Cortana” key-phrases are not configurable © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
25
Take Advantage of Windows Inbox Audio Processing
Echo cancellation, noise suppression, and beam forming It is part of Windows No additional audio driver required No Additional Cost State of the art technology from years of research and development Top performance and quality without tuning Great Performance No tuning or driver updates needed Continuous improvements available in OS updates from WU Low Maintenance Supports both near and far-field Shipped in multiple Microsoft devices (Surface product line, Hololens, Xbox Kinect) Far-field Support
26
Shipped in Broad Ecosystem Devices
WinHEC 2015 12/27/2017 6:26 AM Shipped in Broad Ecosystem Devices © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
27
Far-field Microphone Recommendations
WinHEC 2015 12/27/2017 6:26 AM Far-field Microphone Recommendations Far-field Mic Array Type Number of Mics Recommendation Inbox Audio Processing Support Mic Array Geometry Linear 4 Yes Follow the above mic array geometry if using inbox audio processing Circular 8 Coming soon Use 3rd party solution and follow their spec recommendations. Need to meet quality bar in Microsoft speech spec 2.0 © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
28
Platform algorithm providers
WinHEC 2015 12/27/2017 6:26 AM Testing Requirements There are separate testing requirements for platform algorithm providers and OEMs & ODMs in speech spec 2.0 Partner Type Areas to Test Languages to Test Platform algorithm providers Speech Accuracy All 14 supported locales Voice Activation (both correct and false accepts) OEMs and ODMs US English Voice Activation (correct accept only) US English and Japanese © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
29
Platform Quality Targets for Algorithm Providers
WinHEC 2015 12/27/2017 6:26 AM Platform Quality Targets for Algorithm Providers All “Hey Cortana” keyword spotting algorithms, including Microsoft SW KWS, are tested for performance measurements against the quality bar in the speech spec 2.0 Speech Accuracy Voice Activation (Correct and False Accepts) Release People Scenario Standard Premium HMD Near- field (0.5m) Near- field (0.8m) (4m) 2017 Male, Female, Children (5-12) Quiet 95% Echo 85% 90% Noise (Medium) Echo+Noise (Medium) Noise (Loud) n/a Release People Scenario Standard Premium HMD Near-field (0.5m) Near-field (0.8m) * (4m) 2017 Male, Female, Children (5-12) Quiet 90% Echo Noise (Medium) Echo+Noise (Medium) Noise (Loud) n/a Release People 2017 <= 1 FA per 100 hours of continuous speech © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
30
Device Quality Targets for OEMs and ODMs
WinHEC 2015 12/27/2017 6:26 AM Device Quality Targets for OEMs and ODMs Speech Accuracy Voice Activation (Correct Accepts) Release People Scenario Standard Premium HMD Near- field (0.5m) Near- field (0.8m) Far-field (4m) 2017 Male, Female, Children (5-12) Quiet 95% Echo 85% 90% Noise (Medium) Echo+Noise (Medium) Noise (Loud) n/a Release People Scenario Standard Premium HMD Near-field (0.5m) Near-field (0.8m) Far-field (4m) 2017 Male, Female, Children (5-12) Quiet 90% Echo Noise (Medium) Echo+Noise (Medium) Noise (Loud) n/a 85% Test en-US only Test en-US and ja-JP only © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
31
Audio Labs for Cortana and Speech Testing
WinHEC 2015 12/27/2017 6:26 AM Audio Labs for Cortana and Speech Testing © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
32
Pairing Speech Platform Tests with HLK
WinHEC 2015 12/27/2017 6:26 AM Pairing Speech Platform Tests with HLK Cortana with Voice (CwV) testing done with tools provided System submission created with Specialized PC feature System submission packaged up with the logs from CwV logs System submissions made to SysDev like they are today © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
33
Call to Action SYSDEV Portal Submission
WinHEC 2015 12/27/2017 6:26 AM Call to Action Build high-quality hardware to light up far-field and Wake on Voice (WoV) from Modern Standby Develop Modern Standby and far-field devices for BTS/holiday 2017 Follow the technical guidance in speech specs 2.0 Work with audio driver vendors to implement larger audio capture system buffer size Engage with Microsoft on those devices SYSDEV Portal Submission Submit your Speech Platform test results up to SYSDEV © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
34
Technical Requirement References
WinHEC 2016 12/27/2017 6:26 AM Technical Requirement References Requirement Detail Reference / Spec Speech Platform: Input Device Recommendations & Test Setup specifications 2.0 Audio input device design and development recommendations, test guidance Speech Platform test tools Application and content guidance for testing Microsoft Speech Platform Test Tools Voice Activation Test Sets (Platform provider Tests) FAQ Answers to FAQ Answers to Frequently asked questions about Cortana, such as Wake-On-Voice, , etc. © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
35
Thank You 谢谢 Please follow WinHEC @ WinHEC.com 12/27/2017 6:26 AM
© 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
Similar presentations
© 2024 SlidePlayer.com Inc.
All rights reserved.