Fidelity Test – Import and Export Introduction and Troubleshooting Matthew van Eerde, Windows Sound

Agenda What is Fidelity and how does it work? – Fidelity before export and import were invented. – What export and import are for. – How to run export; how to run import. When does a new log need to be exported? – Submission type – basic/premium, desktop/mobile, etc. – Devices – same hardware, same revisions. – Topology – same inputs, same outputs, same paths through device. Something went wrong. What do I do? – Couldn’t open the import file... why? – The topology is different... why? – Resources – online tips, support.

Fidelity Test – “Run Locally"

Standard DTM setup: – Controller – Studio – System under test Additionally: – Audio Precision analyzer – Audio Precision host + APIB cable – Audio cables Drawbacks: – Audio Precision analyzer is a large commitment – Cost of running test can be disproportionate to benefit

Fidelity Test – “Export”

Same setup as "run locally“: – Test results exported to a specially-decorated log file. – This is the same as the.wtl (Windows Test Log) file on the DTM controller, but with an added hash. – The hash ensures that the file has not changed since it was created. Hash: SHA-1/Encrypted 408e96f2f46e35f916454ac3fad19567b6cb788 (from here on the file is the same as the.wtl file) (this line is important... more on this later) (basic or premium; desktop or mobile) (and so are these... more on this later) (PnP ID plus hash of topology) ()

Fidelity Test – “Import”

Fidelity Test - “Import” Setup changes from "run locally“: – No AP analyzer, APIB cable, or audio cables necessary! – "AP host" is still necessary, but for DTM Scheduler reasons only. It takes no active role in the test - it is just a dumb machine. – Fidelity Test must be able to reach and read the log file from System Under Test. What about the log file? – Log file must be exactly the same as when it was exported (even whitespace, line breaks, and case). – Content of log file must match system being tested (more on this later). – Requirements must be compatible between log file and submission (can't import a "basic" pass into a "premium" submission).

When Does a New File Need to be Exported? A premium submission can only import a premium log. A basic submission can import either a basic or a premium log: – So, it suffices to export only a premium log. The list of PnP IDs of audio devices must match between the system under test and the log file: – So, if you add a USB Webcam, you must export a log that includes the USB Webcam. – If the codec REV_ number is incremented, you must export a new log. – If the SUBSYS_ number changes, you must export a new log The audio "topology" exposed by each audio device in the system under test must match the corresponding audio topology recorded in the log file: – So, if you can connect a pin on the codec to either a "CD in" or an "AUX", you must export a log each way. – If you have a third-party driver for a UAA-compliant device, export a log with the third-party driver installed and a log with the UAA class driver installed (such as the Microsoft HD Audio class driver).

What Is “Topology?”

The previous slide was a screenshot of a single audio filter, viewed in KSStudio.exe, available as part of the Windows Driver Kit. The topology provides a detailed map of how data flows; for capture, from the hardware through the driver to the operating system; for render, from the operating system through the driver to the hardware. Topology is exposed by the audio driver.

More on Topology Driver changes might change topology: – If a driver version changes, it might or might not affect the topology exposed by any given hardware. Class drivers use other information: – The Microsoft USB audio class driver uses information in the USB descriptor (stored in the USB audio device) to decide what topology to expose. If this changes, the topology might change. – The Microsoft HD Audio class driver uses information in the HD Audio pin configuration registers (usually stored in the motherboard BIOS) to decide what topology to expose. If this changes, the topology will probably change. – Some third-party HD Audio drivers also use the pin configuration registers. Jack presence detection: – Some hardware can detect when a jack is plugged in. – Windows Vista allows the driver to expose filters dynamically. – Some HDMI drivers use this dynamic behavior based on an HDMI handshake. – Always plug in all audio jacks - including HDMI - when running any audio test. – Use mmsys.cpl to verify that there are no "unplugged" audio endpoints.

Best Practices on Topology If you do any of these, export a new log: – Change the USB descriptor in a USB audio device. – Change the HD Audio pin configuration in the BIOS. – Install a new version of an audio driver. Always do this: – Plug in all audio jacks before running any audio test. – Verify in mmsys.cpl that no jacks are "unplugged“. – Verify in mmsys.cpl that the listed audio endpoints are sensible.

Unplugged Endpoint in mmsys.cpl

Problems Reading/Writing File

Could not import from file (...) (error 0x ) Export dry run failed (...) (error 0x ) – 0x is ERROR_FILE_NOT_FOUND – 0x is ERROR_ACCESS_DENIED Fidelity Test runs on the system under test... (has different C:, D:,... drives than the studio machine)... as local user.\DTMLLUAdminUser (will likely not have permission to \\server\share paths) Best practice: Place the file on the C: drive of the system under test

Requirement Mismatch "Requirements in the log do not match requirements for this test pass" – Importing basic log into premium submission? – Importing desktop log into mobile submission... or vice versa. – Look at REQUIREMENTS: and REPLAYED: REQUIREMENTS: log lines. – Note that the Fidelity requirements are different for (basic/premium) (desktop/mobile). – A premium log can be imported into a basic submission, but everything else must match.

Device or Topology Mismatch "Some devices in this SUT, selected for test, do not appear in the log file" "This log file contains devices that do not exist in the SUT" – This means there is a mismatch, either in the list of PnP IDs, or in some device's topology. – Make sure all jacks are plugged in; confirm via mmsys.cpl. – If that doesn't fix it, look at the DEVICE-INFO: and REPLAYED: DEVICE-INFO: lines - compare the PnP IDs. – If those all match, look at the Topology hashes and find the ones that don't match; then look at the Topology: lines and break them out.

Example An import fails with "This log file contains devices that do not exist in the SUT" Compare DEVICE-INFO lines first: DEVICE-INFO: HDAUDIO\FUNC_01&VEN_VVVV&DEV_DDDD&SUBSYS_SSSSSSSS& REV_RRRR (Topology SHA-1 hash: f0a5d91d8a1497db a7535e5e1853) REPLAYED: DEVICE-INFO: HDAUDIO\FUNC_01&VEN_VVVV&DEV_DDDD&SUBSYS_SSSSSSSS& REV_RRRR (Topology SHA-1 hash: 62d a8c8842bdc7e468ed4b4f88591) The PnP IDs have been sanitized, but they matched (trust me.) The Topology hashes, on the other hand, don't!

Example (continued) Topology in SUT Topology: HD Audio CD in capture mixer (Pins: Filter, CD Audio; Nodes: CD Volume, CD Mute; Connections: CD Audio => CD Volume(1), CD Mute(0) => Filter, CD Volume(0) => CD Mute(1); ); HD Audio CD in capture (Pins: Recording Control, Filter; Nodes: Unnamed; Connections: Filter => Unnamed(1), Unnamed(0) => Recording Control; ); HD Audio Headphone mixer (Pins: Filter, Headphones; Nodes: Speakers, Master Mute; Connections: Filter => Speakers(1), Master Mute(0) => Headphones, Speakers(0) => Master Mute(1); ); HD Audio Headphone (Pins: PC Speaker, Filter; Nodes: HD Audio line out; Connections: PC Speaker => HD Audio line out(1), HD Audio line out(0) => Filter; ); HD Audio Speaker mixer (Pins: Filter, Speakers; Nodes: Speakers, Master Mute; Connections: Filter => Speakers(1), Master Mute(0) => Speakers, Speakers(0) => Master Mute(1); ); HD Audio Microphone mixer (Pins: Filter, Microphone; Nodes: Mic Volume, Mic Mute, Microphone Boost; Connections: Microphone => Mic Volume(1), Microphone Boost(0) => Filter, Mic Mute(0) => Microphone Boost(1), Mic Volume(0) => Mic Mute(1); ); HD Audio Microphone (Pins: Recording Control, Filter; Nodes: Unnamed; Connections: Filter => Unnamed(1), Unnamed(0) => Recording Control; ); HD Audio Digital in mixer (Pins: Filter, Digital Input Device (SPDIF); Nodes: ; Connections: Digital Input Device (SPDIF) => Filter; ); HD Audio Digital in (Pins: Filter, Filter; Nodes: SPDIF Interface; Connections: Filter => SPDIF Interface(1), SPDIF Interface(0) => Filter; ); HD Audio SPDIF out mixer (Pins: Filter, Digital Output Device (SPDIF); Nodes: Master Mute; Connections: Filter => Master Mute(1), Master Mute(0) => Digital Output Device (SPDIF); ); HD Audio SPDIF out (Pins: Filter, Filter; Nodes: SPDIF Interface; Connections: Filter => SPDIF Interface(1), SPDIF Interface(0) => Filter; ); Topology from file REPLAYED: Topology: HD Audio CD in capture mixer (Pins: Filter, CD Audio; Nodes: CD Volume, CD Mute; Connections: CD Audio => CD Volume(1), CD Mute(0) => Filter, CD Volume(0) => CD Mute(1); ); HD Audio CD in capture (Pins: Recording Control, Filter; Nodes: Unnamed; Connections: Filter => Unnamed(1), Unnamed(0) => Recording Control; ); HD Audio Headphone mixer (Pins: Filter, Headphones; Nodes: Speakers, Master Mute; Connections: Filter => Speakers(1), Master Mute(0) => Headphones, Speakers(0) => Master Mute(1); ); HD Audio Headphone (Pins: PC Speaker, Filter; Nodes: HD Audio line out; Connections: PC Speaker => HD Audio line out(1), HD Audio line out(0) => Filter; ); HD Audio Speaker mixer (Pins: Filter, Speakers; Nodes: Speakers, Master Mute; Connections: Filter => Speakers(1), Master Mute(0) => Speakers, Speakers(0) => Master Mute(1); ); HD Audio Speaker (Pins: PC Speaker, Filter; Nodes: HD Audio line out; Connections: PC Speaker => HD Audio line out(1), HD Audio line out(0) => Filter; ); HD Audio Microphone mixer (Pins: Filter, Microphone; Nodes: Mic Volume, Mic Mute, Microphone Boost; Connections: Microphone => Mic Volume(1), Microphone Boost(0) => Filter, Mic Mute(0) => Microphone Boost(1), Mic Volume(0) => Mic Mute(1); ); HD Audio Microphone (Pins: Recording Control, Filter; Nodes: Unnamed; Connections: Filter => Unnamed(1), Unnamed(0) => Recording Control; ); HD Audio Digital in mixer (Pins: Filter, Digital Input Device (SPDIF); Nodes: ; Connections: Digital Input Device (SPDIF) => Filter; ); HD Audio Digital in (Pins: Filter, Filter; Nodes: SPDIF Interface; Connections: Filter => SPDIF Interface(1), SPDIF Interface(0) => Filter; ); HD Audio SPDIF out mixer (Pins: Filter, Digital Output Device (SPDIF); Nodes: Master Mute; Connections: Filter => Master Mute(1), Master Mute(0) => Digital Output Device (SPDIF); ); HD Audio SPDIF out (Pins: Filter, Filter; Nodes: SPDIF Interface; Connections: Filter => SPDIF Interface(1), SPDIF Interface(0) => Filter; );

Parsing Topology That's kind of daunting, so let's break it down: each line contains several filters. Each filter looks like: – Filter name (Pins:...; Nodes:...; Connections:...; ); Each filter corresponds to a KSStudio block. Recommended legible layout: Filter name ( Pins: Pin 1, Pin 2,...; (these are the things on the outside of the block) Nodes: Node 1, Node 2,...; (these are the things on the inside of the block) Connections: (Nodes have two sides, Node(0) and Node(1); Pins have one side) Pin 1 => Node 1(1), Node 1(0) => Node 2(0),...; ); Parse both topologies into the legible layout, then use a "diff" tool to see what changed (various tools available).

Example (continued) Topology in SUT HD Audio CD in capture mixer ( Pins: Filter, CD Audio; Nodes: CD Volume, CD Mute; Connections: CD Audio => CD Volume(1), CD Mute(0) => Filter, CD Volume(0) => CD Mute(1); ); HD Audio CD in capture ( Pins: Recording Control, Filter; Nodes: Unnamed; Connections: Filter => Unnamed(1), Unnamed(0) => Recording Control; );... (etc.) Topology from file HD Audio CD in capture mixer ( Pins: Filter, CD Audio; Nodes: CD Volume, CD Mute; Connections: CD Audio => CD Volume(1), CD Mute(0) => Filter, CD Volume(0) => CD Mute(1); ); HD Audio CD in capture ( Pins: Recording Control, Filter; Nodes: Unnamed; Connections: Filter => Unnamed(1), Unnamed(0) => Recording Control; );... (etc.)

What's the Difference? The following filter was exposed when the exported log file was created, but is not exposed in the system under test: HD Audio Speaker ( Pins: PC Speaker, Filter; Nodes: HD Audio line out; Connections: PC Speaker => HD Audio line out(1), HD Audio line out(0) => Filter; ); Diagnosis: there's something wrong with the "Speaker" endpoint, which is actually a line out. Checking mmsys.cpl confirms the diagnosis; the "Speaker" endpoint is listed as "Not plugged in."

Online Resources Audio Fidelity Testing Windows Logo Kit Requirements Windows Driver Kit / Windows Logo Kit System Error Codes

Disclaimer © 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.