1. Assessing Sleep Wearables and In-Bedroom Devices: CTA Standards Work
Max Hirshkowitz, PhD, D ABSM
NSF, Immediate Past Chairman of Board
Baylor College of Medicine, Full Professor (Emeritus
Stanford University, Consulting Professor

2. Disclosure
Max Hirshkowitz, PhD, has no relevant financial relationships to disclose and does not intend to discuss the off-label/investigative use of any medication.

3. Educational Objectives
Upon concluding this activity, participants should be able to...
To review basic concepts concerning terminology, regulatory issues, measurement, and statistics
To summarize adopted CTA standards and the current status of the currently ongoing work group project

4. Lessons From History

5. Remember… No Technology Starts Out Perfect

6. But… Technology Improves Over Time
Then
Now

7. And then there are obstructionists

8. 1936 The first overnight Polysomnograph Machine Loomis Labs in
Tuxedo Park, NY

9. In Silicon Valley Parlance… In the late 1970’s Polysomnography became the Killer App for diagnosing Sleep Apnea

10. Sleep laboratory control room, circa 1979

11. Digital PSG
Arrives…
Finally
1984

12. Sleep laboratory control room, circa 2009

13. But… Then along came HST (Home Sleep Testing)

14. The Future Isn’t What It Used To Be

15. The War Between PSG & HST Began as the New Millennium DawnedVS

17. The PSG War Against HST was Lost in 2008

18. The Problem Now is That the Baby Got Thrown Out With the Bathwater

19. CTA/ANSI Standards Project for Wearable & In-bedroom Sleep Trackers.
The project has three parts
Phase I: Definitions and Informatics
Phase II: Methods
Phase III: Performance and Validation Criteria

20. So… What are the Objections to Wearable & In-bedroom Sleep Trackers?
I don’t know what the data mean
The data are unreliable
I don’t want to be bothered with it
because of A & B
because the tools I use now are fine… if it ain’t broke, don’t fix it
I don’t have any way to bill for it

21. So… What are the Objections to Wearable & In-bedroom Sleep Trackers?
I don’t know what the data mean
The data are unreliable
I don’t want to be bothered with it
because of A & B
because the tools I use now are fine… if it ain’t broke, don’t fix it
I don’t have any way to bill for it
The CTA/ANSI Standards Project was commissioned to answer objections A & B & by extension C.1

22. Phase I: Definitions & Informatics
ANSI/CTA Standard
Definitions and Characteristics for Wearable Sleep Monitors
ANSI/CTA/NSF
September 2016

23. Five Categories of Terms
Terms in Category A: Terms describing a sleep episodes temporal surround. These are general terms related to the sleeper’s intention and their position in the environment. Because a person may be in bed without an intention to sleep leads to the distinction between these two elements.
Terminology Category
Elemental Measure
Derived Measures A
General terms describing the temporal surround of a sleep episode
Time when individual began Attempting To Sleep (TATS):
TATS Start Time
TATS End Time
Time In Bed (TIB):
TIB Start Time
TIB End Time
Total TATS Duration
Total TIB Duration

24. Five Categories of Terms
Terms in Category B: Terms describing the general sequence of events once there is intention to sleep. Continued wakefulness and eventual sleep will occur. It should be noted that some of the terms in category B have both general and laboratory definitions. However, terms such as wakefulness do not require polysomnographic comparison to verify (e.g., I am awake while I am typing this document). BB
General terms describing basic features of wakefulness and sleep
Awake
Asleep
Awakening from sleep
Brief awakening
Brief moment of sleep (dozing)
Total sleep period duration (TSPD)
Total sleep time (TST)
Sleep maintenance %
Total wakefulness duration
Wakefulness duration after initial sleep onset
Number of awakenings
Number of brief awakenings
Awakening rate per hour
Sleep fragmentation rate
Number of dozing episodes

25. Five Categories of Terms
Terms in Category C: Once the initial sleep and final wakefulness episode determined, One can designate several important metrics based on the relationship between these measures and the temporal surround. These include latency to sleep onset, latency to arising, and sleep efficiency.
Terms derived from basic features of wakefulness, sleep as they relate to the sleep episode and its surround
Initial sleep onset time
Final awakening time
Latency to sleep onset
Latency to arising
Sleep efficiency %

26. Five Categories of Terms
Terms in Category D: These terms are defined in the “AASM manual for the scoring of sleep and associated events: rules, terminology and technical specifications. 1st edition” (Iber et al, 2007) and updated by subsequent revisions. This nomenclature is well established and widely used by sleep researchers and sleep medicine practitioners. These standards are based on polysomnographic (electroencephalogram, electrooculogram, and electromyogram) correlates derived from normal human subjects. To avoid creating linguistic ambiguity, these terms should concord with standardized definitions and will not be redefined here.
Specific terms describing processes occurring during sleep based on polysomnography
REM Sleep N1 N2 N3
CNS Arousal
Number of CNS arousals
CNS arousal rate per hour
Duration, percentage (of TST), and latency from sleep onset for each of the following:
REM

27. Five Categories of Terms
Terms in Category E: Terms proposed here are designed to extend sleep feature description beyond existing standard nomenclature. Alternative terminology is recommended to avoid creating ambiguity with existing polysomnographic nomenclature (provided in category D). Care was taken to avoid using sleep medicine laboratory terminology in order to avoid confusion
Alternate terms for subdividing sleep into different processes
Dream sleep
Core Sleep
Sound Sleep
Restless Sleep
Duration and percentage for each of the following:
Core sleep
Restless sleep % of TST

28. Five Categories of Terms
Terms in Category F: Terms used to describe circadian rhythms are included here. Wrist actigraphs have long been used in circadian rhythm research. Each actigraph differs with respect to precision and accuracy; consequently, each should be evaluated independently.
Terms used to describe the sleep-wake cycle over time periods exceeding 7 days
Circadian Amplitude
Circadian Period length (tau)
Circadian Phase
Relative duration of the active period compared to the dormant period

29. Phase II: Sleep Tracking Methods
Self-Report
Direct or Video Observation
Room/Bed Sensors
Actigraphy (Movement Sensors)
Polysomnography (EEG-EOG-EMG)
Eye Movements (Piezo-Electric, EOG, Room Sensors)
Autonomic Nervous System Measures
BP Blood Pressure
EDA Electrodermal Activity
HR Heart Rate and Regularity
Respiratory Rate & Variability
Body Temperature
Endocrine Variations

30. Phase II: Sleep Tracking Methodology
Notation: D- Directly Measured; I- Inferred from measure indicated; S- Sleep medicine standard
Self-Report
Observation
Room/Bed Sensors
Actigraphy
PSG
ANS
Body Temp
Endocrine
TATS Start Time D I
TATS End Time
TIB Start Time
TIB End Time
Awake S
Asleep
Awakening from Sleep
Brief Awakening
Initial Sleep Onset Time
Final Awakening Time
Brief Moment of Sleep (Dozing)
REM Sleep
N1 Sleep
N2 Sleep
N3 Sleep
CNS Arousal
Dream Sleep
Core Sleep
Restless Sleep
Sound Sleep
Circadian Amplitude
Circadian Period Length (tau)
Circadian Phase

31. Phase III: Performance Criteria
Different terms require different criteria (e.g., stages vs. circadian)
Different terms require different environmental and test conditions
Sample considerations or each measure
Number of subjects
Male and female
Age group
Good vs. Poor Sleepers
Undiagnosed vs. Diagnosed
With Sleep Disorders
With medical, neurological, or psychiatric conditions

32. Phase III: Performance Criteria
Design & Statistics:
Comparisons require temporal concordance (not overall composite agreement)
Paradigm:
Passive monitoring & comparison with a standard
Temporal agreement during an intervention (eg. Forced awakening)
Agreement with self-report after a spontaneous event (“I’m awake”)
Experiments in nature

33. Validation must
be performed
against a reliable
reference standard

34. … How reliable does it have to be?

35. The Problem with Correlations

36. What’s Wrong with Using Overall Agreement?

41. Phase III: Performance Criteria
Design & Statistics:
Comparisons require temporal concordance (not overall composite agreement)
Paradigm:
Passive monitoring & comparison with a standard
Temporal agreement during an intervention (e.g. Forced awakening)
Agreement with self-report after a spontaneous event (“I’m awake”)
Experiments in nature

42. Wearable Self-monitoring Technology

43. Phase III: Performance Criteria
To date, I’ve found only 11 peer- reviewed journal articles addressing consumer wearable devices’ performance
Research reports varied widely with respect to their methods, terminology, standards for testing & sample characteristics
Agreement rates varied

44. So… what is the current status of the CTA Standards Project for Wearable & In-bedroom Sleep Trackers?
Phase I: Definitions and Informatics- Complete
Phase II: Methods- Complete
Phase III: Performance and Validation Criteria- Currently On-going

45. So What? Who Cares?
Consistent self-monitoring is key for behavior change. This includes:
improved nutrition
fitness
weight loss
Wearable or In-bed technology can provide self- monitoring for sleep (like a bathroom scale does for weight)

46. Final Considerations:
These devices are pretty good at estimating
Sleep Duration, Awakenings, Restlessness, and Sleep Timing
They are getting better at assessing
Sleep vs. wakefulness in normal individuals
Given the NSF Sleep Quality Project findings showing little agreement about sleep stage contribution, these devices may be able to estimate sleep quality
Thus, if Sleep Health’s elements are:
sleep duration
sleep quality
sleep satisfaction (which an individual assesses)
absence of sleep disorders…
We are 3/4ths of the way there.

47. Educational Objectives
Participants should now be able to...
To review basic concepts concerning terminology, regulatory issues, measurement, and statistics
To summarize adopted CTA standards and the current status of the currently ongoing work group project

48. Thank You