1. An Efficient Bit Vector Approach to Semantics-based
An Efficient Bit Vector Approach to Semantics-based
Machine Perception in Resource-constrained Devices
This is one example of an application being build from the research I will be discussing
Titled: An efficient bit vector approach to semantics-based machine perception in resource-constrained devices
Cory Henson, Krishnaprasad Thirunarayan, Amit Sheth
Kno.e.sis – Ohio Center of Excellence in Knowledge-enabled Computing
Wright State University, Dayton, OH, USA

2. The Patient of the Future
- Larry Smarr is a professor at the University of California, San Diego
And he was diagnosed with Chrones Disease
What’s interesting about this case is that Larry diagnosed himself
He is a pioneer in the area of Quantified-Self, which uses sensors to monitor physiological symptoms
Through this process he discovered inflamation, which led him to discovery of Chrones Disease
This type of self-tracking is becoming more and more common
The Patient of the Future
MIT Technology Review, 2012

3. What if we could automate this sense making ability?
- and what if we could do this at scale?
… and do it efficiently and at scale

4. Sensing is a key enabler of the Internet of Things
50 Billion Things by 2020 (Cisco)
BUT, how do we make sense of the resulting avalanche
of sensor data?

5. People are good at making sense of sensory input
sense making based on human cognitive models
What can we learn from cognitive models of perception?
The key ingredient is prior knowledge

6. 1 2 Perception Cycle* Explanation Discrimination Prior Knowledge
Translating low-level signals into high-level knowledge
Explanation 1
Observe
Property
Perceive
Feature
perception cycle contains two primary phases
explanation
translating low-level signals into high-level abstractions
inference to the best explanation
discrimination
focusing attention on those properties that will help distinguish between multiple possible explanations
used to intelligently task sensors and collect additional observations (rather than brute force approach of blindly collecting all observations)
Prior Knowledge
Focusing attention on those aspects of the environment that provide useful information 2
Discrimination
* based on Neisser’s cognitive model of perception

7. To enable machine perception,
Semantic Web technology is used to integrate
sensor data with prior knowledge on the Web

8. The Web is becoming a
global knowledge base

9. W3C Semantic Sensor Network (SSN) Ontology
Prior knowledge on the Web
W3C Semantic Sensor Network (SSN) Ontology
Bi-partite Graph

10. W3C Semantic Sensor Network (SSN) Ontology
Prior knowledge on the Web
W3C Semantic Sensor Network (SSN) Ontology
Bi-partite Graph

11. 1 Explanation Explanation
Explanation is the act of choosing the objects or events that best account for a set of observations; often referred to as hypothesis building
Translating low-level signals into high-level knowledge
Explanation 1
Observe
Property
Perceive
Feature
perception cycle contains two primary phases
explanation
translating low-level signals into high-level abstractions
inference to the best explanation
discrimination
focusing attention on those properties that will help distinguish between multiple possible explanations
used to intelligently task sensors and collect additional observations (rather than brute force approach of blindly collecting all observations)

12. Explanation Inference to the best explanation
Explanation is the act of choosing the objects or events that best account for a set of observations; often referred to as hypothesis building
Inference to the best explanation
In general, explanation is an abductive problem; and hard to compute
Finding the sweet spot between abduction and OWL
Single-feature assumption* enables use of OWL-DL deductive reasoner
* An explanation must be a single feature which accounts for
all observed properties

13. Explanation
Explanatory Feature: a feature that explains the set of observed properties
ExplanatoryFeature ≡ ∃ssn:isPropertyOf—.{p1} ⊓ … ⊓ ∃ssn:isPropertyOf—.{pn}
Observed Property
Explanatory Feature
elevated blood pressure
Hypertension
clammy skin
Hyperthyroidism
palpitations
Pulmonary Edema

14. 2 Discrimination Explanation Discrimination
Discrimination is the act of finding those properties that, if observed, would help distinguish between multiple explanatory features
Explanation
Observe
Property
Perceive
Feature
perception cycle contains two primary phases
explanation
translating low-level signals into high-level abstractions
inference to the best explanation
discrimination
focusing attention on those properties that will help distinguish between multiple possible explanations
used to intelligently task sensors and collect additional observations (rather than brute force approach of blindly collecting all observations)
Focusing attention on those aspects of the environment that provide useful information 2
Discrimination

15. Discrimination
Expected Property: would be explained by every explanatory feature
ExpectedProperty ≡ ∃ssn:isPropertyOf.{f1} ⊓ … ⊓ ∃ssn:isPropertyOf.{fn}
Expected Property
Explanatory Feature
elevated blood pressure
Hypertension
clammy skin
Hyperthyroidism
palpitations
Pulmonary Edema

16. Discrimination
Not Applicable Property: would not be explained by any explanatory feature
NotApplicableProperty ≡ ¬∃ssn:isPropertyOf.{f1} ⊓ … ⊓ ¬∃ssn:isPropertyOf.{fn}
Not Applicable Property
Explanatory Feature
elevated blood pressure
Hypertension
clammy skin
Hyperthyroidism
palpitations
Pulmonary Edema

17. Discrimination
Discriminating Property: is neither expected nor not-applicable
DiscriminatingProperty ≡ ¬ExpectedProperty ⊓ ¬NotApplicableProperty
Discriminating Property
Explanatory Feature
elevated blood pressure
Hypertension
clammy skin
Hyperthyroidism
palpitations
Pulmonary Edema

18. kHealth: knowledge-enabled healthcare
Our Motivation
kHealth: knowledge-enabled healthcare
Through physical monitoring and analysis, our cellphones could act as an early warning system to detect serious health conditions
- With this ability, many problems could be solved
- For example: we could help solve health problems (before they become serious health problems) through monitoring symptoms and real-time sense making, acting as an early warning system to detect problematic health conditions
canary in a coal mine

19. How do we implement machine perception efficiently on a
resource-constrained device?
Use of OWL reasoner is resource intensive
(especially on resource-constrained devices),
in terms of both memory and time
Runs out of resources with prior knowledge >> 15 nodes
Asymptotic complexity: O(n3)
Intelligence distributed at the edge of the network
Requires resource-constrained devices (mobile phones, gateway notes, etc.) to be able to utilize SW technologies

20. intelligence at the edge
Approach 1: Send all sensor observations to the cloud for processing
Approach 2: downscale semantic processing so that each device is capable of machine perception
Intelligence distributed at the edge of the network
Requires resource-constrained devices (mobile phones, gateway notes, etc.) to be able to utilize SW technologies
intelligence at the edge

21. Efficient execution of machine perception
Use bit vector encodings and their operations to encode prior knowledge and execute semantic reasoning
- compute machine perception inferences -- i.e., explanation and discrimination -- of high-complexity on a resource-constrained devices in miliseconds

22. Lifting and lowering knowledge
Translate prior knowledge, observations, and explanations between SW and bit vector representation
lower
lift

23. Explanation: efficient algorithm
INTUITION: The strategy employed relies on the use of the bit vector AND operation to discover and dismiss those features that cannot explain the set of observed properties.
Observed
Property
Previous
Explanatory Feature
Current
Explanatory Feature
Prior Knowledge HN HM PE bp 1 cs pa HN HM PE 1 HN HM PE 1 bp 1 cs pa
AND
=> 1 1
AND
=>

24. Discrimination: efficient algorithm
INTUITION: The strategy employed relies on the use of the bit vector AND operation to discover and assemble those features that discriminate between the explanatory features
Observed
Property
Previous
Explanatory Feature
Current
Explanatory Feature
Discriminating
Property
Prior Knowledge HN HM PE bp 1 cs pa bp 1 cs pa HN HM PE 1 HN HM PE 1 bp cs pa 1
AND
=> 1 1 1
… expected? =
=> FALSE
Is the property
discriminating?
ZERO Bit Vector
… not-applicable? =
=> FALSE

25. Evaluation on a mobile device
Efficiency Improvement
Problem size increased from 10’s to 1000’s of nodes
Time reduced from minutes to milliseconds
Complexity growth reduced from polynomial to linear
O(n3) < x < O(n4)
O(n)

26. 1 Translate low-level data to high-level knowledge
3 ideas to takeaway
1 Translate low-level data to high-level knowledge
Machine perception can be used to convert low-level sensory signals into high-level knowledge useful for decision making
2 Prior knowledge is the key to perception
Using SW technologies, machine perception can be formalized and integrated with prior knowledge on the Web
3 Intelligence at the edge
By downscaling semantic inference, machine perception can execute efficiently on resource-constrained devices

27. Thank you. An Efficient Bit Vector Approach to Semantics-based
An Efficient Bit Vector Approach to Semantics-based
Machine Perception in Resource-constrained Devices
Cory Henson, Krishnaprasad Thirunarayan, Amit Sheth
Kno.e.sis – Ohio Center of Excellence in Knowledge-enabled Computing