1. Consciousness and Cognition
EE141
Cognitive Architectures
Consciousness and Cognition
Janusz A. Starzyk

2. 2018/7/30
2018/7/30
Motivated Learning
Definition: Motivated learning (ML) is pain based motivation, goal creation and learning in embodied agent.
Machine creates abstract goals based on the primitive pain signals.
It receives internal rewards for satisfying its goals (both primitive and abstract).
ML applies to EI working in a hostile environment.
Various pains and external signals compete for attention.
Attention switching results from competition.
Cognitive perception is aided by winner of competition. 2 2

3. Primitive Goal Creation
2018/7/30
Primitive Goal Creation - +
Pain
Dry soil
Primitive
level
open
tank
sit on
garbage
refill
faucet
w. can
water
Dual pain
Reinforcing a proper action

4. Abstract Goal Hierarchy
2018/7/30
Abstract Goal Hierarchy - +
Dry soil
Primitive Level
Level I
Level II
faucet
w. can
open
water
Sensory pathway
(perception, sense)
Motor pathway
(action, reaction)
Level III
tank
refill
Abstract goals are created to reduce abstract pains and to satisfy the primitive goals
A hierarchy of abstract goals is created to satisfy the lower level goals
Activation
Stimulation
Inhibition
Reinforcement
Echo
Need
Expectation

5. Goal Creation Experiment in ML
MOTOR
FUNCTION
SENSOR OBJECT
REDUCES PAIN
INCREASES PAIN
Eat
Food
Hunger
Lack of Food
Buy
Food at
Grocery Store
Lack of Money
Withdraw from
Bank
Account
Overdrawn Account
Work in
The office
Lack of job opportunities
Study at
School -
Play
with
Toys

6. Goal Creation Experiment in ML
2018/7/30
Goal Creation Experiment in ML
Pain signals in CGS simulation

7. Goal Creation Experiment in ML
2018/7/30
Goal Creation Experiment in ML
Action scatters in 5 CGS simulations

8. Goal Creation Experiment in ML
2018/7/30
Goal Creation Experiment in ML
The average pain signals in 100 CGS simulations

9. Goal Creation Experiment in ML
2018/7/30
Goal Creation Experiment in ML
Comparison between GCS and RL

10. Compare RL (TDF) and ML (GCS)
2018/7/30
Compare RL (TDF) and ML (GCS)
Mean primitive pain Pp value as a function of the number of iterations:
- green line for TDF
blue line for GCS.
Primitive pain ratio with pain threshold 0.1

11. Compare RL (TDF) and ML (GCS)
2018/7/30
Compare RL (TDF) and ML (GCS)
Problem solved
Comparison of execution time on log-log scale
TD-Falcon green
GCS blue
Combined efficiency of GCS 1000 better than TDF
Conclusion: embodied intelligence, with motivated learning based on goal creation system, effectively integrates environment modeling and decision making – thus it is poised to cross the chasm

12. Reinforcement Learning Motivated Learning
2018/7/30
2018/7/30
Reinforcement Learning Motivated Learning
Single value function
Various objectives
Measurable rewards
Predictable
Objectives set by designer
Maximizes the reward
Potentially unstable
Action depends on the state of the environment
Learning effort increases with complexity
Always active
Multiple value functions
One for each goal
Internal rewards
Unpredictable
Sets its own objectives
Solves minimax problem
Always stable
Action depends on the states of the environment and agent
Learns better in complex environment than RL
Acts when needed 12 12

13. Primitive needs Water Reservoir Abstract Needs Wash in Water
Drink Water
Irrigate
Dirty
Thirsty
Drought
Primitive Needs

14. Abstract needs Well Public Money Draw own Water Spend Money to Buy
Spend Money to Build
Water
Reservoir
Abstract Needs
Wash in Water
Drink Water
Irrigate
Dirty
Thirsty
Drought
Primitive Needs

15. Build Water Recreation
Abstract needs
Ground Water
Well Building
Wealthy Taxpayers
Tourists' Attractions
Dig a Well
Rise Taxes
Build Ecotourism
Water Supply
Well
Public Money
Build Water Recreation
Draw own Water
Spend Money to Buy
Spend Money to Build
Water
Reservoir
Abstract Needs
Wash in Water
Drink Water
Irrigate
Dirty
Thirsty
Drought
Primitive Needs

16. Abstract needs Drought Reservoir Public Money Tourists' Attractions
Irrigate
Spend Money to Build
Build Ecotourism
Thirsty
Water
Drink Water
Spend Money to Buy
Primitive Needs
Build Water Recreation
Policy
Develop Infrastructure
Wealthy Taxpayers
Rise Taxes
Well
Draw own Water
Dirty
Wash in Water
Well Building
Dig a Well
Abstract Needs
Employment Opportunities
Ground Water
Water Supply
Receive Salary
Resource Management and Planning
Management
Regulate Use
Planning

17. NAC actions
All abstract pain neurons have a bias input B that depends on the state of the environment and the preference (bias) of the agent for or against a certain resource or action performed by other actors referred as NACs (Non-agent characters).
Thus observed bias triggers the pain.

18. Developing Trust
In motivated learning, trust is associated with NACs actions.
If most of the NAC’s actions are desirable (as when a parent takes care of his child, providing it with warmth and comfort) the ML agent’s trust towards the NAC’s actions increases.
Other character features like shyness may be related to overall experience with NAC agents.
If most NACs interactions hurt the ML agent it may develop mistrust to all NACs, and become shy.
If most NACs run away from the agent it may become fearless.
Trust can be computed from
where mi number of pain signal related to action nk

19. Developing Trust
In simulation we can show advantage of developing trust

20. Definition of Machine Consciousness
2018/7/30
Definition of Machine Consciousness
Consciousness is attention driven cognitive perception, motivations, thoughts, plans, and action monitoring.
A machine is conscious IFF besides ability to perceive, act, learn, and remember, it has a working memory (central executive) mechanism that controls all the processes (conscious or subconscious) of the machine;
Photo: 20

21. Consciousness: functional requirements
2018/7/30
Consciousness: functional requirements
Intelligence
Working memory
Attention and attention switching
Mental saccades
Cognitive perception
Cognitive action control
Photo: 21

22. Computational Model of Machine Consciousness
2018/7/30
Computational Model of Machine Consciousness
Semantic memory
Sensory processors
Data encoders/ decoders
Sensory units
Motor skills
Motor processors
Motor units
Emotions, rewards, and sub-cortical processing
Attention switching
Action monitoring
Motivation and goal processor
Planning and thinking
Episodic memory
Queuing and organization of episodes
Episodic Memory & Learning
Working Memory
Sensory-motor
Inspiration: human brain
Photo (brain):

23. Sensory and Motor Hierarchies
Sensory and motor systems appear to be arranged in hierarchies with information flowing between each level of the sensory and motor hierarchies. 23 23

24. 2018/7/30
Sensory- Motor Block
Semantic memory
Sensory processors
Data encoders/ decoders
Sensory units
Motor skills
Motor processors
Motor units
Emotions, rewards, and sub-cortical processing
Sensory-motor
sensory processors integrated with semantic memory
motor processors integrated with motor skills
sub-cortical processors integrated with emotions and rewards

25. Working Memory Platform for the emergence of consciousness
2018/7/30
Working Memory
Platform for the emergence of consciousness
Controls its conscious and subconscious processes
Working memory is driven by
attention switching
learning mechanism
creation and selection of
motivations and goals
ahsmail.uwaterloo.ca/kin356/cexec/cexec.htm

26. Motivation and goal processor
2018/7/30
Working Memory
Attention switching
Action monitoring
Motivation and goal processor
Planning and thinking
Working Memory
Tasks
cognitive perception
attention
attention switching
motivation
goal creation and selection
thoughts
planning
learning, etc.

27. Motivation and goal processor
2018/7/30
Working Memory
Attention switching
Action monitoring
Motivation and goal processor
Planning and thinking
Working Memory
Interacts with other units for
performing its tasks
gathering data
giving directions to other units
No clearly identified decision center
Decisions are influenced by
competing signals representing
motivations, pains, desires, plans, and interrupt signals
need not be cognitive or consciously realized
competition can be interrupted by attention switching signal

28. Attention Switching !!! Attention Attention switching
2018/7/30
Attention Switching !!!
Attention
is a selective process of cognitive perception, action and other cognitive experiences like
thoughts, action planning, expectations, dreams
Attention switching
leads to sequences of cognitive experiences
Comic:

29. 2018/7/30
Attention Switching !!!
Dynamic process resulting from competition between
motivations
sensory inputs
internal thoughts
blog.gigoo.org/.../

30. Attention Switching !!! May be a result of :
2018/7/30
Attention Switching !!!
May be a result of :
deliberate cognitive experience (and thus fully conscious)
subconscious process (stimulated by internal or external signals)
Thus, while paying attention is a conscious experience,
switching attention does not have to be. 30

31. Write to episodic memory
Simplified Cognitive Machine
Formulate episode
Saccade control
Changing perception
Changing environment
Advancement
of a goal?
Yes No
Action control
Changing motivation
Write to episodic memory
Loop 1
Loop 2
Attention spotlight
Associative memory
From virtual game

32. Visual Saccades
Input image A B C D
What
Where

33. 2018/7/30
Mental Saccades
Selected part of the image resulting from an eye saccade.
Perceived input activates object recognition and associated areas of semantic and episodic memory.
This in turn activates memory traces in the global workspace area that will be used for mental searches (mental saccades). 33

34. Mental saccades in a conscious machine
2018/7/30
Mental saccades in a conscious machine
Perceptual saccades
Changing perception
Changing environment
Associative memory No
Action control
Loop 5
Loop 2
Advancement
of a goal?
Yes
Learning
Attention spotlight
Mental saccades
Continue
search?
Loop 1
Plan action?
Action?
Changing motivation
Loop 3
Loop 4 34

35. Comprehensive Cognitive Model
Proposed cognitive system organization
Contains
Semantic, episodic and procedural memories.
WTA attention switching
Visual and mental saccades
Scene building
Action planning
And more…
Figure represents our top-level design model

36. Computational Model: Summary
2018/7/30
Computational Model: Summary
Self-organizing mechanism of emerging motivations and other signals competing for attention is fundamental for conscious machines.
A central executive controls conscious and subconscious processes driven by its attention switching mechanism.
Attention switching is a dynamic process resulting from competition between representations, sensory inputs and internal thoughts
Mental saccades of the working memory are fundamental for cognitive thinking, attention switching, planning, and action monitoring
Photo: 36

37. Computational Model: Implications
2018/7/30
Computational Model: Implications
Motivations for actions are physically distributed
competing signals are generated in various parts of machine’s mind
Before a winner is selected, machine does not interpret the meaning of the competing signals
Cognitive processing is predominantly sequential
winner of the internal competition is an instantaneous director of the cognitive thought process, before it is replaced by another winner
Top down activation for perception, planning, internal thought or motor functions
results in conscious experience
decision of what is observed and where is it
planning how to respond
a train of such experiences constitutes consciousness

38. Neoaxis Implementation VIDEO
2018/7/30
NeoAxis Simulation
Neoaxis Implementation VIDEO

39. Conclusions Consciousness is computational
2018/7/30
Conclusions
Consciousness is computational
Intelligent machines can be conscious 39