Consciousness and Cognition EE141 Cognitive Architectures Consciousness and Cognition Janusz A. Starzyk

2018/7/30 2018/7/30 Motivated Learning Definition: Motivated learning (ML) is pain based motivation, goal creation and learning in embodied agent. Machine creates abstract goals based on the primitive pain signals. It receives internal rewards for satisfying its goals (both primitive and abstract). ML applies to EI working in a hostile environment. Various pains and external signals compete for attention. Attention switching results from competition. Cognitive perception is aided by winner of competition. 2 2

Primitive Goal Creation 2018/7/30 Primitive Goal Creation - + Pain Dry soil Primitive level open tank sit on garbage refill faucet w. can water Dual pain Reinforcing a proper action

Abstract Goal Hierarchy 2018/7/30 Abstract Goal Hierarchy - + Dry soil Primitive Level Level I Level II faucet w. can open water Sensory pathway (perception, sense) Motor pathway (action, reaction) Level III tank refill Abstract goals are created to reduce abstract pains and to satisfy the primitive goals A hierarchy of abstract goals is created to satisfy the lower level goals Activation Stimulation Inhibition Reinforcement Echo Need Expectation

Goal Creation Experiment in ML MOTOR FUNCTION SENSOR OBJECT REDUCES PAIN INCREASES PAIN Eat Food Hunger Lack of Food Buy Food at Grocery Store Lack of Money Withdraw from Bank Account Overdrawn Account Work in The office Lack of job opportunities Study at School - Play with Toys

Goal Creation Experiment in ML 2018/7/30 Goal Creation Experiment in ML Pain signals in CGS simulation

Goal Creation Experiment in ML 2018/7/30 Goal Creation Experiment in ML Action scatters in 5 CGS simulations

Goal Creation Experiment in ML 2018/7/30 Goal Creation Experiment in ML The average pain signals in 100 CGS simulations

Goal Creation Experiment in ML 2018/7/30 Goal Creation Experiment in ML Comparison between GCS and RL

Compare RL (TDF) and ML (GCS) 2018/7/30 Compare RL (TDF) and ML (GCS) Mean primitive pain Pp value as a function of the number of iterations: - green line for TDF blue line for GCS. Primitive pain ratio with pain threshold 0.1

Compare RL (TDF) and ML (GCS) 2018/7/30 Compare RL (TDF) and ML (GCS) Problem solved Comparison of execution time on log-log scale TD-Falcon green GCS blue Combined efficiency of GCS 1000 better than TDF Conclusion: embodied intelligence, with motivated learning based on goal creation system, effectively integrates environment modeling and decision making – thus it is poised to cross the chasm

Reinforcement Learning Motivated Learning 2018/7/30 2018/7/30 Reinforcement Learning Motivated Learning Single value function Various objectives Measurable rewards Predictable Objectives set by designer Maximizes the reward Potentially unstable Action depends on the state of the environment Learning effort increases with complexity Always active Multiple value functions One for each goal Internal rewards Unpredictable Sets its own objectives Solves minimax problem Always stable Action depends on the states of the environment and agent Learns better in complex environment than RL Acts when needed http://www.bradfordvts.co.uk/images/goal.jpg 12 12

Primitive needs Water Reservoir Abstract Needs Wash in Water Drink Water Irrigate Dirty Thirsty Drought Primitive Needs

Abstract needs Well Public Money Draw own Water Spend Money to Buy Spend Money to Build Water Reservoir Abstract Needs Wash in Water Drink Water Irrigate Dirty Thirsty Drought Primitive Needs

Build Water Recreation Abstract needs Ground Water Well Building Wealthy Taxpayers Tourists' Attractions Dig a Well Rise Taxes Build Ecotourism Water Supply Well Public Money Build Water Recreation Draw own Water Spend Money to Buy Spend Money to Build Water Reservoir Abstract Needs Wash in Water Drink Water Irrigate Dirty Thirsty Drought Primitive Needs

Abstract needs Drought Reservoir Public Money Tourists' Attractions Irrigate Spend Money to Build Build Ecotourism Thirsty Water Drink Water Spend Money to Buy Primitive Needs Build Water Recreation Policy Develop Infrastructure Wealthy Taxpayers Rise Taxes Well Draw own Water Dirty Wash in Water Well Building Dig a Well Abstract Needs Employment Opportunities Ground Water Water Supply Receive Salary Resource Management and Planning Management Regulate Use Planning

NAC actions All abstract pain neurons have a bias input B that depends on the state of the environment and the preference (bias) of the agent for or against a certain resource or action performed by other actors referred as NACs (Non-agent characters). Thus observed bias triggers the pain.

Developing Trust In motivated learning, trust is associated with NACs actions. If most of the NAC’s actions are desirable (as when a parent takes care of his child, providing it with warmth and comfort) the ML agent’s trust towards the NAC’s actions increases. Other character features like shyness may be related to overall experience with NAC agents. If most NACs interactions hurt the ML agent it may develop mistrust to all NACs, and become shy. If most NACs run away from the agent it may become fearless. Trust can be computed from where mi number of pain signal related to action nk

Developing Trust In simulation we can show advantage of developing trust

Definition of Machine Consciousness 2018/7/30 Definition of Machine Consciousness Consciousness is attention driven cognitive perception, motivations, thoughts, plans, and action monitoring. A machine is conscious IFF besides ability to perceive, act, learn, and remember, it has a working memory (central executive) mechanism that controls all the processes (conscious or subconscious) of the machine; http://hplusmagazine.com/sites/default Photo: www.spectrum.ieee.org/.../biorobot11f-thumb.jpg 20

Consciousness: functional requirements 2018/7/30 Consciousness: functional requirements Intelligence Working memory Attention and attention switching Mental saccades Cognitive perception Cognitive action control Photo: http://eduspaces.net/csessums/weblog/11712.html http://faculty.virginia.edu/consciousness 21

Computational Model of Machine Consciousness 2018/7/30 Computational Model of Machine Consciousness Semantic memory Sensory processors Data encoders/ decoders Sensory units Motor skills Motor processors Motor units Emotions, rewards, and sub-cortical processing Attention switching Action monitoring Motivation and goal processor Planning and thinking Episodic memory Queuing and organization of episodes Episodic Memory & Learning Working Memory Sensory-motor Inspiration: human brain Photo (brain): http://www.scholarpedia.org/article/Neuronal_correlates_of_consciousness

Sensory and Motor Hierarchies Sensory and motor systems appear to be arranged in hierarchies with information flowing between each level of the sensory and motor hierarchies. 23 23

2018/7/30 Sensory- Motor Block Semantic memory Sensory processors Data encoders/ decoders Sensory units Motor skills Motor processors Motor units Emotions, rewards, and sub-cortical processing Sensory-motor http://www.ourbabynews.com/wp-content sensory processors integrated with semantic memory motor processors integrated with motor skills sub-cortical processors integrated with emotions and rewards

Working Memory Platform for the emergence of consciousness 2018/7/30 Working Memory Platform for the emergence of consciousness Controls its conscious and subconscious processes Working memory is driven by attention switching learning mechanism creation and selection of motivations and goals http://www.unifesp.br/dpsicobio/eventos/workingmemory/ ahsmail.uwaterloo.ca/kin356/cexec/cexec.htm

Motivation and goal processor 2018/7/30 Working Memory Attention switching Action monitoring Motivation and goal processor Planning and thinking Working Memory Tasks cognitive perception attention attention switching motivation goal creation and selection thoughts planning learning, etc. http://prodinstres.pbworks.com

Motivation and goal processor 2018/7/30 Working Memory Attention switching Action monitoring Motivation and goal processor Planning and thinking Working Memory Interacts with other units for performing its tasks gathering data giving directions to other units No clearly identified decision center Decisions are influenced by competing signals representing motivations, pains, desires, plans, and interrupt signals need not be cognitive or consciously realized competition can be interrupted by attention switching signal http://www.resourceroom.net/

Attention Switching !!! Attention Attention switching 2018/7/30 Attention Switching !!! http://www.mukyaa.com Attention is a selective process of cognitive perception, action and other cognitive experiences like thoughts, action planning, expectations, dreams Attention switching leads to sequences of cognitive experiences http://brandirons.com/ Comic: http://lonewolflibrarian.wordpress.com/2009/08/05/attention-and-distraction-what-are-you-paying-attention-to-08-05-09/

2018/7/30 Attention Switching !!! Dynamic process resulting from competition between motivations sensory inputs internal thoughts blog.gigoo.org/.../ http://www.cs.miami.edu

Attention Switching !!! May be a result of : 2018/7/30 Attention Switching !!! May be a result of : deliberate cognitive experience (and thus fully conscious) subconscious process (stimulated by internal or external signals) Thus, while paying attention is a conscious experience, switching attention does not have to be. 30

Write to episodic memory Simplified Cognitive Machine Formulate episode Saccade control Changing perception Changing environment Advancement of a goal? Yes No Action control Changing motivation Write to episodic memory Loop 1 Loop 2 Attention spotlight Associative memory From virtual game

Visual Saccades Input image A B C D What Where

2018/7/30 Mental Saccades Selected part of the image resulting from an eye saccade. Perceived input activates object recognition and associated areas of semantic and episodic memory. This in turn activates memory traces in the global workspace area that will be used for mental searches (mental saccades). 33

Mental saccades in a conscious machine 2018/7/30 Mental saccades in a conscious machine Perceptual saccades Changing perception Changing environment Associative memory No Action control Loop 5 Loop 2 Advancement of a goal? Yes Learning Attention spotlight Mental saccades Continue search? Loop 1 Plan action? Action? Changing motivation Loop 3 Loop 4 http://cdn-3.lifehack.org/wp-content 34

Comprehensive Cognitive Model Proposed cognitive system organization Contains Semantic, episodic and procedural memories. WTA attention switching Visual and mental saccades Scene building Action planning And more… Figure represents our top-level design model

Computational Model: Summary 2018/7/30 Computational Model: Summary Self-organizing mechanism of emerging motivations and other signals competing for attention is fundamental for conscious machines. A central executive controls conscious and subconscious processes driven by its attention switching mechanism. Attention switching is a dynamic process resulting from competition between representations, sensory inputs and internal thoughts Mental saccades of the working memory are fundamental for cognitive thinking, attention switching, planning, and action monitoring Photo: http://www.prlog.org/10313829-homeless-man-earns-250000-after-viewing-prosperity-consciousness-video-subliminal-mind-training.html 36

Computational Model: Implications 2018/7/30 Computational Model: Implications Motivations for actions are physically distributed competing signals are generated in various parts of machine’s mind Before a winner is selected, machine does not interpret the meaning of the competing signals Cognitive processing is predominantly sequential winner of the internal competition is an instantaneous director of the cognitive thought process, before it is replaced by another winner Top down activation for perception, planning, internal thought or motor functions results in conscious experience decision of what is observed and where is it planning how to respond a train of such experiences constitutes consciousness

Neoaxis Implementation VIDEO 2018/7/30 NeoAxis Simulation Neoaxis Implementation VIDEO https://www.youtube.com/watch?feature=player_embedded&v=nhXXZgVY67E&hd=1

Conclusions Consciousness is computational 2018/7/30 Conclusions Consciousness is computational Intelligent machines can be conscious 39