Safely Crowd-Sourcing Critical Mass for a Self-Improving Human-Level Learner (”Seed AI”) Mark R. Waser Digital Wisdom Institute

Goal To safely create a human-level (or higher) artificial general intelligence (AGI)  Without debating what intelligence is  Without debating consciousness, emotions, etc.  Without constantly re-inventing the wheel Through self-improvement/learning

Self  Required for self-improvement  The complete loop of a process (or a physical entity) modifying itself  The mere fact of being self-referential causes a self, a soul, a consciousness, an “I” to arise out of mere matter (Hofstadter, I Am a Strange Loop)  Must, particularly if indeterminate in behavior, necessarily and sufficiently be defined an entity rather than an object  Humans innately tend to do this with the “pathetic fallacy”  Tri-partite  Physical hardware  “Personal” memory/knowledge base  Currently running processes

Learning  Learning is the functional integration of knowledge  A “learner” must be capable of integrating all acquired knowledge into its world model and skill portfolio to a sufficient extent that it is both immediately usable and can be built upon.  “Memorization” is NOT learning (data only – Watson)  Mere “algorithm execution” is NOT learning  *unless* it is self-modifying the algorithms recursively  “Discovery” is not necessary for learning  Although that is a skill that will be learned quickly enough

 Information integration theory (Tononi 2004) claims that consciousness is one and the same as a system’s capacity to integrate information  A “knowledge integrator”  incorporates knowledge into it’s world model  “understands” and therefore can predict  refactors knowledge and data for usability (/speed)  A word problem solver  and, eventually, an analogy builder Knowledge Integration

Critical Mass A reasonably achievable minimal set of initial cognitive and learning characteristics such that a learner starting anywhere above the critical knowledge will acquire the vital knowledge that a typical human learner would be able to acquire. Samsonovich 2011  Effectively, once a learner truly knows how to learn, it is capable of learning anything – subject, of course, to time and other constraints.  Thus, a learner above critical mass is a “seed AGI” fully capable of growing into a full-blown human-level (or, more likely, higher-level) artificial general intelligence.

Critical Components I: Self-Knowledge & Reflection  A self must know itself to be a self  Composed of three parts:  The running processes  The personal memory/knowledge base  The physical hardware  Must start with:  A competent model of each  Sensors to detect changes and their effects

Critical Components II: Explicit Goals  Do not defect from the community  Do not become too large/powerful  Acquire and integrate knowledge  Instrumental goals

Critical Components III: Reliability  Self-Control, Integrity, Autonomy, Responsibility  In “predictive control” of its own state and that of the physical objects that support it  Note: This is a marked deviation from the human example.

Humans are....  Evolved to self-deceive in order to better deceive others (Trivers 1991)  Unable to directly sense agency (Aarts et al. 2005)  Prone to false illusory experiences of self- authorship (Buehner and Humphreys 2009)  Unable to correctly retrieve the reasoning behind moral judgments (Hauser et al. 2007)  Almost always unaware of what morality is and why it should be practiced....

The Function of Morality “to suppress or regulate selfishness and make cooperative social life possible” J. Haidt & S. Kesebir Handbook of Social Psychology

Architecture Processes will be divided into three main classes:  Operating system processes  Subconscious/tool processes  One serial consciousness/learner process (CLP) The CLP will be able to create, modify and/or influence many of the subconscious/tool processes. The CLP will NOT be given access to modify operating system processes  Indeed, it will have multiple/redundant logical, emotional & moral reasons to seriously convince it not to even try

Operating System Architecture  Open, Pluggable, Service-Oriented/Message-Passing  Quickly adopt novel input streams  Handle resource requests and allocation  Provide connectivity between components  Safety Features  Act as a “black box” security monitor capable of reporting problems without the consciousness’s awareness  Able to “manage” the CLP by manipulating the amount of processor time and memory available to it (assuming that the normal subconscious processes are unable to do so)  Other protections against hostile humans, inept builders, and the learner itself may be implemented as well  Robot Operating System (ROS)

Automated Predictive World Model An active copy of the CLP’s world model  Is the most important subconscious process  Will serve as an interface to the “real” world  CLP effectively is a homunculous (subjective consciousness?)  Will be both reactive and predictive  Will generate “anomaly interrupts” upon deviations from expectations as an approach to solving the “brittleness” problem (Perlis 2008)  Will contain certain relatively immutable concepts to serve as anchors both for emotions and for ensuring safety (trigger patterns – Ohman et al. 2001)

Anchors & Emotions  Anchors create a multiple attachment point model which is much safer than the single-point-of-failure, top-down-only approach of “machine enslavement” advocated by the SIAI (Yudkowsky 2001)  Emotions will be generated by the subconscious processes as “actionable qualia” to inform the CLP and will also bias the selection and urgency tags of information relayed via the predictive model  Violations of the cooperative social living “moral” system will result in a flood of urgently–tagged anomaly interrupts demanding that consciousness resources be expended to “solve the problem”

Conscious Learning Process (CLP)  The goal is to provide as many optional structures and standards to support and speed development as much as possible while not restricting possibilities beyond what is absolutely required for safety.  We believe the best way to do this is with a blackboard system similar to Learning IDA (Baars and Franklin 2007).  The CLP acts like the Governing Board of the Policy Governance model (Carver 2006) to create a coherent, consistent, integrated narrative plan of action to fulfill the goals of the larger self.

A Social Media/Crowd-Sourcing Plan  Open Architecture  Open Source Operating System (ROS)  Pluggable Modules/Services/Interfaces  “Blackboard” Consciousness  Community-designed/vetted goals/morality  Community-designed/vetted anchor points  Emotions & Attention as safety mechanisms  Critical mass composed of reflection-capable mix & match components  Scripts to build from base to any level  Contests/Gamification to design & build

Safely Crowd-Sourcing Critical Mass for a Self-Improving Human-Level Learner (”Seed AI”) Mark R. Waser Digital Wisdom Institute