2SummaryThis is a project-based robotics course, containing both introductory as well as more advanced concepts. It presents a broad overview of robotic manipulation and also focuses on fundamentals such as robot kinematics, dynamics, control and planning. This year, the course will focus on concepts in support of major competitions. The course is divided between two areas:Robot KinematicsRotations, Translations and Homogeneous TransformationsParameterizations and SingularitiesRobot kinematics and the JacobianRobot staticsMulti-fingered graspingNonholonomic and parellel robotsRobot Dynamics and ControlRobot dynamical modelsComputed Torque ControlAdvanced Robot ControlKinematic Control of Nonholonomic RobotsTrajectory PlanningForce Control and Physical Human Robot InteractionVisual Servoing and Visual Human Robot Interaction
3Tentative Syllabus Week 1 - Jan 20, 22, Lectures 1,2 Intro to robotics: History of robotics ; Examples of robotsRobotics Jargon ; Course ProjectsWeek 2 - Jan 27, 29 Lectures 3,4Review of basics: Matrix algebraIntro to Robot Kinematics: Geometric issues ; Frames ; NotationWeek 3 - Feb 3, 5, Lectures 5,6Homogeneous Transforms, Parameterizations and SingularitiesWeek 4 -Feb 10, 12, Lectures 7,8Robot Kinematics : DH framework ; Forward Kinematics ; Inverse KinematicsDiscussion on other Kinematic Representations
4Tentative Syllabus Week 5 - Feb 17, 19, Lectures 9, 10 Robot Manipulability and Singularities; StaticsWeek 6 - Feb 24, 26, Lectures 11, 12Manipulability, redundancy, parallel robots.Week 7 - March 3, 5, Lectures 13, 14Robot statics, compliance, stiffnessRobot grasping and multi-fingered manipulationWeek 8 - March 9-13Spring BreakWeek 9 - March 17, 19, Lectures 15, 16Nonholonomic constraints and wheeled mobile robots.
5Tentative Syllabus Week 10 - March 24, 26, Lectures 17, 18 Robot dynamics: Euler-Lagrange formulationRobot dynamics: Newton Euler formulationMidterm (take-home) handed out March 26, due March 31 will cover kinematics and staticsWeek 11 - March 31, April 2, Lectures 19, 20Examples of dynamical equationsProperties of robot dynamic equationsWeek 12 -April 7, 9 Lectures 21, 22Basic robot control, modeling and control of joints.Concepts in stability, passivity with applications to robot control.Week 13 - April 14, 16, Lectures 22, 23Computed torque control.Trajectory generation and following for manipulators.
6Tentative Syllabus Week 14 - April 21, 23, Lectures 24, 25 Robot Control using Lyapunov TheoryRobust and advanced control for robot manipulators.Week 15 - April 28, 30, Lectures 26, 27Introduction to Force control.Introduction to robot vision and visual servo controlWeek 16 - May 5, 7,Project Presentations, Project dueWeek 17 - May 12Final exam (in-class, comprehensive).
7TextbooksRobot Modeling and Control, by M.W. Spong, S. Hutchinson, M. Vidyasagar (2005) (required)Mathematical Introduction to Robotic Manipulation by Richard M. Murray, Zexiang Li, S. Shankar Sastry (free download at link)Robot Manipulator Control: Theory and Practice (Control Engineering, 15) by Frank L. Lewis, et al (on reserve)Introduction to Robotics: Mechanics and Control (3rd Edition) by John J. Craig (on reserve)Autonomous Mobile Robots, by S. S. Ge, F. L. Lewis (on reserve).Robotics: Modelling, Planning and Control (Advanced Textbooks in Control and Signal Processing) [Hardcover], Bruno Siciliano, Lorenzo Sciavicco, Luigi Villani, Giuseppe Oriolo ( Publisher: Springer; 2nd Printing. edition (February 11, 2011), ISBN-10: , ISBN-13:MATLAB Robotics Toolbox by Peter Corke, free downloadROS and GAZEBO simulation environments, by Willow Garage and OSRF, free download
8Course Refresher Reading Assignments Math: linear/matrix algebra, trigonometry, differential equations.Controls: stability, linear & nonlinear systems.Physics: mechanics, force & motion.Programming: MATLAB, ROSReading assignments will be posted online. Assigned reading material may be useful for certain exam problems.
9Honor CodeMissed deadlines for take-home exam and project proposal and report: Maximum grade drops 10% per late dayAcademic Dishonesty will not be tolerated. All exams and reports are individual assignments. Your take-home exam and reports will be carefully scrutinized to ensure a fair grade for everyone.Attendance and Drop Policy: Attendance is not mandatory. However, if you skip classes, you will find the homework and exams more difficult. Assignments are going to be posted here, however, due to the pace of the lectures, copying someone else's notes may be an unreliable way of making up an absence. You are responsible for all material covered in class regardless of absences.
10Exams and Class Projects Midterm, final, individual work only.Project – team project only.Course project – A list of topics will be provided. Stop by and discuss the class project with me during office hours early in the semester. The project work is collaborative. It can be applied or can involve programming.IEEEAmazon Picking ChallengeMobile Microrobotics ChallengeHumanitarian Robotics ChallengeOther possible projects: Prosthetics, Humanoid Robots, and other robots at UTARI and NGS
11Research Credits: Dr. Woo Ho Lee Humanoid RobotsResearchCredits: Dr. Woo Ho Lee
12ATLAS Biped humanoid robot (28DOF) Height: 1.88m Weight: 150kg Hydraulic actuationCompany: Boston Dynamics (purchased by Google)Government Furnished Equipment for DARPA Robotics ChallengeSensors: LIDAR, stereo sensors
13THOR Biped humanoid robot (34DOF) Open source platform: Available for purchase after DRCInstitution: Virginia TechDevelop for DARPA Robotics Challenge
14iCub Humanoid robot for human cognition and AI 53DOF including 18 DOF handsPrice: 250K EurosHeight: 1m, Weight: 22kgOpen source humanoid robot platformCAD, software, and BOMInstitution: IIT funded by EUAdopted by 20 laboratories worldwide. 25 iCubs in Europe, USA, …
15Asimo Biped humanoid robot (34DOF) Lease price: $150K / month Height: 1.3m, Weight: 54kgOperation time: 1hourManufacturer: Honda in JapanMarket: Research and education
16Hubo II Biped humanoid robot (38DOF) Price: $400K Height: 1.3m, Weight: 50kgTwo embedded computers1 for real time control1 for communication and application programs (Window XP)Company:Rainbow Co (KAIST spin-off) in KoreaMarket: Research and educationUsed at DARPA Robotics Challenge
17HRP-4 Biped humanoid robot (34DOF) Price: $300K Height: 1.5m Weight: 39kgOperation: 34 minuteOS: LinuxManufacturer: Kawada industries in Japan
18Nao Humanoid robot (25DOF) Price: $16K Open source software Used in the Robot Soccer World CupOver 200 academic institutionsCompany:Aldebaran Robotics in FranceFounded in 2005Market:EducationResearch
19DARWIN Open platform Humanoid robot Price: $10K 20 DOFs Used in the RoboCup.Company:Robotis in KoreaFounded in 1999Market:EducationToy
20Zeno Facial expression humanoid robot (36DOF including 11DOF Head) Frubber robotic skinPrice: $17.5KCompany:Hanson RoboticsFounded in 2003Market: Education & entertainment
21PR2 Personal robot Open source software: ROS Price: $400K (~50 robots sale)Company:Willow GarageFunded by founder. Seek a self sustaining business modelFounded in 2006Market: Education
23Personal Robotics Affordable Applications TelepresenseEducationEntertainmentGlobal personal robotics market$18B in 2015 [Source: Grishin Robotics]
24Double iPad equipped teleconferencing robot Price: $1999 (preorder) Company:Double RoboticsSeed fund from Y Combinator$250K funding from Grishin RoboticsFounded in 20116 manMarket: TelepresenceHospitals, schools, retail stores, museums900+ Pre-orders
25SpheroRobot Ball You Control With Your Smartphone (iPhone and Android)Price: $129Company:OrbotixSeed fund from TechStars$15M fundingFounded in 201035 peopleBluetooth communicationMarket: EntertainmentPresident Obama’s visit
26Romo Robot with an iPhone for brain Price: $149 Company: RomotiveSeed fund from TechStars$6.5M fundingFounded in 201116 peopleBluetooth communicationMarket: EntertainmentTelepresence, Autonomous navigation
27Cubelets Robot construction kit Reconfigurable robots Price: $160 - $520CompanyModular Robotics$3.5M fundingFounded in 20084 peopleMarket: Education
28mOwayduino Arduino based robot Programmable with Arduino IED Startup in SpainFund: indiegogoMarket: Education and Toy
29RK-1 Arduino based mobile robot Arduino + iOS/Android Price: £175 Company: My Mobile RobotsFund: KickstarterOne man company in LondonMarket: Entertainment
30Disciplines in EE Controls and Robotics Robotics – examples of projectsSystems Approach and Related Concepts -Feedback Control BasicsFeedback Control HistoryRobotics Basics and HistoryRobotics Projects at NGS
31Signals and Systems Signal: System: Any time dependent physical quantityElectrical, Optical, MechanicalSystem:Object in which input signals interact to produce output signals.Some have fundamental properties that make linear systems predictable:Sinusoid in, sinusoid out of same frequency (when transients settle)Double the amplitude in, double the amplitude out (when initial state conditions are zero)
32System ModelingBuilding mathematical models based on observed data, or other insight for the system.Parametric models (analytical): ODE, PDENon-parametric models: graphical models - plots, look-up cause-effect tablesMental models – Driving a car and using the cause-effect knowledgeSimulation models – Many interconnect subroutines, objects in video game
33Types of Models White Box Black Box Gray Box – combination of the two derived from first principles laws: physical, chemical, biological, economical, etc.Examples: RLC circuits, MSD mechanical models (electromechanical system models).Black Boxmodel is entirely derived from measured dataExample: regression (data fit)Gray Box – combination of the two
34White Box Systems: Electrical Defined by Electro-Magnetic Laws of Physics: Ohm’s Law, Kirchoff’s Laws, Maxwell’s EquationsExample: Resistor, Capacitor, Inductor
35RLC Circuit as a SystemKirchoff’s Voltage Law (KVL):
36Linear Time-Invariant Models Continuous-time linear dynamical system (LDSC) has the formdx/dt= A(t)x(t) + B(t)u(t),y(t) = C(t)x(t) + D(t)u(t)where:t R denotes timex(t) Rn is the state (vector)u(t) Rm is the input or controly(t) Rp is the output
37Linear System Description in Frequency Domain Purpose of Frequency Domain Analysis:Convert Differential equations into Algebraic EquationsInterconnect systems using block diagramsUse graphical tools to discover of influence behavior of systems
38EE-Specific Diagrams Block Diagram Model: Helps understand flow of information (signals) through a complex systemHelps visualize I/O dependenciesEquivalent to a set of linear algebraic equations.Based on a set of primitives:Transfer Function Summer/Difference Pick-off pointUU2+U1U1+U2UU+
40Automatic ControlControl: process of making a system variable converge to a reference valueIf r=ref_value=changing - servo (tracking control)If r=ref_value=constant - regulation (stabilization)Open loop vs. closed loop (feedback) control+ry+yControllerK(s)PlantG(s)ControllerK(s)+PlantG(s)r-Sensor GainH(s)
41Brief History of Feedback Control The key developments in the history of mankind that affected the progress of feedback control were:1. The preoccupation of the Greeks and Arabs with keeping accurate track of time. This represents a period from about 300 BC to about 1200 AD. (Primitive period of AC)2. The Industrial Revolution in Europe, and its roots that can be traced back into the 1600's. (Primitive period of AC)3. The beginning of mass communication and the First and Second World Wars. (1910 to 1945). (Classical Period of AC)4. The beginning of the space/computer age in (Modern Period of AC).
42Primitive Period of ACFloat Valve for tank level regulators Drebbel incubator furnace control (1620)(antiquity)
43Primitive Period of ACJames Watt Fly-Ball Governor For regulating steam engine speed (late 1700’s)
44Classical Period of ACStability Analysis: Maxwell, Routh, Hurwitz, Lyapunov (before 1900).Electronic Feedback Amplifiers with Gain for long distance communications (Black, 1927)Stability analysis in frequency domain using Nyquist criterion (1932), Bode Plots (1945).PID controller (Callender, 1936) – servomechanism controlRoot Locus (Evans, 1948) – aircraft controlMost of the advances were done in Frequency Domain.
45Modern Period of AC Time domain analysis (state-space) Bellmann, Kalman: linear systems (1960)Pontryagin: Nonlinear systems (1960) – IFACOptimal controlsH-infinity control (Doyle, Francis, 1980’s) – loop shaping (in frequency domain).MATLAB (1980’s to present) has implemented math behind most control methods.
46Feedback Control Role of feedback: Reduce sensitivity to system parameters (robustness)Disturbance rejectionTrack desired inputs with reduced steady state errors, overshoot, rise time, settling time (performance)Systematic approach to analysis and designSelect controller based on desired characteristicsPredict system response to some inputSpeed of response (e.g., adjust to workload changes)Approaches to assessing stability
47Feedback System Block Diagram Temperature control system
48Feedback System Block Diagrams Automobile Cruise Control
49Key Transfer Functions Reference+SControllerPlant–TransducerJlh: Can we make the diagram bigger and the title smaller?
51Effect of pole locations Oscillations(higher-freq)Im(s)Faster DecayFaster BlowupRe(s)(e-at)(eat)Negative feedbackPole at -1/A (stable)Positive feedbackPole at 1/A (unstable)
52Basic Control Actions: u(t) Jlh: Can we give more intuition on control actions. This seems real brief considering its importance.
53Summary of Basic Control Proportional controlMultiply e(t) by a constantPI controlMultiply e(t) and its integral by separate constantsAvoids bias for stepPD controlMultiply e(t) and its derivative by separate constantsAdjust more rapidly to changesPID controlMultiply e(t), its derivative and its integral by separate constantsReduce bias and react quickly
54Conclusion: Control Systems Abstraction is the basis for system level thinking. Abstraction requires advanced mathematics, and it is especially required of Electrical and Computer Engineers.Control Theory contains abstractions and generalizations able to guarantee predictable performance of systems under control.Negative feedback offers numerous advantages: noise rejection, robustness to plant variations, dynamical tracking performance.Examples of popular control schemes include Proportional-Integral-Derivative (PID) schemes.Modern control is primarily based on time-domain analysis of state-equations using matrices.Control engineers can find jobs in any industry. Control concepts can be applied in any engineering industry.
55Robots as Complex Systems Controlled by Feedback G. Bekey definition: an entity that can sense, think and act.Classification: manipulators, mobile robots, mobile manipulators.SenseThinkActRobot
56Robot Subsystems A mechanical structure. For manipulators this structure consists of a set of rigid bodies (links), connected by means of articulations (joints). Links and joints can also be described in terms of an arm (for mobility), a wrist (for dexterity) and an end-effector (for performing the task).For mobile robots, the structure consists of a chassis with a locomotion mechanism, in the form of legs, wheels, rotor blades, etc.Actuators. These set the robot in motion through actuation of its joints, and are typical electric or hydraulic.Sensors. These measure the status of the manipulator (propriceptive sensors) and the status of the environment (heteroceptive sensors).A control system. This enables control and supervision of the robot, and is usually a computer with a graphical user interface, a pendant, or a remote controller.
57Recent Developments in Robot Software Platforms for Standardization 2009- Willow Garage Launches Robot Operating System (ROS)2011- National Instruments Introduces the Labview Robotics Modulewe sense several inputs we need to do something with themwe need to make use of the inputs information that we have to generate meaningful outputso we need some kind of coordination between interpretation of input to the action of output actuatorit’s called “Orchestrator”Orhcestration means:……………………..
58IEEE Robotics Societies and Competitions Websites to watch:IEEE Robotics & Automation SocietyJoin it for $15Check out events and information at:IEEE Region 5 Robotics CompetitionBoth a paper and a robot course contest.For next year (2014)Mobile Microrobotics IEEE ICRACompete with the world’s smallest robots
59Conclusion: RoboticsRobotics uses advanced concepts in control to connect sensors with actuators.Robots can be classified as manipulators (e.g. robotic arms), mobile robots, mobile manipulators.Major disciplines in robotics are: kinematics, dynamics, planning, control, perception, and cognition.Robotics is a multidisciplinary field, including computer scientists, mechanical engineers, electrical engineering, industrial engineers, etc.
60Research in Multiscale Robotics at Next Gen Systems (NGS) Group Tools and FundamentalsEstablished TechnologiesEmerging TechnologiesRoboticsControl SystemsManufacturing & AutomationMicrosystems & MEMSNanotechnologyBiotechnologyModeling & SimulationControl TheoryAlgorithmsAssistive RobotsHuman-like robotsDistributed and wireless sensor systemsSmall-scaleRobotics &ManufacturingMicromanufacturingMicroroboticsMicroassemblyMicropackagingSensor & Actuator ArraysNanoManufacturingNew applicationsfor robot systems
62New Initiatives and Projects at NGS and UTA Research Institute NSF Projects in Assistive technologies, including robots for helping children with Cerebral Palsy w/ CSE Dept. (2009-).UTA-UNTHSC joint project to study robot therapy and early diagnosys of Autism Spectrum Disorders – TXMRC consortium (2011-).UTARI Consortium in Assistive Robotics (2012-) in collaboration with Qinetiq-North America.UTARI Participation in DARPA Robotics Challenge in collaboration with RESquared, Inc. (Oct. 2012).New NSF Project through the National Robotics Initiative aiming at creating the next generation Robotic Skin (Oct 2012) for co-Robots.Plans under way for M.S. Degree program in UAS and Robotics at UTA’s College of Engineering (2013).
63Wafer Scale Microfactory (Micro-Nano) “From a few robots+controllers to many µrobots via assembly and die bonding”µparts, nparts inµcontroller IC diesAssembliesoutµrobot MEMS diesController + Robotµparts, nparts in
65Making the Microfactory by Automated 3D Microassembly Larger number of robotsMeasurement uncertainty, measurement range,Time delaysFewer embedded sensors, low SNRManufacturing uncertainty, inacurate robot models)Environmental effects (stiction, temperature)Control Challenges:
66Mobile Microbotics Challenge 2011 Hosted at IEEE International Conference on Robotics and Automation, Shanghai, China, May 10, 2011.7 Qualified Teams: France (FEMTO-ST), Italy (IIT), Univ. of Waterloo (CA), 4 US Universities (Stevens, Hawaii, Maryland, UTAMaximum robot size: 600 microns sphere.UTA Microrobots, 2011MobilityChallengeMicroAssemblyEventVibration and Laser Actuated
67UTA Vibot Control Using National Instruments PXI-8196 Microrobot pose (x, y, θ) from NI-1742 Smart CameraExchange of pose data with the control interface VI via shared variablesUser control of square wave output through PXI-5201 Arbitrary Waveform Generator (AWG). Output frequency to piezoelectric actuator. PXI 7831 FPGA RIOData logging via control interface VIUTA Microrobotics Team videoPXI-8196 controllerrobotposex, y, θPZTActuatorarena andmicrorobotimageuser controlcontrol interface VIsquare waveamplitude & frequency
68Advanced Human Robot Interaction Realistic & Intuitive Human-Robot InteractionCo-botics w/ Physical InteractionReal-Time Visual Feedbackand Facial ExpressionsAdvanced Human-Robot InterfacesRobot Touch HRIVisual HRINeptune Control throughNeural HeadbandZeno Video
72New Project: Physical HRI with Robotic Skin Popa, Celik-Butler, Butler, Lewis, Bugnariu National Robotics Initiative1) Devices: “distributed sensors” Integration of multi-modal, multi-resolution, MEMS skin sensors to include tactile, thermal, pressure, acceleration, and distance IR sensing.2) System Design: “where to place sensors on robot?”Maximize statistical information metrics related to robot perception.3) Control and Learning: “both human and robot learn during interaction” Learning algorithms and adaptive impedance control for efficient use of new sensor technology to sense human intent and control the interaction.4) Co-Robot performance: “how does this technology help humans?” The impact of the new technology to humans will be assessed, including the safety, level of assistance to several targeted user groups, ease of use, aesthetics, and therapeutical benefits of this technology.UTA NanoFab Bolometer and Pressure Sensitive SkinRobot and Skin SimulationFabrication & Integration of Skin/Garment HardwareInteraction LearningPerceivedImpedanceMeasurement & SimulationpHRIIterate Designs & AlgorithmsInitial sensor prototypesRobotic hardwareTask requirements
73Neptune: Assistive Mobile Manipulation System with intuitive human-machine interfaces Point Cloud overlaid with the virtual model (left), and detected obstacles (right) during dynamic motion planning
74Neptune: VideoMobile robot control using brain activity sensor device
75Zeno – Socially Assistive Robot Zeno is half-meter tall robot-child character capable of facial expression, head-eye motion, arm gesturing, mimicking and dynamic walking in the near future.We are developing a high performance embedded “motor cortex” for Zeno, controlling interactive motion, and applying it for treatment of Autism.
76RoboKind Zeno R-30 Current Hardware 4 DOF11 DOFOnboard Computer4GB NAND flash and 16GB SD Micro Class 101.6GHz Intel Atom Z5301GB RAMWiFi N at 150mb/sec and Giga-bit EthernetUbuntu LinuxTwo HD cameras720p, 30 fpsTwo microphones18 DOF
77System Description Robotic Arm Model Torso Neck Eyes Eyelids Mouth 4 DOFHuman arm contains 7 DOFTorsoYaw rotationNeckRoll, pitch, yaw rotationEyesPan and tiltEyelidsMouthOpen and closeBody PartDOFServo TypeArms8Dynamixel RX-28Torso1Neck3AX-12HS-5245MGHS-B1MGEyes2Cirrus CS- 101 STDEyelidsMouthHS-65MG
78Modes of Operation Dynamic Interactive: Kinect based motion Dynamic Interactive Mode RepresentationDynamic Interactive: Kinect based motionCan generate a more human-like motion during therapyCan generate any desired motion with a desired joint velocity generated by the human motion as long as it does not exceed the servo’s velocity constraintsSmooth motion that resembles human-like movement235 ms robot response timeHuman visual response time msDynamic Interactive Mode System Block Diagram
79DARPA Robotics Challenge 2012 Track B Team: RE2, Inc., UT Arlington, Soar Technologies, Inc.UTA Team MembersDan O. Popa, Ph.D., Associate Prof. of Electrical EngineeringGian Luca Mariottini, Ph.D., Assistant Prof. of Computer Science and Eng.Alan Bowling, Assistant Prof. of Mechanical and Aerospace EngineeringFrank L. Lewis, Professor of Electrical EngineeringKamesh Subbarao, Associate Prof. of Mechanical and Aerospace Engineering
80DRC Synopsys Connection with National Robotics Intiative (NRI) DoD Disaster Relief Scenario inspired by Katrina, Gulf Spill and Fukushima – robotic solution without using custom fixtures and tooling, aims to increase the number of innovatorsScenario (time limits under 1 hour for each event)1. Drive a utility vehicle at the site (no door, but regular vehicle)2. Travel dismounted across rubble (something a human can do)3. Remove debris blocking an entryway (up to 5kg boulders)4. Open a door and enter a building (open door by handle)5. Climb an industrial ladder and traverse an industrial walkway (ladder goes all the way to floor, therefore no arm pull-up strength required)6. Use a tool to break through a concrete panel (human tools like jackhammer)7. Locate and close a valve near a leaking pipe8. Replace a component such as a cooling pump
81DRC TechnologyHuman - Predictive Models – Supervised Autonomy – Robot - EnvironmentQuestions to address:What kind of I/O from human to robots or virtual robot modelsWhat kind of messages to robot over degraded communication linksWhat kind of autonomy/mobility/manipulation/energy algorithms for robot.Human Factors Leading to DARPA Project:Disaster environments, even degraded, have been engineered for humans.No shortage of human tools (vehicles, power tools) that should be useable by robotsNo expert roboticists required to operate robots (operation in natural way) – therefore some human intiution for robots is requiredGazebo Simulation Environment – Open Source Robotics FoundationAtlas Petman GFE – Boston Dynamics
82Lecture 1: Intro to Robotics Outline:Origins of robotics in the scifi artistic genreDefinition of robotsManipulators and mobile robotsHistory of robotics with timelineOverview of robotics research at ARRI-UTABasic robotics concepts
83History of RoboticsRobotics was first introduced into our vocabulary by Czech playwright Karel Capek in his 1920’s play Rossum’s Universal Robots.The word “robota” in Czech means simply work. Robots as machines that resemble people, work tirelessly, and revolt against their creators.The same myth/concept is found in many books/movies today:“Terminator”, “Star-Wars” series.Mary Shelley’s 1818 Frankenstein.Frankenstein & The Borg are examples of “cybernetic organisms”.Cybernetics is a discipline that was created in the late 1940’s by Norbert Wiener, combining feedback control theory, information sciences and biology to try to explain the common principles of control and communications in both animals and machines.“Behavioral robotics”: organisms as machines interacting with their environment according to behavioral models.
84History of RoboticsShould robots look like humans? “anthropomorphic or humanoid robots”.Need for these machines to also be intelligent - link to “Artificial Intelligence (AI)”.Need for humans to create machines similar to them is rooted in religious beliefs, recommended reading “God in the Machine” by Anne FoerstIt is not the appearance of the robot that most connects it to humans: HAL in “Space Odyssey 2001”, Lt. Data in “Startrek-TNG”, R2D2 and C3PO in “Star Wars”. Which one is more “likeable” and why?
85History of RoboticsRobots need not look like humanoids, but they make use of:Strong & precise articulated arms to accomplish tasks that were performed by humans – “articulated robots”, or “manipulators”. Fear that they will replace human laborers.Use of mobility to reposition the robot from one location to another, “mobile robots”. This can be done by locomotion like humans do (“legged robots”), but most likely it will use other means such as wheels (“wheeled robots”).Robotics is a multi-disciplinary field. Best robotics researchers and engineers will touch upon all disciplines:Mechanical Engineering – concerned primarily with manipulator/mobile robot design, kinematics, dynamics, compliance and actuation.Electrical Engineering – concerned primarily with robot actuation, electronic interfacing to computers and sensors, and control algorithms.Computer Science – concerned primarily with robot programming, planning, and intelligent behavior.
86Definition of RobotsAccording to the Robotics Industries Association (RIA): “A robot is a reprogrammable multifunctional manipulator designed to move material, parts, tools, or specialized devices through variable programmed motions for the performance of a variety of tasks (Jablonski and Posey, 1985)”.This definition underscored the reprogrammability of robots, but it also just deals with manipulators and excludes mobile robots.Close relationship with the concept of “automation”, the discipline that implements principles of control in specialized hardware. Three levels of implementation:Rigid automation – factory context oriented to the mass manufacturing of products of the same type. Uses fixed operational sequences that cannot be altered.Programmable automation – factory context oriented to low-medium batches of different types of products. A programmable system allows for changing of manufacturing sequences.Flexible automation – evolution of programmable automation by allowing the quick reconfiguration and reprogramming of the sequence of operation. Flexible automation is often implemented as “Flexible robotic workcells” (Decelle 1988, Pugh 1983). Reprogramming/retooling the robots changes the functionality of the workcell.
87Definition of RobotsAccording to the Japanese Industrial Robot Association (JIRA), robots can be classified as follows:Class 1: manual handling device – a device with several DOF’s actuated by the operator.Class 2: fixed sequence robot – similar to fixed automation.Class 3: variable sequence robot – similar to programmable automation.Class 4: playback robot – the human performs tasks manually to teach the robot what trajectories to follow.Class 5: numerical control robot – the operator provides the robot with the sequence of tasks to follow rather than teach it.Class 6: intelligent robot – a robot with the means to understand its environment, and the ability to successfully complete a task despite changes in the surrounding conditions where it is performed.Another definition describes robotics as the intelligent connection between perception and action (Brady 1985). This is an overly inclusive definition.Yet another definition, which focuses on mobile robots (Arkin 1998) is “A robot is a machine able to extract information from its environment, and use this knowledge to move safely, in a meaningful and purposive manner”.
88Definition of Robots Sense Think Act Robot G. Bekey definition: an entity that can sense, think and act.Extensions: communicate, imitate, collaborateClassification: manipulators, mobile robots, mobile manipulators.SenseThinkActRobot
89ManipulatorsIndustrial manipulators were born after WWII out of earlier technologies:Teleoperators. Teleoperators, or remotely controlled mechanical manipulator, were developed at first by Argonne and Oak Ridge National Labs to handle radioactive materials. These devices are also called “master-slave”, and consisted of a “master” arm being guided through mechanical links to mimic the motion of a “slave” arm that is operated by the user. Eventually, the mechanical links were replaced by electrical or hydraulic links.Numerically controlled milling machines (CNC). CNC machines were needed because of machining needs for very complex and accurate shapes, in particular aircraft parts.
90Mobile RobotsMobile robots were born out of “unmanned vehicles”, which also appear in WWII (for example an unmanned plane dropped the atomic bomb at Nagasaki).Unmanned Aerial Vehicles (UAV), Underwater Vehicles (UUV) and Ground Vehicles (UGV).Because tethered mobile vehicles could not move very far, and radio communications were limited, an approach to mobile robots is to endow them with the necessary control and decision capability - “autonomy”Autonomous Underwater/Ground/Aerial Vehicles (AUV/AGV/AAV).Unlike manipulators, we do not think of a remotely controlled toy as a mobile robot, suggesting that one of the fundamental aspects of mobile robotics is the capacity for autonomous operation.
94Robot History Timeline – first electric and hydraulic teleoperators are developed by General Electric and General Mills. Force feedback is added to prevent the crushing of glass containers during manipulation.CNC machine tools for accurate milling of aircraft parts are introduced.1953 – W. Grey Walter applies cybernetics principles to a robotic design called “machine speculatrix”, which became a robotic tortoise. The simple principles involved were:Parsimony: simple is better. Simple reflexes are the basis of robot behavior.Exploration or speculation: the system never remains still except when recharging. Constant motion is needed to keep it from being trapped.Attraction: the system is motivated to move towards objects or light.Aversion: the system moves away from certain objects, such as obstacles.Discernment: the system can distinguish between productive and unproductive behavior, adapting itself to the situation.
95G. Walter Grey's tortoise These vehicles had a light sensor, touch sensor, propulsion motor, steering motor, and a two vacuum tube analog computer.
96Robot History Timeline 1954 – George Devol replaced the slave manipulator in a teleoperator with the programmability of the CNC controller, thus creating the first “industrial robot”, called the “Programmable Article Transfer Device”.1955 – The Darmouth Summer Research Conference marks the birth of AI. Marvin Minsky, from the AI lab at MIT defines an intelligent machine as one that would tend to “build up within itself an abstract model of the environment in which it is placed. If it were given a problem, it could first explore solutions within the internal abstract model of the environment and then attempt external experiments”. This approach dominated robotics research for the next 30 years.Joseph Engleberger, a Columbia physics student buys the rights to Devol’s robot and founds the Unimation Company.1961 – The first Unimate robot is installed in a Trenton, NJ General Motors plant to tend a die casting machine. The key was the reprogrammability and retooling of the machine to perform different tasks. The Unimate robot was an innovative mechanical design based on a multi-degree of freedom cantilever beam. The beam flexibility presented challenges for control. Hydraulic actuation was eventually used to alleviate precision problems.
98Robot History Timeline 1962 – 1963 – The introduction of sensors is seen as a way to enhance the operation of robots. This includes force sensing for stacking blocks (Ernst, 1961), vision system for binary decision for presence of obstacles in the environment (McCarthy 1963), pressure sensors for grasping (Tomovic and Boni, 1962). Robot interaction with an unstructured environment at MIT’s AI lab (Man and Computer – MAC project).1968 – Kawasaki Heavy Industries in Japan acquires a license for Unimate.1968 – Shakey, a mobile robot is developed by SRI (Stanford Research Institute). It was placed in a special room with specially colored objects. A vision system would recognize objects and pushed objects according to a plan. This planning software was STRIPS, and it maintained and updated a world model. The robot had pan/tilt and focus for the camera, and bump sensors.– The Stanford Arm is developed, along with the first language for programming robots - WAVE.
99Robot History Timeline Late 1970’s – First assembly applications of robotics are considered: water pumps – Paul and Bolles, typewriter – Will and Grossman, Remote Center of Compliance gripper (RCC) developed at Draper Labs.1970’s – Innovation in the type of robots introduced: Unimation 2000, Cincinnati Milacron (“The tomorrow tool, T3”) – the first computer controlled manipulator, the PUMA (“Programmable Universal Machine for Assembly”) by Unimation, the SCARA (“Selective compliant articulated robot for Assembly”) introduced in Japan and the US (by Adept Technologies).1972 – First snake-like robot – ACM III – Hirose – Tokyo Inst. Of Tech.1977 – Development of mobile robot Hilaire at Laboratoise d’Automatique et d’Analyse des Systemes (LAAS) in Toulouse, France. This mobile robot had three wheels and it is still in use.1970’s – JPL develops its first planetary exploration Rover using a TV camera, laser range finder and tactile sensors.
102Robot History Timeline 1980’s – Innovation in improving the performance of robot arms – feedback control to improve accuracy, program compliance, the introduction of personal computers as controllers, and commercialization of robots by a large number of companies: KUKA (Germany), IBM 7535, Adept Robot (USA), Hitachi, Seiko (Japan).Early 1980’s – Multi-fingered hands developed, Utah-MIT arm (16 DOF) developed by Steve Jacobsen, Salisbury’s hand (9 dof).– Stanford cart/CMU rover developed by Hans Moravec, later on became the Nomad mobile robot.1980’s – Legged and hopping robots (BIPER – Shimoyama) and Raibert 1986.– V. Braitenberg revived the tortoise mobile robots of W. Grey Walter creating autonomous robots exhibiting behaviors. Hogg, Martin and Resnick at MIT create mobile robots using LEGO blocks (precursor to LEGO Mindstorms). Rodney Brooks at MIT creates first insect robots at MIT AI Lab – birth of behavioral robotics.
103KUKAThey can load, unload, deburr, flame-machine, laser, weld, bond, assemble, inspect, and sort.
104IBM 7535IBM 7535 Manufacturing System provided it advanced programming functions, including data communications, programmable speed.
106Nomad mobile robotThe XR4000 is an advanced mobile robot system that incorporates state of the art drive, control, networking, power management, sensing, communication and software development technologies.
107Rensselaer Polytechnic Institute CAT Robots CAT-Mobile: Autonomous Tractor-Trailer Robot (Wen, Divelbiss, Popa)Tetrobot: Modular & Reconfigurable Stewart Platform(Sanderson & Lee)
108Robot History Timeline 1990’s – Humanoid robots – Cog, Kismet (MIT), Wasubot, WHL-I – Japan, Honda P2 (1.82m, 210kg), and P3 (1.6m, 130kg), ASIMO.1990’s – Entertainment and Education Robots – SARCOS (“Jurassic Park”), Sony AIBO, LEGO Mindstorms, Khypera, Parallax.ROBOCUP, the competition simulating the game of soccer played by two teams of robots having been held around the world since 1997 (Osaka) .1990’s – Introduction of space robots (manipulators as well as rovers – the MARS rover 1996), parallel manipulators (Stewart-Gough Platforms), multiple manipulators, precision robots (“Robotworld”), surgical robots (“RoboDoc”), first service robots (as couriers in hospitals, etc)
112Kismet – MIT AI LabKismet consists of a head with large eyes with eyelids, bushy eyebrows, rubber lips, and floppy ears.
113Cog – MIT AI LabCog is a humanoid robot. It has a torso, arms and a head but no legs. Cog's torso does not have a spine but it can bend at the waist from side-to-side and from front-to-back and can twist its torso the same way a person can. Cog's arms also move in a natural way.
114Hierarchical family of robots (K-Team - Switzerland) Khepera (6 in)Koala (20 in)Alice (1 in)
115ARV Wall-Climbing Robot for Fuselage Inspection
116Robot History Timeline 2000’s – IRobot introduces the first autonomous vacuum – “Roomba”.2000’s – Mini and micro robots, “Smart Dust” – Berkeley, UTA, EPFL/Lausanne, microfactories.2000’s – Military applications - Robotic assistants for dangerous environments and reconnaissance, AUV’s and UUV’s, etc.2000’s – Intuitive Surgical introduces the Da Vinci surgical robot.2000’s – Robotic Deployment of Sensor Networks
1172006: Microsoft Introduces Robotics Studio What is Microsoft Robotics Studio?A window-based environment that is used to create robotics applicationWhat does Microsoft Robotics Studio do?Consider Robotics Application where we have several sensory inputs andneeded to be processed to command Actuators outputMicrosoft Robotics Studio provide a programmatic way to create anorchestrator that manage robotics applications (“Service”)InputsActuatorswe sense several inputs we need to do something with themwe need to make use of the inputs information that we have to generate meaningful outputso we need some kind of coordination between interpretation of input to the action of output actuatorit’s called “Orchestrator”Orhcestration means:……………………..Multiple Sensory InputsOrchestratorMultiple Actuator OutputsOrchestration: “The task of consuming sensory input from a variety of sources and as a result manipulating a set of actuatorsto respond to the sensory input.”
123Human-Machine Interaction Research Study of physiological effects of a robot on a human.Measurements takenHeart Rate (electrocardiograph)RespirationPerspirationPulse WaveBlood PressureThey used the above data to analyze the emotional state of the human participated in the experiment. They also have a 12 DOFs serial mechanism to capture a human motion so that they could incorporate motion with the data from the measurements. The experiment was performed by putting a human wearing all the measuring devices in the working environment with a robot to have interactions with humanoid robot and they interpreted all the data from measurements to indicate the participant’s stress level.now I’m going to give some examples on humanoid researchthese are current research topic that leading research groups are working on right nowthis research focuses on the study of physiological effects of a robot on a human. They assume that when a human working with a robot for a long time it can cause stress to that human so they built a device that can attach human body so they can take these measurements-heart rate-Pulse Wave-Blood Pressure-RespirationThe experiment was performed by putting a human wearing all the measuring devices in the working environment with a robot to have interactions with humanoid robot and they interpreted all the data from measurements to indicate the participant’s stress level. They recorded certain actions that significantly increases or decreases the stress level of the human.the goal is to investigate what kind of action can decrease the stress level the mostK. Itoh, H. Miwa, Y. Nukariya, M. Zecca, H. Takanobu, S. Roccella, M. C. Carrozza, P. Dario and A. Takanishi, Development of a Bioinstrumentation System in the Interaction Between a Human and a Robot. IEEE/RSJ International Conference on Intelligent Robots and Systems October 2006.
124Social Interactive Robot Research This research proposed a facial expression imitation system that consists of two part; facial expression recognition and facial express generation through a robot. The facial expression part is able to classify basic facial expressions includes; neutral, happiness, sadness, anger, surprise, disgust and fear.In this paper, they developed an artificial facial expression imitation system that can recognize human’s facial expression and able to imitate human’s facial expression. They proposed a method of classifier that was suitable to implement to recognize human facial expression in real timethe facial expression imitation system that consists of two part; facial expression recognition and facial express generation through a robot.the system can classify basic facial expressions includes; neutral, happiness, sadness, anger, surprise, disgust and fear.S. Sosnowski, A. Bitterman, K. Kuhnlenz, and M. Buss, Design and Evaluation of Emotion DisplayEDDIE. IEEE/RSJ International Conference on Intelligent Robots and Systems October 2006.
125Research on Dynamics of Humanoid Robots Whole body motion is very complicated, so when a humanoid robot is to be controlled remotely through teleoperation it is more intuitive to use marionette device to control the humanoid robotbase on the idea of teleoperation the research try to come up with the way to interface the humanoid robot with some kind of device that easy to manipulatesince the humanoid robot have many degree of freedom . it would be very difficult to control the motion of the robot with slidersWhole body motion is very complicated, so when a humanoid robot is to be controlled remotely through teleoperation it is more intuitive to use marionette device to control the humanoid robotT. Takubo, K. Inoue and T. Arai, Wholebody Teleoperation for Humanoid Robot by Marionette System. IEEE/RSJ International Conference on Intelligent Robots and Systems October 2006.
126Research on Human-Robot Interaction Haptic DevicesRobot Assisted Minimal Invasive SurgeryParallel/Redundant Haptic DevicesMulti-Fingered Haptic Interfacesome interesting topics includesRobot Assisted Minimal Invasive Surgerywhich is the top picture on the rightParallel/Redundant Haptic Deviceswhich is the left pictureMulti-Fingered Haptic Interfacewhich is the bottom one on the right
127Robotics Applications Today, commercial robots are used routinely in the following applications:Industrial Manufacturing – “Transforming objects” - arc/spot welding, milling/drilling, glueing/sealing, laser/water jet cutting, grinding, deburring, screwing, painting, and assembly.Material Handling: “Pick and Place”- palletizing (placing objects on a pellet in an ordered way), warehouse loading/unloading, part sorting, packaging, electronic chip pick and place, hazardous material handling.Measurement: object finding, contour finding, inspection, 3D registration.Entertainment robotics: animated figures, flight simulator, robotic pets.Service robotics: robotic aids for handicapped people, artificial limbs, robotic vacuum, courier.Military robotics: defusing explosive devices, scout robots, UAVs.Surgical Robotics: drilling, suturing, cauterizing, tool holding.
128Robotics Applications Robot prices continue to drop compared to the cost of human labor.In the year 2000, 78% of all robots installed in the US were welding or material-handling robots.