1. EE 4315/EE5325 Robotics Lectures: Tue-Thu, 11-12:20 pm, NH111
Instructors: Indika Wijayasinghe Ph.D. &
Dan Popa, Ph.D., Associate Professor, EE
Office hours: Tue/Thu 9:00 –11:00 pm, NH250
Tue/Wed 1:30pm-3:30 pm, NH543
Course info:
Grading policy:
Homework – 15%
Midterm exam (take home) – 20%
Final project w/ presentation – 40%
Final exam (in class) – 25%
Project grade is composed of:
Project Proposal (February 12) – 5%
Project Progress Report (March 19) – 10%
Project Final Presentation (May 5, May 7) – 5%
Project Report / Paper (May 7) – 20%

2. Summary
This is a project-based robotics course, containing both introductory as well as more advanced concepts. It presents a broad overview of robotic manipulation and also focuses on fundamentals such as robot kinematics, dynamics, control and planning. This year, the course will focus on concepts in support of major competitions. The course is divided between two areas:
Robot Kinematics
Rotations, Translations and Homogeneous Transformations
Parameterizations and Singularities
Robot kinematics and the Jacobian
Robot statics
Multi-fingered grasping
Nonholonomic and parellel robots
Robot Dynamics and Control
Robot dynamical models
Computed Torque Control
Advanced Robot Control
Kinematic Control of Nonholonomic Robots
Trajectory Planning
Force Control and Physical Human Robot Interaction
Visual Servoing and Visual Human Robot Interaction

3. Tentative Syllabus Week 1 - Jan 20, 22, Lectures 1,2
Intro to robotics: History of robotics ; Examples of robots
Robotics Jargon ; Course Projects
Week 2 - Jan 27, 29 Lectures 3,4
Review of basics: Matrix algebra
Intro to Robot Kinematics: Geometric issues ; Frames ; Notation
Week 3 - Feb 3, 5, Lectures 5,6
Homogeneous Transforms, Parameterizations and Singularities
Week 4 -Feb 10, 12, Lectures 7,8
Robot Kinematics : DH framework ; Forward Kinematics ; Inverse Kinematics
Discussion on other Kinematic Representations

4. Tentative Syllabus Week 5 - Feb 17, 19, Lectures 9, 10
Robot Manipulability and Singularities; Statics
Week 6 - Feb 24, 26, Lectures 11, 12
Manipulability, redundancy, parallel robots.
Week 7 - March 3, 5, Lectures 13, 14
Robot statics, compliance, stiffness
Robot grasping and multi-fingered manipulation
Week 8 - March 9-13
Spring Break
Week 9 - March 17, 19, Lectures 15, 16
Nonholonomic constraints and wheeled mobile robots.

5. Tentative Syllabus Week 10 - March 24, 26, Lectures 17, 18
Robot dynamics: Euler-Lagrange formulation
Robot dynamics: Newton Euler formulation
Midterm (take-home) handed out March 26, due March 31 will cover kinematics and statics
Week 11 - March 31, April 2, Lectures 19, 20
Examples of dynamical equations
Properties of robot dynamic equations
Week 12 -April 7, 9 Lectures 21, 22
Basic robot control, modeling and control of joints.
Concepts in stability, passivity with applications to robot control.
Week 13 - April 14, 16, Lectures 22, 23
Computed torque control.
Trajectory generation and following for manipulators.

6. Tentative Syllabus Week 14 - April 21, 23, Lectures 24, 25
Robot Control using Lyapunov Theory
Robust and advanced control for robot manipulators.
Week 15 - April 28, 30, Lectures 26, 27
Introduction to Force control.
Introduction to robot vision and visual servo control
Week 16 - May 5, 7,
Project Presentations, Project due
Week 17 - May 12
Final exam (in-class, comprehensive).

7. Textbooks
Robot Modeling and Control, by M.W. Spong, S. Hutchinson, M. Vidyasagar (2005) (required)
Mathematical Introduction to Robotic Manipulation by Richard M. Murray, Zexiang Li, S. Shankar Sastry (free download at link)
Robot Manipulator Control: Theory and Practice (Control Engineering, 15) by Frank L. Lewis, et al (on reserve)
Introduction to Robotics: Mechanics and Control (3rd Edition) by John J. Craig (on reserve)
Autonomous Mobile Robots, by S. S. Ge, F. L. Lewis (on reserve).
Robotics: Modelling, Planning and Control (Advanced Textbooks in Control and Signal Processing) [Hardcover], Bruno Siciliano, Lorenzo Sciavicco, Luigi Villani, Giuseppe Oriolo ( Publisher: Springer; 2nd Printing. edition (February 11, 2011), ISBN-10: , ISBN-13:
MATLAB Robotics Toolbox by Peter Corke, free download
ROS and GAZEBO simulation environments, by Willow Garage and OSRF, free download

8. Course Refresher Reading Assignments
Math: linear/matrix algebra, trigonometry, differential equations.
Controls: stability, linear & nonlinear systems.
Physics: mechanics, force & motion.
Programming: MATLAB, ROS
Reading assignments will be posted online. Assigned reading material may be useful for certain exam problems.

9. Honor Code
Missed deadlines for take-home exam and project proposal and report: Maximum grade drops 10% per late day
Academic Dishonesty will not be tolerated. All exams and reports are individual assignments. Your take-home exam and reports will be carefully scrutinized to ensure a fair grade for everyone.
Attendance and Drop Policy: Attendance is not mandatory. However, if you skip classes, you will find the homework and exams more difficult. Assignments are going to be posted here, however, due to the pace of the lectures, copying someone else's notes may be an unreliable way of making up an absence. You are responsible for all material covered in class regardless of absences.

10. Exams and Class Projects
Midterm, final, individual work only.
Project – team project only.
Course project – A list of topics will be provided. Stop by and discuss the class project with me during office hours early in the semester. The project work is collaborative. It can be applied or can involve programming.
IEEE
Amazon Picking Challenge
Mobile Microrobotics Challenge
Humanitarian Robotics Challenge
Other possible projects: Prosthetics, Humanoid Robots, and other robots at UTARI and NGS

11. Research Credits: Dr. Woo Ho Lee
Humanoid Robots
Research
Credits: Dr. Woo Ho Lee

12. ATLAS Biped humanoid robot (28DOF) Height: 1.88m Weight: 150kg
Hydraulic actuation
Company: Boston Dynamics (purchased by Google)
Government Furnished Equipment for DARPA Robotics Challenge
Sensors: LIDAR, stereo sensors

13. THOR Biped humanoid robot (34DOF)
Open source platform: Available for purchase after DRC
Institution: Virginia Tech
Develop for DARPA Robotics Challenge

14. iCub Humanoid robot for human cognition and AI
53DOF including 18 DOF hands
Price: 250K Euros
Height: 1m, Weight: 22kg
Open source humanoid robot platform
CAD, software, and BOM
Institution: IIT funded by EU
Adopted by 20 laboratories worldwide. 25 iCubs in Europe, USA, …

15. Asimo Biped humanoid robot (34DOF) Lease price: $150K / month
Height: 1.3m, Weight: 54kg
Operation time: 1hour
Manufacturer: Honda in Japan
Market: Research and education

16. Hubo II Biped humanoid robot (38DOF) Price: $400K
Height: 1.3m, Weight: 50kg
Two embedded computers
1 for real time control
1 for communication and application programs (Window XP)
Company:
Rainbow Co (KAIST spin-off) in Korea
Market: Research and education
Used at DARPA Robotics Challenge

17. HRP-4 Biped humanoid robot (34DOF) Price: $300K Height: 1.5m
Weight: 39kg
Operation: 34 minute
OS: Linux
Manufacturer: Kawada industries in Japan

18. Nao Humanoid robot (25DOF) Price: $16K Open source software
Used in the Robot Soccer World Cup
Over 200 academic institutions
Company:
Aldebaran Robotics in France
Founded in 2005
Market:
Education
Research

19. DARWIN Open platform Humanoid robot Price: $10K 20 DOFs
Used in the RoboCup.
Company:
Robotis in Korea
Founded in 1999
Market:
Education
Toy

20. Zeno Facial expression humanoid robot (36DOF including 11DOF Head)
Frubber robotic skin
Price: $17.5K
Company:
Hanson Robotics
Founded in 2003
Market: Education & entertainment

21. PR2 Personal robot Open source software: ROS
Price: $400K (~50 robots sale)
Company:
Willow Garage
Funded by founder. Seek a self sustaining business model
Founded in 2006
Market: Education

22. Personal and Toy Robots
Market

23. Personal Robotics Affordable Applications
Telepresense
Education
Entertainment
Global personal robotics market
$18B in 2015 [Source: Grishin Robotics]

24. Double iPad equipped teleconferencing robot Price: $1999 (preorder)
Company:
Double Robotics
Seed fund from Y Combinator
$250K funding from Grishin Robotics
Founded in 2011
6 man
Market: Telepresence
Hospitals, schools, retail stores, museums
900+ Pre-orders

25. Sphero
Robot Ball You Control With Your Smartphone (iPhone and Android)
Price: $129
Company:
Orbotix
Seed fund from TechStars
$15M funding
Founded in 2010
35 people
Bluetooth communication
Market: Entertainment
President Obama’s visit

26. Romo Robot with an iPhone for brain Price: $149 Company:
Romotive
Seed fund from TechStars
$6.5M funding
Founded in 2011
16 people
Bluetooth communication
Market: Entertainment
Telepresence, Autonomous navigation

27. Cubelets Robot construction kit Reconfigurable robots
Price:  $160 - $520
Company
Modular Robotics
$3.5M funding
Founded in 2008
4 people
Market: Education

28. mOwayduino Arduino based robot Programmable with Arduino IED
Startup in Spain
Fund: indiegogo
Market: Education and Toy

29. RK-1 Arduino based mobile robot Arduino + iOS/Android Price: £175
Company: My Mobile Robots
Fund: Kickstarter
One man company in London
Market: Entertainment

30. Disciplines in EE Controls and Robotics
Robotics – examples of projects
Systems Approach and Related Concepts -
Feedback Control Basics
Feedback Control History
Robotics Basics and History
Robotics Projects at NGS

31. Signals and Systems Signal: System:
Any time dependent physical quantity
Electrical, Optical, Mechanical
System:
Object in which input signals interact to produce output signals.
Some have fundamental properties that make linear systems predictable:
Sinusoid in, sinusoid out of same frequency (when transients settle)
Double the amplitude in, double the amplitude out (when initial state conditions are zero)

32. System Modeling
Building mathematical models based on observed data, or other insight for the system.
Parametric models (analytical): ODE, PDE
Non-parametric models: graphical models - plots, look-up cause-effect tables
Mental models – Driving a car and using the cause-effect knowledge
Simulation models – Many interconnect subroutines, objects in video game

33. Types of Models White Box Black Box Gray Box – combination of the two
derived from first principles laws: physical, chemical, biological, economical, etc.
Examples: RLC circuits, MSD mechanical models (electromechanical system models).
Black Box
model is entirely derived from measured data
Example: regression (data fit)
Gray Box – combination of the two

34. White Box Systems: Electrical
Defined by Electro-Magnetic Laws of Physics: Ohm’s Law, Kirchoff’s Laws, Maxwell’s Equations
Example: Resistor, Capacitor, Inductor

35. RLC Circuit as a System
Kirchoff’s Voltage Law (KVL):

36. Linear Time-Invariant Models
Continuous-time linear dynamical system (LDSC) has the form
dx/dt= A(t)x(t) + B(t)u(t),
y(t) = C(t)x(t) + D(t)u(t)
where:
t  R denotes time
x(t)  Rn is the state (vector)
u(t)  Rm is the input or control
y(t)  Rp is the output

37. Linear System Description in Frequency Domain
Purpose of Frequency Domain Analysis:
Convert Differential equations into Algebraic Equations
Interconnect systems using block diagrams
Use graphical tools to discover of influence behavior of systems

38. EE-Specific Diagrams Block Diagram Model:
Helps understand flow of information (signals) through a complex system
Helps visualize I/O dependencies
Equivalent to a set of linear algebraic equations.
Based on a set of primitives:
Transfer Function Summer/Difference Pick-off point U U2 + U1
U1+U2 U U +

39. Control System Block Diagram

40. Automatic Control
Control: process of making a system variable converge to a reference value
If r=ref_value=changing - servo (tracking control)
If r=ref_value=constant - regulation (stabilization)
Open loop vs. closed loop (feedback) control + r y + y
Controller
K(s)
Plant
G(s)
Controller
K(s) +
Plant
G(s) r -
Sensor Gain
H(s)

41. Brief History of Feedback Control
The key developments in the history of mankind that affected the progress of feedback control were:
1. The preoccupation of the Greeks and Arabs with keeping accurate track of time. This represents a period from about 300 BC to about 1200 AD. (Primitive period of AC)
2. The Industrial Revolution in Europe, and its roots that can be traced back into the 1600's. (Primitive period of AC)
3. The beginning of mass communication and the First and Second World Wars. (1910 to 1945). (Classical Period of AC)
4. The beginning of the space/computer age in (Modern Period of AC).

42. Primitive Period of AC
Float Valve for tank level regulators Drebbel incubator furnace control (1620)
(antiquity)

43. Primitive Period of AC
James Watt Fly-Ball Governor For regulating steam engine speed (late 1700’s)

44. Classical Period of AC
Stability Analysis: Maxwell, Routh, Hurwitz, Lyapunov (before 1900).
Electronic Feedback Amplifiers with Gain for long distance communications (Black, 1927)
Stability analysis in frequency domain using Nyquist criterion (1932), Bode Plots (1945).
PID controller (Callender, 1936) – servomechanism control
Root Locus (Evans, 1948) – aircraft control
Most of the advances were done in Frequency Domain.

45. Modern Period of AC Time domain analysis (state-space)
Bellmann, Kalman: linear systems (1960)
Pontryagin: Nonlinear systems (1960) – IFAC
Optimal controls
H-infinity control (Doyle, Francis, 1980’s) – loop shaping (in frequency domain).
MATLAB (1980’s to present) has implemented math behind most control methods.

46. Feedback Control Role of feedback:
Reduce sensitivity to system parameters (robustness)
Disturbance rejection
Track desired inputs with reduced steady state errors, overshoot, rise time, settling time (performance)
Systematic approach to analysis and design
Select controller based on desired characteristics
Predict system response to some input
Speed of response (e.g., adjust to workload changes)
Approaches to assessing stability

47. Feedback System Block Diagram
Temperature control system

48. Feedback System Block Diagrams
Automobile Cruise Control

49. Key Transfer Functions
Reference + S
Controller
Plant –
Transducer
Jlh: Can we make the diagram bigger and the title smaller?

50. Transient Response Characteristics

51. Effect of pole locations
Oscillations
(higher-freq)
Im(s)
Faster Decay
Faster Blowup
Re(s)
(e-at)
(eat)
Negative feedback
Pole at -1/A (stable)
Positive feedback
Pole at 1/A (unstable)

52. Basic Control Actions: u(t)
Jlh: Can we give more intuition on control actions. This seems real brief considering its importance.

53. Summary of Basic Control
Proportional control
Multiply e(t) by a constant
PI control
Multiply e(t) and its integral by separate constants
Avoids bias for step
PD control
Multiply e(t) and its derivative by separate constants
Adjust more rapidly to changes
PID control
Multiply e(t), its derivative and its integral by separate constants
Reduce bias and react quickly

54. Conclusion: Control Systems
Abstraction is the basis for system level thinking. Abstraction requires advanced mathematics, and it is especially required of Electrical and Computer Engineers.
Control Theory contains abstractions and generalizations able to guarantee predictable performance of systems under control.
Negative feedback offers numerous advantages: noise rejection, robustness to plant variations, dynamical tracking performance.
Examples of popular control schemes include Proportional-Integral-Derivative (PID) schemes.
Modern control is primarily based on time-domain analysis of state-equations using matrices.
Control engineers can find jobs in any industry. Control concepts can be applied in any engineering industry.

55. Robots as Complex Systems Controlled by Feedback
G. Bekey definition: an entity that can sense, think and act.
Classification: manipulators, mobile robots, mobile manipulators.
Sense
Think
Act
Robot

56. Robot Subsystems A mechanical structure.
For manipulators this structure consists of a set of rigid bodies (links), connected by means of articulations (joints). Links and joints can also be described in terms of an arm (for mobility), a wrist (for dexterity) and an end-effector (for performing the task).
For mobile robots, the structure consists of a chassis with a locomotion mechanism, in the form of legs, wheels, rotor blades, etc.
Actuators. These set the robot in motion through actuation of its joints, and are typical electric or hydraulic.
Sensors. These measure the status of the manipulator (propriceptive sensors) and the status of the environment (heteroceptive sensors).
A control system. This enables control and supervision of the robot, and is usually a computer with a graphical user interface, a pendant, or a remote controller.

57. Recent Developments in Robot Software Platforms for Standardization
2009- Willow Garage Launches Robot Operating System (ROS)
2011- National Instruments Introduces the Labview Robotics Module
we sense several inputs we need to do something with them
we need to make use of the inputs information that we have to generate meaningful output
so we need some kind of coordination between interpretation of input to the action of output actuator
it’s called “Orchestrator”
Orhcestration means:……………………..

58. IEEE Robotics Societies and Competitions
Websites to watch:
IEEE Robotics & Automation Society
Join it for $15
Check out events and information at:
IEEE Region 5 Robotics Competition
Both a paper and a robot course contest.
For next year (2014)
Mobile Microrobotics IEEE ICRA
Compete with the world’s smallest robots

59. Conclusion: Robotics
Robotics uses advanced concepts in control to connect sensors with actuators.
Robots can be classified as manipulators (e.g. robotic arms), mobile robots, mobile manipulators.
Major disciplines in robotics are: kinematics, dynamics, planning, control, perception, and cognition.
Robotics is a multidisciplinary field, including computer scientists, mechanical engineers, electrical engineering, industrial engineers, etc.

60. Research in Multiscale Robotics at Next Gen Systems (NGS) Group
Tools and Fundamentals
Established Technologies
Emerging Technologies
Robotics
Control Systems
Manufacturing & Automation
Microsystems & MEMS
Nanotechnology
Biotechnology
Modeling & Simulation
Control Theory
Algorithms
Assistive Robots
Human-like robots
Distributed and wireless sensor systems
Small-scale
Robotics &
Manufacturing
Micromanufacturing
Microrobotics
Microassembly
Micropackaging
Sensor & Actuator Arrays
NanoManufacturing
New applications
for robot systems

61. NGS Focus: Advanced Robot Systems
NEXT GENERATION ROBOTICS
Assistive Robotics | Microrobotics | Networked Robotics
INTEGRATION
Design
Simulation
Networking
Learning
Control
Diagnostics
OPTIMIZATION
NGS
Motion Control | Human-Robot Interaction | Sensors & Actuators
MICRO & Nano MANUFACTURING
Packaging | Assembly | Testing | Reliability

62. New Initiatives and Projects at NGS and UTA Research Institute
NSF Projects in Assistive technologies, including robots for helping children with Cerebral Palsy w/ CSE Dept. (2009-).
UTA-UNTHSC joint project to study robot therapy and early diagnosys of Autism Spectrum Disorders – TXMRC consortium (2011-).
UTARI Consortium in Assistive Robotics (2012-) in collaboration with Qinetiq-North America.
UTARI Participation in DARPA Robotics Challenge in collaboration with RESquared, Inc. (Oct. 2012).
New NSF Project through the National Robotics Initiative aiming at creating the next generation Robotic Skin (Oct 2012) for co-Robots.
Plans under way for M.S. Degree program in UAS and Robotics at UTA’s College of Engineering (2013).

63. Wafer Scale Microfactory (Micro-Nano)
“From a few robots+controllers to many µrobots via assembly and die bonding”
µparts, nparts in
µcontroller IC dies
Assemblies
out
µrobot MEMS dies
Controller + Robot
µparts, nparts in

64. Wafer Scale Microfactory for Nanotechnology

65. Making the Microfactory by Automated 3D Microassembly
Larger number of robots
Measurement uncertainty, measurement range,
Time delays
Fewer embedded sensors, low SNR
Manufacturing uncertainty, inacurate robot models)
Environmental effects (stiction, temperature)
Control Challenges:

66. Mobile Microbotics Challenge 2011
Hosted at IEEE International Conference on Robotics and Automation, Shanghai, China, May 10, 2011.
7 Qualified Teams: France (FEMTO-ST), Italy (IIT), Univ. of Waterloo (CA), 4 US Universities (Stevens, Hawaii, Maryland, UTA
Maximum robot size: 600 microns sphere.
UTA Microrobots, 2011
Mobility
Challenge
Micro
Assembly
Event
Vibration and Laser Actuated

67. UTA Vibot Control Using National Instruments PXI-8196
Microrobot pose (x, y, θ) from NI-1742 Smart Camera
Exchange of pose data with the control interface VI via shared variables
User control of square wave output through PXI-5201 Arbitrary Waveform Generator (AWG). Output frequency to piezoelectric actuator. PXI 7831 FPGA RIO
Data logging via control interface VI
UTA Microrobotics Team video
PXI-8196 controller
robot
pose
x, y, θ
PZT
Actuator
arena and
microrobot
image
user control
control interface VI
square wave
amplitude & frequency

68. Advanced Human Robot Interaction
Realistic & Intuitive Human-Robot Interaction
Co-botics w/ Physical Interaction
Real-Time Visual Feedback
and Facial Expressions
Advanced Human-Robot Interfaces
Robot Touch HRI
Visual HRI
Neptune Control through
Neural Headband
Zeno Video

69. Land-Based Mobile Wireless Sensors

70. Hubo (KAIST, Hanson Robotics, Inc. and UTA) - 2006
Face uses – 38 motors
Advanced AI
Simulates Expression in Real time

71. Quickskin ™ Basic Demo on 6DOF CRS Robot

72. New Project: Physical HRI with Robotic Skin Popa, Celik-Butler, Butler, Lewis, Bugnariu National Robotics Initiative
1) Devices: “distributed sensors” Integration of multi-modal, multi-resolution, MEMS skin sensors to include tactile, thermal, pressure, acceleration, and distance IR sensing.
2) System Design: “where to place sensors on robot?”Maximize statistical information metrics related to robot perception.
3) Control and Learning: “both human and robot learn during interaction” Learning algorithms and adaptive impedance control for efficient use of new sensor technology to sense human intent and control the interaction.
4) Co-Robot performance: “how does this technology help humans?” The impact of the new technology to humans will be assessed, including the safety, level of assistance to several targeted user groups, ease of use, aesthetics, and therapeutical benefits of this technology.
UTA NanoFab Bolometer and Pressure Sensitive Skin
Robot and Skin Simulation
Fabrication & Integration of Skin/Garment Hardware
Interaction Learning
Perceived
Impedance
Measurement & Simulation
pHRI
Iterate Designs & Algorithms
Initial sensor prototypes
Robotic hardware
Task requirements

73. Neptune: Assistive Mobile Manipulation System with intuitive human-machine interfaces
Point Cloud overlaid with the virtual model (left), and detected obstacles (right) during dynamic motion planning

74. Neptune: Video
Mobile robot control using brain activity sensor device

75. Zeno – Socially Assistive Robot
Zeno is half-meter tall robot-child character capable of facial expression, head-eye motion, arm gesturing, mimicking and dynamic walking in the near future.
We are developing a high performance embedded “motor cortex” for Zeno, controlling interactive motion, and applying it for treatment of Autism.

76. RoboKind Zeno R-30 Current Hardware
4 DOF
11 DOF
Onboard Computer
4GB NAND flash and 16GB SD Micro Class 10
1.6GHz Intel Atom Z530
1GB RAM
WiFi N at 150mb/sec and Giga-bit Ethernet
Ubuntu Linux
Two HD cameras
720p, 30 fps
Two microphones
18 DOF

77. System Description Robotic Arm Model Torso Neck Eyes Eyelids Mouth
4 DOF
Human arm contains 7 DOF
Torso
Yaw rotation
Neck
Roll, pitch, yaw rotation
Eyes
Pan and tilt
Eyelids
Mouth
Open and close
Body Part
DOF
Servo Type
Arms 8
Dynamixel RX-28
Torso 1
Neck 3
AX-12
HS-5245MG
HS-B1MG
Eyes 2
Cirrus CS- 101 STD
Eyelids
Mouth
HS-65MG

78. Modes of Operation Dynamic Interactive: Kinect based motion
Dynamic Interactive Mode Representation
Dynamic Interactive: Kinect based motion
Can generate a more human-like motion during therapy
Can generate any desired motion with a desired joint velocity generated by the human motion as long as it does not exceed the servo’s velocity constraints
Smooth motion that resembles human-like movement
235 ms robot response time
Human visual response time [9] ms
Dynamic Interactive Mode System Block Diagram

79. DARPA Robotics Challenge 2012
Track B Team: RE2, Inc., UT Arlington, Soar Technologies, Inc.
UTA Team Members
Dan O. Popa, Ph.D., Associate Prof. of Electrical Engineering
Gian Luca Mariottini, Ph.D., Assistant Prof. of Computer Science and Eng.
Alan Bowling, Assistant Prof. of Mechanical and Aerospace Engineering
Frank L. Lewis, Professor of Electrical Engineering
Kamesh Subbarao, Associate Prof. of Mechanical and Aerospace Engineering

80. DRC Synopsys Connection with National Robotics Intiative (NRI)
DoD Disaster Relief Scenario inspired by Katrina, Gulf Spill and Fukushima – robotic solution without using custom fixtures and tooling, aims to increase the number of innovators
Scenario (time limits under 1 hour for each event)
1. Drive a utility vehicle at the site (no door, but regular vehicle)
2. Travel dismounted across rubble (something a human can do)
3. Remove debris blocking an entryway (up to 5kg boulders)
4. Open a door and enter a building (open door by handle)
5. Climb an industrial ladder and traverse an industrial walkway (ladder goes all the way to floor, therefore no arm pull-up strength required)
6. Use a tool to break through a concrete panel (human tools like jackhammer)
7. Locate and close a valve near a leaking pipe
8. Replace a component such as a cooling pump

81. DRC Technology
Human - Predictive Models – Supervised Autonomy – Robot - Environment
Questions to address:
What kind of I/O from human to robots or virtual robot models
What kind of messages to robot over degraded communication links
What kind of autonomy/mobility/manipulation/energy algorithms for robot.
Human Factors Leading to DARPA Project:
Disaster environments, even degraded, have been engineered for humans.
No shortage of human tools (vehicles, power tools) that should be useable by robots
No expert roboticists required to operate robots (operation in natural way) – therefore some human intiution for robots is required
Gazebo Simulation Environment – Open Source Robotics Foundation
Atlas Petman GFE – Boston Dynamics

82. Lecture 1: Intro to Robotics
Outline:
Origins of robotics in the scifi artistic genre
Definition of robots
Manipulators and mobile robots
History of robotics with timeline
Overview of robotics research at ARRI-UTA
Basic robotics concepts

83. History of Robotics
Robotics was first introduced into our vocabulary by Czech playwright Karel Capek in his 1920’s play Rossum’s Universal Robots.
The word “robota” in Czech means simply work. Robots as machines that resemble people, work tirelessly, and revolt against their creators.
The same myth/concept is found in many books/movies today:
“Terminator”, “Star-Wars” series.
Mary Shelley’s 1818 Frankenstein.
Frankenstein & The Borg are examples of “cybernetic organisms”.
Cybernetics is a discipline that was created in the late 1940’s by Norbert Wiener, combining feedback control theory, information sciences and biology to try to explain the common principles of control and communications in both animals and machines.
“Behavioral robotics”: organisms as machines interacting with their environment according to behavioral models.

84. History of Robotics
Should robots look like humans? “anthropomorphic or humanoid robots”.
Need for these machines to also be intelligent - link to “Artificial Intelligence (AI)”.
Need for humans to create machines similar to them is rooted in religious beliefs, recommended reading “God in the Machine” by Anne Foerst
It is not the appearance of the robot that most connects it to humans: HAL in “Space Odyssey 2001”, Lt. Data in “Startrek-TNG”, R2D2 and C3PO in “Star Wars”. Which one is more “likeable” and why?

85. History of Robotics
Robots need not look like humanoids, but they make use of:
Strong & precise articulated arms to accomplish tasks that were performed by humans – “articulated robots”, or “manipulators”. Fear that they will replace human laborers.
Use of mobility to reposition the robot from one location to another, “mobile robots”. This can be done by locomotion like humans do (“legged robots”), but most likely it will use other means such as wheels (“wheeled robots”).
Robotics is a multi-disciplinary field. Best robotics researchers and engineers will touch upon all disciplines:
Mechanical Engineering – concerned primarily with manipulator/mobile robot design, kinematics, dynamics, compliance and actuation.
Electrical Engineering – concerned primarily with robot actuation, electronic interfacing to computers and sensors, and control algorithms.
Computer Science – concerned primarily with robot programming, planning, and intelligent behavior.

86. Definition of Robots
According to the Robotics Industries Association (RIA): “A robot is a reprogrammable multifunctional manipulator designed to move material, parts, tools, or specialized devices through variable programmed motions for the performance of a variety of tasks (Jablonski and Posey, 1985)”.
This definition underscored the reprogrammability of robots, but it also just deals with manipulators and excludes mobile robots.
Close relationship with the concept of “automation”, the discipline that implements principles of control in specialized hardware. Three levels of implementation:
Rigid automation – factory context oriented to the mass manufacturing of products of the same type. Uses fixed operational sequences that cannot be altered.
Programmable automation – factory context oriented to low-medium batches of different types of products. A programmable system allows for changing of manufacturing sequences.
Flexible automation – evolution of programmable automation by allowing the quick reconfiguration and reprogramming of the sequence of operation. Flexible automation is often implemented as “Flexible robotic workcells” (Decelle 1988, Pugh 1983). Reprogramming/retooling the robots changes the functionality of the workcell.

87. Definition of Robots
According to the Japanese Industrial Robot Association (JIRA), robots can be classified as follows:
Class 1: manual handling device – a device with several DOF’s actuated by the operator.
Class 2: fixed sequence robot – similar to fixed automation.
Class 3: variable sequence robot – similar to programmable automation.
Class 4: playback robot – the human performs tasks manually to teach the robot what trajectories to follow.
Class 5: numerical control robot – the operator provides the robot with the sequence of tasks to follow rather than teach it.
Class 6: intelligent robot – a robot with the means to understand its environment, and the ability to successfully complete a task despite changes in the surrounding conditions where it is performed.
Another definition describes robotics as the intelligent connection between perception and action (Brady 1985). This is an overly inclusive definition.
Yet another definition, which focuses on mobile robots (Arkin 1998) is “A robot is a machine able to extract information from its environment, and use this knowledge to move safely, in a meaningful and purposive manner”.

88. Definition of Robots Sense Think Act Robot
G. Bekey definition: an entity that can sense, think and act.
Extensions: communicate, imitate, collaborate
Classification: manipulators, mobile robots, mobile manipulators.
Sense
Think
Act
Robot

89. Manipulators
Industrial manipulators were born after WWII out of earlier technologies:
Teleoperators. Teleoperators, or remotely controlled mechanical manipulator, were developed at first by Argonne and Oak Ridge National Labs to handle radioactive materials. These devices are also called “master-slave”, and consisted of a “master” arm being guided through mechanical links to mimic the motion of a “slave” arm that is operated by the user. Eventually, the mechanical links were replaced by electrical or hydraulic links.
Numerically controlled milling machines (CNC). CNC machines were needed because of machining needs for very complex and accurate shapes, in particular aircraft parts.

90. Mobile Robots
Mobile robots were born out of “unmanned vehicles”, which also appear in WWII (for example an unmanned plane dropped the atomic bomb at Nagasaki).
Unmanned Aerial Vehicles (UAV), Underwater Vehicles (UUV) and Ground Vehicles (UGV).
Because tethered mobile vehicles could not move very far, and radio communications were limited, an approach to mobile robots is to endow them with the necessary control and decision capability - “autonomy”
Autonomous Underwater/Ground/Aerial Vehicles (AUV/AGV/AAV).
Unlike manipulators, we do not think of a remotely controlled toy as a mobile robot, suggesting that one of the fundamental aspects of mobile robotics is the capacity for autonomous operation.

91. Anthropomorphic Robots

92. Animal-like Robots

93. Unmanned Vehicles

94. Robot History Timeline
– first electric and hydraulic teleoperators are developed by General Electric and General Mills. Force feedback is added to prevent the crushing of glass containers during manipulation.
CNC machine tools for accurate milling of aircraft parts are introduced.
1953 – W. Grey Walter applies cybernetics principles to a robotic design called “machine speculatrix”, which became a robotic tortoise. The simple principles involved were:
Parsimony: simple is better. Simple reflexes are the basis of robot behavior.
Exploration or speculation: the system never remains still except when recharging. Constant motion is needed to keep it from being trapped.
Attraction: the system is motivated to move towards objects or light.
Aversion: the system moves away from certain objects, such as obstacles.
Discernment: the system can distinguish between productive and unproductive behavior, adapting itself to the situation.

95. G. Walter Grey's tortoise
These vehicles had a light sensor, touch sensor, propulsion motor, steering motor, and a two vacuum tube analog computer.

96. Robot History Timeline
1954 – George Devol replaced the slave manipulator in a teleoperator with the programmability of the CNC controller, thus creating the first “industrial robot”, called the “Programmable Article Transfer Device”.
1955 – The Darmouth Summer Research Conference marks the birth of AI. Marvin Minsky, from the AI lab at MIT defines an intelligent machine as one that would tend to “build up within itself an abstract model of the environment in which it is placed. If it were given a problem, it could first explore solutions within the internal abstract model of the environment and then attempt external experiments”. This approach dominated robotics research for the next 30 years.
Joseph Engleberger, a Columbia physics student buys the rights to Devol’s robot and founds the Unimation Company.
1961 – The first Unimate robot is installed in a Trenton, NJ General Motors plant to tend a die casting machine. The key was the reprogrammability and retooling of the machine to perform different tasks. The Unimate robot was an innovative mechanical design based on a multi-degree of freedom cantilever beam. The beam flexibility presented challenges for control. Hydraulic actuation was eventually used to alleviate precision problems.

97. UNIMATE robot

98. Robot History Timeline
1962 – 1963 – The introduction of sensors is seen as a way to enhance the operation of robots. This includes force sensing for stacking blocks (Ernst, 1961), vision system for binary decision for presence of obstacles in the environment (McCarthy 1963), pressure sensors for grasping (Tomovic and Boni, 1962). Robot interaction with an unstructured environment at MIT’s AI lab (Man and Computer – MAC project).
1968 – Kawasaki Heavy Industries in Japan acquires a license for Unimate.
1968 – Shakey, a mobile robot is developed by SRI (Stanford Research Institute). It was placed in a special room with specially colored objects. A vision system would recognize objects and pushed objects according to a plan. This planning software was STRIPS, and it maintained and updated a world model. The robot had pan/tilt and focus for the camera, and bump sensors.
– The Stanford Arm is developed, along with the first language for programming robots - WAVE.

99. Robot History Timeline
Late 1970’s – First assembly applications of robotics are considered: water pumps – Paul and Bolles, typewriter – Will and Grossman, Remote Center of Compliance gripper (RCC) developed at Draper Labs.
1970’s – Innovation in the type of robots introduced: Unimation 2000, Cincinnati Milacron (“The tomorrow tool, T3”) – the first computer controlled manipulator, the PUMA (“Programmable Universal Machine for Assembly”) by Unimation, the SCARA (“Selective compliant articulated robot for Assembly”) introduced in Japan and the US (by Adept Technologies).
1972 – First snake-like robot – ACM III – Hirose – Tokyo Inst. Of Tech.
1977 – Development of mobile robot Hilaire at Laboratoise d’Automatique et d’Analyse des Systemes (LAAS) in Toulouse, France. This mobile robot had three wheels and it is still in use.
1970’s – JPL develops its first planetary exploration Rover using a TV camera, laser range finder and tactile sensors.

100. Snake-like robot
                                                                                           
A. Hirose (Tokyo IT)

101. Snake (MIT) and Swimming (Eel) Robot (UHK)

102. Robot History Timeline
1980’s – Innovation in improving the performance of robot arms – feedback control to improve accuracy, program compliance, the introduction of personal computers as controllers, and commercialization of robots by a large number of companies: KUKA (Germany), IBM 7535, Adept Robot (USA), Hitachi, Seiko (Japan).
Early 1980’s – Multi-fingered hands developed, Utah-MIT arm (16 DOF) developed by Steve Jacobsen, Salisbury’s hand (9 dof).
– Stanford cart/CMU rover developed by Hans Moravec, later on became the Nomad mobile robot.
1980’s – Legged and hopping robots (BIPER – Shimoyama) and Raibert 1986.
– V. Braitenberg revived the tortoise mobile robots of W. Grey Walter creating autonomous robots exhibiting behaviors. Hogg, Martin and Resnick at MIT create mobile robots using LEGO blocks (precursor to LEGO Mindstorms). Rodney Brooks at MIT creates first insect robots at MIT AI Lab – birth of behavioral robotics.

103. KUKA
They can load, unload, deburr, flame-machine, laser, weld, bond, assemble, inspect, and sort.

104. IBM 7535
IBM 7535 Manufacturing System provided it advanced programming functions, including data communications, programmable speed.

105. Utah-MIT arm

106. Nomad mobile robot
The XR4000 is an advanced mobile robot system that incorporates state of the art drive, control, networking, power management, sensing, communication and software development technologies.

107. Rensselaer Polytechnic Institute CAT Robots
CAT-Mobile: Autonomous Tractor-Trailer Robot (Wen, Divelbiss, Popa)
Tetrobot: Modular & Reconfigurable Stewart Platform
(Sanderson & Lee)

108. Robot History Timeline
1990’s – Humanoid robots – Cog, Kismet (MIT), Wasubot, WHL-I – Japan, Honda P2 (1.82m, 210kg), and P3 (1.6m, 130kg), ASIMO.
1990’s – Entertainment and Education Robots – SARCOS (“Jurassic Park”), Sony AIBO, LEGO Mindstorms, Khypera, Parallax.
ROBOCUP, the competition simulating the game of soccer played by two teams of robots having been held around the world since 1997 (Osaka) .
1990’s – Introduction of space robots (manipulators as well as rovers – the MARS rover 1996), parallel manipulators (Stewart-Gough Platforms), multiple manipulators, precision robots (“Robotworld”), surgical robots (“RoboDoc”), first service robots (as couriers in hospitals, etc)

109. Lego Mindstorms

110. Asimo
Honda announced the development of new technologies for the next-generation ASIMO humanoid robot, targeting a new level of mobility.
               

111. Entertainment robots from SARCOS

112. Kismet – MIT AI Lab
Kismet consists of a head with large eyes with eyelids, bushy eyebrows, rubber lips, and floppy ears.

113. Cog – MIT AI Lab
Cog is a humanoid robot. It has a torso, arms and a head but no legs. Cog's torso does not have a spine but it can bend at the waist from side-to-side and from front-to-back and can twist its torso the same way a person can. Cog's arms also move in a natural way.

114. Hierarchical family of robots (K-Team - Switzerland)
Khepera (6 in)
Koala (20 in)
Alice (1 in)

115. ARV Wall-Climbing Robot for Fuselage Inspection

116. Robot History Timeline
2000’s – IRobot introduces the first autonomous vacuum – “Roomba”.
2000’s – Mini and micro robots, “Smart Dust” – Berkeley, UTA, EPFL/Lausanne, microfactories.
2000’s – Military applications - Robotic assistants for dangerous environments and reconnaissance, AUV’s and UUV’s, etc.
2000’s – Intuitive Surgical introduces the Da Vinci surgical robot.
2000’s – Robotic Deployment of Sensor Networks

117. 2006: Microsoft Introduces Robotics Studio
What is Microsoft Robotics Studio?
A window-based environment that is used to create robotics application
What does Microsoft Robotics Studio do?
Consider Robotics Application where we have several sensory inputs and
needed to be processed to command Actuators output
Microsoft Robotics Studio provide a programmatic way to create an
orchestrator that manage robotics applications (“Service”)
Inputs
Actuators
we sense several inputs we need to do something with them
we need to make use of the inputs information that we have to generate meaningful output
so we need some kind of coordination between interpretation of input to the action of output actuator
it’s called “Orchestrator”
Orhcestration means:……………………..
Multiple Sensory Inputs
Orchestrator
Multiple Actuator Outputs
Orchestration: “The task of consuming sensory input from a variety of sources and as a result manipulating a set of actuators
to respond to the sensory input.”

118. USC Mobile Robots
Robot teams (A. Howard)

119. Flying Insect (UCB)

120. Solar AUV II
SAUV-II from Autonomous Underwater Research Institute (AUSI) – New Hampshire

121. Hierarchical family of robots (UMN)
Scout & Ranger Series

122. The Humanoids are Coming

123. Human-Machine Interaction Research
Study of physiological effects of a robot on a human.
Measurements taken
Heart Rate (electrocardiograph)
Respiration
Perspiration
Pulse Wave
Blood Pressure
They used the above data to analyze the emotional state of the human participated in the experiment. They also have a 12 DOFs serial mechanism to capture a human motion so that they could incorporate motion with the data from the measurements. The experiment was performed by putting a human wearing all the measuring devices in the working environment with a robot to have interactions with humanoid robot and they interpreted all the data from measurements to indicate the participant’s stress level.
now I’m going to give some examples on humanoid research
these are current research topic that leading research groups are working on right now
this research focuses on the study of physiological effects of a robot on a human. They assume that when a human working with a robot for a long time it can cause stress to that human so they built a device that can attach human body so they can take these measurements
-heart rate
-Pulse Wave
-Blood Pressure
-Respiration
The experiment was performed by putting a human wearing all the measuring devices in the working environment with a robot to have interactions with humanoid robot and they interpreted all the data from measurements to indicate the participant’s stress level. They recorded certain actions that significantly increases or decreases the stress level of the human.
the goal is to investigate what kind of action can decrease the stress level the most
K. Itoh, H. Miwa, Y. Nukariya, M. Zecca, H. Takanobu, S. Roccella, M. C. Carrozza, P. Dario and A. Takanishi, Development of a Bioinstrumentation System in the Interaction Between a Human and a Robot. IEEE/RSJ International Conference on Intelligent Robots and Systems October 2006.

124. Social Interactive Robot Research
This research proposed a facial expression imitation system that consists of two part; facial expression recognition and facial express generation through a robot. The facial expression part is able to classify basic facial expressions includes; neutral, happiness, sadness, anger, surprise, disgust and fear.
In this paper, they developed an artificial facial expression imitation system that can recognize human’s facial expression and able to imitate human’s facial expression. They proposed a method of classifier that was suitable to implement to recognize human facial expression in real time
the facial expression imitation system that consists of two part; facial expression recognition and facial express generation through a robot.
the system can classify basic facial expressions includes; neutral, happiness, sadness, anger, surprise, disgust and fear.
S. Sosnowski, A. Bitterman, K. Kuhnlenz, and M. Buss, Design and Evaluation of Emotion Display
EDDIE. IEEE/RSJ International Conference on Intelligent Robots and Systems October 2006.

125. Research on Dynamics of Humanoid Robots
Whole body motion is very complicated, so when a humanoid robot is to be controlled remotely through teleoperation it is more intuitive to use marionette device to control the humanoid robot
base on the idea of teleoperation the research try to come up with the way to interface the humanoid robot with some kind of device that easy to manipulate
since the humanoid robot have many degree of freedom . it would be very difficult to control the motion of the robot with sliders
Whole body motion is very complicated, so when a humanoid robot is to be controlled remotely through teleoperation it is more intuitive to use marionette device to control the humanoid robot
T. Takubo, K. Inoue and T. Arai, Wholebody Teleoperation for Humanoid Robot by Marionette System. IEEE/RSJ International Conference on Intelligent Robots and Systems October 2006.

126. Research on Human-Robot Interaction
Haptic Devices
Robot Assisted Minimal Invasive Surgery
Parallel/Redundant Haptic Devices
Multi-Fingered Haptic Interface
some interesting topics includes
Robot Assisted Minimal Invasive Surgery
which is the top picture on the right
Parallel/Redundant Haptic Devices
which is the left picture
Multi-Fingered Haptic Interface
which is the bottom one on the right

127. Robotics Applications
Today, commercial robots are used routinely in the following applications:
Industrial Manufacturing – “Transforming objects” - arc/spot welding, milling/drilling, glueing/sealing, laser/water jet cutting, grinding, deburring, screwing, painting, and assembly.
Material Handling: “Pick and Place”- palletizing (placing objects on a pellet in an ordered way), warehouse loading/unloading, part sorting, packaging, electronic chip pick and place, hazardous material handling.
Measurement: object finding, contour finding, inspection, 3D registration.
Entertainment robotics: animated figures, flight simulator, robotic pets.
Service robotics: robotic aids for handicapped people, artificial limbs, robotic vacuum, courier.
Military robotics: defusing explosive devices, scout robots, UAVs.
Surgical Robotics: drilling, suturing, cauterizing, tool holding.

128. Robotics Applications
Robot prices continue to drop compared to the cost of human labor.
In the year 2000, 78% of all robots installed in the US were welding or material-handling robots.

129. Robotics Applications

130. Textbook Readings
Chapter 1 from textbooks