Quick Overview of Robotics and Computer Vision

Computer Vision Agent Environment camera Light ?

Computer Vision Applications Light is probably the most valuable information that humans sense about their environment, and CV has a ton of applications: 1.Object classification 2.3D Reconstruction 3.Motion Analysis

Bag-of-visual-words processing: A common CV pipeline for object classification Converts an image into a vector, much like the bag-of-words model for text. After that, you can apply standard ML techniques to build a classifier.

Another common pipeline: Feature detection + correspondences 1.Find “features” in an image. These are typically things like corners, edges, or just very distinctive image patches.

Another common pipeline: Feature detection + correspondences 1.Find “features” in an image. 2.Find correspondences between features in similar images. Blue line shows a correspondence between features in stereo images.

Another common pipeline: Feature detection + correspondences 1.Find “features” in an image. 2.Find correspondences between features in similar images. 3.Recover depth

Improved Stereo Vision Using two (or more) cameras to recover the “depth” of pixels in a scene is called Stereo Vision or Stereo Reconstruction. Some techniques to improve it: -Structured light (like the Kinect) -Other sensors (eg, laser range finders) feature=player_embedded

Robotics: Putting it all together Agent Environment cameras Light, Laser, GPS  Representation of beliefs and goals  Estimation of state based on previous state and sensor information  Planning of actions to reach goals  Learning from previous examples to improve over time. motors Motion

Robotics Quiz For each of the following problems, which technique from our class would you use to solve it? 1.Determine the position of the robot, based on the sensor reading at the current time step, and the belief about the position at the previous time step.

Robotics Quiz For each of the following problems, which technique from our class would you use to solve it? 2.Given a road map, a belief about the starting position, and actions for driving forward and turning, determine a sequence of actions to get you to a desired point on the map.

Robotics Quiz For each of the following problems, which technique from our class would you use to solve it? 3.Given a camera feed aimed at an object on a table, come up with a sequence of robotic arm/finger motions that will pick up the object.

Robotics Quiz For each of the following problems, which technique from our class would you use to solve it? 4.Given a set of examples of robotic car maneuvers in a fixed area (several city blocks, say), some that successfully got to their goal, and some that didn’t (crashed or failed to reach the goal), determine the maneuver for each location in the area.

Additional Robotics Questions Representation: How complicated is it (how many variables/dimensions do you need) to represent the position and orientation of a robot car on a map? How many variables/dimensions do you need to represent the position, orientation, change in position, and change in orientation of a robot car on a map? The position and orientation are called the kinematic state of a robot. The position, orientation, change in position, and change in orientation are called the dynamic state of a robot.

Answer: Additional Robotics Questions Representation: How complicated is it (how many variables/dimensions do you need) to represent the position and orientation of a robot car on a map? How many variables/dimensions do you need to represent the position, orientation, change in position, and change in orientation of a robot car on a map? For a 2D world, the kinematic state requires 3 variables (dimensions): 1 for position on the x axis, 1 for position on the y axis, and 1 for the compass direction (or angle away from North). For a 2D world, the dynamic state requires 5 variables (dimensions): 3 for the kinematic state, 1 for forward velocity, and 1 for “turning velocity”, or yaw.

Additional Robotics Questions 3D Representation: How many variables/dimensions are needed for the kinematic and dynamic states of a robot helicoptor? Many-D Representation: How many variables/dimensions do you need for a robotic arm that has six rotation joints?

Answer: Additional Robotics Questions 3D Representation: How many variables/dimensions are needed for the kinematic and dynamic states of a robot helicoptor? 6 for kinematic, 12 for dynamic. Many-D Representation: How many variables/dimensions do you need for a robotic arm that has six rotation joints? The answer really depends on the orientation of the joints with respect to one another. But if each joint provides independent motion from the previous joints, you would need 6 variables for kinematic state (angle of each joint), plus 6 additional variables for dynamic state (change in the angle of each joint).

Some robotic videos $3000 DARPA robot hand (very cheap for a robot hand): mbedded&v=NvhCk6BvLBE Weird robot: