1. Robot Homing Workshop

2. 1.The Camera Prospective Projection Modely' r r’ z y x x'
(x’,y’) f
Image Plane
inverted
Object point
(x,y,z)
Figure 1.1
The camera pinhole model

3. Image plane: shifted and rotatedX Y x y r x’ y’
(x,y,z)
(x’,y’) Y’ X’ r’ Z
Figure 1.2
Image plane: shifted and rotated

4. 2.Affine Invariant Shape Correspondence Model
Let Shapes C and C’ be described by points lying on their boundaries:
Supposed the following relation exits:
C’ is an affine transformation version of C (Or vice versa)

5. For any three points
from shape C and their
corresponding points
from shape C’ there exists
the relation:
( Determinant notation )
The triangle area created by is
(2.1)

6. 2.1 Shape Feature Vector Let’s look at For each boundary point
we define the feature vector
Let
Where
(-/+ cyclic around N)
is the triangle area created by

7. can be groups into 3 parts:
local features
global features
and again local features

8. is normalized to
Now for shape C we can define the set:
For
in a similar way we have:
According to 2.1
we get

9. For each point
and its affine-transformed corresponding
point
there exits

10. 2.2 Global Shift Index
Let Shape C’ be an affine-transformed version of C.
If the points from the shape’s boundary have been taken ‘smoothly’ then:
( + is cyclic around N )
S is the global shift Index.

11. Estimating s Estimation of s means finding the most suitable number
to be the value of s.
G is
diagonal matrix
The chosen value for s is

12. 2.3 Similarity Matching of Shapes
Shapes C and C’ are considered similar if they are the same or if one shape is an affine-transformed version of the other.
Let’s define
It is assumed that the feature vector set
is a unique characteristic of
of a particular shape (or its affine-transformed version)
for similar shapes
is quite small (ideally 0)
for different shapes
should be larger

13. Local Shift Index If shapes C and C’ are similar, for each point on C
the suitable point
on C’ can be found locally:
Let n be the number of times when
For similar shapes n should be close to N.

14. Similarity Matching Function
For shapes C and C’ If
C and C’ are similar
C and C’ are different If

15. 3.Robot Homing Workshop

16. 3.1 Workshop Hardware and Software
Matlab: Image Processing
Lejos
vfm
Matlab: Getting Image

17. 3.2 Workshop Hypothesis Camera Pinhole Model
Image received by the camera is twisted according to:
The camera (the robot) is ‘far enough’ from the arena walls
The ratio
is approximately constant
The received shape image is an affine-transformed version of the shape printout.
The similarity matching function can be applied to identify the home shape.

18. 3.3 The Matlab Program Aspects
Traveling around the arena and grabbing the wall images
Analyzing the received images and retrieving the shape figures
Determine whether the retrieved shape is the home shape
Traveling towards the home shape when it has been detected

19. 4. Analyzing The Camera Images
4.1 Image Segmentation
Converting the RGB Image Into a gray level one.
Applying Canny Edge Detector with threshold on the converted image.
RGB Image
Gray Level Image
Applying Edge Detection

20. An Image containing two 8-connected elements
Finding 8-connectivity Image Elements. 1 2 3 8 4 7 5 6
Pixel’s 8 neighbours
An Image containing two 8-connected elements

21. 8-Connectivity Element Finding Algorithm
  1.  Scan the image left to right, top to bottom.
2. If pixel value is not 0, then
(a) If only one of its upper and left neighbours (neighbours 1,2,3,8 the previous figure )
has a label, than copy it.
(b) If all neighbours have the same label, then copy the label.
(c) If they have different labels, then copy one of them and enter the labels in
the equivalence table as equivalent labels.
(d) Otherwise assign a new label to this pixel and enter this label in the
equivalence table.
  3. If there are more pixels to consider, then go to step 2.
  4. Find the lowest label for each equivalent set in the equivalence table.
5. Scan the picture. Replace each label by the lowest label in its equivalent set.

22. Suitable Image Element
4.2 Analyze of Image Elements
Image Element can be one of the following groups:
A whole shape Boundary
Shape Image
Suitable Image Element

23. 2. Part of A shape Boundary 3. Arena Corners 4. Arena wall
Image of partial shape & arena corners
Image of the arena wall
Suitable Image Elements
Image Elements due to blank wall image

24. 4.2.1 Wall Detection 4.2.2 Close Line Detection
There are “too many” Elements in the image.
4.2.2 Close Line Detection
A whole shape boundary is a smooth close line.(Hamiltonian circle)
The Image Elements due to the arena corners are open lines.
We need to find out weather the image element is a closed line .
We should find the element boundary.
Image Pixel
Image Interior Pixel
Image Boundary Pixel

25. Boundary Following Algorithm
1. Find the starting pixel
for the region using a systematic scan,
say from left to right and from top to bottom of the image.
2. Let the current pixel in the boundary tracking be denoted by c.
Set
and let the 4-neighbour to the left of s be 1 2 3 8 4 7 5 6
Pixel’s 8 neighbours
3. Let the 8-neighbours of c starting with b in the clockwise order be
. Find
, for the first i that is in S.
4. Set
and
5. Repeat Steps 3 and 4 until

26. returns its boundary pixels ordered clockwise.
Applying the algorithm to a whole shape boundary (“smooth” closed line)
returns its boundary pixels ordered clockwise.
If the image element is a smooth closed line the first boundary pixel to be
come across again is only the starting one c.
This Algorithm doesn’t enable to distinguish between open lines due to
the arena corners and open lines due to partial boundary of a shape
(which may be of interest).
Image
Image Elements

27. 4.2.3 Detection of the Home Shape
The boundary of the shape considered home is a closed line.
The (synthetic) home figure is stored in the robot data base.(simply a file)
and its boundary is of course closed.
We can use the the matching algorithm described above.
Sampling in “smooth manner” N points from the home boundary as
in the data base.
Sampling in “smooth manner” N point from the received shape
boundary (suitable element pixels)
Applying the matching algorithm

28. Problems: The original home shape as in the data base
The shape as received by
the robot camera
The received shape is twisted and only approximation of an affine transformation.
The sample points from the received shape can not be promised to be
the corresponding affine transformation version of the sampled points from the
original home shape.
How to determine
and

29. Partial Solutions:
Intuitively the above problems mainly affect the values of the local features and
and have less influence on global features.The matrix reduces the weight of
the local features.
The values of
and
should be found experimentally.

30. 4.2.4 Calculation Performed on a Received Shape.
Weight of its boundary ( number of pixels)
The area of the shape bounding box
The shape’s center of mass
4.2.5 A shape at the image margins
It results in an open line element at the image margins
Elements at the margins should be treated.
Problem: Elements can also be due to the arena corners.

31. Element in Image Margins?
Edge Detection
Segmentation
Element
Wall?
Closed Shape?
Calculate:
Boundary Pixel Number
Bounding Box Area
Center of Mass
Home?
yes
Image Analysis

32. 5. Scanning For the Home Shape
For Traveling inside the arena we need to deduce the 3D orientation out of 2D images.
Implementing spatial orientation is not an easy task and beyond the scope…
The Scanning process is very limited and primitive.
Detecting the home shape is not promised.
5.1 The Scanning Cycle
The search process is divided into finite number of so called scanning cycles.
The search is terminated after all the scanning cycles have been exhausted,
regardless the home finding.

33. Home Scanning Procedure
Start
Scan cycle
Closed
Shape
Found
Home
Wall
Detected
Should find a
shape OR
Something in
top margin
Get an Image
Analyze it
Move towards
home
Check if the shape
is the smallest
Refine Location
Check if should find
shape in next image
Scan cycle ends
Turn towards the
Smallest image
Rotate the camera
in small angle
Scan cycles exhausted
Move forwards
Finish
Scanning
yes no
Home Scanning Procedure

34. 5.2 The Refine Location Procedure.
The procedure handles two case:
There is an element at the top margin of the image so it might be a part of a shape
to which the robot camera is to close.
The current image analysis yields no close shape whereas according to the previous
image a shape should have been found.(element at the left/right margin)
Problem: Because of the insufficient segmentation the elements can be due to
the arena walls,texture and etc.
A finite number of trials should be taken in order to find a shape.

35. Refine Location Case 1 Begin End IterNum  1 Home Found 0
and
Element at top margin
Begin
Go backwards
Get an image
Analyze it
Closed
Shape
Found
Home
IterNumIterNum +1
HomeFound1
yes no
Move towards
home
End
Refine Location Case 1

36. Refine Location Case 2 yes no i = 1 Turn back in small angle i<=2
Home
Found
Closed
Shape
HomeFound1
i = 1
Turn back
in small angle
Get an image
Analyze it
ii +1
Return to the
initial orientation
i<=2 no
yes
Move towards
home
End
Refine Location Case 2

37. 5.3 Moving towards Home Procedure
Once the home shape has been detected,the robot moved towards it.
Problem: The home shape doesn’t have to be in the image center so while moving towards it
the shape might be out of the sight field of the camera.
Calculating the home shape center of mass.
The robot moves forward in small “steps” checking after each step if the home shape
is still in camera field of sight .
If not, The robot moves backwards to its previous location and turns a “little” towards
the home shape according to the location of the shape’s center of mass.
Problem: The robot shouldn’t approach the home shape too much,otherwise the
affine transformation approximation won’t be correct to be used and
the matching algorithm won’t be consistent.
Calculation the area of the home shape bounding box.
If the area is “too large” the robot stops and doesn’t try to get closer.

38. yes Moving towards home no Begin End Move forwards Get an image
Close enough to home
Move forwards
Get an image
Analyze it
Home
Found
Move back
Turn back to
initial orientation
Turn towards
home
yes no
Moving towards home