1. VISION BASED CONTROL OF AN AUTONOMUS BLIMP WITH ACTUATOR SATURATION USING PULSE WIDTH MODULATION
Korkut BEKIROGLU
Mario Sznaier
Bahram Shafai
Master Thesis Defense
I would like to thanks everybody for coming my thesis defense. Today I will be presenting my master thesis. The topics is Actuator saturation control for the blimp with pulse width modulation via visual feedback.
My name is K.B. My advisers are Professor Mario Sznaier and Bahram Shafai from Department of the Electrical and Computer Engineering.
The blimp is an Small scale of zeppelin

2. Outline Motivation & Objective System Overview and Mechanical Design
Identification of The System and Controller Design
Algorithms
Software of The System
Conclusion and Future Works
Thorough out this talk, We will briefly explain that why we made this study and then we will take quick overview to the system and mechanical design, then I will start theory of the system with identification of the system and Controller design.
Following part will include, I will briefly explain PWM algorithm. After that I will explain software of the system or how the GUI can be used to control the blimp automatically and manually. Finally I will wrap up with conclusion and future works

3. Motivation
Unmanned Aerial Vehicles (UAV) are used for military and civilian applications for surveillance and monitoring.
Object tracking and navigation of UAV are one of the most researched areas of the combination of the computer vision and control engineering.
Is very motivation for us. Actually its low speed and low altitude is an advantageous for monitoring and surveillance such as rush hour traffic to control it and fighting the forest fires. It collects data with fitted camera or thermal sensor.
These are very famous application areas for the blimp.
Additionally Object tracking and navigation of UAV are one of the most researched areas of the combination of the computer vision and control engineering is very challenging.

4. Objective Verification of The Mathematical Model of The Blimp.
Design a controller and PWM for an unmanned blimp to track a selected object inside of a room, using only horizontal motion.
Coding a Software to Drive The System Automatically and Manually.
First of all, after we establish all necessary components, it was also tough to connect parallel port with transmitter-receiver circuit couples. We decided an approximate mathematical model of the blimp, after we calculated it, we need to verified it by comparing with real data and calculated data. Next step became controller design for this calculated model. After Following section of controller design is that developing a software to control the blimp.

5. System Overview The camera mounted with an angle to the gondola.
Hardware Design
The camera mounted with an angle to the gondola.
Connection of the parallel port and Transmitter-Receiver Circuit.
Set up to frame grabber to the code
Software Design
Identification, Controller and PWM design
Software Design
Identification
Controller & PWM
At the beginning, We started with hardware design of the system. We mounted a camera with an angle to the gondola in order to maximize the field of view of the camera.
We made connection of the parallel port of computer and Transmitter circuit which necessary to send command to the receiver circuit of the propellers. Transmitter circuit was a remote controller of toy blimp. We modified it to connect the computer.
We then collected displacement data to verified mathematical model of the blimp.
Finally we improved the controller and PWM algorithms for the blimp and we implemented these controller and PWM to the system.
Implementation

6. Mechanical Design of The Blimp
Selection of the envelope of the blimp
Selection of the batteries
Wireless Camera and its receiver.
Selection of the Transmitter and Receiver for the Propellers.
Higher volume of the envelope, higher force is needed to drive the system. It effects robustness of the system. It is also related with maximum payload of the gondola.
Selection of the batteries effects size of the envelope and system dynamics indirectly. Lightest batteries need to be chosen.
It effects system algorithm. Any noisy image can break the algorithm.
They should also work properly or reasonably fast.

7. Mechanical Design of The Blimp
The connection to the transmitter circuit via the parallel port.
Pin Number
Pin(4)
Pin(5)
Pin(6)
Pin(7)
Forward (12) 1
Backward (48)
Spin Left (24)
Spin Right (36)
Left (8)
Right (4)
The figure shows connection of the transmitter circuit with parallel port of the computer.
The table also shows the output of the parallel port for the motion direction of the blimp. It shows that which number should be sent to pins of the parallel port. In other words, which pins should be on to drive the system to the decided direction.

8. How The System Works
Camera collects the live images and sends the captured image data to the camera receiver.
The captured image is received by the frame grabber to be processed by the code.
Camshift algorithm processes the frame to track the selected object.
The output of the camshift algorithm is compared with the reference signal to calculate the error signal. (In the code)
The wireless camera

9. How The System Works
Controller processes the error signal and controller output is sent to the PWM to form the required pulse signal to be sent to the actuators.
Output of the PWM is sent to the parallel port of the computer.
After transmitter circuit receives the data from parallel port, it sends the commands to receiver circuit
Actually for the transmitter circuit.

10. How The System Works
Motors receive the command and they drive the blimp according to the desired direction with modulated speed .
Camera grabs another frame and the same loop is carried out.

11. Sensor Camera + Camshift u vX u
Camera +
Camshift Y v Z
Real time sensing, real time location information of the target object.
The camshift algorithm, or Continuously Adaptive Mean-Shift algorithm, is a modified version of the Meanshift algorithm, developed by Intel.
The camshift uses color based histogram to define characteristics of the target object and mean-shift algorithm to search the image.
The basic idea of the mean-shift algorithm is a simple iterative procedure that shifts the each data points towards the highest average of the data points in its neighborhood. The real time mean-shift algorithm endeavors to maximize the correlation between two statistical distributions.
Real Time sensing, Real time location information,
Mean-shift algorithm was modified to cope with dynamically changing of the color probability distributions derived from image sequences.
It was actually developed as a fast and efficient way to track a given face in the presence of noise then it used to track other colorful objects as well.

12. Verification of The Mathematical model
Input-Output Relations (Transfer Function)
Newtonian laws of motion (Mathematical Model)
Aerodynamic Drag Force
Its inertia
Directly proportional with Density of fluid rho
Cross section area A
There are other factors affecting the drag force including shape, texture, viscosity, compressibility, lift and the boundary layer separation.
Coefficient of drag C

13. Approximate Model of The Blimp
Nonlinearity (Velocity Saturation) of the system because of the drag force
Double integrator because of its inertia
Because of aerodynamic force, there is a saturation nonlinearity on its velocity state, in other words, it’s velocity will be constant after a certain time.

14. Output Plots of The System
Displacement of the system
Second order Polynomial in an interval 3-11 (Linear part of the saturation)
Linear function after 11 second (After velocity of the system is saturated)
These figures show displacement of the system. We collected these data to verify the approximate model of the system. We drew a second order polynomial in an interval 3-11 seconds.. This is actually linear part of the saturation. Then we drew a linear function after 11 second which is after system is saturated.
This is very important information that shows when the system saturates.

15. Controller Design
Controller design for double integrator. (P-PI-PD-PID Controllers)
Jury's stability test and bilinear transformation were used to decide range of the gains.
When T=0.6 sec, Kp=0.2 was chosen and Kd=1 was calculated. K=0.6
1.Proportional (P) controller, it is clear that the roots of the system move to the outside of the unit circle which can be seen from the root locus of the system.
2.Proportional-Integrator (PI) type of controller would not change anything since it only increases the system type but does not bring any additional control action.
3.The PID type of controller might stabilize the system but degrade the performance of the tracker by
creating unnecessary overshoot because of the extra integrator that it brings to the overall system.

16. Controller Implementation
System is type 2 and steady state error is zero.
Controller
Adjustement of the controller parameters.Kp=2,Kd=3 and K=1 when T=0.6 sec
Time domain controller,
We don’t want too much overshoot since target object should be kept inside of the screen. Otherwise system becomes unstable.

17. Controller Implementation and Closed Loop
The continuous approximate model of the system,
The discrete controller,
The sampling element,
Zero Order Hold (ZOH),
The saturation block between two integrators,
The PWM block
It is consist of

18. Reference Input Reference input.
Decide a bigger frame to decide percentage level of the PWM.
Axis X Y
Xrefmin 40
Xrefmax 80
Yrefmin 60
Yrefmax
100
Reference input is a set. The output of the camshift is compared with this set.
We can not choose a single point because of its inertia.
Due to the problems explained in Chapter 2 and 3 such as big envelope for the propellers, sensitivity and noisy image, the reference input cannot be a single point.

19. PWM Algorithm
One larger frame was experimentally set around the reference frame.
After deciding the motion direction, percentage level of PWM is calculated
Maxpwm=12*10 and Maxpwm2=12*5
Forward-Backward
PWM*100
Right-Left
X<20 1
Y<30
20<X<40
Cont.out/Maxpwm2
30<Y<60
Cont.out/Maxpwm
40<X<80
60<Y<100
80<X<100
100<Y<130
100<X<120
130<Y<160
First of all current position of the blimp should be decided. This is a critical information to decide new motion direction of the blimp. Then algorithm decides that how much propellers need to make a thrust for the blimp according to current position of target object.

20. PWM Algorithm The average value of power to operate the propellers.
When the blimp moves towards to the reference frame, the blimp has to be made thrust with as small as possible power according to closeness to the reference frame.
If it doesn’t push the blimp with full force, the camera can loose the target object in the screen.

21. Graphical User Interface
Automatic and Manual Control of The Blimp
Functionality for Selection of The Target Object
Buttons for Identification and Object Save and Load
Left top buttons to control the blimp manually. The control buttons are meant to be ON or OFF the motors.
One extra Fw Step Experiment button was added with the purpose of giving step input in order to use it for the verification process of the mathematical model of the blimp.
Save button saves x, y coordinates and time after press the step experiment button.
Press video button to see the video and press stop button to stop showing the video.
It’s proportion of the current density to density of the selected object.

22. Discussion
The verification of the mathematical model of the blimp was made.
This study demonstrated an effective way to track any target object with UAV by using camshift algorithm, PD controller and the PWM
The blimp control software was also developed to control the blimp as an indoor application.
The study shows how the PWM overcomes the nonlinearity of a system.

23. Future Work Better identification is needed for more uniform dynamics
Each single propeller dynamics and other components dynamics should be defined properly.
Define aerodynamic drag force with known parameters.
Replaced camera and receiver
Minimize the noisy images.
Improve Camshift Algorithm for real time
More powerful propellers for the envelope.

24. References Developing a Low-Cost Autonomous Indoor
Blimp. R. Sanz P. Gonzalez, W. Burgard and J.L. Fernandez. Journal of Physical Agents.
An autonomous blimp for a surveillance system. Kazushi Fujitanit Takanori Fukaot and Take Kanadet. Intelligent Robots and Systems.
Autonomous blimp with real-time visual tracking and image projection. Kyungjoon Lee Seungyong Oh, Sungchul Kang and Euntai Kim. Intelligent Robots and Systems.
Computer vision-based navigation for autonomous blimps.Vijay Kumar Lucio De Souza Coelho, Mario Fernando Montenegro Campos. Computer Graphics, Image Processing, and Vision.

25. References Mean shift, mode seeking, and clustering.
Yizong Cheng. Applications of Computer Vision.
Real time face and object tracking as a component of a perceptual user interface. Gary R. Bradski. Applications of Computer Vision.
Controller design for linear unstable systems with position and rate actuator saturation. Patrice Langouet Sophie Tarbouriech, Germain Garcia. IEEE International Conference on Control Applications.
Stability regions for linear systems with saturating controls via circle and popov criteria. C. Burgat C. Pittet, S. Tarbouriech. Conference on Decision Control.

26. Demo

27. Thanks Prof. Mario Sznaier Prof. Bahram Shafai Prof. Octavia Camps
Prof. Rifat Sipahi
Mustafa Ayazoglu, Necmiye Ozay
Robust Systems Laboratory
My wife and family
I would like to sincerely thanking Prof. Mario Sznaier for giving me the opportunity to work on this thesis and providing the facilities.
I would also like to thank Prof. Bahram Shafai for advising me whenever I needed.
I would also like to thank Prof. Rifat Sipahi for sharing with me his ideas for this thesis.
I would also like to thank Mustafa Ayazoglu and Necmiye Ozay all of the help and explanations.
Finally, I would like to thank my wife, Saliha, for all of her love and support and for putting up with me through all of the stressful times.