1. THE TECHNICAL VISION SYSTEM FOR DIAGNOSIS OF THE HARVESTING UNIT OF THE ROBOT FOR GATHERING WILD PLANTS
A.I. Kuznetsov, A.V. Tyryshkin
National Research Tomsk Polytechnic University Department of Control Systems and Mechatronics
Tomsk, Russia
2017

2. Topicality Advantages of using robots in agriculture:
productivity increase;
cost reduction;
reduction of labor intensity of service;
improving the quality of products.

3. Fig.1. Prognosis of dynamics of the market of agricultural robots
Topicality
Fig.1. Prognosis of dynamics of the market of agricultural robots

4. Topicality
Reserves of cranberry marshes of Tomsk region more 10 million tons
Manually collects no more than 15% of the entire crop.
The use of robots promises increased collection efficiency.

5. Topicality The collection unit can often fail.
In the event of a malfunction, the robot will continue to move, but the cranberries will not be collected.
The diagnostic system will prevent the work from being "idle".

6. Review of existing solutions
Fig.2.AGROBOT HUELVA SW6010
Fig.3. Octinion
Fig.4. Shibuya Seiki
Fig.5. BoniRob
All existing systems are designed to work in artificial conditions.

7. Goals and objectives Main idea:
create diagnosis system for solving problem controlling the operation of harvesting unit.
Function:
diagnosis of the harvesting unit
determination of the efficiency of berry picking;
increase the convenience of servicing the robot;
improving the robot's economy.
Goals:
choose of the principle of the system;
choose of the realization method;
development of the work algorithm;
realization of the algorithm;
debugging and adjustment.

8. Choose of the principle of the system
Monitoring the details
Control of the main function
Benefits:
simplicity of the diagnostic algorithm;
completeness of information about the state of the parts of the node.
Benefits:
tolerance to various design options;
the possibility of collecting statistical data;
lack of direct contact with responsible details;
ease of installation and transfer of the system.

9. Choose of the realization method
Main function of harvesting unit is – decrease in the number of berries

10. Figure of the proposed solution
Computer
Harvesting unit
Autonomous platform
Camera №1
Camera №2
Fig.6. Harvesting robot

11. Development of the work algorithm
Main functions:
determination of the malfunction of the harvesting unit;
determining the efficiency of the harvesting unit;
control zones without berries;
setting the parameters of the recognized berries;
setting options for displaying error messages.

12. Algorithm
Fig.7. Block diagram

13. Development of the function of recognition and counting of berries
One of the main distinctive features of cranberries is the color.

14. Development of the function of recognition and counting of berries
Fig.8. Dependence of the color of the berries on different lighting conditions

15. Setting cranberry options
Select a sample of the berries from the photo.
Select a color sample from the tone ruler.

16. Development of the function of recognition and counting of berries
Fig.9. Recognition results for a given color

17. Development of the function of recognition and counting of berries
Fig.10. The graph of the dependence of the amount of the recognized berries on the size of the color range of the berries

18. Development of the function of recognition and counting of berries

19. Development of the function of recognition and counting of berries
Noises:
presence of parasitic objects of similar color;
complex lighting conditions that require a larger range of berry color.

20. Development of the function of recognition and counting of berries
A contour is a line that delimits an object and characterizes its shape.
Border - the place of sharp changes in color, brightness or other color coordinates.

21. Development of the function of recognition and counting of berries
Выделение границ с использованием контрастности
Fig.11. Searching for borders on image with transformation by the Laplace operator

22. Methods of preprocessing
Fig.12. Contrast
Fig.13. Sharpness

23. Methods of preprocessing
Fig.14. Searching for borders with contrast preprocessing
Fig.15. Searching for borders with sharpness preprocessing

24. Development of the function of recognition and counting of berries
To determine the conformity of the form, the Hafa transformation method is used.

25. Development of the function of recognition and counting of berries

26. Service information The average efficiency factor is used.
Finding zones without berries.
Fig.16. Service information

27. Advantages Simplify the maintenance of the robot.
Increase the working time.
Access to statistical indicators.
Savings of money due to the lack of a permanent operator.

28. Conclusion
A system for diagnosing of the harvesting unit of the robot for gathering wild plants was developed.
Algorithms for detecting and counting cranberries in the image are created.
Complex external conditions are taken into account.
The output of service information

29. THANK YOU FOR ATTENTION
A.I. Kuznetsov, A.V. Tyryshkin
National Research Tomsk Polytechnic University Department of Control Systems and Mechatronics
Tomsk, Russia
2017