1. Intelligent off-road vehicles Martin Servin, Department of Physics, 2008-04-02 www.umu.se\proj\ifor www.umu.se\proj\ifor

2. Outline Background to the field Overview IFOR Autonomous navigation Crane automation Simulator based design Feel free to ask questions and make comments and proposals!

3. A sample of technological gems… Mars rover – extreme teleoperation Deep Blue – reasoning computer DARPA Grand Challenge – competition with autonomous vehicles QRIO – balancing robot Parthenon – virtual 3D reconstruction HCI – retinal display

4. The off-road challange Demand for new technology Increased productivity Increased safety and work environment Environmental sustainability The forestry challange Complex work processes to automate – no computer beats the human in running a harvester Rough environment with big variations – sensor vision in forest, robust and sustainable system handling vibrations, moist and dirt Vision from forestry industry “2025 – Ingen man i maskinen, ingen hand på spakarna “

5. An initiavive for R&D for intelligent off-road technology Initiated by the industry in 2001 Collaboration between academia and industry = a forum for R&D and a collection of projects focused at IFOR technology What is IFOR ? Academia: Umeå University Swedish University of Agricultural Sciences Skogforsk Industry: Komatsu Forest Holmen Skog Sveaskog BAE Systems Hägglunds LKAB + network of other research centers and companies

6. Technology vision Improved work environment Increased productivity and cut costs Increased safety Reduced environmental impact Technology:Goals: Control algorithms and modeling Interaction – man, machine and environment Sensor vision Localization and map building 200120252010 Automation of routine work processes Crane tip control Unmanned vehicles

7. Activities and projects Autonomous navigation Dr Thomas Hellström 1 PhD students Computing Science Department Smart Crane Control Prof Anton Shiriaev 1 FoAss, 1 PostDoc, 3 PhD Department of Applied Physics and Electronics Vehicle simulators Dr Martin Servin In collaboration with VRlab at UmU Miscellanious - Seminars and workshops - Experiments and pre-studies - Student projects Equipment Forest machines – valmet forwarder and harvester Full sized in-door hydraulic crane Portable prototyping hardware for feedback control Sensors (dgps, laser radar, hydraulic pressure, stereo camera,…) Simulator systems Funding > 25 MSEK since 2001 Kempe Foundation, Sveaskog, Vinnova, Komatsu Forest, Sparbanksstiftelsen Norrland, Umeå University, LKAB, BAE Systems Hägglunds Other actors SLU Skogforsk Applied Mathematics – Prof Mats G Larsson Design Institute UCIT / ProcessIT Innovations

8. Autonomous navigation Dr Thomas Hellström 1 PhD students - unmanned transportation of logs - localization, path tracking and path planning - RTK-DPGS with cm accuracy - laser scanners, radars,... - machine learning and sensor fusion - first prototype demonstrated in Dec 2005 - ”Simulator in the loop” Autumn 2008 we are running the student DBT-projects: - Sensor vision and remote operation - Simulation of terrain vehicle with autonomous abilities

9. www.cs.umu.se/research/ifor/IFORnav/videos.htm

10. Smart Crane Control Prof Anton Shiriaev – Control System Theory 1 FoAss, 1 PostDoc, 3 PhD - motion planning, motion control for mechanical systems - feedback design for hydraulically actuated cranes - crane tip control - optimized motions – speed and stability - semi-automation, e.g. automatich loading - VR-enabled remote operation - portable prototyping hardware for feedback control Recent results: - motion faster and more stable than human operator – Valmet forwarder - demonstrated automatic loading in lab Grant from “Stiftelsen för strategisk forskning” for crane control using only hydraulic measurements at Komatsu Forest 1 industrial PhD have been granted (?) - Komatsu Forest and Umeå University splitting the costs 50-50 – Semi-autonomous harvester control system

11.  Fast crane motion.avi Motion faster and more stable than human operator is possible!

12.  Virtual Environment Teleoperation Click control.avi Detection of rotating log.avi

13. Visual Simulation of Machine Concepts for Forest Biomass Harvesting Martin Servin, A. Backman, K. Bodin - Umeå University, Sweden U. Bergsten, D. Bergström, T. Nordfjell, I. Wästerlund - Swedish University of Agricultural Sciences, Sweden B. Löfgren - Skogforsk (the Forestry Research Institute of Sweden) VRIC 2008 – 10th International Conference on Virtual Reality (Laval Virtual)

14. Outline Simulator-based design Forest biomass harvesting –concept machine and work method Experiments in simulator environment –system and procedure –purpose: find optimal harvesting technique and machine design Training simulator technology – also for concieving new machines concepts and work methods

15. Simulator-based design (SBD) Simulation tools are converging – R&D process impoves – cross-disciplinary participation Extension of virtual prototyping and simulation to include human-in-the-loop Fast and sheap Simulators – complex yet controllable environments Figure from T. Alm ”Simulator-based design” (2007). End customer Manufacturer Designer Researcher Engineer Simulator training

16. Application of SBD to: Forest biomass harvesting Increasing demand for forest biomass Early harvesting/thinning is becoming profitable Large volumes and areas, small income per unit, energy consumption Crucial to use optimized technology – economically and environmentally sustainable Uncertain on what solution to choose for thinning Virtual and real prototypes are important!

17. New harvesting methods in dense forest stands Early harvesting = thinning + biomass harvesting - single-tree harvesting - multi-tree harvesting - geometric area based felling strip roads 3 m wide every 15-20 m corridors 1x10 m 10 trees, 6 m, 50 kg collect in piles of 50 trees

18. Machine concept for harvesting in dense forest stands Size: 4x2 m, 2.5 ton, 8m reach Mobility: indv 4W on pendulum arms Harvester head: multi-tree vs blade Control and HMI: boom-tip control, semi-autonomous, teleoperation (direct or VE), laser scanner & stereo camera, dynamic 3D maps from sattelite and AUV

19. Experiments in simulator environment - system and procedure System components software: Colosseum3D (OSG, Vortex – AgX Multiphysics, lua,…) hardware: full simulator environment (screen projection, authentic chair and joysticks, motion platform) or portable case, convential multicore PC models: data from real forest stands in 3D terrain, vehicle = 20 rigid bodies coupled by kinemtaic constraints (wheel suspension, crane joints,…) vehicle automation and HMI module: vehicle control, automation, sensor, 3D-map engine and HMI interface The application requires advanced real-time physics: terramechanics, stacking, hydraulics,…

20. Experiments in simulator environment - system and procedure Experiment procedure Task: do harvest thinning in a given dense forest stand Variations: - forest stand (distribution, species, topology) - harvester head (single, multi, sword) - vehicle (existing machines, new proposals) - automation and HMI (manual, semi-automatic, fully auto) - operator Measurements: - time per biomass unit in kg (strip road, corridor, tree, move to pile, positioning, transport) - energy consumption - work environment Optimize: find optimal mechine design and work method – data from simulator experiments used in logistics computation