1 Early Pest Detection in Greenhouses Vincent Martin, Sabine Moisan INRIA Sophia Antipolis Méditerranée, Pulsar project-team, France

2 2/12 Motivation: reduce pesticide use Agricultural issues: Temperature and hygrometric conditions inside a greenhouse favor frequent and rapid attacks of bioagressors (insects, spider mites, fungi). Difficult to know starting time and location of such attacks. Reduce time overhead of workers in charge of greenhouse biological monitoring Understand better pest population behaviors Computer vision issues: Automatically identify and count populations to allow rapid decisions Improve and cumulate knowledge of greenhouse attack history

3 3/12 DIViNe 1 : A Decision Support System 1 Detection of Insects by a Video Network Identification and counting of pests Manual methodDIViNe system Result delivery Up to 2 daysNear real-time Advantages Discrimination capacity Autonomous system, temporal sampling, cost Disadvantages Need of a specialized operator (taxonomist); precision vs. time Predefined insect types; video camera installation

4 4/12 Proposed Approach Intelligent Acquisition Intelligent Acquisition Detection Classification Tracking Behaviour Recognition Behaviour Recognition Regions of interest Pest identification Pest trajectories Scenarios (laying, predation…) Image sequences with moving objects Pest counting results Current work Future work Automatic vision system for in situ, non invasive, and early detection Based on a video sensor network Lined up with cognitive vision research (machine learning, a priori knowledge…)

5 5/12 First DIViNe Prototype Network of 5 wireless video cameras (protected against water projection and direct sun). In a 130 m 2 greenhouse at CREAT planted with 3 varieties of roses. Observing sticky traps continuously during daylight. High image resolution (1600x1200 pixels) at up to 10 frames per second. 400€

6 6/12 Intelligent Acquisition Module Scheduled image sequence acquisition: at specific time intervals, on motion detection Distant tuning of each sensor settings (resolution, frame rate) Storage and retrieval of relevant video data

7 7/12 Detection Module Handle illumination changes due to sun rotation, shadows, reflection… Adapt algorithms to deal with different image contexts Cloudy context with reflections and low contrast Sunny context with shadows and high contrast video clip

8 8/12 Detection Module: Preliminary Results Acquisition: sticky trapDetection without context adaptation with context adaptation Weakly supervised learning to acquire context knowledge from global image characteristics Context identification for background model selection video clip

9 9/12 Classification Module: Preliminary Results Regions labeled according to insect types based on a visual concept ontology: Domain Class WhiteFly SuperClass Bioagressor { ShapeConcept Descriptors: circularity [ ] excentricity [ ] rectangularity [ ] elongation [ ] convexity [ ] compacity [ ] ColorConcept Descriptors: saturation [ ] lightness [ ] hue [ ] SizeConcept Descriptors: area [ ] length [ ] width [ ] }

10 10/12 Conclusion and Future Work A greenhouse equipped with a video camera network A software prototype: Intelligent image acquisition Pest detection (few species) Future: Detect more species Observe directly on plant organs (e.g. spider mites) Behaviour recognition Integrated biological sensor See

11 11/12 Behavior description based on a generic declarative language relying on a video event ontology Scenario models based on the concepts of states and events related to interesting objects. state = spatiotemporal property valid at a given instant and stable on a time interval. event = meaningful change of state. scenario = combination of primitive states and events by using logical, spatial or temporal constraints between objects, events, and states. Behavior Recognition Module Laying scenario example state: insideZone( Insect, Zone ) event: exitZone( Insect, Zone ) state: rotating( Insect ) scenario: WhiteflyPivoting( Insect whitefly, Zone z ) { A: insideZone( whitefly, z ) // B: rotating( whitefly ); constraints: duration( A ) > duration( B ); } scenario: EggAppearing( Insect whitefly, Insect egg, Zone z ) { insideZone( whitefly, z ) then insideZone( egg, z ); } main scenario: Laying( Insect whitefly, Insect egg, Zone z ) { WhiteflyPivoting( whitefly, z ) // loop EggAppearing( egg, z ) until exitZone( whitefly, z ); then send(”Whitefly is laying in ” + z.name); }

12 12/12 Plant Organs Monitoring Issues: Plant motion estimation ( + need of auto focus sensors) Non planar field of view choice of the sensor positions