1 Detection, tracking and sizing of fish of in data from DIDSON multibeam sonars Helge Balk 1, Torfinn Lindem 1, Jan Kubečka 2 1 Department of Physics, University of Oslo, PO.Box Blindern, NO-0317 Oslo, Norway 2 Biology Centre of Czech Academy of Sciences, Institute of Hydrobiology, Na sadkach 7, CZ Ceske Budejovice, Czech Republic.

2 CFD AND DIDSON tracking 3D approach Detection methods Echogram approach Introduction Conclusion Inc.Video methods

3 Placing Norway on the map University of Oslo No Biological institute Cz

Our main interest u As usual to find out abot the fish u How many u How big u What are they doing 4

Equipment that may be used Resons-Seabat Coda Octopus Echoscope DIDSON Simrad MS70 Simrad SM2000 Split beam

DIDSON u Dual frequency Identification SONnar u Developed for military underwater tasks like diver night vision and mine searching u Become popular for fish studies u Identification ability u Can see pictures of the fish. u Fish size from geometry, not from TS 6

Our aim u Develoop a target detector for DIDSON data u Can vi use the Cross Filter Detector CFD develooped for ordinary echogram u If not, can we optimise it to fit the DIDSON data u Or is there something to learn from the video world 7

8 Dual-Frequency Identification Sonar (DIDSON)

DIDSON problems u Low snr, u Low dynamic span, u Not calibrated, u Not veldefined sample volume u Only x,z, but no y position information 9

10 DIDSON inside

11 Examples of data

12 CFD AND DIDSON Tracking Echogram approach 3D approach Detection methods Aim, material and methods Introduction Conclusion

Detection theory - methods u Edgebased u Gradient operators u Linking Edge u Thresholding u Constant, u Addaptive, u Stastistical u Relaxation u If this is a fish pixel, then… 13

Cross Filter Detector (CFD) a Filter 1 Variance c Comparator Input echogram Filter 2 b Evaluator Traces Signal a Signal b Signal c Combine Evaluator Filter direction

CFD –Addaptive thresholding Main challenge: Find the optimal threshold signal threshold

Detection methods 16 Foreground filter Background filter Comparator variance Evaluator Background Modelling Comparator Evaluator Video Echogram Crossfilter detector Common video processing How to fit the Crossfilter to video like data? Can we learn something from the video world?

Background modelling. – the most important part. u Recursive u Approximated median u Kalmann filter u Mixture of Gausians u Non recursive u Previous picture u Median u Linear predictive u Nonparametric Background Modelling Comparator Evaluator Video Common video processing

Background modelling. – the most important part. u Three best u 1 Mixture of Gausians u 2 Median u 3 Approximated median 18 Ching, Cheung and Kamath found u Not much difference u App. Median much faster and simpler than the others Sen-Ching S. Cheung and Chandrika Kamath Center for Applied Scientic Computing Lawrence Livermore National Laboratory, Livermore, CA 94550

Comparator 19 Background Modelling Comparator Evaluator Video Common video processing

Evaluator u Morfological filter u Recognise fish on size and shape u May use higher order statistics u Connect parts of targets 20 Background Modelling Comparator Evaluator Video Common video processing

21 CFD AND DIDSON Tracking 3D approach Detection methods Echogram approach Introduction Summary Inc.Video methods

22 Echogram approach Amplitude Detector Gain 96-Ch Multi beam-viewer Amp-Echogram Multi  1 beam Echogram generator

23 Generate echograms and apply the Cross-Filter a) Mean echogram u At each range bin extract mean values from a selected number of beams. Like an ordinary transducer with controllable opening angle b) Max Intensity u At each range bin, select the sample from the beam with highest intensity How to combine many beams into one ?

24 Generating Echograms from multi beam Data recorded by Debby Burwen a) Averaging a number of beams 10x12 deg b) Pick the beam with strongest intensity Many beams  1 beam

25 Testing the CFD on many to 1 beam echograms Echogram approach

26 Echogram approach works well until density becomes too high We want to push the density limit Echogram approach

27 CFD AND DIDSON Tracking Echogram approach 3D approach The original Cross filter Aim, material and methods Introduction Summary

28 Adding a third dimension u Work directly on the multi beam data u Want to detect more than one target in the same range bin 3d-trace 2d-trace time width range 3D approach

29 We added the beam dimension to the filters DDF New Running window operators 2D 3D Beam. nr Range Ping Range 3D approach

Test foreground filter Frame Beam Range operator size

1 Test Background filter Frame Beam Range operator size

32 Testing cross filter on a small trout in Fisha River Max Intensity echogram

CFD with filters and threshold Forefilt 3 x 3 x 3 Back filt 3 x 3 x 3 Threshold Offset=20

34 Evaluator can take away unwanted targets

35

36 CFD AND DIDSON Tracking 3D approach Detection methods Echogram approach Introduction Summary Inc.Video methods

37 Extended the background filter with an approximated median operator (N. McFarlane and C. Schoeld 1995) ddf Q

38 And extended the comparator with alternatives Background Foreground If ( a - b )>T ) a b detection Threshold

Background subtraction Forefilt 3 x 3 x 3 Back filt 3 x 3 x 3 App.Median Threshold Offset=20

40 CFD AND DIDSON Tracking 3D approach Detection methods Echogram approach Introduction Summary Inc.Video methods

41 The initial idea was to detect traces directly by clustering Cluster of overlapping fish pictures ( Work well in the echogram approach )

42 But data often showed traces split up in individual fish pictures Clustering worked for big slow fish Tracker needed for fast fish Center of gravity track

43 Special predictor can be made for multi beam data Special predictor can be formed from the DIDSON fish picture In addition to traditional predictors are available such as Alpha Beta and Kalman Fish center line predictor

44 CFD AND DIDSON Tracking 3D approach Detection methods Echogram approach Introduction Summary Inc.Video methods

Summary 45 Background Modelling Comparator Evaluator Video Common video processing Foreground filter Background filter Comparator variance EvaluatorEchogram Crossfilter detector DIDSON Best method Tracker 3D-Foreground filter Comparator Evaluator Background Modelling

Summary 46 DIDSON Best method for moving targets Tracker 3D-Foreground filter Background Modelling ComparatorEvaluator u Needed in most cases u Need for various predictors depending on data Improved foreground signal Approximated Median ( a - b )>T ) a b 3D better than 2D Optimise on improving foreground

Run demo now if time 47

And that was it! Thanks for the attention! Questions? 48 CFD AND DIDSON Tracking 3D approach Detection methods Echogram approach Introduction Summary Inc.Video methods