1. Slides at www.cs.ucr.edu/~eamonn/public/cs170guest.ppt Eamonn Keogh eamonn@cs.ucr.edu

2. What do... Bill Gates Sir Richard Branson Larry Page Mohammed Saleh Bin Laden …have in common?

3. They each wrote to me in the last 3 months about my paper on Bayesian classification of insects… Let us Review Bayesian classification Discuss the paper

4. Naïve Bayes Classifier Thomas Bayes 1702 - 1761

5. Grasshoppers Katydids The Classification Problem (informal definition) Given a collection of annotated data. In this case 5 instances Katydids of and five of Grasshoppers, decide what type of insect the unlabeled example is. Katydid or Grasshopper?

6. Thorax Length AbdomenLength AntennaeLength MandibleSize Spiracle Diameter Leg Length For any domain of interest, we can measure features Color {Green, Brown, Gray, Other} Has Wings?

7. Antenna Length 10 123456789 1 2 3 4 5 6 7 8 9 Grasshoppers Katydids Abdomen Length Let’s get lots more data…

8. Antenna Length 10 123456789 1 2 3 4 5 6 7 8 9 Katydids Grasshoppers With a lot of data, we can build a histogram. Let us just build one for “Antenna Length” for now…

9. We can leave the histograms as they are, or we can summarize them with two normal distributions. Let us us two normal distributions for ease of visualization in the following slides…

10. p(c j | d) = probability of class c j, given that we have observed d 3 Antennae length is 3 We want to classify an insect we have found. Its antennae are 3 units long. How can we classify it? We can just ask ourselves, give the distributions of antennae lengths we have seen, is it more probable that our insect is a Grasshopper or a Katydid. There is a formal way to discuss the most probable classification…

11. 10 2 P( Grasshopper | 3 ) = 10 / (10 + 2)= 0.833 P( Katydid | 3 ) = 2 / (10 + 2)= 0.166 3 Antennae length is 3 p(c j | d) = probability of class c j, given that we have observed d

12. 9 3 P( Grasshopper | 7 ) = 3 / (3 + 9)= 0.250 P( Katydid | 7 ) = 9 / (3 + 9)= 0.750 7 Antennae length is 7 p(c j | d) = probability of class c j, given that we have observed d

13. 6 6 P( Grasshopper | 5 ) = 6 / (6 + 6)= 0.500 P( Katydid | 5 ) = 6 / (6 + 6)= 0.500 5 Antennae length is 5 p(c j | d) = probability of class c j, given that we have observed d

14. Bayes Classifiers That was a visual intuition for a simple case of the Bayes classifier, also called: Idiot Bayes Naïve Bayes Simple Bayes We are about to see some of the mathematical formalisms, and more examples, but keep in mind the basic idea. previously unseen instance Find out the probability of the previously unseen instance belonging to each class, then simply pick the most probable class.

15. Bayes Classifiers Bayesian classifiers use Bayes theorem, which says p(c j | d ) = p(d | c j ) p(c j ) p(d) p(c j | d) = probability of instance d being in class c j, This is what we are trying to compute p(d | c j ) = probability of generating instance d given class c j, We can imagine that being in class c j, causes you to have feature d with some probability p(c j ) = probability of occurrence of class c j, This is just how frequent the class c j, is in our database p(d) = probability of instance d occurring This can actually be ignored, since it is the same for all classes

16. Assume that we have two classes malefemale c 1 = male, and c 2 = female. We have a person whose sex we do not know, say “drew” or d. malefemale malefemale Classifying drew as male or female is equivalent to asking is it more probable that drew is male or female, i.e which is greater p(male | drew) or p(female | drew) malemalemale p(male | drew) = p(drew | male ) p(male) p(drew) (Note: “Drew can be a male or female name”) What is the probability of being called “drew” given that you are a male? What is the probability of being a male? What is the probability of being named “drew”? (actually irrelevant, since it is that same for all classes) Drew Carey Drew Barrymore

17. p(c j | d) = p(d | c j ) p(c j ) p(d) Officer Drew NameSex DrewMale ClaudiaFemale DrewFemale Female AlbertoMale KarinFemale NinaFemale SergioMale This is Officer Drew (who arrested me in 1997). Is Officer Drew a Male or Female? Luckily, we have a small database with names and sex. We can use it to apply Bayes rule…

18. male p(male | drew) = 1/3 * 3/8 = 0.125 3/8 3/8 female p(female | drew) = 2/5 * 5/8 = 0.250 3/8 3/8 Officer Drew p(c j | d) = p(d | c j ) p(c j ) p(d) NameSex DrewMale ClaudiaFemale DrewFemale Female AlbertoMale KarinFemale NinaFemale SergioMale Female Officer Drew is more likely to be a Female.

19. Officer Drew IS a female! Officer Drew male p(male | drew) = 1/3 * 3/8 = 0.125 3/8 3/8 female p(female | drew) = 2/5 * 5/8 = 0.250 3/8 3/8

20. NameOver 170 CM EyeHair lengthSex DrewNoBlueShortMale ClaudiaYesBrownLongFemale DrewNoBlueLongFemale DrewNoBlueLongFemale AlbertoYesBrownShortMale KarinNoBlueLongFemale NinaYesBrownShortFemale SergioYesBlueLongMale p(c j | d) = p(d | c j ) p(c j ) p(d) So far we have only considered Bayes Classification when we have one attribute (the “antennae length”, or the “name”). But we may have many features. How do we use all the features?

21. To simplify the task, naïve Bayesian classifiers assume attributes have independent distributions, and thereby estimate p(d|c j ) = p(d 1 |c j ) * p(d 2 |c j ) * ….* p(d n |c j ) The probability of class c j generating instance d, equals…. The probability of class c j generating the observed value for feature 1, multiplied by.. The probability of class c j generating the observed value for feature 2, multiplied by..

22. To simplify the task, naïve Bayesian classifiers assume attributes have independent distributions, and thereby estimate p(d|c j ) = p(d 1 |c j ) * p(d 2 |c j ) * ….* p(d n |c j ) p( officer drew |c j ) = p(over_170 cm = yes|c j ) * p(eye =blue|c j ) * …. Officer Drew is blue-eyed, over 170 cm tall, and has long hair Female p( officer drew | Female) = 2/5 * 3/5 * …. Male p( officer drew | Male) = 2/3 * 2/3 * ….

23. p(d 1 |c j ) p(d 2 |c j ) p(d n |c j ) cjcj The Naive Bayes classifiers is often represented as this type of graph… Note the direction of the arrows, which state that each class causes certain features, with a certain probability …

24. Naïve Bayes is fast and space efficient We can look up all the probabilities with a single scan of the database and store them in a (small) table… SexOver190 cmMaleYes0.15 No0.85 FemaleYes0.01 No0.99 cjcj … p(d 1 |c j ) p(d 2 |c j ) p(d n |c j ) SexLong HairMaleYes0.05 No0.95 FemaleYes0.70 No0.30 SexMale Female

25. An obvious point. I have used a simple two class problem, and two possible values for each example, for my previous examples. However we can have an arbitrary number of classes, or feature values AnimalMass >10 kgCatYes0.15 No0.85 DogYes0.91 No0.09 PigYes0.99 No0.01 cjcj … p(d 1 |c j ) p(d 2 |c j ) p(d n |c j ) AnimalCat Dog Pig ColorCatBlack0.33 White0.23 Brown0.44 DogBlack0.97 White0.03 Brown0.90 PigBlack0.04 White0.01 Brown0.95

26. The Naïve Bayesian Classifier has a piecewise quadratic decision boundary Grasshoppers Katydids Ants Adapted from slide by Ricardo Gutierrez-Osuna

27. What is the Deadliest Animal? (not counting humans!)

28. What is the Deadliest Animal? 500+ per year 200+ per year 10+ per year 300+ per year 1,000,000+ per year 50,000+ per year

29. What is the Deadliest Animal? 50,000+ per year 500+ per year 200+ per year 10+ per year 300+ per year 1,000,000+ per year

30. One penny weights about the same as one thousand mosquitoes How can something so small be so deadly?

31. What is Malaria? Malaria is a disease that involves high fevers, shaking chills, joint pain, flu-like symptoms. In some cases it can produce coma and death. There are more than 225 million cases of malaria each year, killing around 1-million people.

32. Where does Malaria come from? Malaria has been known since ancient times. Many believed it came from “bad air” (Italian: mala aria, “bad air”) 500 years ago, a handful of people believed that insects might be involved in human diseases. Hortus Sanitatis (The Garden of Health) 1497

33. It was Sir Ronald Ross, an British army surgeon working in India, who proved in 1897 that malaria is transmitted by mosquitoes. Sir Ronald Ross received the 1902 Nobel Prize for Physiology or Medicine for his work (This was somewhat controversial, as many others made similar discoveries around the same time )

34. Malaria Parasites 1 st Vector Initial Human host Liver infection Blood infection 2 nd Vector Next Human host Malaria Transmission Cycle

35. We get malaria from mosquitoes We get malaria from humans

36. There are 3,528 kinds of mosquitoes Only a handful of species take human blood Only the females take human blood There are 100 trillion mosquitoes alive today Mosquitoes have been around for at least 100 million years We know this from fossil records/DNA studies Mosquitoes have spread malaria for at least 35 million years We know this from insects found in amber The Mosquito

38. Where does malaria cause problems?

39. www.worldmapper.org

40. Given that we have known for over one hundred years how Malaria is spread, where is the magic pill or immunization? For a variety of reasons, a cure or immunization continues to alluded mankind. However there are some interventions that can help In the 20th century, smallpox killed 400 million people worldwide, it is now eradicated. Polio is almost eradicated.

41. Interventions to Mitigate Malaria The use of insecticidal treated mosquito nets Spraying of insecticides (including controversial chemicals such as DDT ) Introduction of fish/turtles/crustaceans to eat mosquito larva The introduction of dragonflies which eat adult mosquitoes. Habitat reduction by draining ponds and pools Use of chemical films to reduce the surface tension of water (drowning the pupa)... and hundreds more proven or tentative ideas

42. Interventions Cost Money! Even cheap solutions have hidden costs Insecticidal treated mosquito nets are cheap to make, but… To make mosquito nets work, you need educators, incentive programs, maintenance etc “...aid agencies and non-governmental organizations are quietly grappling with a problem: Data suggest that nearly half of Africans who have access to the nets refuse to sleep under them” (LA Times May-2-2010).

43. We need to know where/when the problem is the greatest. Where are the insects? Which insects are they? When did they arrive? The classic solution? Use sticky traps –Inaccurate –Costly –Long time lag Planning interventions requires knowledge

44. My Research I believe that we can count and classify insects with sensors. The classification problem –Must be cheap –Must be low powered –Must be accurate

45. 00.511.522.533.544.5 x 10 4 -0.2 -0.1 0 0.1 0.2 One second of audio from our sensor. The Common Eastern Bumble Bee (Bombus impatiens) takes about one tenth of a second to pass the laser. Background noise Bee begins to cross laserBee has past though the laser UCR Wingbeat Sensor

46. 00.511.522.533.544.5 x 10 4 -0.2 -0.1 0 0.1 0.2 01002003004005006007008009001000 0 1 2 3 4 x 10 -3 Single-Sided Amplitude Spectrum of Y(t) Frequency (Hz) |Y(f)| One second of audio from the laser sensor. Only Bombus impatiens (Common Eastern Bumble Bee) is in the insectary. Background noise Bee begins to cross laser Peak at 197Hz Harmonics 60Hz interference

47. 01002003004005006007008009001000 0 1 2 3 4 x 10 -3 Frequency (Hz) |Y(f)| 01002003004005006007008009001000 Frequency (Hz) 01002003004005006007008009001000 Frequency (Hz)

48. 0 100 200300400500600700 Wing Beat Frequency Hz

49. 0 100 200300400500600700 Wing Beat Frequency Hz

50. 400500600700 Anopheles stephensi: Female mean =475, Std = 30 Aedes aegyptii : Female mean =567, Std = 43 517 If I see an insect with a wingbeat frequency of 500, what is it?

51. 400500600700 517 12.2% of the area under the pink curve 8.02% of the area under the red curve What is the error rate? Can we get more features?

52. A mosquito flying across the laser beam 4008001200 0 Wingbeat frequency at 354 Hz Amplitude Spectrum Can we discriminate the sex of mosquitoes? (here Aedes aegypti ) Predicted Class Can we discriminate mosquitoes from non- mosquitoes ? Predicted Class Can we discriminate between mosquito species? Predicted Class femalemale Ae. aegypti Tipula An. gambiae Ae. aegypti Actualfemale 9,9946 Actual Ae. aegypti 9,9990 Actual An. gambiae 9,97914 Classmale 39,997 ClassTipula 1800 Class Ae. aegypti 219,986

53. 17 Bee count 21 Mosquito count 2134 Housefly count

54. 17 Bee count 21 Mosquito count 2134 Housefly count Spay the dark red areas at dusk

55. Using Pesticides Intelligently If we are not sure where the insects are… If we know where the insects are… High cost (fuel/pesticide) Damage to environment Low cost Little damage to environment

56. You won 10000K Congratulations! You won the Nigerian lottery! Please send me your name and more than four hours! Upcoming dentist app Dear Eamonn Don’t forget your dentist appoint Many thanks, joe here's we hue opposings and ents thers that fled of the undispriz'd coward to suffer Lose Weight Now! here's we hue opposings and ents thers that fled of the undispriz'd coward to suffer Make money fast! here's we hue opposings and ents thers that fled of the undispriz'd coward to suffer Upcoming dentist app Dear Eamonn Don’t forget your dentist appoint next Wednesday the 17 Upcoming dentist app Dear Eamonn Don’t forget your dentist appoint next Wednesday the 17 You won 10000K Congratulations! You won the Nigerian lottery! Please send me your name and address Spam Filter Spam FolderEmail Folder The holy grail of computational entomology With a single click of a button, we can take action on our emails/spam. We can delete, forward, save them etc

57. 17 Bee count 21 Mosquito count 2134 Housefly count Can we do this with insects? Can we “delete” mosquitoes (Kill them) Can we “forward” bees (Make them go North and pollinate almonds etc) The holy grail of computational entomology

58. Controlling Insects with Computers Insect cyborgs are cool, but not the answer. We need to be able to control insects at a cost of say 1,000,000 insects per penny.

59. Controlling Insects with Computers We already control some insects with technology/computers We are exploiting the fact that moths evolved without artificial lights. What else can we exploit?

60. The more we learn about nature, the more we can improve human the human condition

61. 10 12345678 9 1 2 3 4 5 6 7 8 9 100 1020304050607080 90 100 10 20 30 40 50 60 70 80 90 10 12345678 9 1 2 3 4 5 6 7 8 9 Which of the “Pigeon Problems” can be solved by a decision tree?

62. Advantages: –Fast to train (single scan). Fast to classify –Not sensitive to irrelevant features –Handles real and discrete data –Handles streaming data well Disadvantages: –Assumes independence of features Advantages/Disadvantages of Naïve Bayes

63. Summary of Naïve Bayes The Naïve Bayes classifier is a fantastic tool for solving real world problems. The are lots of jobs, research opportunities and entrepreneurial opportunities for people that can apply it in real world situations.