1. 1 Algorithms for Port of Entry Inspection for WMDs Fred S. Roberts DyDAn Center Rutgers University

2. 2 Port of Entry Inspection Algorithms Goal: Find ways to intercept illicit nuclear materials and weapons destined for the U.S. via the maritime transportation system Currently inspecting only small % of containers arriving at ports Even inspecting 8% of containers in Port of NY/NJ might bring international trade to a halt (Larrabbee 2002)

3. 3 Port of Entry Inspection Algorithms Aim: Develop decision support algorithms that will help us to “optimally” intercept illicit materials and weapons subject to limits on delays, manpower, and equipment Find inspection schemes that minimize total “cost” including “cost” of false alarms (“false positives”) and failed alarms (“false negatives”) Mobile Vacis: truck- mounted gamma ray imaging system

4. 4 Port of Entry Inspection Algorithms My work on port of entry inspection has gotten me and my students to some remarkable places. Me on a Coast Guard boat in a tour of the harbor in Philadelphia

5. 5

6. 6 Sequential Decision Making Problem Stream of containers arrives at a port The Decision Maker’s Problem: Which to inspect? Which inspections next based on previous results? Approach: –“decision logics” –combinatorial optimization methods –Builds on ideas of Stroud and Saeger at Los Alamos National Laboratory –Need for new models – and methods

7. 7 Sequential Diagnosis Problem Such sequential diagnosis problems arise in many areas: –Communication networks (testing connectivity, paging cellular customers, sequencing tasks, …) –Manufacturing (testing machines, fault diagnosis, routing customer service calls, …) –Medicine (diagnosing patients, sequencing treatments, …)

8. 8 Sequential Decision Making Problem Containers arriving to be classified into categories. Simple case: 0 = “ok”, 1 = “suspicious” Inspection scheme: specifies which inspections are to be made based on previous observations

9. 9 Sequential Decision Making Problem 0’s and 1’s suggest binary digits (bits) Bit String: A sequence of bits: 0001, 1101, … Boolean Function: A function that assigns to each bit string a 0 or a 1. Bit String xB(x) 001 010B(00) = 1, B(10) = 0 100 111

10. 10 Sequential Decision Making Problem Bit strings are of course crucial carriers of information in modern computers They are used to encode detailed instructions and are translated to a sequence of on-off instructions for switches in the computer. We will use them differently.

11. 11 Counting Bit Strings and Boolean Functions To understand the difficulty of the container inspection problem, we will need to count bit strings and Boolean functions. Product Rule of Counting: If something can happen in n 1 ways, and no matter how the first thing happens, a second thing can happen in n 2 ways, then the two things together can happen in n 1 x n 2 ways.

12. 12 Counting Bit Strings and Boolean Functions Product Rule of Counting (More Generally): If something can happen in n 1 ways, and no matter how the first thing happens, a second thing can happen in n 2 ways, and no matter how the first two things happen, a third thing can happen in n 3 ways, and …, then all the things together can happen in n 1 x n 2 x n 3 x … ways.

13. 13 Counting Bit Strings and Boolean Functions How many bit strings are there of 3 bits? There are 2 choices for the first bit. No matter how it is chosen, there are 2 choices for the second bit. No matter how the first 2 bits are chosen, there are 2 choices for the third bit. Get 2 x 2 x 2 = 2 3 = 8

14. 14 Counting Bit Strings and Boolean Functions How many bit strings are there of 3 bits? 000, 001, 010, 011, 100, 101, 110, 111 How many bit strings are there of 5 bits? 2525 How many bit strings are there of n bits? 2n2n

15. 15 Counting Bit Strings and Boolean Functions How many Boolean functions are there of 3 variables?

16. 16 Counting Bit Strings and Boolean Functions How many Boolean functions are there of 3 variables? There are 2 3 = 8 bit strings of 3 variables For each such bit string, the Boolean function assigns it one of 2 different values (0 or 1).

17. 17 Counting Bit Strings and Boolean Functions How many Boolean functions are there of 3 variables? There are 2 3 = 8 bit strings of 3 variables For each such bit string, the Boolean function assigns it one of 2 different values (0 or 1). There are 2 8 = such Boolean functions. 2 2 3

18. 18 Counting Bit Strings and Boolean Functions How many Boolean functions are there of n variables?

19. 19 Counting Bit Strings and Boolean Functions How many Boolean functions are there of n variables? There are 2 n bit strings of n variables For each such bit string, the Boolean function assigns it one of 2 different values (0 or 1).

20. 20 Counting Bit Strings and Boolean Functions How many Boolean functions are there of n variables? There are 2 n bit strings of n variables For each such bit string, the Boolean function assigns it one of 2 different values (0 or 1). There are such Boolean functions. 2 2 n

21. 21 Counting Bit Strings and Boolean Functions How big is ? When n = 4, this is 65,536 2 2 n

22. 22 Counting Bit Strings and Boolean Functions The problem of making a detailed design of a digital computer usually involves finding a practical circuit implementation of certain functional behavior. A computer device implements a Boolean function. In the early days of computing, the goal was to create a catalogue listing a good circuit implementation of each Boolean function. This would require 65,536 entries!

23. 23 Counting Bit Strings and Boolean Functions Of course, luckily, by symmetry, some of these examples are “equivalent” For example, if we interchange two variables. In the early days of computing (1951), a group from the Harvard Computation Laboratory painstakingly listed all 65,536 Boolean functions of 4 variables and determined which were equivalent. (There were 222 types.)

24. 24 Sequential Decision Making Problem For Container Inspection Containers have attributes, each in a number of states Sample attributes: –Levels of certain kinds of chemicals or biological materials –Whether or not there are items of a certain kind in the cargo list –Whether cargo was picked up in a certain port

25. 25 Sequential Decision Making Problem Currently used attributes: –Does ship’s manifest set off an “alarm”? –What is the neutron or Gamma emission count? Is it above threshold? –Does a radiograph image come up positive? –Does an induced fission test come up positive? Gamma ray detector

26. 26 Sequential Decision Making Problem We can imagine many other attributes Research at DyDAn is concerned with general algorithmic approaches. We seek a methodology not tied to today’s technology. Detectors are evolving quickly.

27. 27 Sequential Decision Making Problem Simplest Case: Attributes are in state 0 or 1 Then: Container is a bit string like 011001 So: Classification is a decision function F that assigns each binary string to a category. 011001F(011001) If attributes 2, 3, and 6 are present, assign container to category F(011001).

28. 28 Sequential Decision Making Problem If there are two categories, 0 and 1, decision function F is a Boolean function. Example: F(000) = F(111) = 1, F(abc) = 0 otherwise This classifies a container as positive iff it has none of the attributes or all of them. 1 =

29. 29 Sequential Decision Making Problem What if there are three categories, 0 ½, and 1?. Example: F(000) = 0, F(111) = 1, F(abc) = 1/2 otherwise This classifies a container as positive if it has all of the attributes, negative if it has none of the attributes, and uncertain if it has some but not all of the attributes. How many such decision functions are there if there are three attributes?

30. 30 Sequential Decision Making Problem What if there are three categories, 0 ½, and 1? How many such decision functions are there if there are three attributes?

31. 31 Sequential Decision Making Problem What if there are three categories, 0 ½, and 1?. How many such decision functions are there if there are three attributes? There are 2 3 bit strings of length 3.

32. 32 Sequential Decision Making Problem What if there are three categories, 0 ½, and 1?. How many such decision functions are there if there are three attributes? There are 2 3 bit strings of length 3. For each of them, there are 3 choices for the value of F.

33. 33 Sequential Decision Making Problem What if there are three categories, 0 ½, and 1?. How many such decision functions are there if there are three attributes? There are 2 3 bit strings of length 3. For each of them, there are 3 choices for the value of F. Thus, there are such decision functions. 3 2 3

34. 34 Sequential Decision Making Problem Given a container, test its attributes until know enough to calculate the value of F. An inspection scheme tells us in which order to test the attributes to minimize cost. Even this simplified problem is hard computationally.

35. 35 Sequential Decision Making Problem This assumes F is known. Simplifying assumption: Attributes are independent. At any point we stop inspecting and output the value of F based on outcomes of inspections so far. Complications: May be precedence relations in the components (e.g., can’t test attribute a 4 before testing a 6. Or: cost may depend on attributes tested before. F may depend on variables that cannot be directly tested or for which tests are too costly.

36. 36 Sequential Decision Making Problem Such problems are hard computationally. There are many possible Boolean functions F. Even if F is fixed, problem of finding a good classification scheme (to be defined precisely below) is NP-complete – it is hard in a precise computer science sense.

37. 37 Sensors and Inspection Lanes n types of sensors measure presence or absence of the n attributes. Many copies of each sensor. Complication: different characteristics of sensors. Entities come for inspection. Which sensor of a given type to use? Think of inspection lanes and waiting on line for inspection Besides efficient inspection schemes, could decrease costs by: –Buying more sensors –Change allocation of containers to sensor lanes.

38. 38 Trees A tree for us is a directed graph. It has nodes (vertices). Directed edges or arcs head from a vertex to another. There are no “cycles” (you can’t double back on yourself) We will deal with rooted trees: One node is a root. All arcs point downwards in our diagrams, starting from the root. If each node has two or zero outgoing (downwards) arcs, we have a binary tree. Nodes with no outgoing arcs are called leaves.

39. 39 Binary Decision Tree Approach Sensors measure presence/absence of attributes: so 0 or 1 Use two categories: 0, 1 Binary Decision Tree: –Nodes are sensors or categories –Two arcs exit from each sensor node, labeled left and right. –Take the right arc when sensor says the attribute is present, left arc otherwise

40. 40 Binary Decision Tree Approach Reach category 1 from the root only through the path a 0 to a 1 to 1. Container is classified in category 1 iff it has both attributes a 0 and a 1. Corresponding Boolean function: F(11) = 1, F(10) = F(01) = F(00) = 0. Figure 1

41. 41 Binary Decision Tree Approach Reach category 1 from the root only through the path a 1 to a 0 to 1. Container is classified in category 1 iff it has both attributes a 0 and a 1. Corresponding Boolean function: F(11) = 1, F(10) = F(01) = F(00) = 0. Note: Different tree, same function Figure 1

42. 42 Binary Decision Tree Approach Reach category 1 from the root only through the path a 0 to 1 or a 0 to a 1 to 1. Container is classified in category 1 iff it has attribute a 0 or attribute a 1. Corresponding Boolean function: F(11) = 1, F(10) = F(01) = 1, F(00) = 0. Figure 1

43. 43 Binary Decision Tree Approach Can you find another binary tree that calculates the same Boolean function? Sure, just interchange nodes a 0 and a 1 Figure 1

44. 44 Binary Decision Tree Approach Reach category 1 from the root by: a 0 L to a 1 R a 2 R 1 or a 0 R a 2 R1 Container classified in category 1 iff it has a 1 and a 2 and not a 0 or a 0 and a 2 and possibly a 1. Corresponding Boolean function: F(111) = F(101) = F(011) = 1, F(abc) = 0 otherwise. Figure 2

45. 45 Binary Decision Tree Approach This binary decision tree corresponds to the same Boolean function F(111) = F(101) = F(011) = 1, F(abc) = 0 otherwise. However, it has one less observation node a i. So, it is more efficient if all observations are equally costly and equally likely. Figure 3

46. 46 Binary Decision Tree Approach So we have seen that a given Boolean function may correspond to different binary decision trees. How do we find a binary decision tree corresponding to a Boolean function? How do we find a least cost one? Port of Long Beach

47. 47 Binary Decision Tree Approach So we have seen that a given Boolean function may correspond to different binary decision trees. How do we find a binary decision tree corresponding to a Boolean function F? Brute force method: Start with root a 0, put both left and right arcs to an a 1. From each a 1 put both left and right arcs to a 2, etc. At the end, each path through the tree corresponds to a bit string x and let that path end at F(x).

48. 48 Binary Decision Tree Approach a0a0 a1a1 a1a1 a2a2 a2a2 a2a2 a2a2 F(010)

49. 49 Binary Decision Tree Approach a0a0 a1a1 a1a1 a2a2 a2a2 a2a2 a2a2 xF(x)x 11110111 11010100 10100010 10010000 000 110 1 1

50. 50 Binary Decision Tree Approach Even if the Boolean function F is fixed, the problem of finding the “least cost” binary decision tree for it is very hard (NP-complete). For small n = number of attributes, can try to solve it by trying all possible binary decision trees corresponding to the Boolean function F. Even for n = 4, not practical. (n = 4 at Port of Long Beach-Los Angeles) Port of Long Beach

51. 51 Binary Decision Tree Approach Promising Approaches: Heuristic algorithms, approximations to optimal. Special assumptions about the Boolean function F. For “monotone” Boolean functions, integer programming formulations give promising heuristics. Stroud and Saeger (Los Alamos National Lab) enumerate all “complete,” monotone Boolean functions and calculate the least expensive corresponding binary decision trees. Their method practical for n up to 4, not n = 5.

52. 52 Binary Decision Tree Approach Monotone Boolean Functions: Given two bit strings x 1 x 2 …x n, y 1 y 2 …y n Suppose that x i  y i for all i implies that F(x 1 x 2 …x n )  F(y 1,y 2 …y n ). Then we say that F is monotone. Then 11…1 has highest probability of being in category 1.

53. 53 Binary Decision Tree Approach Monotone Boolean Functions: Given two bit strings x 1 x 2 …x n, y 1 y 2 …y n Suppose that x i  y i for all i implies that F(x 1 x 2 …x n )  F(y 1,y 2 …y n ). Then we say that F is monotone. Example: n = 4, F(x) = 1 iff x has at least two 1’s. F(1100) = F(0101) = F(1011) = 1, F(1000) = 0, etc. Is this monotone? Yes

54. 54 Binary Decision Tree Approach Incomplete Boolean Functions: Boolean function F is incomplete if F can be calculated by finding at most n-1 attributes and knowing the value of the input string on those attributes Example: F(111) = F(110) = F(101) = F(100) = 1, F(000) = F(001) = F(010) = F(011) = 0. F(abc) is determined without knowing b (or c). F is incomplete.

55. 55 Binary Decision Tree Approach Complete, Monotone Boolean Functions: Stroud and Saeger: algorithm for enumerating binary decision trees implementing complete, monotone Boolean functions. Feasible to implement up to n = 4. Then you can find least cost tree by enumerating all binary decision trees corresponding to a given Boolean function and repeating this for all complete, monotone Boolean functions.

56. 56 Binary Decision Tree Approach Complete, Monotone Boolean Functions: Stroud and Saeger: algorithm for enumerating binary decision trees implementing complete, monotone Boolean functions. n = 2: –There are 6 monotone Boolean functions. –Only 2 of them are complete, monotone –There are 4 binary decision trees for calculating these 2 complete, monotone Boolean functions.

57. 57 Binary Decision Tree Approach Complete, Monotone Boolean Functions: n = 3: –9 complete, monotone Boolean functions. –60 distinct binary trees for calculating them –Counting methods more complicated than simple ones we have described before –All counts here are from Stroud and Saeger

58. 58 Binary Decision Tree Approach Complete, Monotone Boolean Functions: n = 4: –114 complete, monotone Boolean functions. –11,808 distinct binary decision trees for calculating them. –(Compare 1,079,779,602 BDTs for all Boolean functions)

59. 59 Binary Decision Tree Approach Complete, Monotone Boolean Functions: n = 5: –6894 complete, monotone Boolean functions –263,515,920 corresponding binary decision trees. Combinatorial explosion! Need alternative approaches; enumeration not feasible! (Even worse: compare 5 x 10 18 BDTs corresponding to all Boolean functions)

60. 60 Cost Functions So far, we have figured one binary decision tree is cheaper than another if it has fewer nodes. This is oversimplified. There are more complex costs involved than number of sensors in a tree.

61. 61 Cost Functions Stroud-Saeger method applies to more sophisticated cost models, not just cost = number of sensors in the BDT. Using a sensor has a cost: –Unit cost of inspecting one item with it –Fixed cost of purchasing and deploying it –Delay cost from queuing up at the sensor station Preliminary problem: disregard fixed and delay costs. Minimize unit costs.

62. 62 Cost Functions Simplification so far: Disregard characteristics of population of entities being inspected. Only count number of observation (attribute) nodes in the tree. Unit Cost Complication: How many nodes of the decision tree are actually visited during average container’s inspection? Depends on “distribution” of containers. Answer can depend on probability of sensor errors and probability of bomb in a container.

63. 63 Cost Functions: Delay Costs Tradeoff between fixed costs and delay costs: Add more sensors cuts down on delays. More sophisticated models describe the process of containers arriving There are differing delay times for inspections Use “queuing theory” to find average delay times under different models

64. 64 Cost Functions: Delay Costs There are differing delay times for inspections Use “queuing theory” to find average delay times under different models Explorations for Students: Explore the fundamental principles of “queueing theory”

65. 65 Cost Functions: Unit Costs Tree Utilization Complication: Assume cost depends on how many nodes of BDT are actually visited during an “average” container’s inspection. (This is sum of unit costs.) Depends on characteristics of population of entities being inspected. I.e., depends on “distribution” of containers. In our early models, we assume we are given probability of sensor errors and probability of bomb in a container. This allows us to calculate “expected” cost of utilization of the tree C util. (Details omitted here.)

66. 66 Cost Functions Cost of false positive: Cost of additional tests. –If it means opening the container, it’s very expensive. Cost of false negative: –Complex issue. –What is cost of a bomb going off in Manhattan?

67. 67 Cost Functions: Sensor Errors One Approach to False Positives/Negatives: Assume there can be Sensor Errors Simplest model: assume that all sensors checking for attribute a i have same fixed probability of saying a i is 0 if in fact it is 1, and similarly saying it is 1 if in fact it is 0. More sophisticated analysis later describes a model for determining probabilities of sensor errors. Notation: X = state of nature (bomb or no bomb) Y = outcome (of sensor or entire inspection process).

68. 68 Probability of Error for The Entire Tree State of nature is zero (X = 0), absence of a bomb State of nature is one (X = 1), presence of a bomb Probability of false positive (P(Y=1|X=0)) for this tree is given by Probability of false negative (P(Y=0|X=1)) for this tree is given by A B C 0 1 0 1 A B C 0 1 0 1 P(Y=1|X=0) = P(Y A =1|X=0) * P(Y B =1|X=0) + P(Y A =1|X=0) *P(Y B =0|X=0)* P(Y C =1|X=0) P falsepositive P(Y=0|X=1) = P(Y A =0|X=1) + P(Y A =1|X=1) *P(Y B =0|X=1)*P(Y C =0|X=1) P falsenegative

69. 69 Probability of Error for The Entire Tree State of nature is zero (X = 0), absence of a bomb Probability of false positive (P(Y=1|X=0)) for this tree is given by A B C 0 1 0 1 P(Y=1|X=0) = P(Y A =1|X=0) * P(Y B =1|X=0) + P(Y A =1|X=0) *P(Y B =0|X=0)* P(Y C =1|X=0) P falsepositive Note the use of the product rule: probability is ratio between number of ways something we are interested in can happen and total number ways things can happen.

70. 70 Probability of Error for The Entire Tree State of nature is zero (X = 0), absence of a bomb Probability of false positive (P(Y=1|X=0)) for this tree is given by A B C 0 1 0 1 P(Y=1|X=0) = P(Y A =1|X=0) * P(Y B =1|X=0) + P(Y A =1|X=0) *P(Y B =0|X=0)* P(Y C =1|X=0) P falsepositive Also uses the sum rule of counting: If one event can happen in n 1 ways and a second event in n 2 different ways, then there are n 1 +n 2 ways in which either the first event or the second event can occur (but not both).

71. 71 Aside: Illustrating the Sum Rule For example, how many bit strings are there of 4 bits so that there are either exactly three 1’s or exactly three 0’s? For bit strings of three 1’s, we have four choices for the place to put the 0, so there are 4 such strings. For bit strings with three 0’s, there are similarly 4 such strings. By the sum rule, there are 8 bit strings with exactly three 1’s or exactly three 0’s, since it is impossible to have both. 1110, 1101, 1011, 0111, 0001, 0010, 0100, 1000

72. 72 Cost Function used for Evaluating the Decision Trees. C Tot = C FalsePositive *P FalsePositive + C FalseNegative *P FalseNegative + C util C FalsePositive is the cost of false positive (Type I error) C FalseNegative is the cost of false negative (Type II error) P FalsePositive is the probability of a false positive occurring P FalseNegative is the probability of a false negative occurring C util is the expected cost of utilization of the tree.

73. 73 Cost Function used for Evaluating the Decision Trees. C FalsePositive is the cost of false positive (Type I error) C FalseNegative is the cost of false negative (Type II error) P FalsePositive is the probability of a false positive occurring P FalseNegative is the probability of a false negative occurring C util is the expected cost of utilization of the tree. P FalsePositive and P FalseNegative are calculated from the tree. C util is calculated from tree and probabilities of bomb in container and probability of sensor errors. C FalsePositive, C FalseNegative are input – given information.

74. 74 Stroud Saeger Experiments Stroud-Saeger ranked all trees formed from 3 or 4 sensors A, B, C and D according to increasing tree costs. Used cost function defined above. Values used in their experiments: –C A =.25; P(Y A =1|X=1) =.90; P(Y A =1|X=0) =.10; –C B = 10; P(Y C =1|X=1) =.99; P(Y B =1|X=0) =.01; –C C = 30; P(Y D =1|X=1) =.999; P(Y C =1|X=0) =.001; –C D = 1; P(Y D =1|X=1) =.95; P(Y D =1|X=0) =.05; –Here, C i = unit cost of utilization of sensor i. Also fixed were: C FalseNegative, C FalsePositive, P(X=1)

75. 75 Sensitivity Analysis When parameters in a model are not known exactly, the results of a mathematical analysis can change depending on the values of the parameters. It is important to do a sensitivity analysis: let the parameter values vary and see if the results change. So, do the least cost trees change if we change values like probability of a bomb, cost of a false positive, etc?

76. 76 Stroud Saeger Experiments: Our Sensitivity Analysis We have explored sensitivity of the Stroud- Saeger conclusions to variations in values of the three parameters: C FalseNegative, C FalsePositive, P(X=1) (Work of Anand, Madigan, Mammone, Pathak, Roberts) We estimated high and low values for the parameters.

77. 77 Stroud Saeger Experiments: Our Sensitivity Analysis –C FalseNegative was varied between 25 million and 10 billion dollars Low and high estimates of direct and indirect costs incurred due to a false negative. –C FalsePositive was varied between $180 and $720 Cost incurred due to false positive (4 men * (3 -6 hrs) * (15 – 30 $/hr) –P(X=1) was varied between 1/10,000,000 and 1/100,000

78. 78 Stroud Saeger Experiments: Our Sensitivity Analysis –C FalseNegative was varied between 25 million and 10 billion dollars Low and high estimates of direct and indirect costs incurred due to a false negative. –C FalsePositive was varied between $180 and $720 Cost incurred due to false positive (4 men * (3 -6 hrs) * (15 – 30 $/hr) –P(X=1) was varied between 1/10,000,000 and 1/100,000 –Extra assignment: think about whether these ranges are reasonable and how one would determine them?

79. 79 Stroud Saeger Experiments: Our Sensitivity Analysis n = 3 (use sensors A, B, C) Varied the parameters C FalseNegative, C FalsePositive, P(X=1) We chose the value of one of these parameters from the interval of values Then explored the highest ranked tree as the other two were chosen at random in the interval of values. 10,000 experiments for each fixed value. We looked for the variation in the top-ranked tree and how the top-rank related to choice of parameter values. Very surprising results.

80. 80 Frequency of Top-ranked Trees when C FalseNegative and C FalsePositive are Varied 10,000 randomized experiments (randomly selected values of C FalseNegative and C FalsePositive from the specified range of values) for the median value of P(X=1). The above graph has frequency counts of the number of experiments when a particular tree was ranked first or second or third and so on. Only three trees (7, 55 and 1) ever came first. 6 trees came second, 10 came third, 13 came fourth.

81. 81 10,000 randomized experiments for the median value of C FalsePositive. Only 2 trees (7 and 55) ever came first. 4 trees came second. 7 trees came third. 10 and 13 trees came 4 th and 5 th respectively. Frequency of Top-ranked Trees when C FalseNegative and P(X=1) are Varied

82. 82 10,000 randomized experiments for the median value of C FalseNegative. Only 3 trees (7, 55 and 1) ever came first. 6 trees came second. 10 trees came third. 13 and 16 trees came 4 th and 5 th respectively. Frequency of Top-ranked Trees when P(X=1) and C FalsePositive are Varied

83. 83 Most Frequent Tree Groups Attaining the Top Three Ranks. Trees 7, 9 and 10 A B C 0 1 0 1 B A C A 1 01 0 0 B A A C 1 01 0 0 All the three decision trees have been generated from the same Boolean function. Both Tree 9 and Tree 10 are ranked second and third more than 99% of the times when Tree 7 is ranked first.

84. 84 Most Frequent Tree Groups Attaining the Top Three Ranks Trees 55, 57 and 58 A 1 C B 1 0 1 B 1 C A 1 0 1 B 1 A C 1 0 1 All three trees correspond to the same Boolean function. Tree ranked 57 is second 96% of the times and tree 58 is third 79 % of the times when tree 55 is ranked first.

85. 85 Most Frequent Tree Groups Attaining the Top Three Ranks Trees 1, 3, and 2 A B C 0 0 0 1 A C B 0 0 0 1 B A C 0 0 0 1 All three trees correspond to the same Boolean function. Tree 3 is ranked second 98% of times and tree 2 is ranked third 80 % of the times when tree 1 is ranked first.

86. 86 Stroud Saeger Experiments: Sensitivity Analysis: 4 Sensors Second set of computer experiments: n = 4 (use sensors, A, B, C, D). Same values as before. Experiment 1: Fix values of two of C FalseNegative, C FalsePositive, P(X=1) and vary the third through their interval of possible values. Experiment 2: Fix a value of one of C FalseNegative, C FalsePositive, P(X=1) and vary the other two. Do 10,000 experiments each time. Look for the variation in the highest ranked tree.

87. 87 Tree Structure For Top Trees 1 a b b dc d1 c d 1 1 0 01 0 1 Tree number 11485 1 a b b c d1 c d 1 1 0 0 1 c 0 d 1 0 Tree number 10129

88. 88 Tree Structure For Top Trees 1 a b b dc d1 c d 1 1 0 01 0 1 Tree number 11485 1 a b b c d1 c d 1 1 0 0 1 c 0 d 1 0 Tree number 10129 Explorations for Students: Compare the Boolean functions corresponding to these two trees. See where they differ. How close are they? Can you make closeness precise?

89. 89 Modeling Sensor Errors One Approach to Sensor Errors: Modeling Sensor Operation Threshold Model: –Sensors have different discriminating power –Many use counts (e.g., Gamma radiation counts) –See if count exceeds threshold –If so, say attribute is present.

90. 90 Conclusions from Sensitivity Analysis Considerable lack of sensitivity to modification in parameters for trees using 3 or 4 sensors. Very few optimal trees. Very few Boolean functions arise among optimal and near-optimal trees. Surprising results.

91. 91 Some Research Challenges Explain why conclusions are so insensitive to variation in parameter values. Explore the structure of the optimal trees and compare the different optimal trees. Develop less brute force methods for finding optimal trees that might work if there are more than 4 attributes. Develop methods for approximating the optimal tree. Pallet vacis

92. 92 Closing Remark Recall that the “cost” of inspection includes the cost of failure, including failure to foil a terrorist plot. There are many ways to lower the total “cost” of inspection: -Use more efficient orders of inspection. -Find ways to inspect more containers. -Find ways to cut down on delays at inspection lanes.

93. 93 Research Team Saket Anand, Rutgers, ECE graduate student Endre Boros, Rutgers, Operations Research Elsayed Elsayed, Rutgers, Ind. & Systems Engineering Liliya Fedzhora, Rutgers, Operations Res. grad. student Paul Kantor, Rutgers, Schl. of Infor. & Library Studies Abdullah Karaman, Rutgers Ind. & Syst. Eng. grad. student Alex Kogan, Rutgers, Business School Paul Lioy, Rutgers/UMDNJ, Environmental and Occupational Health and Sciences Institute David Madigan, Rutgers, Statistics Richard Mammone, Rutgers, Center for Advanced Information Processing S. Muthukrishnan, Rutgers, Computer Science Saumitr Pathek, Rutgers ECE graduate student Richard Picard, Los Alamos, Statistical Sciences Group Fred Roberts, Rutgers, DIMACS Center Kevin Saeger, Los Alamos, Homeland Security Phillip Stroud, Los Alamos, Systems Engineering and Integration Group Hao Zhang, Rutgers Ind. & Systems Eng., graduate student

94. 94 Collaborators on Sensitivity Analysis: Saket Anand David Madigan Richard Mammone Saumitr Pathak Research Support: Office of Naval Research National Science Foundation Los Alamos National Laboratory: Rick Picard Kevin Saeger Phil Stroud

95. 95 Mathematics can help with homeland security!