Naïve Bayes based Model Billy Doran

“If the model does what people do, do people do what the model does?”

Bayesian Learning Determines the probability of a hypothesis H given a set of data D: Ρ(Η|D) = P(D|H) P(H)⁄P(D)

P(H) is the prior probability of H. The probability of observing H for the whole data set P(H|D) is the posterior probability of H. This means that given the Data D what is the probability of the hypothesis H. P(D) is the prior probability of observing D. It is constant throughout the data set and can be ignored. P(D|H) is the likelihood of observing the data given the hypothesis. Does the hypothesis reproduce the data?

Maximum a Posteriori Probability In order to classify an example as belonging to one category or another we aim to find the maximal value of P(H|D) For example we can take the training pattern, if we want to find the probability that this example belongs to category A the posterior probability is: P(Category A|A,X,C)

Naïve Bayes The Naïve Bayes algorithm allows us to assume conditional independence of the dimensions. This means that we consider each dimension in terms of its probability given the category: P(A,B|Cat A) = P(A|Cat A)P(B|Cat A) Using this information we are able to build a table of the Conditional Probabilities for each dimension

Conditional Probability Table Probabilities for Category A ABC Dimension Dimension Dimension P(Dimension1=A|Category A) is 4/6, which is

Calculation In order to get the scores for the pattern we first find P(A|A,B,C)=P(A|A)P(B|A)P(C|A)P(A) =0.666*0.1666*0.1666*0.375= P(B|A,B,C)=0.166*0.5*0.1*0.375= P(C|A,B,C)=0.1*0.1*0.833*0.375= Next we normalise the score to get a value in the range [0-1] A= /( ) = 0.52

Conjunctions In order to calculate the conjunction of categories we find the joint probability of the two categories P(A&B) = P(A)P(B) This is similar to the Prototype Theory for conjunctions.

Training Data ABCA&BA&CB&CSingleJoint A A A A BAB AAB B B B B C C C C C C

Training Data The model is almost perfectly consistent learner, meaning that it reproduces the original training data with 100% accuracy. For the conjunction examples #5 and #6 it classifies them as B and A respectively. They obtain a significantly higher score in the AB conjunction than in the AC or BC conjunctions. This seems to suggest that these two examples are more representative of one member of the conjunction than the other.

Test Data ABCA&BA&CB&CSingleJoint A>B>CAB>AC>BC C>B>ABC>AC>AB C>A>BAC>BC>AB C>B>ABC>AC>AB C>B=AAB=AC>AC

Graphs: Comparing Experimental results to Model results

Test Data The results are generally consistent with the experimental data. Except for #3 and #4: For #3 the experiment predicts AC>AB>BC, while the model generates AC>BC>AB For #4 the experimental data predicts C>B>A, the model gives B>C>A

Statistical Analysis The average for the correlation between the model and experimental data was R=0.88 At alpha =0.05 and df = n-2, this was significant. #1 0.82, #2 0.93, #3 0.85, #4 0.84, #5 0.88, #6 0.92

Unusual Predictions How would the model handle ? Output: A > B > C, AC > AB > BC Is it possible to ask the model about triple conjunction? Example: Model predicts: C>B=A, AB=AC>ABC>BC

Conclusion Naïve Bayes produces a good hypothesis of how people learn category classification. The use of probabilities matches well with the underlying logic of the correlations between the dimensions and categories. Creating a Causal Network might be an informative way to investigate further the interactions between the individual dimensions.

Limitations As the model uses a version of prototype to calculate its conjunction it is not able to capture overextension. To rectify this The formulae: Can be used to approximate overextension, where C is the category and KC is the set of non-C categories.

Limitations The model, also, does not take into account negative evidence. While it captures the general trend of the categories it does not, for example, represent the strength of negativity for Category C in test pattern #5 This pattern is very similar to the conjunction patterns given in the training data. The strong negative reaction seems to be caused by the association between these conjunctions and categories A and B.

The End