USER INTERFACE USER INTERFACE January 12, 2006 Intern 박지현 Performance analysis of filtering software using Signal Detection Theory Ashutosh Deshmukh, Balaji Rajagopalan

USER INTERFACE USER INTERFACE Performance analysis of software filters - Single software filters - Multi-method software filters Introduction Signal Detection Theory (SDT) Conclusion 1 C o n t e n t s 3 4 2

USER INTERFACE USER INTERFACE Introduction Today, individuals face less of a problem caused by lack of information, as in the pre-Internet era, but more of information overload and lack of the ability to control the flow of information. For example, families want to protect young children from pornographic sites. The purpose of this paper is to analytically evaluate performance of filtering software based on Signal Detection Theory (SDT).

USER INTERFACE USER INTERFACE :: Signal Detection Theory Model Signal Detection Theory (SDT)

USER INTERFACE USER INTERFACE Signal Detection Theory (SDT)  The decision maker sets the criterion value based on the prior probabilities of the observation being signal or noise and the benefits associated with hits and correct identifications and costs associated with misses and false alarms. Criterion Value =  The decision maker sets the criterion value based on the prior probabilities of the observation being signal or noise and the benefits associated with hits and correct identifications and costs associated with misses and false alarms. Criterion Value = :: The benefits and costs The decision maker classifies an event as signal or noise in two steps. ‘A likelihood ratio’ and ‘the benefits and costs’.  The decision to accept a hypothesis is taken by comparing this likelihood ratio with to a criterion C: Likelihood ratio (LR) = = If LR(S :N) ≥ C, then select S If LR(S :N) < C, then select N.  The decision to accept a hypothesis is taken by comparing this likelihood ratio with to a criterion C: Likelihood ratio (LR) = = If LR(S :N) ≥ C, then select S If LR(S :N) < C, then select N. :: A likelihood ratio c LR

USER INTERFACE USER INTERFACE LR = ( ) ( ) Performance analysis of software filters The objective of filtering software is to block unacceptable websites from the user. :: Single software filters  If LR ≥ classify the website as unacceptable.  If LR< classify the website as acceptable.  If LR ≥ classify the website as unacceptable.  If LR< classify the website as acceptable. α LR

USER INTERFACE USER INTERFACE Performance analysis of software filters The objective of filtering software is to block unacceptable websites from the user. :: Multi-method software filters LR =≥ If we assume the costs associated with miss rates and false alarm rates are equal.. LR ≥ α (classify the website as unacceptable. ) If hit rate = 75%, false alarm rate = 25% the base rates of unacceptable websites = 1% *1% ≥ 1 So five filters are required to make a correct decision. If the hit rate is 80% and false alarm rate is 20% then only four filters are required to make a correct decision. If hit rate = 75%, false alarm rate = 25% the base rates of unacceptable websites = 1% *1% ≥ 1 So five filters are required to make a correct decision. If the hit rate is 80% and false alarm rate is 20% then only four filters are required to make a correct decision. For example X

USER INTERFACE USER INTERFACE Conclusion In an extremely dynamic environment such as the internet, software filters are still quite useful in the business and family environment. An alternative suggested by the SDT framework is the use of a multi-method based software filters.