1. Dynamic Category Profiling for Text Filtering and Classification
Rey-Long Liu
Dept. of Medical Informatics
Tzu Chi University
Taiwan, R.O.C.

2. Goal
Promote both precision and recall of text filtering and classification by
Finding more suitable features (terms) to measure the degree of acceptance (DOA) of a document d w.r.t. a category c
Deriving a method to make filtering and classification decisions based on the DOA estimation

3. Motivation
Previous techniques often find and employ those features (terms) that are
Representative for a category
Discriminative to distinguish a category
Unfortunately, content overlapping was often ignored
A document d may be classified into a category c only if their share the same content to some extent

4. An example Two categories
c1: computer networks
c2: computer animations
Previous techniques tend to employ the term like “network” and “animation” as features
They are both representative & discriminative
Unfortunately, the term “computer” may NOT be selected, but
It is helpful to filter out those documents that are about network but NOT computers (e.g. traffic network)

5. To discriminate c from others To validate content overlapping
Therefore, features of a category should be selected dynamically when a document is entered
To discriminate c from others
To validate content overlapping
Features that correlate with c
Features that correlate with other categories
Features that appear in c but do not appear in d
Features that do not appear in c but appear in d
Underlying classifier
Considered
Not considered
Dynamic profiling

6. The method: DP4FC
DP4FC: Dynamic Profiling for Filtering & Classification
Associating various classifiers with DP4FC
Training
Document for TF & TC
Documents for Threshold Tuning
DP4FC
Filtered Documents
Classified Documents
Integrated TF & TC
Underlying Classifier
Documents for Classifier Building
Classifier Building
Threshold Tuning
DOA Estimation by Dynamic Profiling
Testing
DOA Estimation

7. DOA estimation by dynamic profiling
Procedure DOAEstimationByDP(c, d), where
(1) c is a category,
(2) d is a document for thresholding or testing
Return: DOA value of d with respect to c
Begin
(1) DOAbyDP = 0;
(2) For each term t in c but not in d, do
(2.1) DOAReduction = Support(t, c)  log2(IDF of t in training data and d);
(2.2) DOAbyDP = DOAbyDP - DOAReduction;
(3) For each term t in d but not in c, do
(3.1) DOAReduction = Support(d, c)  log2(IDF of t in training data and d);
(3.2) DOAbyDP = DOAbyDP - DOAReduction;
(4) Return DOAbyDP;
End.

8. Making a filtering and classification decision
Two thresholds are derived
One is based on the DOA values produced by the underlying classifier
The other is based on the DOA values produced by DP4FC
A document may be classified into a category only if its DOA values are greater than or equal to the corresponding thresholds of the category

9. Experiment Aspects Settings (1) Source of experimental data
(A) Reuter-21578
(B) Yahoo text hierarchy
(2) Split of test data
(A) In-space test data (for evaluating TC)
(B) Out-space test data (for evaluating TF)
(3) Split of the training data for classifier building (CB) and threshold tuning (TT)
(A) 50% for CB; 50% for TT (with 2-fold cross validation)
(B) 80% for CB; 20% for TT (with 5-fold cross validation)
(4) Parameter settings for the classifier
Different sizes of feature sets on which the classification methodologies were built

10. The underlying classifier
The Rocchio’s classifier (RO)
1*DocPDoc/|P|  2*DocNDoc/|N|
P and N are the sets of positive and negative documents, respectively
RO is often tested in both text filtering and classification
Parameter setting
1=16; 2=4
Previous studies showed that this setting is good for RO

11. Evaluation criteria For text classification: For text filtering
Precision (P), Recall (R), and F1=2PR/(P+R)
For text filtering
Filtering Ratio (FR)
# out-space documents filtered out / # out-space documents
Average Misclassifications (AM)
# misclassifications / # out-space documents misclassified into the category space

12. Results
Performance (in F1) in processing in-space documents

13. Performance (in FR) in processing out-space documents

14. Performance (in AM) in processing out-space documents

15. Conclusion For each category, most documents should be filtered out
Content overlapping between a document and a category is thus important
It measures how d talks about those contents not in c, and vice versa
Unfortunately, it is often ignored by previous techniques
It calls for dynamic profiling for each category
With dynamic profiling, the classifier’s performance may be both better & more stable in both filtering & classification

16. Thank you