1. Presented by: Prof. Ali Jaoua
Feature Extraction Based on Isolated Labels: Application for Automatic News Categorization
Presented by: Prof. Ali Jaoua

2. Agenda Introduction Financial Watch Project Related Work
Problems:
Reducing dimensionality of space of words
Exploring pertinent features
X2-statistic, Document Frequency Thresholding, Information gain
Proposed Approach
Basic Aspect & Preprocessing phase: (Empty words, Segmentation, Stemming,)
Conceptual Approach & Mathematical background
Conclusion & future work

3. Financial watch project objectives:
Develop a financial news text mining platform (sources from: web-pages, blogs and forum postings, messages, etc.)
Collect, analyze and extract relevant concepts and events from Arabic and English financial news
Trigger alerts to potential users of the system assisting them to take decisions
 Identify and alert decision-makers
Identify the best tools to be integrated into the FinancialWatch platform.

4. Why news should process ???
Applications
Health Care Delivery IE systems
Intelligence Gathering IE systems
IE for disaster response
Financial Watch System

5. Financial Watch Project

6. Automatic Text Categorization
Definition and History
Classify a set of documents into one or more predefined categories or classes.
Definition of set of alerting systems
Approach:
Supervised Text Categorization
Definition of financial news ontology
Semantics characterizing the concept labels
Use of Support Vector Machine (SVM) for evaluation

7. Proposed Approach Semantic Filtering Exploring pertinent features
Segmentation
Stop word Removal
Synonym and Stemming Process
Synonyms in English text: Loading Wordnet3 in database to process synonyms
Stemming process:
Arabic Stemmer
English Stemmer
Exploring pertinent features
Conceptual Approach & Mathematical background

8. Algorithm of Text Categorization System
Training Set
Global Feature List
TF.IDF Weighting
Document Representation
Training Vectors
Training Phase
Step 2: Obtain Training Vectors
Perform stop word elimination and stemming.
Apply TF.IDF weighting
Represent a document as vector.
Step 3: Apply SVM Classifier
Apply SVM classifier for each pair of classes of training vectors and obtain k(k-1)/2 classifiers.
Elimination of stop words
Stemming
Classifier 1
Classifier k(k- 1)/2
SVM Classifier ……

9. Formal Context Coverage based on Isolated Labels
Algorithm
Map textual document into binary relation R.
Compute the fringe relation Rd ∩ R◦R−1◦R associated with a binary context R and for all pairs (x; y) in Rd, only different generated concepts are added to the coverage C.
If R is still not covered by C, then it is extended by additional composed properties or labels.
Second and third steps are repeated until obtaining a complete coverage of the binary relation R.

10. FCA Background Formal Information Representation(1/2)
Formal Context I: (A,B,R)
{} {A1, A2, A3, A4, A5}
{B2}{A1, A2, A3, A4}
{B3}{A3, A4, A5}
{B1, B2}{A1, A2}
{B2, B3}{A3, A4}
{B3, B4}{A4, A5}
{B2, B3, B4}{A4}
{B1, B2, B3, B4}{}
Lattice of Context I R B1 B2 B3 B4 A1 1 A2 A3 A4 A5
A binary relation R between two finite sets D and T is a subset of the Cartesian product DXT.
A Rectangle: of R is a Cartesian product of two sets (A,B) such that AD, BT and AXBR.
If AXB  A´XB´ R, we have A = A´ and B = B´  the Rectangle is maximal
If AXB  A´XB´ R, we have A = A´ and B = B´  the Rectangle is maximal (Concept)
Concepts c1 and c2 are connected iff c1 ≤ c2 and ∄ c3 such that c1 ≤ c3 ≤ c2.

11. Background Formal Information Representation(2/2)
Formal Context I: (A,B,R)
{} {A1, A2, A3, A4, A5}
{B2}{A1, A2, A3, A4}
{B3}{A3, A4, A5}
{B1, B2}{A1, A2}
{B2, B3}{A3, A4}
{B3, B4}{A4, A5}
{B2, B3, B4}{A4}
{B1, B2, B3, B4}{}
Lattice of Context I R B1 B2 B3 B4 A1 1 A2 A3 A4 A5
Minimal Coverage: {C4,C5,C6} C4 C5 C6
Pseudo Concept (A4, B3)
Is the union of all concepts containing
(A4,B3): PS = I(B3.R-1) o R o I(A4.R)
The set of all concepts of a formal context and the partial ordering can be represented graphically using a concept lattice.
A concept lattice consists of nodes that represent the concepts and edges connecting these nodes.
The nodes for concepts c1 and c2 are connected if and only if c1 ≤ c2 and there is no other concept c3 such that c1 ≤ c3 ≤ c2.

12. Formal Context Coverage based on Isolated Labels
Example
D= {a b c d e. a b e. a b e. a b c d e. b d. a b c d e. a b c d e. d f g. b f g.}.
First Iteration
Isolated Points: {(x1, a), (x1, e), (x2 a), (x2, e), (x3, c)}
Concepts: C0={x0; x1; x2; x5; x6} × {a; b; e}. Labels are a, e.
C1={x0; x3; x5; x6} × {b; c} Label is c.

13. Formal Context Coverage based on Isolated Labels
Next Iteration (Coverage is not reached)
Expand
Repeat same steps until obtaining full coverage of R.
Concepts: C0={x0; x1; x2; x5; x6} × {a; b; e}. Labels are a, e.
C1={x0; x3; x5; x6} × {b; c} Label is c.
C2={x0; x4; x5; x6} × {b; d} Label is b.d.
C3={x7} × {d; f; g} Label is d.f.
C4={x8} × {b; f; g} Label is b.f.
Features: { a, e, c, b.d, d.f, b.f}

14. Formal Context Coverage based on Isolated Labels
Example
إرتفعت الأرباح الصافية لمصرف الراجحي السعودي بنسبة 10% خلال الربع الثاني من العام الحالي مقارنة مع الفترة ذاتها من العام الماضي. وبلغ صافي ربح الشركة خلال الربع الثالث 24مليون ريال مقابل صافي ربح بمقدار 21.8 مليون ريال للربع المماثل من العام السابق. كما بلغ إجمالي الربح خلال الربع الثالث 27 مليون ريال مقابل 24 مليون ريال عن نفس الفترة من العام السابق أي بارتفاع قدره 12.5%. وبلغ إجمالي الربح التشغيلي خلال الربع الثالث 24مليون ريال مقابل 21مليون ريال للربع المماثل عن نفس الفترة من العام السابق.
Apply Stop Word Elimination and Stemming
رفع ربح صفا صرف رجح سعد نسب ربع ثني عام حال قرن فتر عام مضي. بلغ صفا ربح شرك ربع ثلث مليون ريال قبل صفا ربح قدر مليون ريال ربع مثل عام سبق. بلغ جمل ربح ربع ثلث مليون ريال مليون ريال فتر عام سبق رفع. بلغ جمل ربح شغل ربع ثلث مليون ريال قبل مليون ريال ربع مثل فترعام سبق.
Labels (Features)
} رفع.ربح، صرف، رجح، سعد، نسب، قرن، مضي، ثني، شرك، قدر {

15. Illustrative example: original text

16. Illustrative example: (cont.)
Textual document after preprocessing
Features Extracted

17. Illustrative example: (cont.)
Stemmed Words

18. Methodology Algorithms
Find minimal coverage
Use of composed properties to reach a coverage
Find pertinent information in of quality and reasonable efficiency ..
Management Change
Transaction
Performance
Others

19. Evaluation – English Texts

20. Evaluation – Arabic Texts

21. Conclusion & Future work
New methods were proposed to handle dimensional reduction of the space of words
FCA is used for its simplicity and effectiveness as an analysis tool
Domain information is used to reduce the structuring noise and improve labeling process
Applications like corpus organization, text summarization and feature-extraction were experimented in context of the FWATCH project
Evaluation results show the significance of the new methods for incrementally updating a stable information store

22. Thank You Q & A