5 th -6 th December th Meeting Paris WP2: NERC

5 th -6 th December th Meeting Paris D2.3 Currently a nearly-finished draft. Reports on NERC v.2 NERC v.2 still deals only with 1 st domain but name matching and normalisation now added. Contains system documentation and evaluation results for ENERC, HNERC and INERC so far: FNERC still to come. New in this version is the use of the NERC-based demarcator between NERC and FE: this affects evaluation.

5 th -6 th December th Meeting Paris D2.3: NERC Systems Various improvements to ENERC, HNERC and INERC: tokenisation, lexical resources etc. Addition of name matching and normalisation but issue raised about whether these are best done as part of NERC or FE.

5 th -6 th December th Meeting Paris Normalisation & Name Matching Normalisation best performed after FE –For efficiency: only normalise units that are part of facts –FE disambiguates certain entities (e.g. SPEED) and this helps normalisation Name Matching could be done in NERC or FE –For co-referential entities within the same product description we need name matching before FE –For other entities, it can be done after FE where it will be helped by FE disambiguation (e.g. SOFT_OS).

5 th -6 th December th Meeting Paris Normalisation & Name Matching HNERC: name matching of coreferential entities within same product description – after NERC and Demarcator but before FE. All other name matching and normalisation after FE. INERC: name matching as part of NERC: integrated into the ontology look-up process. ENERC: both modules operate on entities and encode results as attributes on the entities (which can then be inherited by the facts). Can be done after NERC or FE (or both).

5 th -6 th December th Meeting Paris Evaluation Annotators of gold standard were instructed only to annotate entities which are part of product descriptions. Demarcation now happens after NERC, therefore the NERC modules annotate entities throughout the page. Evaluation of NERC against the gold standard gives false measure of precision. Evaluation of NERC+Demarcation combined would give accurate measure if the Demarcator is totally accurate, but not otherwise.

5 th -6 th December th Meeting Paris D2.3: NERC Evaluation HNERCINERCENERC RecPrec F-score RecPrecF-scoreRecPrec F-score MANUF MODEL PROC SOFT_OS

5 th -6 th December th Meeting Paris WP2 Tasks Finish D2.3 Finalise DTD for 2 nd domain and start annotating as soon as 2 nd domain corpus is ready. Build and evaluate NERC v.3 which deals with both domains.