1. Semantic and phonetic automatic reconstruction of medical dictations STEFAN PETRIK, CHRISTINA DREXEL, LEO FESSLER, JEREMY JANCSARY, ALEXANDRA KLEIN,GERNOT KUBIN, JOHANNES MATIASEK, FRANZ PERNKOPF, HARALD TROST

2. About ASR in Medical Dictation. Reconstruction of literal transcripts to improve. Retraining of Standard Language Model with reconstructed transcripts.

3. Introduction  ASR eliminated the need to type whole documents instead the medical transcriptionists just need to do post-processing which too is a bit time consuming and tedious  To train the ASR system to perform better literal transcripts are required

4. A way of getting over this problem is unsupervised or lightly supervised learning. Although these methodologies reduce WER they do not give explanations for the mismatch

5. SPARC approach A framework for reconstruction of literal transcripts from the recognition result and final medical reports Available data: Audio Files, Draft Transcripts and Final report Error Categorization: recognition error or Reformulation

7. Phonetics similardissimilar SemanticssimilarMatchReformulation dissimilarCorrectionReformulation and correction The SPARC approach to text reconstruction. Based on semantic and phonetic similarity measurements, chunks of written and recognized text can be classified as either matches, reformulations, corrections or a combination of both.

8. Data Description Define a mismatch region as a contiguous sequence of mismatch edit operations in order to establish correspondences between matched words. A statistical study of a corpus of 80,000 medical reports with 38 million words revealed an average length of 2.3 words for a mismatch region and an average occurrence of 3.6 times for this region within the corpus.

9. Text alignment During alignment, both input documents are viewed as sequences of tokens. A generalized Levenshtein alignment algorithm is then applied to these sequences. Based on the algorithm three actions are performed: Substitution Insertion Deletion

10. Kitten  Sitting 1.Sitten  substitution 2.Sittin  Substitution 3.Sitting  Addition

12. Similarity measures Similarity measure are used in both text alignment and reconstruction. These measures are consulted by the levenshtein algorithm to improve accuracy The measures are used to condition the reconstruction rules.

13. Semantic similarity To measure semantic similarity the following resources are used a reference: Unified Medical Language System WordNet lexical database The morphosyntactic information from the lexicon was worked into the finite-state transducer that is used as a morphological lexicon.

14. 7 identical (modulo case or 5 synonymous 2 same UMLS semantic type 6 same root (only inflection) parent(word1,word2) or parent(word2,word1) 1 direct hierarchical relation between semantic types 4 morphologically derived0 no similarity at all 3 conceptual siblings

15. Phonetic similarity Requires three sets of input: Phonetic symbol sequence from the recognized text The orthographic word sequence from the recognized text Word sequence from the written text

17. Automatic phonetic transcription In a first step, the written text is transferred to the phonetic domain with automatic phonetic transcription (APT). This is done by a simple lexicon lookup A simple regular expression syntax was defined to encode the possibly many speaking and pronunciation variants in a single string

18. Automatic syllabification An online automatic syllabification algorithm was implemented to determine syllables directly from the texts. For syllabification, a number of competing syllabification constraints are applied to the input words. But due to lack of primary stress information the “noonset” Constraint was removed

20. Combined similarity measurement The semantic and phonetic scoring functions are used as building blocks for a combined scoring function that best exhibits the behavior that is required for further processing. Semantic matching is applied in the first place to filter out clear cases and avoid overstretched regions of correspondence, before phonetic matching is used to find detailed matches.

21. Text reconstruction Deformatting Identifying and retracing moved blocks Application of reconstruction rules Reconstructing moved blocks

23. Rule engine The reconstruction rules that are interpreted by this engine provide a mechanism for inspecting a sliding window that is moved over multiple columns according to their alignment rule rulename match -w/window regexp/ # inspect sliding window do # specify reconstruction result done

25. Rule definitions 1.Baseline: 2.Recognised-only 3.Written-only 4.Context (CTX) 5.Overlap, greedy 6.Overlap, selective

26. Experiments The quality of the reconstruction was tested. The speech recognition performance using reconstructed texts is measured

27. Reconstruction quality Metrics: Recall = | COR | / |COR | + | MISS | Precision = | COR | / |COR | + |WRONG| harmonic mean F1

30. Rule-based vs. data-driven reconstruction k-NN NN SVM

32. Automatic speech recognition The SPARC method was compared to the standard method for language model training and to a random generation of reconstructed text.