Complex Client Interactions using Semantic Search

Specific Knowledge and State The client has multiple interacting states which need to be reconciled with the Call Centre knowledge - held in complex text. Examples – health insurance, tax, social security

Handling a Huge Diversity These are areas which do not suit the simple call centre approach – a manual, a call script, a checklist The state of the client must be dynamically matched to the complexities of the relation with the client – it could be legislative or contractual, and is held in a textual form which would take hours to read carefully Health funds in particular have the problem of thousands of individual contracts with employers, with lots of differences among them, and then millions of employee certificates – a huge diversity

Client State Information about the state of the client may come from: The organisation database What the client or provider tells you about what has happened What the client or provider tells you about what might happen These pieces of information are used to update the state of the insurance policy or tax estimate – the policy is no longer just words, but is an active working structure

Text Query The system should allow the operator to enter free text queries We should expect the interface to handle defined terms – terms like “covered expense” that are defined in the documentation, and then “non- covered expense” whereas “non-network provider” It gets tricky – the user won’t know exactly what are defined terms, so the system needs to figure out whether a defined term or the word in its ordinary sense is meant Is ASD covered?

What the Query Can Ask A query can ask: To find context in the document – where is… mentioned? To determine an object – which is the primary plan? To find a logical state – is aromatherapy covered? To find a value – how much is the copay for this operation? It is useful to have a search mechanism which can seamlessly handle any of these different types of queries – particularly when there is a sequence – you can’t work out how much without first working out whether it is covered, which means you had to find the relevant section and the things that are excluded The full range of queries is only possible in semantic searching

What the Query Looks Like The query is turned into a structure using interrogative grammar patterns – they are only turned on when in query mode The search then goes down the link – in this case, for an existential value, where the “done” will match any action relation, and the “it” will match any object or relation (except a person) Searching can be on objects, relations, logical states or numeric values

Four Approaches If the system can read and interpret free text queries, there are four possible approaches: Semantic analysis of query, then simple keyword searching of text Searching using switched document structure Searching of semantic structure A dialog, where the searched structure changes during the interaction What would be copay and out of pocket for heart surgery costing $45,000

Simple Searching Even for simple searching, the query needs to be elaborated to find the various ways the same query can be phrased Is ASD covered Do you exclude ASD How much do you pay for ASD Some questions implicitly include several steps – simple searching Is not good at this, so there is a strong (hidden) constraint on how much meaning can be packed into the query Prolotherapy is specifically excluded, but it could be called “proliferation therapy” or “regenerative injection therapy” – these have to be caught too

Simple Searching One problem with simple searching is that the document has a strong logical structure – sentences have applicability controlled by their location in some part of the document structure. We are interested in claims for pregnancy: No insurance is extended to a child born as a result of a covered person's pregnancy. The statement is in the Extension for Total Disability clause – it is only applicable there and would be quite erroneous if found with a simple word search for a covered person

Simple Gets Technical We have a clause which says: Metastatic cancers are not covered if resulting from a pre-existing condition. Fred rings up – “the cancer has spread to the other leg”. This meets the definition of metastatic, but if you don’t know, the system can’t tell you. If it has a searchable dictionary entry which says: Main Entry: metastasis 1 the spread of a disease-producing agency (as cancer cells or bacteria) from the initial or primary site of disease to another part of the body; also : the process by which such spreading occurs 2 : a secondary malignant tumor resulting from metastasis the system could work out what Fred was saying, and no, it isn’t covered. It had to take the simple meaning and search the meaning of the word in the exclusion using multiple relations.

Simple Gets Technical 2 We have a Defined Term: Advanced imaging, for the purpose of this definition, includes Magnetic Resonance Imaging (MRI), … and Computed Tomography (CT) imaging. And then Advanced imaging … in a health care practitioner's office Network health care practitioner 100% benefit payable after network provider deductible Someone rings up about an MRI, it is necessary to find its inclusion in a Defined Term, then go looking for the benefits for the Defined Term – keyword searching Is not good at this. Some background knowledge is also essential – a CAT scan is a common name for a CT image.

Searching Using Document Structure The next level of searching is to still use key words extracted from a semantic analysis of the query to search the document, but control the areas of text that can be searched depending on the state of the contract and the state of the client – if they are paid up, don’t look at the termination clause unless they ask about it This approach can drastically reduce the number of hits on irrelevant material – as an example, a simple search on pregnancy in a health fund certificate found 18 hits, a structured search found 6 – bringing the operator much closer to an answer The method is still relying at its lowest level on finding words in text, so it can’t do implied steps, calculations or other necessary tricks

Automatic Reading The next level of complexity is to use semantic analysis on the query, then use the extracted structure to search the document structure. The system needs to automatically read the document, transforming it into an active semantic structure. This is done long before the call occurs. When a call comes in, the current state of the client is used to update the extracted structure, turning off all the parts that are not currently applicable and setting variables in the structure – states and values – that are relevant to the client. The Call Centre operator then queries the updated structure using free text queries which are also automatically read and converted into structure – the query handling can handle multiple search steps and calculations What would be copay and out of pocket for ASD?

Semantic Search Once the semantic structure has been extracted, the search can be highly specific, using the relations among the objects in the query to find the precise area in the structure that matches what the query is about, taking into account synonyms, antonyms and mapping between relations like buy and sell Automatic reading of large documents is slow – about the same speed as an attentive human, so it has to be done beforehand. Many documents are mostly the same – boilerplate – so this can greatly speed up the reading task, where structure already extracted from one document can be loaded into the structure for a new document

Reading Using Semantics The text is parsed using all the knowledge available to the system – knowledge about the domain or area covered by the text – financial, legislative, medical, including what it has already learned from the document it is reading The system knows how to unravel difficult legalistic text

Typical Legal/Medical Health insurance certificates are an uneasy mix of legal terms – “for the purposes of this definition” – medical science – “Magnetic Resonance Angiography” – and medical terms – “auto islet cells”, “Placenta previa”. To provide support, the system needs to integrate these areas, and provide analysis across the integration. The people who contribute to one area will usually know little about the other areas, and it is easy to get lost in a large document.

Definition of Metastasis The structural definition of a technical word. There are two meanings – the process of spreading or the resultant tumor. The definition can be used in searching to find equivalent descriptions - “the cancer spread”

Unravelling Definitions A large document will have defined terms, and the definitions are often circular or tangled Policy means the document describing the benefits we provide as agreed to by the policyholder. Policyholder means the legal entity identified as the policyholder in the policy who establishes and endorses an employee benefit plan for insurance coverage. “Policy” is defined in terms of the Policyholder, and “Policyholder” is defined in terms of Policy – this is a mild example of what can be a serious problem in understanding the text – note that the Policyholder is defined in terms of the policyholder – this is a self-referential error which the system has to deal with

Document Structure Complex documents have a strong logical structure, which needs to be considered when searching for answers. Parts of the structure can “disappear” or be voided by particular circumstances

Semantic Structure Extracting the semantic structure isn’t a trivial exercise – it is also a good time to fix the semantic errors in the text to prevent garbage getting into the Call Centre

What Sort of Errors? Even in documents that have been closely read by specialists, you can expect inconsistencies, errors, ambiguities and weak statements: Defined terms not defined or defined in terms of themselves or the form changing because someone didn’t check, or pasted something from another document which had different definitions Document references that don’t contain the target relation – “The benefits listed in Section 5” Something described differently in different parts of the document Debris from editing that was never cleaned up – missing indexed list items Lack of completeness - “person over 50 with normal risk” – what happens for person with high risk? These things are guaranteed to confuse when an answer is needed quickly, so are cleaned up beforehand

Searching Searching is much easier in a structure than in text, but we have to allow for many different ways of asking the same question Something can be excluded, not included, left out, omitted, forgot to put it in, didn’t think it was necessary If we force the operator to use the exact terms, many things will be missed – it has to be resilient

Dialog The next step, once the semantic structure is available, is to provide dialog capability – that is, the system updates its state each time it gives an answer, so the next answer is dependent on all that has gone before for this client interaction – the information that has been given in the queries, any database lookups prompted by those queries, any suggestions that have been accepted (it has to work out which suggestions haven’t been accepted) This capability can be made directly accessible to the user – the ability to use free text queries on a combination of complex text and database information greatly extends the usefulness of a customer portal What would be copay and out of pocket for ASD? Copay $3200, OOP $4500 What if I waited three months? Copay $1200, OOP $2500

Support for Call Centres We can provide support for Call Centre operators directly from complex free text – eliminating guesses, greatly reducing mistakes, reducing operator stress The same semantic structures can be used for a Customer Web Portal Active Structure technology from Interactive Engineering Pty Ltd