1. Knowledge Acquisition and Modelling
Knowledge Acquisition and Elicitation
Ref: Knowledge Acquisition in Practice: A step by step guide, Milton, Springer-Verlag

2. Knowledge Engineering
Transfer View
Human knowledge transferred to knowledge base
=>knowledge exists and is accessible
Typically interviews and task execution and observation used for KA
End result set of rules that exercise knowledge made explicit
Modelling View
Need to build models
Incremental, evolutionary process
Model is an approximation of reality
Models are subjective

3. KA Typology 3 KA techniques natural techniques contrived techniques
modelling techniques
interviews
observation techniques
group meetings
questionnaires
unstructured interview
semi-structured
interview
structured interview
card sorting
three card trick
rep grid technique
constrained tasks
20-questions
commentating
teach back
limited time
limited information
laddering
process mapping
concept mapping
state diagram mapping
KA Typology 3

4. Natural Techniques

5. Interview Techniques
Knowledge engineer asks questions of the expert or end user.
Essential method for acquiring explicit conceptualisations and knowledge, but poor for tacit knowledge.
Variations:
Unstructured interview
Free flowing, used in early stages of elicitation, can produce basics of knowledge domain, basically broad chat
Semi-structured interview
Main technique for elicitation
Pre-defined questions sent to expert prior to interview, supplementary questions asked at interview. Can be used as part of validation.
Structured interview
Pre-defined set of questions, can simply be filling in a questionnaire at the interview.

6. Interview Techniques Dependent on
The questions asked
Ability of the expert to articulate the knowledge
Model built on knowledge elicited during interview
Model reviewed by the expert

7. Modelling Techniques

8. Modelling Techniques Use of knowledge models with experts
Used as validation and refinement
Can show a basic model to an expert and prompt them to modify.
Can show a complete model of knowledge provided by one expert to a second expert to cross-validate.
Can create one from scratch with an expert – start with a blank page

9. Model Based Knowledge Acquisition
Each model emphasizes certain aspects of the system to be built and abstracts from others.
Each model is indicative of one view of the world
Models provide a decomposition of knowledge- engineering tasks:
while building one model, the knowledge engineer can temporarily neglect certain other aspects.
The knowledge-engineering space of choices and tools can to some extent be controlled by the introduction of a number of models 9

10. Knowledge Modelling Process10

11. Knowledge Modelling Use skeletal models Or generic tasks
Generic tasks are templates of problem-solving activities that can be configured together to describe any intelligent activity.
Modelling Frameworks 11

12. Knowledge Modelling
At least five different types of knowledge are distinguished:
Tasks-goals
correspond to the goals that must be achieved during problem solving.
Problem-solving methods
ways to achieve the goals described in tasks. In some knowledge modelling frameworks, problem-solving methods define subtasks to which other problem solving methods can be applied. We will call such a decomposition a task instance.
Inferences
describe the primitive reasoning steps in the problem solving process.
Ontologies
describe the structure and vocabulary of the static domain knowledge.
Domain knowledge
refers to a collection of statements about the domain.

13. Principles Divide and conquer.
Configuration of an adequate model set for a specific application.
Models evolve through well defined states.
The model set supports project management.
Model development is driven by project objectives and risk.
Models can be developed in parallel. 13

14. Recommended Reading Knowledge Engineering: Principles and Methods
Rudi Studer, V. Richard Benjamins and Dieter Fense
Data & Knowledge Engineering (1998)
Volume: 25, Issue: 1-2, Publisher: Elsevier
uder98knowledge.pdf

15. Contrived Techniques

16. Knowledge Capture – Specialised Techniques
Contrived Techniques
Primarily for deep, tacit knowledge
Involve the expert performing tasks they would not normally do as part of their job.
Most of these techniques come from psychology

17. Knowledge Capture – Specialised Techniques
Important types:
Concept (card) Sorting
Three Card Trick (Triadic)
Repertory Grid Technique
Constrained Tasks
20-questions
Commentary
Teach Back
Usually involve expert doing two types of task:
Tasks they normally perform
Commentary is useful here
Tasks designed to probe the expert
Concept sorting or Triadic

18. Concept (Card) Sorting
Way of finding out how an expert compares and orders concepts
Can reveal knowledge about classes, properties and relations
Works best in small groups
Simplest form is card sorting
Collection of concepts (or other knowledge objects) are written on separate cards
Cards sorted into piles by an expert in to piles - each card in a pile must have something in common
Each time the cards are sorted it will be based on an attribute and each pile will represent a value
Enables significant elicitation of properties and dimensions
Used to capture concept knowledge and tacit knowledge
Use in conjunction with triadic method
Can also sort objects or pictures instead of cards

19. Concept Sorting – How To ?
Decide what classes of concepts you want to explore (in particular their properties – attributes and values)
Write the name of each concept on a separate card
At the session explain to the expert what is going to happen
Ask the expert to name the piles
Write down (or record) the results of the sort
Collect the cards and ask the expert to sort again
Repeat until the expert can’t sort anymore

20. Triadic Elicitation Method (3 card trick)
Used to capture the way in which an expert views the concepts in a domain.
Present three random concepts and ask in what way two of them are similar but different from the other one.
Answer will give an attribute.
A good way of acquiring tacit knowledge.
How does it work ?
Select 3 cards at random
Identify which 2 cards are the most similar?
– Why?
– What makes them different from the third card?
Helps to determine the characteristics of our classes
Picking 3 cards forces us into identifying differences between them
There will always be two that are “closer” together
Although which two cards that is may differ depending on your perspective 20

21. Triadic Elicitation – How To?
Explain to the expert that you are trying a technique to draw out deeper knowledge
Collect all cards previously used
Shuffle cards and randomly select 3
Place them on the table, two close together one further away
Ask how the two close together are similar and the other different
Write down (or record) what the expert says using an attribute
Use the results to find another card sort to find the values of all concepts for this attribute
If the expert can’t identify an attribute, just pick another 3 cards
Repeat until the expert can think of no more differences

22. 20-Questions Expert asks questions of the engineer
The Knowledge Engineer thinks of an object/concept in the domain
Expert asks yes/no questions to the knowledge engineer in order to deduce an answer.
Knowledge Engineer
notes the questions and the order in which they are asked
need not know much about the domain, or have an answer in mind, just answer “yes” or “no” randomly
The questions asked provide a good way of quickly acquiring attributes in a prioritised order.
Can provide an insight into the key aspects, properties or categories and their relative priorities.
Note that the main purpose of this exercise is not really to try and find out what the Engineer is thinking of, but to determine the important properties! 22

23. 20-Questions – How To?
Decide on set of concepts you need to explore in more detail
Explain to the expert what is going on
Ask the expert to imagine that you the engineer have the same level of knowledge they do about the set of concepts
Instruct the expert that they should ask the least number of questions to deduce the answer
Engineer can only answer yes and no
Explain that the best way is to ask questions which split the concepts in half so that each question eliminates half the possible solutions
Start
As each question is asked write it down (or record it)
When a number of questions have been asked take the expert back to an earlier question and change the answer you gave to prompt the expert to ask further questions
After the session extract the attributes and values (or new concepts) from the questions asked and these will be added to the knowledge base

24. Laddering
Involves the construction, modification and validation of trees.
Accessing personal construct system
Take a group of things and ask what they have in common
Then what other ‘siblings’ (brothers/sisters) there might be
A valuable method for acquiring concept knowledge and, to a lesser extent, process knowledge.
Can make use of various trees:
concept tree
composition tree
attribute tree
process tree
decision tree
cause tree
Starting with some ‘seed’ item you grow a network of how the domain items
relate to each other by asking a number of ‘directional’ questions such as:
To move towards the specific:
“How can you tell it’s a ?”
“Can you give examples of ?” 24

25. Example
Source: Bourne and Jenkins , Eliciting Managers' Personal Values: An Adaptation of the Laddering Interview Method, Organizational Research Methods, SAGE 2005

26. Concept Tree
Hierarchical diagram of concepts showing classes and members
Activities to create
Move nodes (concepts) around the tree
Add new node
Deleting nodes
Renaming nodes
Difficulty is avoiding the problems which requires knowing:
All links on the tree represent an ‘is-a’ relationship
Terminology to describe the tree
What classes to use in the tree
Naming conventions to use
How to deal with complex cases (e.g. multiple parents, synonyms)

27. Concept Tree – ‘is-a’ relationship
Is-a = is a type of
Different to ERDs
vehicle
traffic
ship
traffic issues
lorry
car
steam ship
sailing ship
shipping lanes
pollution
congestion
Road safety
What are the mistakes in this tree?

28. Concept Tree - Terminology
Root node
Leaf node
Branch
Parent
Children
Descendants

29. Concept Tree – What classes to use?
Class is a concept which has children on a tree
Other concepts are related to it by an is-a relationship
To develop classes use either a top-down or bottom-up approach
Top-down start with a set of general classes and refine
Bottom-up start to develop classes by grouping those concepts that are similar

30. Repertory Grid technique
Used to elicit attributes for a set of concepts
Used to rate concepts against attributes using a numerical scale
Uses statistical analysis to arrange and group similar concepts and attributes
Allows the expert to provide a rating of each concept for an attribute in concept sorting
A useful way of capturing concept knowledge and tacit knowledge
When many ratings are provided using many attributes statistics can be applied to find clusters and correlations
Requires special software
This technique is based on a model of how people regard the world, called ‘personal construct theory’,
devised by psychologists. The idea is that each domain element in some set of interest is classified
according to a set of constructs that apply to all the elements of the set to some degree – a construct
is a linear scale with two extreme values, such as heavy/light, cheap/expensive, wet/dry, conscientious/
dilatory or whatever.
Every construct of the set of constructs that apply to the domain set is then expressed as a numeric
scale, using the same range of numbers each time, typically 1—5 or 1—9, with the centre of the scale
representing some kind of medium value for that construct. A grid is constructed and the expert is
asked to assign a number for each construct for each domain element, showing how s/he would place
that domain element on that construct scale.
An example will help to make this clearer. Imagine that the domain elements are certain types
of crime: petty theft, burglary, drug-dealing, murder, mugging and rape. It should be obvious from
figure 3.1 that the various constructs are being ranked on a scale of 1—5 and that it is not always the
value on the left-hand side of a scale that corresponds to the value 1. 30

31. Repertory Grid – How To? 1st stage 2nd stage 3rd stage 4th stage
Concepts are selected (between 6 & 15)
Set of approx. same no. of attributes is also required
Should be such that values can be rated on a continuous scale (e.g. small to large)
Chosen from knowledge previously elicited
2nd stage
Expert provides a rating for each concept against each attribute
Numerical scale is used
3rd stage
Ratings are applied to cluster analysis to create a visual representation of the ratings called a focus grid
Concepts with similar scores will be grouped together, attributes with similar scores will be grouped
4th stage
Engineer walks expert through the results to gain feedback and prompt for further knowledge about the groupings
If needed more concepts and attributes are rated and included in the grid

32. Repertory Grid Example
Domain elements are certain types of crime: petty theft, burglary, drug-dealing, murder, mugging and rape.
This is one expert’s view on the issue.
Consider carefully whether any pair of attributes are very similar, by comparing horizontal lines in this grid.
The closest is probably the personal/impersonal one and the major/petty one.
Beware, when making this comparison, that the expert may have inadvertently ‘inverted’ the scale for just one of two similar constructs.
For example, in the example the major/petty construct has a value of 5 for ‘major’. If the expert had chosen 1 instead, and 5 for ‘petty’, then this construct and the personal/impersonal one would look very different.
Further analysis may lead you to omit one pairing of constructs.
Following that you would draw up a table showing how similar or dissimilar each domain element is from the others.
For example, when the absolute-value metric is used, the (numeric) difference.
Grid shows various constructs are being ranked on a scale of 1—5 and that it is not always the value on the left-hand side of a scale that corresponds to the value 1. 32

33. Constrained Tasks
Expert performs a task they would normally do, but with constraints.
Variations:
limited time
limited data
Useful for focusing the expert on essential knowledge and priorities 33

34. Commentary and protocol generation
Expert provides a running commentary of their own or another’s task performance.
A valuable method for acquiring process knowledge and tacit knowledge.
Variations:
self-reporting
imaginary self-reporting
self-retrospective
shadowing
retrospective shadowing 34

35. 35

36. Knowledge Analysis and Modelling

37. Knowledge Analysis
Identifying the elements needed to build the knowledge base
Concepts
Things that constitute a domain
Main elements of the k-base
Attributes
Qualities or features belonging to a class of concepts
Values
Specific qualities or features of a concept that differentiate it from other concepts
Relations
Way in which concepts are associated with one another

38. Concepts Physical concepts Pieces of information
Products, components, machines
Pieces of information
Plans, goals, requirements
Sources of information
Documents, databases, websites
People and roles
Experts, roles of experts
Organisations and groups
Producers, suppliers, consumers, departments
Areas of knowledge
Marketing, physics, chemistry
Functions
Purpose of components or roles
Tasks
Activities performed by experts
Issues
Problems, solutions, advantages, disadvantages
Physical phenomena
Mechanisms and forces
Other issues
Constraints, behaviours, states

39. Attributes Of physical objects Of information Of people
Shape, age
Of information
Source, format, importance
Of people
Gender, age, personality
Of organisation
Size, turnover, product range

40. Values Come in different varieties Dependent on type Categorical
Adjective, number, sentence, paragraphs, hyperlinks, images, pictures
Categorical
For values that are adjectives
Numerical
For values that are numbers
Text
For values that are one or two sentences
Hypertext
For values that are chunks of hypertext

41. Relations Has part Performs Followed by Requires Causes Produces
Can have an inverse relation
Short exercise
Think of something that illustrates each one of these

42. Knowledge modelling
K-model = way of viewing the knowledge in the k-base
Each model provides a different perspective on the knowledge
Helps clarify the ‘mess’ that is the knowledge
Can be used in elicitation

43. Trees Diagram showing hierarchical arrangement of nodes Node = concept
Link = relationship
Concept tree
Composition tree
Cause tree
Mixed tree

44. Concept tree Each link is an is-a relation Taxonomy
Read from right to left
Taken from

45. Other types of tree Composition tree Process tree Attribute tree
All links are has-part
Used to show components and sub-components of a concept
Process tree
Special form of composition tree
All nodes are tasks
Attribute tree
Shows attributes and values to describe a concept
Mixed tree
Contains more than one type of relation

46. Matrices Attribute matrix
Presents set of properties of a concept (attributes and values)
Concepts on vertical axis
Attributes and values on horizontal axis

47. Relationship matrix
Shows two sets of concepts related to one another using a specified relationship
Cells show which pairs of concepts have the relationship

48. Maps Shows an arrangement of nodes linked by arrows
Each node represents concept
Link represents relationship
Concept maps
Process maps

49. Concept map
Many different types

50. Knowledge Analysis – How to?
How do you identify concepts from interview transcripts and documents?
Need some codification
Highlighters – different colours for different things

51. Typical project 47 steps proposed by Milton
Knowledge Acquisition in Practice: A step by step guide, Milton, Springer-Verlag
Phase I
Start, scope and plan the project
Phase II
Initial capture and modelling
Phase III
Detailed capture and modelling
Phase IV
Share and store knowledge

52. Phase I – Start, Scope and Plan
Identify a project
How it can benefit, what it involves
Gather opinions from relevant people
Document ideas as project proposal
Seek agreement on proposal from key people
Start knowledge capture
With domain experts break the domain into different topics and rank against key criteria
Identify a proposed scope and finalise
Identify sources of knowledge
Define and understand the type of project to be able to create a schedule
Collate the proposal, scope and schedule into a project plan and disseminate with other materials to team

53. Phase II – Initial Capture and Modelling
Learn the basics of the domain from documents and informal conversation with experts
Prepare for semi-structured interviews then execute and transcribe
Analyse results to identify concepts, create a concept tree to develop a taxonomy and validate with experts
Create a k-page for each concept
K-page = 2 column table showing all knowledge associated with a concept
Create a glossary
Build a meta-model showing the relationships between concepts and relationships
Build appropriate k-models
Continue with validation models

54. Phase III – Detailed Capture and Modelling
Use further interviews and modelling to capture more detailed knowledge
Finalise k-model
Prepare prototype end product used to carry out assessment exercise with sample of end-users
Produce a completion plan defining what needs to be done to complete the project
Use specialised techniques to do detailed knowledge capture
If needed cross-validate between experts and resolve conflicts

55. Phase IV – Share and store knowledge
Define and create format of end-product
Create provisional end-product using k-base
Give to experts for full validation
Create final end-product and release for use
After use for some time assess impact on organisation and document it
Conduct complete product review to learn lessons and make suggestions to change methodology

56. Ensure end-product is useful, usable and used
End-users must find
Find product useful
Find product easy to use
Actually use it